If you're seeing DNS issues with Phabricator (for example, inexplicable DNS failures connecting to RDS or OAuth hosts) in the last day or so, it may be because of your system has betrayed you and silently replaced libc with a haunted libc that has fewer remote execution vulnerabilities but more spooky ghosts.

This error might look like one of these messages, or might be some other DNS-related error:

php_network_getaddresses: getaddrinfo failed: Name or service not known
[cURL/6] (...) <CURLE_COULDNT_RESOLVE_HOST> There was an error resolving the server hostname. ...

You can detect if this "upgrade" occurred by looking at the end of /var/log/dpkg.log to see if it records a recent libc upgrade. Such an upgrade might look like this:

$ tail /var/log/dpkg.log
...
2017-03-21 06:52:17 status installed libc-bin:amd64 2.19-0ubuntu6.10
...

There are conflicting reports of how to remedy this, but repeatedly performing these steps until the ghosts leave appears to be a fix:

• Restart php-fpm, nginx, and/or apache.
• Restart the host.

Collected ghost stories:

• https://news.ycombinator.com/item?id=13921226
• https://forums.aws.amazon.com/thread.jspa?messageID=774198&tstart=0
• https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/1674733