Page MenuHomePhabricator

Mac OS X Mojave, SIP, Code Signing, and Apache
Open, LowPublic

Description

After upgrading to Mojave, existing PHP modules built on an older system version fail to load:

PHP Warning: PHP Startup: Unable to load dynamic library '/Users/epriestley/src/apcu-5.1.12/apcu-5.1.12/modules/apcu.so' - dlopen(/Users/epriestley/src/apcu-5.1.12/apcu-5.1.12/modules/apcu.so, 0x0009): code signature in (/Users/epriestley/src/apcu-5.1.12/apcu-5.1.12/modules/apcu.so) not valid for use in process: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed. in Unknown on line 0

This is broadly a System Integrity Protection (SIP) thing. Googling for this error reveals a lot of "disable SIP" (probably very bad advice) and "reinstall the thing" (probably practical, but not very helpful in this case).

I currently assume SIP is "Probably a Good Idea", although I have a very limited understanding of it and largely reserve judgement. My very fuzzy sense here is that SELinux is a bit more of a cautionary tale, where enabling SELinux breaks every application silently and the path back to the light is very unclear (see T4947). In contrast, SIP seems to generally produce obvious and reasonably clear errors, which is a major point in its favor.

An initial workaround for this narrow case (PHP extension binaries you built locally) is to do perform ad-hoc signature with -s -:

$ codesign -s - /Users/epriestley/src/apcu-5.1.12/apcu-5.1.12/modules/apcu.so

However, that only gets us one level deeper:

PHP Warning: PHP Startup: Unable to load dynamic library '/Users/epriestley/src/apcu-5.1.12/apcu-5.1.12/modules/apcu.so' - dlopen(/Users/epriestley/src/apcu-5.1.12/apcu-5.1.12/modules/apcu.so, 0x0009): code signature in (/Users/epriestley/src/apcu-5.1.12/apcu-5.1.12/modules/apcu.so) not valid for use in process: mapped file has no Team ID and is not a platform binary (signed with custom identity or adhoc?) in Unknown on line 0

It looks like the net effect here is "ad-hoc signatures are not sufficient".

This technical note appears to have an overview of the system as a whole:

https://developer.apple.com/library/archive/technotes/tn2206/_index.html

...although this guide is more clear:

https://developer.apple.com/library/archive/documentation/Security/Conceptual/CodeSigningGuide/Procedures/Procedures.html

If you are generating a code signing identity from scratch, Apple strongly recommends you use the Certificate Assistant component of Keychain Access

This appears to be Keychain AccessCertificate AssistantCreate a Certificate..., then pick "Code Signing".

You can then sign with:

$ codesign -s "name of the certificate" /Users/epriestley/src/apcu-5.1.12/apcu-5.1.12/modules/apcu.so

...but I'm still getting the "file has no Team ID" error.


I think this is a bit more complex than it appears. I believe what's happening here is something like this?

  • Apple has enabled library validation for the httpd binary under Mojave and signed it?
  • This means that Apache will only load modules signed by Apple?
  • So it doesn't matter how we sign some php-extension.so, it won't match the signature on the parent binary so it can't dl(...).

Everyone seems to be working around this issue by replacing the Apple version of Apache with the Homebrew version. Presumably, this version is built without library validation so it can load modules.

See also:

        if [[ $OS_VERSION == "10.14" ]]; then
		echo "****"
		echo "[WARNING]"
		echo "Detected macOS Mojave 10.14. There are serious issues with it, due to the original apache not loading"
		echo "foreign libraries anymore. PHP within apache will most certainly not work anymore if you proceed!"
		echo "The cli version still will."
        echo "See this issue at https://github.com/liip/php-osx/issues/249 for details and discussion"

...so this appears to be a huge mess.

I think options today are either:

  1. (Lots of Work) Replace Apache with a version that doesn't use Library Validation, from Homebrew, by building from source, or via some other mechanism.
  2. (Very Bad) Disable SIP.
  3. (Partial Workaround) Just don't run any PHP extensions.

This is actually potentially quite bad because:

  • It affects both the system httpd and the system php.
  • The system php is not built with the pcntl extension.
  • We require pcntl to do some stuff in arc, notably signal handling.

We also require pcntl to run daemons. So there are limits to how far we can get here.

(The errors above come out of php, but Apache produces similar errors:)

PHP Warning: PHP Startup: Unable to load dynamic library '/Users/epriestley/src/apcu-5.1.12/apcu-5.1.12/modules/apcu.so' - dlopen(/Users/epriestley/src/apcu-5.1.12/apcu-5.1.12/modules/apcu.so, 9): no suitable image found. Did find:\n\t/Users/epriestley/src/apcu-5.1.12/apcu-5.1.12/modules/apcu.so: code signature in (/Users/epriestley/src/apcu-5.1.12/apcu-5.1.12/modules/apcu.so) not valid for use in process using Library Validation: mapped file has no Team ID and is not a platform binary (signed with custom identity or adhoc?)\n\t/Users/epriestley/src/apcu-5.1.12/apcu-5.1.12/modules/apcu.so: stat() failed with errno=25 in Unknown on line 0

Event Timeline

epriestley created this task.

This doesn't help if you're committed to that Apache life, but FWIW I'm using nginx 1.15.8 via Homebrew on Mojave without any issues.

I'm more about not living that Homebrew life.

I don't really have any concrete issues with it, but it feels like after I brew it hides all the system libraries and headers or something and I have to brew forever after or I won't be able to compile/link/load anything? This may be fixed by now or might be something I imagined, and I think I haven't hit too many issues recently (maybe with Python versions?) but since some stuff has to build from source (forked sshd, xhprof) I can't brew everything.

I've generally been able to get Apache + PHP + extensions building from source in the past without too many issues, and it's very occasionally useful for debugging something (at least once, I rebuilt php after adding some printf() to the internals -- maybe when hunting down PCRE segfaults?).

Let's compile Gentoo nine times (Gentoo flavor is macOS Mojave 10.14.2):

$ brew install curl
$ brew install apache2
$ brew install libiconv
$ brew install zlib
php-7.3.1/ $ ./configure --with-mysqli --enable-pcntl --with-curl=/usr/local/opt/curl/ --with-apxs2=/usr/local/opt/httpd/bin/apxs --with-openssl=/usr/local/opt/openssl/ --enable-mbstring --without-iconv --with-zlib=/usr/local/opt/zlib
php-7.3.1/ $ make
php-7.3.1/ $ sudo make install

PHP

  • Extensions load without errors!
  • Pass --without-iconv since we don't need it and it can't find the library.
  • Pass --with-mysqli since the default loadout has neither mysql nor myslqi.
  • Pass --enable-pcntl so the daemons will work.
  • Daemons start! And immediately exit?
  • Pass --with-curl since we need cURL.
checking for cURL in default path... not found
configure: error: Please reinstall the libcurl distribution -
      easy.h should be in <curl-dir>/include/curl/
  • Fixed by brew install curl plus --with-curl=/usr/local/opt/curl/.
  • Pass --with-apxs2=/usr/local/opt/httpd/bin/apxs to get the Apache SAPI built.
Required PHP extensions are not installed.
Install these 3 PHP extension(s):

openssl
mbstring
iconv
  • We could turn mbstring and iconv into soft requirements, and there's a TODO elsewhere.
  • We need iconv, so get rid of --without-iconv. Instead, use --with-iconv=/usr/local/opt/libiconv/.
  • We can get openssl with --with-openssl=/usr/local/opt/openssl/, which brew install apache2 installed as a dependency.
  • Use --enable-mbstring to get mbstring.
Undefined symbols for architecture x86_64:
  "_libiconv", referenced from:
      _php_iconv_string in iconv.o
      __php_iconv_strlen in iconv.o
      _zif_iconv_substr in iconv.o
      __php_iconv_strpos in iconv.o
      _zif_iconv_mime_encode in iconv.o
      __php_iconv_appendl in iconv.o
      _php_iconv_stream_filter_append_bucket in iconv.o
      ...
  "_libiconv_close", referenced from:
      _php_iconv_string in iconv.o
      __php_iconv_strlen in iconv.o
      _zif_iconv_substr in iconv.o
      __php_iconv_strpos in iconv.o
      _zif_iconv_mime_encode in iconv.o
      __php_iconv_mime_decode in iconv.o
      _php_iconv_stream_filter_factory_create in iconv.o
      ...
  "_libiconv_open", referenced from:
      _php_iconv_string in iconv.o
      __php_iconv_strlen in iconv.o
      _zif_iconv_substr in iconv.o
      __php_iconv_strpos in iconv.o
      _zif_iconv_mime_encode in iconv.o
      __php_iconv_mime_decode in iconv.o
      _php_iconv_stream_filter_factory_create in iconv.o
      ...
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
  • Okay, let's remove the iconv dependency then? Put --without-iconv back.
  • We need --with-zlib=/usr/local/opt/zlib for some unit tests in arc.

PHP 7.3.1 is exiting abruptly for me (no error message, return code 0) on preg_match('(\\G\s+)sS', '');. This occurs in the daemons. I can't reproduce this in a separate test script. I have no idea what's going on.

We also need --enable-zip to get the zip extension, to get the ZipArchive class, so "Export to Excel" works. See upcoming change on T13342.

For LDAP, brew install openldap + --with-ldap=/usr/local/opt/openldap/.

This runs into https://bugs.php.net/bug.php?id=76433 which seems to be a hopeless mess.

It seems make is linking against the intentionally-broken system LDAP library (https://bugs.php.net/bug.php?id=76403), not the legitimate brew LDAP library, and apparently there's some even bigger mess with cURL depending on the intentionally-broken system LDAP (https://github.com/Homebrew/homebrew-core/issues/32916).

Applying the patch here (https://bugs.php.net/patch-display.php?bug_id=76433&patch=macos-ldap&revision=latest) to skip all the troublesome calls seems to "work". Yikes.

With PHP 7.4, it seems OpenSSL (and various other libraries) have moved to pkg-config, which is a tool that uses the PKG_CONFIG_PATH environmental variable to locate packages. This impacts OpenSSL and libxml2.

The version of libxml2 "bundled" with MacOS deep inside /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/libxml2/ does not have pkg-config support, so:

$ brew install libxml2

A previously-bundled library ("oniguruma") is no longer bundled with PHP. This library is used for some kind of multibyte regex behavior. It can be disabled with --disable-mbregex. It's not immediately clear if that disables the /u regex flag (which we use) or some other behavior.

This command ultimately produces a build:

$ export PKG_CONFIG_PATH="/usr/local/opt/openssl/lib/pkgconfig:/usr/local/opt/libxml2/lib/pkgconfig"
$ ./configure \
  --with-mysqli \
  --enable-pcntl \
  --with-curl=/usr/local/opt/curl/ \
  --with-apxs2=/usr/local/opt/httpd/bin/apxs \
  --with-openssl=/usr/local/opt/openssl/ \
  --enable-mbstring \
  --without-iconv \
  --with-zlib=/usr/local/opt/zlib \
  --with-zip \
  --disable-mbregex