php 8.1 and earlier crash immediately when compiled with Xcode 16 clang on macOS 15

[ryandesign]

Description

Hi, I'm the maintainer of php in MacPorts. macOS 15 and Xcode 16 were recently released, and following a user report, I discovered that php 8.1 and earlier crash when compiled there. php 8.2 and later don't have this problem, so I'm hoping to discover what change in php 8.2 fixed this so that I can backport it to earlier versions.

The crash is observed during installation:

Generating phar.php
/bin/sh: line 1: 55243 Segmentation fault

For php 8.1, this crash is ignored and the installation process continues, but the installed executable crashes when trying to do anything (php, php -v, etc.). For php 8.0 and earlier, the crash stops the installation process.

From the MacPorts ticket, I've seen two different crash logs—one a segmentation fault with no backtrace showing that execution has passed to an illegal address 0x1e8:

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x00000000000001e8
Exception Codes:       0x0000000000000001, 0x00000000000001e8

Termination Reason:    Namespace SIGNAL, Code 11 Segmentation fault: 11
Terminating Process:   exc handler [75775]

VM Region Info: 0x1e8 is not in any region.  Bytes before following region: 4509498904
      REGION TYPE                    START - END         [ VSIZE] PRT/MAX SHRMOD  REGION DETAIL
      UNUSED SPACE AT START
--->  
      __TEXT                      10cc98000-10d683000    [  9.9M] r-x/r-x SM=COW  /opt/local/var/macports/*/php

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   ???                           	             0x1e8 ???


Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x00007fd29780c900  rbx: 0x000000010e205000  rcx: 0x0000000000000003  rdx: 0x000000010e205020
  rdi: 0x000000010e205000  rsi: 0x000000010e200000  rbp: 0x000000000000001a  rsp: 0x00007ff7b3266be0
   r8: 0x000000010e205000   r9: 0x000000000000006c  r10: 0x00000000001ff800  r11: 0x0000000000000030
  r12: 0x0000000000000000  r13: 0x000000010d6989c0  r14: 0x00007ff7b3266c00  r15: 0x000000010ce78ecd
  rip: 0x00000000000001e8  rfl: 0x0000000000010202  cr2: 0x00000000000001e8
  
Logical CPU:     0
Error Code:      0x00000014 (no mapping for user instruction read)
Trap Number:     14

and the other an abort trap implicating an unnamed (inline?) function called from zend_hash_find:

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000

Termination Reason:    Namespace SIGNAL, Code 6 Abort trap: 6
Terminating Process:   php [40488]

Application Specific Information:
abort() called


Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	    0x7ff813d7db52 __pthread_kill + 10
1   libsystem_pthread.dylib       	    0x7ff813db7f85 pthread_kill + 262
2   libsystem_c.dylib             	    0x7ff813cd8b19 abort + 126
3   libsystem_malloc.dylib        	    0x7ff813bd7ab1 malloc_vreport + 857
4   libsystem_malloc.dylib        	    0x7ff813bdb58b malloc_report + 151
5   php                           	       0x10cf5e27d .LL31 + 203
6   php                           	       0x10cef6498 zend_hash_find + 128
7   php                           	       0x10cf752b1 lookup_class_ex + 312
8   php                           	       0x10cf70823 zend_perform_covariant_type_check + 885
9   php                           	       0x10cf75ae8 zend_do_perform_implementation_check + 554
10  php                           	       0x10cf755f8 do_inheritance_check_on_method + 344
11  php                           	       0x10cf722df do_interface_implementation + 520
12  php                           	       0x10ceec276 zend_class_implements + 216
13  php                           	       0x10cf5d6fd zend_register_weakref_ce + 316
14  php                           	       0x10cf703be zend_register_default_classes + 34
15  php                           	       0x10cef7ba5 zm_startup_core + 101
16  php                           	       0x10cee9ff9 zend_startup_module_ex + 259
17  php                           	       0x10ceea38f zend_startup_module_zval + 12
18  php                           	       0x10cef55cd zend_hash_apply + 87
19  php                           	       0x10ce87a9c php_module_startup + 2180
20  php                           	       0x10cfbb90d php_cli_startup + 13
21  php                           	       0x10cfb9666 main + 1356
22  dyld                          	    0x7ff813a2a2cd start + 1805


Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000000  rbx: 0x0000000000000006  rcx: 0x00007ff7b3203dc8  rdx: 0x0000000000000000
  rdi: 0x0000000000000103  rsi: 0x0000000000000006  rbp: 0x00007ff7b3203df0  rsp: 0x00007ff7b3203dc8
   r8: 0x000000000000002e   r9: 0x0000000000000000  r10: 0x0000000000000000  r11: 0x0000000000000246
  r12: 0x0000000000000000  r13: 0x0000000000000050  r14: 0x0000000000000103  r15: 0x0000000000000016
  rip: 0x00007ff813d7db52  rfl: 0x0000000000000246  cr2: 0x0000000000000000
  
Logical CPU:     0
Error Code:      0x02000148 
Trap Number:     133

On one machine, running php produced this possibly helpful diagnostic:

php(54330,0x7ff8522ffbc0) malloc: *** error for object 0x2cf607000: pointer being freed was not allocated
php(54330,0x7ff8522ffbc0) malloc: *** set a breakpoint in malloc_error_break to debug

PHP Version

8.1.30

Operating System

macOS 15.0

[cmb69]

macOS 15 and Xcode 16 were recently released

I assume that you don't have this environment; otherwise a git bisect should find the commit(s).

The first issue might be related to OPcache JIT.

[iluuu1994]

I ran a quick test-build in CI but couldn't reproduce the issue: https://github.com/iluuu1994/php-src/actions/runs/11144438755/job/30971704251 (ignore the failing last step)

A bisect would indeed be helpful. Usually, opcache is disabled in CLI by default, and JIT is disabled entirely. It also seems to occur during startup.

@ryandesign Are these debug builds? If not, can you try that? As usual, ASan/UBSan would be useful. Also, is this a problem that only appeared on macOS 15.0? This sounds like a compiler issue.

I don't have a Mac, but I think @kocsismate and @devnexen do. They might be able to help.

[kocsismate]

I still have MacOS 14.4, so I check also check what happens before and after an upgrade 😎

[ryandesign]

I assume that you don't have this environment; otherwise a git bisect should find the commit(s).

I have reproduced the issue locally on a 13-year-old dual core machine but before I have it spend a day bisecting I thought I'd ask if the issue is familiar to anyone.

Are these debug builds? If not, can you try that?

They are not. I can try that.

As usual, ASan/UBSan would be useful.

I don't know what that is.

Also, is this a problem that only appeared on macOS 15.0? This sounds like a compiler issue.

So far I am only aware of the issue occurring with macOS 15.0 + Xcode 16.0, not on earlier macOS versions with earlier Xcode versions. Xcode 16.0 is also compatible with macOS 14 ≥ 14.5 but I have not tested whether the issue occurs then. I could not say at this point whether the defect is in the compiler, the OS, or php's code.

[kocsismate]

I also managed to reproduce the issue with a debug build + ASAN, running on MacOS 14.4 + XCode 16:

/Users/kocsismate/dev/php-src/ext/pcre/pcre2lib/sljit/sljitNativeARM_64.c:1890:46: runtime error: left shift of 3 by 30 places cannot be represented in type 'int'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /Users/kocsismate/dev/php-src/ext/pcre/pcre2lib/sljit/sljitNativeARM_64.c:1890:46 in 
Assertion failed: ((zval_gc_flags((p)->u.type_info) & ((1<<7)|(1<<8))) != (1<<7)), function zend_gc_delref, file zend_types.h, line 1210.

==85256==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x00010d50baff at pc 0x000104d3e89c bp 0x00016b5a4ff0 sp 0x00016b5a4fe8
READ of size 16 at 0x00010d50baff thread T0
    #0 0x1026b6898 in ffcps_0 pcre2_jit_neon_inc.h:265
    #1 0x1073f74c4  (<unknown module>)
    #2 0x10262a7c4 in php_pcre2_jit_match pcre2_jit_match.c:165
    #3 0x102a4d7e8 in php_pcre_replace_impl php_pcre.c:1664
    #4 0x102a4cf10 in php_pcre_replace php_pcre.c:1606
    #5 0x102a76fd8 in php_replace_in_subject php_pcre.c:2176
    #6 0x102a55e38 in preg_replace_common php_pcre.c:2314
    #7 0x102a544a0 in zif_preg_replace php_pcre.c:2371
    #8 0x104c80890 in ZEND_DO_ICALL_SPEC_RETVAL_USED_HANDLER zend_vm_execute.h:1297
    #9 0x10470a104 in execute_ex zend_vm_execute.h:55622
    #10 0x10470b8e0 in zend_execute zend_vm_execute.h:60188
    #11 0x104532ea4 in zend_execute_scripts zend.c:1857
    #12 0x103fa2328 in php_execute_script main.c:2551
    #13 0x105881e34 in do_cli php_cli.c:965
    #14 0x10587e444 in main php_cli.c:1367
    #15 0x19a43e0dc  (<unknown module>)

0x00010ae0c3ef is located 7 bytes after 104-byte region [0x00010ae0c380,0x00010ae0c3e8)
allocated by thread T0 here:
    #0 0x107d3f124 in wrap_malloc+0x94 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x53124)
    #1 0x1042b75c0 in __zend_malloc zend_alloc.c:3088
    #2 0x1042ab068 in tracked_malloc zend_alloc.c:2799
    #3 0x1042b62c4 in _malloc_custom zend_alloc.c:2459
    #4 0x1042b5e7c in _emalloc zend_alloc.c:2578
    #5 0x103c7920c in zend_string_alloc zend_string.h:152
    #6 0x103c59e60 in zend_string_init zend_string.h:174
    #7 0x103c5c444 in php_trim_int string.c:824
    #8 0x103c5d384 in php_do_trim string.c:855
    #9 0x103c5c560 in zif_trim string.c:862
    #10 0x104c80890 in ZEND_DO_ICALL_SPEC_RETVAL_USED_HANDLER zend_vm_execute.h:1297
    #11 0x10470a104 in execute_ex zend_vm_execute.h:55622
    #12 0x10470b8e0 in zend_execute zend_vm_execute.h:60188
    #13 0x104532ea4 in zend_execute_scripts zend.c:1857
    #14 0x103fa2328 in php_execute_script main.c:2551
    #15 0x105881e34 in do_cli php_cli.c:965
    #16 0x10587e444 in main php_cli.c:1367
    #17 0x19a43e0dc  (<unknown module>)

SUMMARY: AddressSanitizer: heap-buffer-overflow pcre2_jit_neon_inc.h:265 in ffcps_0
Shadow bytes around the buggy address:
  0x00010ae0c100: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x00010ae0c180: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x00010ae0c200: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x00010ae0c280: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x00010ae0c300: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x00010ae0c380: 00 00 00 00 00 00 00 00 00 00 00 00 00[fa]fa fa
  0x00010ae0c400: fa fa fa fa fa fa 00 00 00 00 00 00 00 00 00 00
  0x00010ae0c480: 00 00 00 00 fa fa fa fa fa fa fa fa 00 00 00 00
  0x00010ae0c500: 00 00 00 00 00 00 00 00 00 00 fa fa fa fa fa fa
  0x00010ae0c580: fa fa 00 00 00 00 00 00 00 00 00 00 00 00 00 03
  0x00010ae0c600: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
    #0 0x1008b2898 in ffcps_0 pcre2_jit_neon_inc.h:265
    #1 0x1055f34c4  (<unknown module>)
    #2 0x1008267c4 in php_pcre2_jit_match pcre2_jit_match.c:165
    #3 0x100c497e8 in php_pcre_replace_impl php_pcre.c:1664
    #4 0x100c48f10 in php_pcre_replace php_pcre.c:1606
    #5 0x100c72fd8 in php_replace_in_subject php_pcre.c:2176
    #6 0x100c51e38 in preg_replace_common php_pcre.c:2314
    #7 0x100c504a0 in zif_preg_replace php_pcre.c:2371
    #8 0x102e7c890 in ZEND_DO_ICALL_SPEC_RETVAL_USED_HANDLER zend_vm_execute.h:1297
    #9 0x102906104 in execute_ex zend_vm_execute.h:55622
    #10 0x1029078e0 in zend_execute zend_vm_execute.h:60188
    #11 0x10272eea4 in zend_execute_scripts zend.c:1857
    #12 0x10219e328 in php_execute_script main.c:2551
    #13 0x103a7de34 in do_cli php_cli.c:965
    #14 0x103a7a444 in main php_cli.c:1367
    #15 0x19a43e0dc  (<unknown module>)

Apparently, PCRE2 Lib is to blame... And I guess the difference between PHP 8.1- and PHP 8.2+ is Christoph's commit: kocsismate@32cceb7 that updated the library.

[iluuu1994]

@kocsismate Do you get no errors with --without-pcre-jit? If so, Ryan might try the same.

[kocsismate]

@iluuu1994 I have several extensions enabled, but I don't hava errors anymore.

[ryandesign]

/Users/kocsismate/dev/php-src/ext/pcre/pcre2lib/sljit/sljitNativeARM_64.c:1890:46: runtime error: left shift of 3 by 30 places cannot be represented in type 'int'

Based on the filename I assume this is arm64 code. My tests were on x86_64 so the problem is at least not specific to arm64.

Apparently, PCRE2 Lib is to blame... And I guess the difference between PHP 8.1- and PHP 8.2+ is Christoph's commit: kocsismate@32cceb7 that updated the library.

For php in MacPorts we use MacPorts pcre2, not the pcre2 bundled with php—at least that's what I intend for it to do. At least looking at the library linkage, it appears to have been successful:

% otool -L /opt/local/bin/php81
/opt/local/bin/php81:
	/usr/lib/libresolv.9.dylib (compatibility version 1.0.0, current version 1.0.0)
	/opt/local/lib/libncurses.6.dylib (compatibility version 6.0.0, current version 6.0.0)
	/opt/local/lib/libbz2.1.0.dylib (compatibility version 1.0.0, current version 1.0.8)
	/usr/lib/libnetwork.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1351.0.0)
	/opt/local/lib/libxml2.2.dylib (compatibility version 16.0.0, current version 16.4.0)
	/opt/local/lib/libpcre2-8.0.dylib (compatibility version 14.0.0, current version 14.0.0)
	/opt/local/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.3.1)
	/opt/local/lib/libedit.0.dylib (compatibility version 1.0.0, current version 1.74.0)
	/opt/local/lib/libargon2.1.dylib (compatibility version 0.0.0, current version 0.0.0)
% 

All MacPorts php versions from php73 onward link with the same MacPorts pcre2 library, so that would not appear to explain why php82 and later work and php81 and earlier don't, unless php still uses part of its bundled pcre2 even when told to use an external pcre2.

@kocsismate Do you get no errors with --without-pcre-jit? If so, Ryan might try the same.

MacPorts pcre2 is built configured with --enable-jit. Rebuilding it with --disable-jit causes all of the php versions—both those that crashed before and those that worked—to fail with the same error:

% php81 -v

Fatal error: Unable to start pcre module in Unknown on line 0
% php82 -v

Fatal error: Unable to start pcre module in Unknown on line 0
% php83 -v

Fatal error: Unable to start pcre module in Unknown on line 0
%

I have not yet tried rebuilding any of the phps with the JITless pcre2 nor have I tried php's --without-pcre-jit configure option.

[cmb69]

@kocsismate, ASan may report false positives for PCRE JIT code.

[NattyNarwhal]

FWIW, I can't seem to reproduce this on my system (M1 Pro, macOS 14.7, Xcode 16, MacPorts). Were you using a specific set of ./configure options?

[ryandesign]

Yes I also had problems reproducing this when trying to run git bisect doing builds outside of MacPorts until I added in configure options that are used in MacPorts.

I was initially using these flags and could not reproduce the problem:

--disable-all \
--disable-cgi \
--with-external-pcre=/opt/local \

I'm now reproducing the problem using:

--disable-all \
--disable-cgi \
--disable-fpm \
--enable-bcmath \
--enable-ctype \
--enable-dom \
--enable-filter \
--enable-libxml \
--enable-pdo \
--enable-session \
--enable-simplexml \
--enable-tokenizer \
--enable-xml \
--enable-xmlreader \
--enable-xmlwriter \
--with-bz2=/opt/local \
--with-external-pcre=/opt/local \
--with-libxml \
--with-mhash=/opt/local \
--with-zlib=/opt/local \

I have not tried to figure out which of those flags make the problem appear because it takes so very long for this old machine to build php.

[ryandesign]

Also I am setting CFLAGS="-Os -Wno-error=implicit-function-declaration -Wno-error=implicit-int -Wno-error=incompatible-function-pointer-types -Wno-error=int-conversion". I tried --enable-debug initially but when I couldn't reproduce the problem I noticed it was setting -O0 so I removed it, thinking that might be hiding the problem. I used -Os because that is what MacPorts uses. After that, I found that I needed to add the additional configure flags mentioned above to reproduce the problem.

I found a faster machine to run git bisect on and it identified 6339938 as the commit that fixed the problem.