Skip to content

releases property attributes of internal classes #11980

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Aug 24, 2023

Conversation

ju1ius
Copy link
Contributor

@ju1ius ju1ius commented Aug 15, 2023

Closes #11979

@kocsismate
Copy link
Member

The same fix is in a PR which I opened a short while ago :) e1b618e

@ju1ius
Copy link
Contributor Author

ju1ius commented Aug 15, 2023

The same fix is in a PR which I opened a short while ago :) e1b618e

Looks like our interests converge! 😉

Anyway, a search for attribute yielded no relevant issues or PRs on github, so... 🤷🏻

@iluuu1994
Copy link
Member

@ju1ius Thank you for the change and the test! I can't reproduce the ASAN failure locally on Clang 16 or GCC. CI runs with Clang 14, it's probable that this is a bug in Clang 14. We may try to update Ubuntu 22.10 which has Clang 15. I'm not sure if this issue is fixed there.

@ju1ius
Copy link
Contributor Author

ju1ius commented Aug 17, 2023

CI runs with Clang 14, it's probable that this is a bug in Clang 14

I'm using Clang 14.0.6 locally (from debian unstable repos) and cannot reproduce either (it fails for another unrelated test though... 🤷🏻).

I didn't dig much into the CI configs so mine might slightly differ.

@iluuu1994
Copy link
Member

iluuu1994 commented Aug 17, 2023

@ju1ius Did you configure with --enable-address-sanitizer? CI also sets CFLAGS="-DZEND_TRACK_ARENA_ALLOC" but that probably doesn't matter here. Are you on macOS (since you're using Clang)? Not sure if that makes a difference.

@ju1ius ju1ius force-pushed the ju1ius/free-prop-attrs2 branch from 8c597ae to b24f7df Compare August 17, 2023 17:43
@ju1ius
Copy link
Contributor Author

ju1ius commented Aug 17, 2023

Did you configure with --enable-address-sanitizer

Ah, no. It seemed to me that -fsanitize=address was enough to enable ASAN... I'll try another build to see if that changes anything.

Are you on macOS (since you're using Clang)?

Nope, Debian Sid.

Long story short, I'm writing a library to write PHP modules in Rust and building with clang+lldb allows for cross-language step-debugging sessions (it might also be possible with gcc but I couldn't make it work...).

@iluuu1994
Copy link
Member

iluuu1994 commented Aug 17, 2023

Hmm, weird. I thought -fsanitize=address would work. Clang required the same for LDFLAGS, but omitting that would result in a linker error instead. The benefit of using the configure flag is just that it doesn't use that CFLAG for the various autoconf tests, which can significantly slow them down.

Long story short, I'm writing a library to write PHP modules in Rust

Cool cool 😎

@ju1ius
Copy link
Contributor Author

ju1ius commented Aug 17, 2023

So I had another go:

This is my PHP build script
#!/bin/bash
make distclean
git clean -fx
./buildconf

declare -a CC_FLAGS=(
  -gdwarf-4
  -ggdb3
  -glldb
  -DZEND_RC_DEBUG=1
  -DTSRM_DEBUG=1
  -DZEND_TRACK_ARENA_ALLOC
  -fsanitize=undefined,address
)

declare -a CONFIG_FLAGS=(
  --enable-debug
  --enable-zts
  --enable-address-sanitizer
  --enable-zend_test=shared
  --enable-opcache
  --disable-opcache-jit
  --enable-tokenizer
  --enable-mbstring
  --enable-dom
  --with-libxml
  --enable-xml
  --enable-xmlwriter
  --enable-phar
  --disable-cgi
  --disable-fpm
)

./configure \
  CC=clang \
  CFLAGS="${CC_FLAGS[*]}" \
  "${CONFIG_FLAGS[@]}"

make -j "$(nproc)"
The command to run the test case failing in CI:
./sapi/cli/php -P -q \
  -d "extension_dir=$(pwd)/modules" \
  -d opcache.enable_cli=1 \
  --asan \
  -x -j8 \
  ./sapi/cli/tests/ext_loading.phpt
Produces the following output
Deprecated: escapeshellarg(): Passing null to parameter #1 ($arg) of type string is deprecated in /php-src/run-tests.php on line 686

Deprecated: escapeshellarg(): Passing null to parameter #1 ($arg) of type string is deprecated in /php-src/run-tests.php on line 687

phpdbg: /php-src/Zend/zend_types.h:1290: uint32_t zend_gc_delref(zend_refcounted_h *): Assertion `(zval_gc_flags((p)->u.type_info) & ((1<<7)|(1<<8))) != (1<<7)' failed.
Aborted

=================================================================
==2659060==ERROR: AddressSanitizer: heap-use-after-free on address 0x615000021280 at pc 0x562c2f7de953 bp 0x7ffd7f6ee350 sp 0x7ffd7f6ee348
READ of size 1 at 0x615000021280 thread T0
    #0 0x562c2f7de952 in clean_module_class /php-src/Zend/zend_API.c:3065:10
    #1 0x562c2f831e94 in zend_hash_apply_with_argument /php-src/Zend/zend_hash.c:2071:13
    #2 0x562c2f793de9 in clean_module_classes /php-src/Zend/zend_API.c:3075:2
    #3 0x562c2f792af4 in module_destructor /php-src/Zend/zend_API.c:3112:3
    #4 0x562c2f7974ef in zend_post_deactivate_modules /php-src/Zend/zend_API.c:3221:4
    #5 0x562c2f0d72d2 in php_request_shutdown /php-src/main/main.c:1881:3
    #6 0x562c3075e012 in do_cli /php-src/sapi/cli/php_cli.c:1136:3
    #7 0x562c30755edb in main /php-src/sapi/cli/php_cli.c:1340:18
    #8 0x7fccc89e06c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #9 0x7fccc89e0784 in __libc_start_main csu/../csu/libc-start.c:360:3
    #10 0x562c2d2033c0 in _start (/php-src/sapi/cli/php+0x1e033c0) (BuildId: 34a3533ac6fa040f57d76c4c195af5de37632742)

0x615000021280 is located 0 bytes inside of 512-byte region [0x615000021280,0x615000021480)
freed by thread T0 here:
    #0 0x562c2d285f62 in free (/php-src/sapi/cli/php+0x1e85f62) (BuildId: 34a3533ac6fa040f57d76c4c195af5de37632742)
    #1 0x562c2f694842 in destroy_zend_class /php-src/Zend/zend_opcode.c:513:4
    #2 0x562c2f81dae0 in _zend_hash_del_el_ex /php-src/Zend/zend_hash.c:1435:3
    #3 0x562c2f81adb3 in _zend_hash_del_el /php-src/Zend/zend_hash.c:1462:2
    #4 0x562c2f832100 in zend_hash_apply_with_argument /php-src/Zend/zend_hash.c:2075:5
    #5 0x562c2f793de9 in clean_module_classes /php-src/Zend/zend_API.c:3075:2
    #6 0x562c2f792af4 in module_destructor /php-src/Zend/zend_API.c:3112:3
    #7 0x562c2f7974ef in zend_post_deactivate_modules /php-src/Zend/zend_API.c:3221:4
    #8 0x562c2f0d72d2 in php_request_shutdown /php-src/main/main.c:1881:3
    #9 0x562c3075e012 in do_cli /php-src/sapi/cli/php_cli.c:1136:3
    #10 0x562c30755edb in main /php-src/sapi/cli/php_cli.c:1340:18
    #11 0x7fccc89e06c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

previously allocated by thread T0 here:
    #0 0x562c2d28620e in malloc (/php-src/sapi/cli/php+0x1e8620e) (BuildId: 34a3533ac6fa040f57d76c4c195af5de37632742)
    #1 0x562c2f798deb in do_register_internal_class /php-src/Zend/zend_API.c:3246:34
    #2 0x562c2f798976 in zend_register_internal_class /php-src/Zend/zend_API.c:3321:9
    #3 0x562c2f798928 in zend_register_internal_class_ex /php-src/Zend/zend_API.c:3284:19
    #4 0x7fccc5c5d503 in register_class__ZendTestClass /php-src/ext/zend_test/test_arginfo.h:477:16
    #5 0x7fccc5c5bc78 in zm_startup_zend_test /php-src/ext/zend_test/test.c:892:20
    #6 0x562c2f77c81b in zend_startup_module_ex /php-src/Zend/zend_API.c:2311:7
    #7 0x562c2ebd1cb1 in php_load_extension /php-src/ext/standard/dl.c:242:49
    #8 0x562c2ebcff54 in php_dl /php-src/ext/standard/dl.c:285:6
    #9 0x562c2ebcf756 in zif_dl /php-src/ext/standard/dl.c:69:2
    #10 0x562c2fdc939f in ZEND_DO_ICALL_SPEC_RETVAL_USED_HANDLER /php-src/Zend/zend_vm_execute.h:1337:2
    #11 0x562c2f90da18 in execute_ex /php-src/Zend/zend_vm_execute.h:56986:7
    #12 0x562c2f90ffb8 in zend_execute /php-src/Zend/zend_vm_execute.h:61583:2
    #13 0x562c2f73ccc7 in zend_execute_scripts /php-src/Zend/zend.c:1875:4
    #14 0x562c2f0e8f5a in php_execute_script /php-src/main/main.c:2492:13
    #15 0x562c3075abbe in do_cli /php-src/sapi/cli/php_cli.c:966:5
    #16 0x562c30755edb in main /php-src/sapi/cli/php_cli.c:1340:18
    #17 0x7fccc89e06c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

SUMMARY: AddressSanitizer: heap-use-after-free /php-src/Zend/zend_API.c:3065:10 in clean_module_class
Shadow bytes around the buggy address:
  0x0c2a7fffc200: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c2a7fffc210: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c2a7fffc220: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c2a7fffc230: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c2a7fffc240: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0c2a7fffc250:[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c2a7fffc260: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c2a7fffc270: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c2a7fffc280: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c2a7fffc290: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c2a7fffc2a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==2659060==ABORTING

=====================================================================
PHP         : /php-src/sapi/cli/php
PHP_SAPI    : cli
PHP_VERSION : 8.3.0-dev
ZEND_VERSION: 4.3.0-dev
PHP_OS      : Linux - Linux julius-laptop 6.4.0-2-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.4.4-3 (2023-08-08) x86_64
INI actual  : /php-src
More .INIs  :
---------------------------------------------------------------------
PHP         : /php-src/sapi/phpdbg/phpdbg
---------------------------------------------------------------------
CWD         : /php-src
Extra dirs  :
VALGRIND    : Not used
=====================================================================
Running selected tests.
PASS Extension loading [sapi/cli/tests/ext_loading.phpt]
=====================================================================
Number of tests :     1                 1
Tests skipped   :     0 (  0.0%) --------
Tests warned    :     0 (  0.0%) (  0.0%)
Tests failed    :     0 (  0.0%) (  0.0%)
Tests passed    :     1 (100.0%) (100.0%)
---------------------------------------------------------------------
Time taken      :     1 seconds
=====================================================================

TL;DR

The test passes, ASAN reports an UAF when trying to free _ZendTestClass on shutdown, and an RC_MOD_CHECK assertion fails (trying to decref a persistent zval - which also causes all tests under sapi/phpdbg to fail...).

But nothing that looks like the CI failure...

clang 14.0.6 btw

@ju1ius
Copy link
Contributor Author

ju1ius commented Aug 17, 2023

Also I just rebased this PR, and now the ASAN build fails but for another set of tests (under ext/soap/ while before it failed for sapi/cli/tests/ext_loading.phpt)...

@iluuu1994
Copy link
Member

I'm afraid there's some random component. Can you try switching to Ubuntu 22.10 or 23.04? Just to see if that avoids the issue.

@ju1ius
Copy link
Contributor Author

ju1ius commented Aug 17, 2023

Can you try switching to Ubuntu 22.10 or 23.04?

I'm afraid not since I'm not on Ubuntu in the first place and don't intend to switch to it. 😆

I can try with another clang version though.

@iluuu1994
Copy link
Member

Sorry, I meant in .github/workflows/push.yml of course 😄

@ju1ius
Copy link
Contributor Author

ju1ius commented Aug 17, 2023

Sorry, I meant in .github/workflows/push.yml of course 😄

Yeah I was afraid you'd say that...

@ju1ius
Copy link
Contributor Author

ju1ius commented Aug 17, 2023

Can you try switching to Ubuntu 22.10 or 23.04?

It seems that 22.04 is the highest available version on GH actions.... I can try with ubuntu-latest maybe?

@iluuu1994
Copy link
Member

It seems that 22.04 is the highest available version on GH actions

🙃

I can try with ubuntu-latest maybe?

In that case, ubuntu-latest is an alias to ubuntu-22.04. Welp.

@ju1ius
Copy link
Contributor Author

ju1ius commented Aug 17, 2023

In that case, ubuntu-latest is an alias to ubuntu-22.04. Welp.

Maybe trying to switch to gcc?

Or install a custom LLVM toolchain...

@iluuu1994
Copy link
Member

@ju1ius Sure, for testing that makes sense. I used clang because it produces a significantly faster asan debug build. Tests take much longer to finish on gcc + asan.

@ju1ius
Copy link
Contributor Author

ju1ius commented Aug 17, 2023

Or install a custom LLVM toolchain...

There's KyleMayes/install-llvm-action

@ju1ius ju1ius requested a review from TimWolla as a code owner August 17, 2023 21:01
@iluuu1994 iluuu1994 removed the request for review from TimWolla August 17, 2023 21:13
@ju1ius
Copy link
Contributor Author

ju1ius commented Aug 17, 2023

Maybe trying to switch to gcc?

Nope, same results w/ GCC, and w/ the same set of test cases...

At least it's not so random....

@ju1ius
Copy link
Contributor Author

ju1ius commented Aug 18, 2023

So, after reading through those seemingly related issues:

I tried setting LSAN_OPTIONS=use_tls=0 in 8b2998b.

The build failed due to detected leaks:

ERROR: LeakSanitizer: detected memory leaks

Direct leak of 640 byte(s) in 1 object(s) allocated from:
     #0 0x561e24a8af9e in malloc (/home/runner/work/php-src/php-src/sapi/cli/php+0x288af9e) (BuildId: 58b8fddd1e8b39fcc40a0e4f973e4315aab98514)
     #1 0x7ff18b14f146  (/lib64/ld-linux-x86-64.so.2+0xe146) (BuildId: 61ef896a699bb1c2e4e231642b2e1688b2f1a61e)

Direct leak of 8 byte(s) in 1 object(s) allocated from:
    #0 0x561e24a8af9e in malloc (/home/runner/work/php-src/php-src/sapi/cli/php+0x288af9e) (BuildId: 58b8fddd1e8b39fcc40a0e4f973e4315aab98514)
    #1 0x7ff18b154e4d  (/lib64/ld-linux-x86-64.so.2+0x13e4d) (BuildId: 61ef896a699bb1c2e4e231642b2e1688b2f1a61e)

SUMMARY: AddressSanitizer: 648 byte(s) leaked in 2 allocation(s).

But this time LSAN did not crash...

I have absolutely no idea of what this option really does, nor if it is really OK to use it (especially since the ASAN build runs under ZTS which does use TLS), but it seems to be a commonly adopted workaround for this issue, so if somebody would like to take over...

@ju1ius ju1ius mentioned this pull request Aug 18, 2023
@iluuu1994
Copy link
Member

iluuu1994 commented Aug 18, 2023

Great find! Reading https://maskray.me/blog/2023-02-12-all-about-leak-sanitizer, it sounds like setting use_tls=0 prevents lsan from iterating thread local variables (which I don't think we have atm, since we're using pthread_getspecific in zts) at the end of the program when marking reachable objects. This would explain why the crash occurs on termination. This indeed sounds like an option for us if it solves the problem.

Edit: I missed that this shows leaks for us. I'll look at those more closely in a couple of hours.

@iluuu1994
Copy link
Member

@ju1ius

diff --git a/.github/lsan-suppressions.txt b/.github/lsan-suppressions.txt
index b8f863ce60..9278992166 100644
--- a/.github/lsan-suppressions.txt
+++ b/.github/lsan-suppressions.txt
@@ -1,3 +1,4 @@
 leak:acommon::DictInfoList::elements
 leak:timer_create
 leak:netsnmp_init_mib_internals
+leak:dlopen
diff --git a/run-tests.php b/run-tests.php
index 6f10e4090b..01cae7e44a 100755
--- a/run-tests.php
+++ b/run-tests.php
@@ -586,7 +586,7 @@ function main(): void
                     $lsanSuppressions = __DIR__ . '/.github/lsan-suppressions.txt';
                     if (file_exists($lsanSuppressions)) {
                         $environment['LSAN_OPTIONS'] = 'suppressions=' . $lsanSuppressions
-                            . ':print_suppressions=0';
+                            . ':print_suppressions=0:use_tls=0';
                     }
                     break;
                 case '--repeat':

Can you try it with this? I guess dl uses a thread-local somewhere internally. We shouldn't need to care about that.

@ju1ius
Copy link
Contributor Author

ju1ius commented Aug 18, 2023

Can you try it with this?

@iluuu1994 Hmmm... Unless I'm missing something, I can't see how this is any different from what I did in 8b2998b?

@iluuu1994
Copy link
Member

@ju1ius It's missing the leak:dlopen entry in .github/lsan-suppressions.txt.

@ju1ius
Copy link
Contributor Author

ju1ius commented Aug 18, 2023

@ju1ius It's missing the leak:dlopen entry in .github/lsan-suppressions.txt.

Aaah, right my bad, here we go.

@iluuu1994
Copy link
Member

@ju1ius Lead cannot be suppressed 😉

@ju1ius ju1ius force-pushed the ju1ius/free-prop-attrs2 branch from 1fa9b73 to 3a3337d Compare August 18, 2023 13:49
@iluuu1994
Copy link
Member

iluuu1994 commented Aug 18, 2023

@ju1ius This seems to break when debug symbols for the dynamic lib are stripped. Can you please try adding leak:/ld-*.so to the list in addition to leak:dlopen?

@ju1ius ju1ius force-pushed the ju1ius/free-prop-attrs2 branch from 3a3337d to e265ce7 Compare August 18, 2023 14:16
@ju1ius
Copy link
Contributor Author

ju1ius commented Aug 18, 2023

Seems to do the trick but we still have

ERROR: LeakSanitizer: detected memory leaks

Direct leak of 24 byte(s) in 1 object(s) allocated from:
    #0 0x55661bc8afbe in malloc (/home/runner/work/php-src/php-src/sapi/cgi/php-cgi+0x288afbe) (BuildId: 5ff694f62bc2ec88bb5a5e0c86e3ddc44cc69bd8)
    #1 0x7fa49d6881da  (/lib/x86_64-linux-gnu/libc.so.6+0x901da) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)

SUMMARY: AddressSanitizer: 24 byte(s) leaked in 1 allocation(s).

@iluuu1994
Copy link
Member

@ju1ius Ugh. Another approach we could try is wrapping the responsible code in __lsan_disable() and __lsan_enable() calls. I can try this myself if you wish, I don't want to keep bothering you.

@ju1ius
Copy link
Contributor Author

ju1ius commented Aug 18, 2023

@ju1ius Ugh. Another approach we could try is wrapping the responsible code in __lsan_disable() and __lsan_enable() calls. I can try this myself if you wish, I don't want to keep bothering you.

Doesn't bother me but at that point it's just gonna be you telling me what to do, so you'll probably be much faster on your own... As you wish.

@iluuu1994
Copy link
Member

@ju1ius I will investigate, thanks for your help so far!

@ju1ius ju1ius force-pushed the ju1ius/free-prop-attrs2 branch from bc1b8f5 to ed16b07 Compare August 19, 2023 08:51
@iluuu1994
Copy link
Member

@ju1ius Can you rebase this again? Hopefully the error is gone now.

@ju1ius ju1ius force-pushed the ju1ius/free-prop-attrs2 branch from ed16b07 to d3531da Compare August 24, 2023 10:00
@iluuu1994 iluuu1994 merged commit 3e0e7e3 into php:master Aug 24, 2023
@iluuu1994
Copy link
Member

Thank you @ju1ius!

jorgsowa pushed a commit to jorgsowa/php-src that referenced this pull request Aug 24, 2023
* adds test case for internal class property attribute

* releases property attributes of internal classes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Property attributes of internal classes are never freed.
3 participants