Skip to content

Fix number of elements after packed hash filling #11022

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 6, 2023

Conversation

nielsdos
Copy link
Member

@nielsdos nielsdos commented Apr 5, 2023

After a hash filling routine the number of elements are set to the fill index. However, if the fill index is larger than the number of elements, the number of elements are no longer correct. This is observable at least via count() and var_dump(). E.g. the attached test case would incorrectly show int(17) instead of int(11).

Solve this by only increasing the number of elements by the actual number that got added. Instead of adding a variable that increments per iteration, I wanted to save some cycles in the iteration and simply compute the number of added elements at the end.

I removed the assignment in ZEND_HASH_FILL_GROW() because the number of elements is set anyway at the end of the hash function and this avoids an unnecessary computation.

I discovered this behaviour while fixing GH-11016, where this filling routine is easily exposed to userland via a specialised VM path [1]. Since this seems to be more a general problem with the macros, and may be triggered outside of the VM handlers, I fixed it in the macros instead of modifying the VM to fixup the number of elements.

[1]

php-src/Zend/zend_vm_def.h

Lines 6132 to 6141 in b2c5acb

ZEND_HASH_FILL_PACKED(result_ht) {
ZEND_HASH_PACKED_FOREACH_VAL(ht, val) {
if (UNEXPECTED(Z_ISREF_P(val)) &&
UNEXPECTED(Z_REFCOUNT_P(val) == 1)) {
val = Z_REFVAL_P(val);
}
Z_TRY_ADDREF_P(val);
ZEND_HASH_FILL_ADD(val);
} ZEND_HASH_FOREACH_END();
} ZEND_HASH_FILL_END();

@nielsdos nielsdos requested review from iluuu1994 and dstogov April 5, 2023 19:26
Copy link
Member

@iluuu1994 iluuu1994 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! But let's hear what Dmitry has to say 🙂

@dstogov
Copy link
Member

dstogov commented Apr 5, 2023

The problem was introduced by #9796 that incorrectly used ZEND_HASH_FILL_PACKED() macro.
The comment above ZEND_HASH_FILL_PACKED() tells - "HashTable must have enough free buckets", but zend_hash_extend(result_ht, zend_hash_num_elements(result_ht) + zend_hash_num_elements(ht), 1); doesn't take into account possible holes.

The fix proposed in #11016 looks right and I would add it to this PR.

Actually ZEND_HASH_FILL_PACKED() was useful to fill new tables.
This patch should extend the usability of ZEND_HASH_FILL_PACKED().

In general this looks good, but let me play with this a bit (in worse case by Monday).

Copy link
Member

@dstogov dstogov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update the fix according to suggestions and include the fix for #11016

After a hash filling routine the number of elements are set to the fill
index. However, if the fill index is larger than the number of elements,
the number of elements are no longer correct. This is observable at
least via count() and var_dump(). E.g. the attached test case would
incorrectly show int(17) instead of int(11).

Solve this by only increasing the number of elements by the actual
number that got added. Instead of adding a variable that increments per
iteration, I wanted to save some cycles in the iteration and simply
compute the number of added elements at the end.

I discovered this behaviour while fixing phpGH-11016, where this filling
routine is easily exposed to userland via a specialised VM path [1].
Since this seems to be more a general problem with the macros, and may
be triggered outside of the VM handlers, I fixed it in the macros
instead of modifying the VM to fixup the number of elements.

[1] https://github.com/php/php-src/blob/b2c5acbb010f4bbc7ea9b53ba9bc81d672dd0f34/Zend/zend_vm_def.h#L6132-L6141
@nielsdos nielsdos changed the base branch from PHP-8.1 to master April 6, 2023 17:41
@nielsdos nielsdos requested a review from kocsismate as a code owner April 6, 2023 17:41
@nielsdos nielsdos force-pushed the fix-hash-fill-packed branch from b6dd6fd to 9b6ab08 Compare April 6, 2023 17:41
@nielsdos
Copy link
Member Author

nielsdos commented Apr 6, 2023

Made the changes like requested. Thanks. As per #11021 (comment) I'll keep the PRs separate, and make sure to commit them in the order Dmitry said.

Copy link
Member

@dstogov dstogov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine

@nielsdos nielsdos merged commit 2ef1930 into php:master Apr 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants