Allow processing of SKIPIF output after a nocache tag

[jmikola]

Previously, a nocache tag would result in the entire SKIPIF output being ignored. This allows the remaining SKIPIF output to still be processed by run_test(). This also changes the "nocache" check to use strncasecmp instead of strpos, which is consistent with how other SKIPIF tags are checked (e.g. info, warn).

For context, "nocache" was originally implemented in 4d43cbe, 0074a1d, and 9fc6be2.

I can't speak for other extensions, but in php-src there are only a few uses of nocache. It is manually printed at the end of some SKIPIF blocks in mysqli tests (see: 4d43cbe).

In the MongoDB extension, we recently added nocache to a SKIPIF helper function responsible for dropping a database collection. After this function successfully drops the collection under test, it printed "nocache". I recently discovered that some tests were running when they should have been skipped. This was due to some tests where other helpers were called after this drop function. They were printing "skip" output but it was being ignored by run-tests.php consuming the earlier "nocache" string. PHPC-2064 goes into more detail, but to summarize I came up with three options:

• Re-order function calls in all of our SKIPIF blocks such that the drop helper is always called last. This has since been done in mongodb/mongo-php-driver@2db00e0.
• Change run-tests.php to no longer clear SKIPIF output, which is the purpose of this PR.
• Change our drop helper to no longer print "nocache" immediately but instead defer its printing to the end of the SKIPIF script (e.g. using register_shutdown_function()). I've yet to do this, but it would address an outstanding issue we have where some tests call the drop helper multiple times and "nocache" is printed twice. Before this PR, the second nocache tag would be ignored (as the result was cleared). With this PR, the second "nocache" will BORK the test due to unexpected SKIPIF output.

Some additional thoughts

The last bullet highlights a potential BC break: a test printing "nocache" twice will now BORK instead of executing. If that's a concern I can rework this PR to instead consume all leading "nocache" tags (e.g. a while loop).

Going back to the old internals thread on SKIPIF caching, there was an open question about whether it should have been possible to skip a test and disable caching (not possible with the original implementation, since SKIPIF output was discarded after a nocache tag was consumed). I suppose that is now possible with this PR, provided "nocache" appears first in SKIPIF output.

[ramsey]

If we're targeting PHP-8.1 because this is considered a bug in the way run-tests.php handles SKIPIF with nocache, then shouldn't we target PHP-8.0 instead? Or does the bug not exist in PHP-8.0?

[jmikola]

shouldn't we target PHP-8.0 instead? Or does the bug not exist in PHP-8.0?

I believe this is only relevant in PHP 8.1+, since that is when nocache was implemented (4d43cbe, 0074a1d, and 9fc6be2).

[cmb69]

The last bullet highlights a potential BC break: a test printing "nocache" twice will now BORK instead of executing.

I'm slightly concerned about this if we actually target PHP-8.1.

[jmikola]

I'm slightly concerned about this if we actually target PHP-8.1.

@cmb69: This change should address your previous concern. One or more "nocache" tags will now be consumed exhaustively, which will avoid leaving behind a "nocache" that might otherwise cause the SKIPIF processing in run_test() to BORK.

This also preserves the a ability to "skip a test and disable caching" by emitting "nocache" at the beginning of SKIPIF output, which I mentioned in the PR OP.

[jmikola]

@cmb69: If you have time to review, I'd be curious to know if the most recent update makes this suitable for PHP-8.1 and feasible to merge upwards ahead of the 8.2.0 release (easier to rely on it that way instead of an arbitrary 8.1.x patch).