Skip to content

Commit 0320ad5

Browse files
MaxSemMaxSem
committed
run-tests: use the EXTENSIONS section for skipping
Currently, most skip checks are just for making sure an extension is available. Even with recent addition of skip caching, this makes tests needlessly slow: * Checks for the same extension in its tests can have small differences impacting cacheability. * Even identical skip checks in two tests can still be executed twice if they're run by different workers. To remedy this, I'm repurposing the existing --EXTENSIONS-- section of .phpt files to specify wjich extensions are required for current test to run. Current behavior: 1) If the extension is already visible to PHP, all is good 2) If it isn't, assume it's present as a shared module and attempt to add it via a command line parameter 3) If that works, all is good 4) If it doesn't, PHP fails with a cryptic error message trying to execute the test itself After this commit: 1) and 2) are kept unchanged 3) Check if shared extension file from 2) is actually present 4) Skip the test if it isn't Other benefits include clear skip reasons (vs. sometimes none in many current skip checks) and moving test information from code to metadata, opening more opportunities for search and analysis. Since --EXTENSIONS-- is barely ever used, this change poses no risk of hidden failures. As a demonstration of the new approach, this commit migrates one extension to it. If merged, I will migrate other extensions in subsequent PRs.
1 parent 356ef5b commit 0320ad5

30 files changed

+115
-89
lines changed

‎ext/sodium/tests/bug78114.phpt

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,7 @@
11
--TEST--
22
Bug #78114 (segfault when calling sodium_* functions from eval)
3-
--SKIPIF--
4-
<?php
5-
if (!extension_loaded('sodium')) die('skip sodium extension not available');
6-
?>
3+
--EXTENSIONS--
4+
sodium
75
--FILE--
86
<?php
97
try {

‎ext/sodium/tests/bug78516.phpt

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
11
--TEST--
22
Bug #78516 (password_hash(): Memory cost is not in allowed range)
3+
--EXTENSIONS--
4+
sodium
35
--SKIPIF--
46
<?php
5-
if (!extension_loaded('sodium')) die('skip sodium extension not available');
67
if (!defined('PASSWORD_ARGON2ID')) die('skip PASSWORD_ARGON2ID not available');
78
?>
89
--FILE--

‎ext/sodium/tests/crypto_aead.phpt

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
11
--TEST--
22
Check for libsodium AEAD
3+
--EXTENSIONS--
4+
sodium
35
--SKIPIF--
46
<?php
5-
if (!extension_loaded("sodium")) print "skip extension not loaded";
67
if (!defined('SODIUM_CRYPTO_AEAD_AES256GCM_NPUBBYTES')) print "skip libsodium without AESGCM";
78
?>
89
--FILE--

‎ext/sodium/tests/crypto_auth.phpt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
--TEST--
22
Check for libsodium auth
3-
--SKIPIF--
4-
<?php if (!extension_loaded("sodium")) print "skip"; ?>
3+
--EXTENSIONS--
4+
sodium
55
--FILE--
66
<?php
77
$msg = random_bytes(1000);

‎ext/sodium/tests/crypto_box.phpt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
--TEST--
22
Check for libsodium box
3-
--SKIPIF--
4-
<?php if (!extension_loaded("sodium")) print "skip"; ?>
3+
--EXTENSIONS--
4+
sodium
55
--FILE--
66
<?php
77
$keypair = sodium_crypto_box_keypair();

‎ext/sodium/tests/crypto_generichash.phpt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
--TEST--
22
Check for libsodium generichash
3-
--SKIPIF--
4-
<?php if (!extension_loaded("sodium")) print "skip"; ?>
3+
--EXTENSIONS--
4+
sodium
55
--FILE--
66
<?php
77
$q = sodium_crypto_generichash('msg');

‎ext/sodium/tests/crypto_hex.phpt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
--TEST--
22
Check for libsodium bin2hex
3-
--SKIPIF--
4-
<?php if (!extension_loaded("sodium")) print "skip"; ?>
3+
--EXTENSIONS--
4+
sodium
55
--FILE--
66
<?php
77
$bin = random_bytes(random_int(1, 1000));

‎ext/sodium/tests/crypto_kdf.phpt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
--TEST--
22
Check for libsodium KDF
3-
--SKIPIF--
4-
<?php if (!extension_loaded("sodium")) print "skip"; ?>
3+
--EXTENSIONS--
4+
sodium
55
--FILE--
66
<?php
77
$key = sodium_crypto_kdf_keygen();

‎ext/sodium/tests/crypto_kx.phpt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
--TEST--
22
Check for libsodium-based key exchange
3-
--SKIPIF--
4-
<?php if (!extension_loaded("sodium")) print "skip"; ?>
3+
--EXTENSIONS--
4+
sodium
55
--FILE--
66
<?php
77
$client_seed = sodium_hex2bin('0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef');

‎ext/sodium/tests/crypto_scalarmult.phpt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
--TEST--
22
Check for libsodium scalarmult
3-
--SKIPIF--
4-
<?php if (!extension_loaded("sodium")) print "skip"; ?>
3+
--EXTENSIONS--
4+
sodium
55
--FILE--
66
<?php
77
$n = sodium_hex2bin("5dab087e624a8a4b79e17f8b83800ee66f3bb1292618b6fd1c2f8b27ff88e0eb");

‎ext/sodium/tests/crypto_secretbox.phpt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
--TEST--
22
Check for libsodium secretbox
3-
--SKIPIF--
4-
<?php if (!extension_loaded("sodium")) print "skip"; ?>
3+
--EXTENSIONS--
4+
sodium
55
--FILE--
66
<?php
77
$nonce = random_bytes(SODIUM_CRYPTO_SECRETBOX_NONCEBYTES);

‎ext/sodium/tests/crypto_secretstream.phpt

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
11
--TEST--
22
Check for libsodium secretstream
3+
--EXTENSIONS--
4+
sodium
35
--SKIPIF--
46
<?php
5-
if (!extension_loaded("sodium")) print "skip extension not loaded";
67
if (!defined('SODIUM_CRYPTO_SECRETSTREAM_XCHACHA20POLY1305_ABYTES')) print "skip libsodium without secretbytes";
78
?>
89
--FILE--

‎ext/sodium/tests/crypto_shorthash.phpt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
--TEST--
22
Check for libsodium shorthash
3-
--SKIPIF--
4-
<?php if (!extension_loaded("sodium")) print "skip"; ?>
3+
--EXTENSIONS--
4+
sodium
55
--FILE--
66
<?php
77
$m1 = 'message';

‎ext/sodium/tests/crypto_sign.phpt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
--TEST--
22
Check for libsodium ed25519 signatures
3-
--SKIPIF--
4-
<?php if (!extension_loaded("sodium")) print "skip"; ?>
3+
--EXTENSIONS--
4+
sodium
55
--FILE--
66
<?php
77
$keypair = sodium_crypto_sign_keypair();

‎ext/sodium/tests/crypto_stream.phpt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
--TEST--
22
Check for libsodium stream
3-
--SKIPIF--
4-
<?php if (!extension_loaded("sodium")) print "skip"; ?>
3+
--EXTENSIONS--
4+
sodium
55
--FILE--
66
<?php
77
$nonce = random_bytes(SODIUM_CRYPTO_STREAM_NONCEBYTES);

‎ext/sodium/tests/exception_trace_without_args.phpt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
--TEST--
22
SodiumException backtraces do not contain function arguments
3-
--SKIPIF--
4-
<?php if (!extension_loaded("sodium")) print "skip"; ?>
3+
--EXTENSIONS--
4+
sodium
55
--FILE--
66
<?php
77

‎ext/sodium/tests/inc_add.phpt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
--TEST--
22
increment and add edge cases
3-
--SKIPIF--
4-
<?php if (!extension_loaded("sodium")) print "skip"; ?>
3+
--EXTENSIONS--
4+
sodium
55
--FILE--
66
<?php
77

‎ext/sodium/tests/installed.phpt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
--TEST--
22
Check for sodium presence
3-
--SKIPIF--
4-
<?php if (!extension_loaded("sodium")) print "skip"; ?>
3+
--EXTENSIONS--
4+
sodium
55
--FILE--
66
<?php
77
echo "sodium extension is available";

‎ext/sodium/tests/pwhash_argon2i.phpt

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,9 @@
11
--TEST--
22
Check for libsodium argon2i
3+
--EXTENSIONS--
4+
sodium
35
--SKIPIF--
4-
<?php if (!extension_loaded("sodium")) print "skip";
6+
<?php
57
if (!defined('SODIUM_CRYPTO_PWHASH_SALTBYTES')) print "skip libsodium without argon2i"; ?>
68
--FILE--
79
<?php

‎ext/sodium/tests/pwhash_scrypt.phpt

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,9 @@
11
--TEST--
22
Check for libsodium scrypt
3+
--EXTENSIONS--
4+
sodium
35
--SKIPIF--
4-
<?php if (!extension_loaded("sodium")) print "skip";
6+
<?php
57
if (!defined('SODIUM_CRYPTO_PWHASH_SCRYPTSALSA208SHA256_SALTBYTES')) print "skip libsodium without scrypt"; ?>
68
--FILE--
79
<?php

‎ext/sodium/tests/sodium_error_001.phpt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
--TEST--
22
TypeErrors will not contain param values in backtrace
3-
--SKIPIF--
4-
<?php if (!extension_loaded("sodium")) die("skip ext/sodium required"); ?>
3+
--EXTENSIONS--
4+
sodium
55
--FILE--
66
<?php
77
declare(strict_types=1);

‎ext/sodium/tests/utils.phpt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
--TEST--
22
Check for libsodium utils
3-
--SKIPIF--
4-
<?php if (!extension_loaded("sodium")) print "skip"; ?>
3+
--EXTENSIONS--
4+
sodium
55
--FILE--
66
<?php
77
$a = 'test';

‎ext/sodium/tests/version.phpt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
--TEST--
22
Check for libsodium version
3-
--SKIPIF--
4-
<?php if (!extension_loaded("sodium")) print "skip"; ?>
3+
--EXTENSIONS--
4+
sodium
55
--FILE--
66
<?php
77
echo strlen(SODIUM_LIBRARY_VERSION) >= 5;

‎run-tests.php

Lines changed: 32 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -628,7 +628,7 @@ function main(): void
628628
if (is_dir($testfile)) {
629629
find_files($testfile);
630630
} else {
631-
if (substr($testfile, -5) == '.phpt') {
631+
if (substr($testfile, -5) == '.phpt' || is_file($argv[$i])) {
632632
$test_files[] = $testfile;
633633
} else {
634634
die('Cannot find test file "' . $argv[$i] . '".' . PHP_EOL);
@@ -1958,22 +1958,22 @@ function run_test(string $php, $file, array $env): string
19581958
$temp_dir = str_replace($temp_source, $temp_target, $temp_dir);
19591959
}
19601960

1961-
$main_file_name = basename($file, 'phpt');
1962-
1963-
$diff_filename = $temp_dir . DIRECTORY_SEPARATOR . $main_file_name . 'diff';
1964-
$log_filename = $temp_dir . DIRECTORY_SEPARATOR . $main_file_name . 'log';
1965-
$exp_filename = $temp_dir . DIRECTORY_SEPARATOR . $main_file_name . 'exp';
1966-
$output_filename = $temp_dir . DIRECTORY_SEPARATOR . $main_file_name . 'out';
1967-
$memcheck_filename = $temp_dir . DIRECTORY_SEPARATOR . $main_file_name . 'mem';
1968-
$sh_filename = $temp_dir . DIRECTORY_SEPARATOR . $main_file_name . 'sh';
1969-
$temp_file = $temp_dir . DIRECTORY_SEPARATOR . $main_file_name . 'php';
1970-
$test_file = $test_dir . DIRECTORY_SEPARATOR . $main_file_name . 'php';
1971-
$temp_skipif = $temp_dir . DIRECTORY_SEPARATOR . $main_file_name . 'skip.php';
1972-
$test_skipif = $test_dir . DIRECTORY_SEPARATOR . $main_file_name . 'skip.php';
1973-
$temp_clean = $temp_dir . DIRECTORY_SEPARATOR . $main_file_name . 'clean.php';
1974-
$test_clean = $test_dir . DIRECTORY_SEPARATOR . $main_file_name . 'clean.php';
1975-
$preload_filename = $temp_dir . DIRECTORY_SEPARATOR . $main_file_name . 'preload.php';
1976-
$tmp_post = $temp_dir . DIRECTORY_SEPARATOR . $main_file_name . 'post';
1961+
$main_file_name = basename($file, '.phpt');
1962+
1963+
$diff_filename = $temp_dir . DIRECTORY_SEPARATOR . $main_file_name . '.diff';
1964+
$log_filename = $temp_dir . DIRECTORY_SEPARATOR . $main_file_name . '.log';
1965+
$exp_filename = $temp_dir . DIRECTORY_SEPARATOR . $main_file_name . '.exp';
1966+
$output_filename = $temp_dir . DIRECTORY_SEPARATOR . $main_file_name . '.out';
1967+
$memcheck_filename = $temp_dir . DIRECTORY_SEPARATOR . $main_file_name . '.mem';
1968+
$sh_filename = $temp_dir . DIRECTORY_SEPARATOR . $main_file_name . '.sh';
1969+
$temp_file = $temp_dir . DIRECTORY_SEPARATOR . $main_file_name . '.php';
1970+
$test_file = $test_dir . DIRECTORY_SEPARATOR . $main_file_name . '.php';
1971+
$temp_skipif = $temp_dir . DIRECTORY_SEPARATOR . $main_file_name . '.skip.php';
1972+
$test_skipif = $test_dir . DIRECTORY_SEPARATOR . $main_file_name . '.skip.php';
1973+
$temp_clean = $temp_dir . DIRECTORY_SEPARATOR . $main_file_name . '.clean.php';
1974+
$test_clean = $test_dir . DIRECTORY_SEPARATOR . $main_file_name . '.clean.php';
1975+
$preload_filename = $temp_dir . DIRECTORY_SEPARATOR . $main_file_name . '.preload.php';
1976+
$tmp_post = $temp_dir . DIRECTORY_SEPARATOR . $main_file_name . '.post';
19771977
$tmp_relative_file = str_replace(__DIR__ . DIRECTORY_SEPARATOR, '', $test_file) . 't';
19781978

19791979
if ($temp_source && $temp_target) {
@@ -2057,15 +2057,28 @@ function run_test(string $php, $file, array $env): string
20572057
$extensions = preg_split("/[\n\r]+/", trim($test->getSection('EXTENSIONS')));
20582058
[$ext_dir, $loaded] = $skipCache->getExtensions("$php $pass_options $extra_options $ext_params $no_file_cache");
20592059
$ext_prefix = IS_WINDOWS ? "php_" : "";
2060+
$missing = [];
20602061
foreach ($extensions as $req_ext) {
20612062
if (!in_array($req_ext, $loaded)) {
20622063
if ($req_ext == 'opcache') {
2063-
$ini_settings['zend_extension'][] = $ext_dir . DIRECTORY_SEPARATOR . $ext_prefix . $req_ext . '.' . PHP_SHLIB_SUFFIX;
2064+
$ext_file = $ext_dir . DIRECTORY_SEPARATOR . $ext_prefix . $req_ext . '.' . PHP_SHLIB_SUFFIX;
2065+
$ini_settings['zend_extension'][] = $ext_file;
20642066
} else {
2065-
$ini_settings['extension'][] = $ext_dir . DIRECTORY_SEPARATOR . $ext_prefix . $req_ext . '.' . PHP_SHLIB_SUFFIX;
2067+
$ext_file = $ext_dir . DIRECTORY_SEPARATOR . $ext_prefix . $req_ext . '.' . PHP_SHLIB_SUFFIX;
2068+
$ini_settings['extension'][] = $ext_file;
2069+
}
2070+
if (!is_readable($ext_file)) {
2071+
$missing[] = $req_ext;
20662072
}
20672073
}
20682074
}
2075+
if ($missing) {
2076+
$message = 'Required extension' . ( count($missing) > 1
2077+
? 's missing: ' . implode(', ', $missing)
2078+
: ' missing:' . $missing[0]
2079+
);
2080+
return skip_test($tested, $tested_file, $shortname, $message);
2081+
}
20692082
}
20702083

20712084
// additional ini overwrites

‎tests/run-test/bug75042-2.phpt

Lines changed: 0 additions & 10 deletions
This file was deleted.

‎tests/run-test/bug75042-3.phpt

Lines changed: 0 additions & 13 deletions
This file was deleted.

‎tests/run-test/extensions-nonexistent.phpt

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
--TEST--
2+
phpt EXTENSIONS directive - extension not found
3+
--FILE--
4+
<?php
5+
6+
require __DIR__ . '/self-invoke.inc.php';
7+
8+
run_test('extensions-nonexistent.phpt.txt');
9+
10+
--EXPECTREGEX--
11+
.*tests="1" failures="0" errors="0" skip="1".*

‎tests/run-test/extensions-nonexistent.phpt.txt

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
--TEST--
2+
This test should always skip
3+
--EXTENSIONS--
4+
standard
5+
ExtensionNotFound
6+
--FILE--
7+
<?php
8+
This should never execute
9+
--EXPECT--
10+
This should never succeed

‎tests/run-test/bug75042.phpt renamed to ‎tests/run-test/extensions-shared.phpt

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
--TEST--
2-
phpt EXTENSIONS directive with shared module
2+
phpt EXTENSIONS directive - shared module
3+
--EXTENSIONS--
4+
openssl
35
--SKIPIF--
46
<?php
57
$php = getenv('TEST_PHP_EXECUTABLE');
@@ -8,11 +10,10 @@ if (false !== stripos(`$php -n -m`, 'openssl')) {
810
}
911
$ext_module = ini_get('extension_dir') . DIRECTORY_SEPARATOR . (substr(PHP_OS, 0, 3) === "WIN" ? "php_openssl." : "openssl.") . PHP_SHLIB_SUFFIX;
1012
if( !file_exists($ext_module) ) die('skip openssl shared extension not found');
11-
--EXTENSIONS--
12-
openssl
13+
1314
--FILE--
1415
<?php
1516
var_dump(extension_loaded('openssl'));
16-
?>
17+
1718
--EXPECT--
1819
bool(true)

‎tests/run-test/extensions-static.phpt

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
--TEST--
2+
phpt EXTENSIONS directive - static extension is present
3+
--EXTENSIONS--
4+
standard
5+
--FILE--
6+
<?php
7+
var_dump(extension_loaded('standard'));
8+
--EXPECT--
9+
bool(true)

0 commit comments

Comments
 (0)