mirror of
https://github.com/php/php-src.git
synced 2024-12-12 03:15:29 +08:00
180 lines
6.4 KiB
Plaintext
180 lines
6.4 KiB
Plaintext
[IMPORTANT NOTICE]
|
|
------------------
|
|
Failed tests usualy indicate a problem with your local system setup
|
|
and not within PHP itself (at least for official PHP release versions).
|
|
You may decide to automaticaly submit a test summary to our QA workflow
|
|
at the end of a test run.
|
|
Please do *not* submit a failed test as a bug or ask for help on why
|
|
it failed on your system without providing substantial backup information
|
|
on *why* the test failed on your special setup. Thank you :-)
|
|
|
|
|
|
[Testing Basics]
|
|
----------------
|
|
The easiest way to test your PHP build is to run "make test" from the
|
|
command line after successfully compiling. This will run the complete
|
|
tests for all enabled functionalities and extensions using the PHP
|
|
CLI binary.
|
|
To execute test scripts, you must build PHP with some SAPI, then you
|
|
type "make test" to execute all or some test scripts saved under
|
|
"tests" directory under source root directory.
|
|
|
|
Usage:
|
|
make test
|
|
|
|
"make test" basically executes "run-tests.php" script
|
|
under the source root (parallel builds will not work). Therefore you
|
|
can execute the script as follows:
|
|
|
|
TEST_PHP_EXECUTABLE=sapi/cli/php \
|
|
sapi/cli/php [-c /path/to/php.ini] run-tests.php [ext/foo/tests/GLOB]
|
|
|
|
|
|
[Which "php" executable "make test" look for]
|
|
---------------------------------------------
|
|
You must use TEST_PHP_EXECUTABLE environment variable to explicitly
|
|
select the php executable to be used to run the tests. That can either
|
|
be the CLI or CGI executable.
|
|
|
|
"make test" executes "run-tests.php" script with "php" binary. Some
|
|
test scripts such as session must be executed by CGI SAPI. Therefore,
|
|
you must build PHP with CGI SAPI to perform all tests.
|
|
|
|
NOTE: PHP binary executing "run-tests.php" and php binary used for
|
|
executing test scripts may differ. If you use different PHP binary for
|
|
executing "run-tests.php" script, you may get errors.
|
|
|
|
|
|
[Which php.ini is used]
|
|
-----------------------
|
|
"make test" uses the same php.ini file as it would once installed.
|
|
The tests have been written to be independent of that php.ini file,
|
|
so if you find a test that is affected by a setting, please report
|
|
this, so we can address the issue.
|
|
|
|
|
|
[Which test scripts are executed]
|
|
---------------------------------
|
|
"run-tests.php" ("make test"), without any arguments executes all
|
|
test scripts by extracting all directories named "tests"
|
|
from the source root and any subdirectories below. If there are files,
|
|
which have a "phpt" extension, "run-tests.php" looks at the sections
|
|
in these files, determines whether it should run it, by evaluating
|
|
the 'SKIP' section. If the test is eligible for execution, the 'FILE'
|
|
section is extracted into a ".php" file (with the same name besides
|
|
the extension) and gets executed.
|
|
When an argument is given or TESTS environment variable is set, the
|
|
GLOB is expanded by the shell and any file with extension "*.phpt" is
|
|
regarded as a test file.
|
|
|
|
Tester can easily execute tests selectively with as follows.
|
|
|
|
Examples:
|
|
./sapi/cli/php run-tests.php ext/mbstring/*
|
|
./sapi/cli/php run-tests.php ext/mbstring/020.phpt
|
|
|
|
|
|
[Test results]
|
|
--------------
|
|
Test results are printed to standard output. If there is a failed test,
|
|
the "run-tests.php" script saves the result, the expected result and the
|
|
code executed to the test script directory. For example, if
|
|
ext/myext/tests/myext.phpt fails to pass, the following files are created:
|
|
|
|
ext/myext/tests/myext.php - actual test file executed
|
|
ext/myext/tests/myext.log - log of test execution (L)
|
|
ext/myext/tests/myext.exp - expected output (E)
|
|
ext/myext/tests/myext.out - output from test script (O)
|
|
ext/myext/tests/myext.diff - diff of .out and .exp (D)
|
|
|
|
Failed tests are always bugs. Either the test is bugged or not considering
|
|
factors applying to the tester's environment, or there is a bug in PHP.
|
|
If this is a known bug, we strive to provide bug numbers, in either the
|
|
test name or the file name. You can check the status of such a bug, by
|
|
going to: http://bugs.php.net/12345 where 12345 is the bug number.
|
|
For clarity and automated processing, bug numbers are prefixed by a hash
|
|
sign '#' in test names and/or test cases are named bug12345.phpt.
|
|
|
|
NOTE: The files generated by tests can be selected by setting the
|
|
environment variable TEST_PHP_LOG_FORMAT. For each file you want to be
|
|
generated use the character in brackets as shown above (default is LEOD).
|
|
The php file will be generated always.
|
|
|
|
NOTE: You can set environment variable TEST_PHP_DETAILED to enable
|
|
detailed test information.
|
|
|
|
[Automated testing]
|
|
If you like to keep up to speed, with latest developments and quality
|
|
assurance, setting the environment variable NO_INTERACTION to 1, will not
|
|
prompt the tester for any user input.
|
|
|
|
Normally, the exit status of "make test" is zero, regardless of the results
|
|
of independent tests. Set the environment variable REPORT_EXIT_STATUS to 1,
|
|
and "make test" will set the exit status ("$?") to non-zero, when an
|
|
individual test has failed.
|
|
|
|
Example script to be run by cron(1):
|
|
========== qa-test.sh =============
|
|
#!/bin/sh
|
|
|
|
CO_DIR=$HOME/cvs/php5
|
|
MYMAIL=qa-test@domain.com
|
|
TMPDIR=/var/tmp
|
|
TODAY=`date +"%Y%m%d"`
|
|
|
|
# Make sure compilation enviroment is correct
|
|
CONFIGURE_OPTS='--disable-all --enable-cli --with-pcre'
|
|
export MAKE=gmake
|
|
export CC=gcc
|
|
|
|
# Set test environment
|
|
export NO_INTERACTION=1
|
|
export REPORT_EXIT_STATUS=1
|
|
|
|
cd $CO_DIR
|
|
cvs update . >>$TMPDIR/phpqatest.$TODAY
|
|
./cvsclean ; ./buildconf ; ./configure $CONFIGURE_OPTS ; $MAKE
|
|
$MAKE test >>$TMPDIR/phpqatest.$TODAY 2>&1
|
|
if test $? -gt 0
|
|
then
|
|
cat $TMPDIR/phpqatest.$TODAY | mail -s"PHP-QA Test Failed for $TODAY" $MYMAIL
|
|
fi
|
|
========== end of qa-test.sh =============
|
|
|
|
NOTE: the exit status of run-tests.php will be 1 when
|
|
REPORT_EXIT_STATUS is set. The result of "make test" may be higher
|
|
than that. At present, gmake 3.79.1 returns 2, so it is
|
|
advised to test for non-zero, rather then a specific value.
|
|
|
|
|
|
[Creating new test files]
|
|
-------------------------
|
|
Writing test file is very easy if you are used to PHP.
|
|
See the HOWTO at http://qa.php.net/write-test.php
|
|
|
|
|
|
[How to help us]
|
|
----------------
|
|
If you find bug in PHP, you can submit bug report AND test script
|
|
for us. You don't have to write complete script, just give us test
|
|
script with following format. Please test the script and make sure
|
|
you write the correct ACTUAL OUTPUT and EXPECTED OUTPUT before you
|
|
submit.
|
|
|
|
<?php
|
|
/*
|
|
Bug #12345
|
|
substr() bug. Do not return expected string.
|
|
|
|
ACTUAL OUTPUT
|
|
XYXA
|
|
|
|
EXPECTED OUTPUT
|
|
ABCD
|
|
*/
|
|
|
|
$str = "XYZABCD";
|
|
echo substr($str,3,7);
|
|
|
|
?>
|