2013-11-26 05:03:06 +08:00
|
|
|
# Test framework for git. See t/README for usage.
|
2005-05-14 13:50:32 +08:00
|
|
|
#
|
|
|
|
|
# Copyright (c) 2005 Junio C Hamano
|
|
|
|
|
#
|
2010-04-16 21:53:59 +08:00
|
|
|
# This program is free software: you can redistribute it and/or modify
|
|
|
|
|
# it under the terms of the GNU General Public License as published by
|
|
|
|
|
# the Free Software Foundation, either version 2 of the License, or
|
|
|
|
|
# (at your option) any later version.
|
|
|
|
|
#
|
|
|
|
|
# This program is distributed in the hope that it will be useful,
|
|
|
|
|
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
|
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
|
|
|
# GNU General Public License for more details.
|
|
|
|
|
#
|
|
|
|
|
# You should have received a copy of the GNU General Public License
|
|
|
|
|
# along with this program. If not, see http://www.gnu.org/licenses/ .
|
2005-05-14 13:50:32 +08:00
|
|
|
|
2012-06-25 13:01:35 +08:00
|
|
|
# Test the binaries we have just built. The tests are kept in
|
|
|
|
|
# t/ subdirectory and are run in 'trash directory' subdirectory.
|
|
|
|
|
if test -z "$TEST_DIRECTORY"
|
|
|
|
|
then
|
|
|
|
|
# We allow tests to override this, in case they want to run tests
|
|
|
|
|
# outside of t/, e.g. for running tests on the test library
|
|
|
|
|
# itself.
|
|
|
|
|
TEST_DIRECTORY=$(pwd)
|
2013-11-18 12:12:43 +08:00
|
|
|
else
|
|
|
|
|
# ensure that TEST_DIRECTORY is an absolute path so that it
|
|
|
|
|
# is valid even if the current working directory is changed
|
|
|
|
|
TEST_DIRECTORY=$(cd "$TEST_DIRECTORY" && pwd) || exit 1
|
2012-06-25 13:01:35 +08:00
|
|
|
fi
|
|
|
|
|
if test -z "$TEST_OUTPUT_DIRECTORY"
|
|
|
|
|
then
|
|
|
|
|
# Similarly, override this to store the test-results subdir
|
|
|
|
|
# elsewhere
|
|
|
|
|
TEST_OUTPUT_DIRECTORY=$TEST_DIRECTORY
|
|
|
|
|
fi
|
|
|
|
|
GIT_BUILD_DIR="$TEST_DIRECTORY"/..
|
|
|
|
|
|
2012-09-18 01:06:19 +08:00
|
|
|
################################################################
|
|
|
|
|
# It appears that people try to run tests without building...
|
|
|
|
|
"$GIT_BUILD_DIR/git" >/dev/null
|
|
|
|
|
if test $? != 1
|
|
|
|
|
then
|
|
|
|
|
echo >&2 'error: you do not seem to have built git yet.'
|
|
|
|
|
exit 1
|
|
|
|
|
fi
|
|
|
|
|
|
2012-06-25 13:01:35 +08:00
|
|
|
. "$GIT_BUILD_DIR"/GIT-BUILD-OPTIONS
|
|
|
|
|
export PERL_PATH SHELL_PATH
|
|
|
|
|
|
2012-09-22 12:55:10 +08:00
|
|
|
# if --tee was passed, write the output not only to the terminal, but
|
|
|
|
|
# additionally to the file test-results/$BASENAME.out, too.
|
|
|
|
|
case "$GIT_TEST_TEE_STARTED, $* " in
|
|
|
|
|
done,*)
|
|
|
|
|
# do not redirect again
|
|
|
|
|
;;
|
test-lib: add --verbose-log option
The "--verbose" option redirects output from arbitrary
test commands to stdout. This is useful for examining the
output manually, like:
./t5547-push-quarantine.sh -v | less
But it also means that the output is intermingled with the
TAP directives, which can confuse a TAP parser like "prove".
This has always been a potential problem, but became an
issue recently when one test happened to output the word
"ok" on a line by itself, which prove interprets as a test
success:
$ prove t5547-push-quarantine.sh :: -v
t5547-push-quarantine.sh .. 1/? To dest.git
* [new branch] HEAD -> master
To dest.git
! [remote rejected] reject -> reject (pre-receive hook declined)
error: failed to push some refs to 'dest.git'
fatal: git cat-file d08c8eba97f4e683ece08654c7c8d2ba0c03b129: bad file
t5547-push-quarantine.sh .. Failed -1/4 subtests
Test Summary Report
-------------------
t5547-push-quarantine.sh (Wstat: 0 Tests: 5 Failed: 0)
Parse errors: Tests out of sequence. Found (2) but expected (3)
Tests out of sequence. Found (3) but expected (4)
Tests out of sequence. Found (4) but expected (5)
Bad plan. You planned 4 tests but ran 5.
Files=1, Tests=5, 0 wallclock secs ( 0.01 usr + 0.01 sys = 0.02 CPU)
Result: FAIL
One answer is "if it hurts, don't do it", but that's not
quite the whole story. The Travis tests use "--verbose
--tee" so that they can get the benefit of prove's parallel
options, along with a verbose log in case there is a
failure. We just need the verbose output to go to the log,
but keep stdout clean.
Getting this right turns out to be surprisingly difficult.
Here's the progression of alternatives I considered:
1. Add an option to write verbose output to stderr. This is
hard to capture, though, because we want each test to
have its own log (because they're all run in parallel
and the jumbled output would be useless).
2. Add an option to write verbose output to a file in
test-results. This works, but the log is missing all of
the non-verbose output, which gives context.
3. Like (2), but teach say_color() to additionally output
to the log. This mostly works, but misses any output
that happens outside of the say() functions (which isn't
a lot, but is a potential maintenance headache).
4. Like (2), but make the log file the same as the "--tee"
file. That almost works, but now we have two processes
opening the same file. That gives us two separate
descriptors, each with their own idea of the current
position. They'll each start writing at offset 0, and
overwrite each other's data.
5. Like (4), but in each case open the file for appending.
That atomically positions each write at the end of the
file.
It's possible we may still get sheared writes between
the two processes, but this is already the case when
writing to stdout. It's not a problem in practice
because the test harness generally waits for snippets to
finish before writing the TAP output.
We can ignore buffering issues with tee, because POSIX
mandates that it does not buffer. Likewise, POSIX
specifies "tee -a", so it should be available
everywhere.
This patch implements option (5), which seems to work well
in practice.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-21 18:48:00 +08:00
|
|
|
*' --tee '*|*' --va'*|*' --verbose-log '*)
|
2013-04-30 02:16:21 +08:00
|
|
|
mkdir -p "$TEST_OUTPUT_DIRECTORY/test-results"
|
|
|
|
|
BASE="$TEST_OUTPUT_DIRECTORY/test-results/$(basename "$0" .sh)"
|
test-lib: add --verbose-log option
The "--verbose" option redirects output from arbitrary
test commands to stdout. This is useful for examining the
output manually, like:
./t5547-push-quarantine.sh -v | less
But it also means that the output is intermingled with the
TAP directives, which can confuse a TAP parser like "prove".
This has always been a potential problem, but became an
issue recently when one test happened to output the word
"ok" on a line by itself, which prove interprets as a test
success:
$ prove t5547-push-quarantine.sh :: -v
t5547-push-quarantine.sh .. 1/? To dest.git
* [new branch] HEAD -> master
To dest.git
! [remote rejected] reject -> reject (pre-receive hook declined)
error: failed to push some refs to 'dest.git'
fatal: git cat-file d08c8eba97f4e683ece08654c7c8d2ba0c03b129: bad file
t5547-push-quarantine.sh .. Failed -1/4 subtests
Test Summary Report
-------------------
t5547-push-quarantine.sh (Wstat: 0 Tests: 5 Failed: 0)
Parse errors: Tests out of sequence. Found (2) but expected (3)
Tests out of sequence. Found (3) but expected (4)
Tests out of sequence. Found (4) but expected (5)
Bad plan. You planned 4 tests but ran 5.
Files=1, Tests=5, 0 wallclock secs ( 0.01 usr + 0.01 sys = 0.02 CPU)
Result: FAIL
One answer is "if it hurts, don't do it", but that's not
quite the whole story. The Travis tests use "--verbose
--tee" so that they can get the benefit of prove's parallel
options, along with a verbose log in case there is a
failure. We just need the verbose output to go to the log,
but keep stdout clean.
Getting this right turns out to be surprisingly difficult.
Here's the progression of alternatives I considered:
1. Add an option to write verbose output to stderr. This is
hard to capture, though, because we want each test to
have its own log (because they're all run in parallel
and the jumbled output would be useless).
2. Add an option to write verbose output to a file in
test-results. This works, but the log is missing all of
the non-verbose output, which gives context.
3. Like (2), but teach say_color() to additionally output
to the log. This mostly works, but misses any output
that happens outside of the say() functions (which isn't
a lot, but is a potential maintenance headache).
4. Like (2), but make the log file the same as the "--tee"
file. That almost works, but now we have two processes
opening the same file. That gives us two separate
descriptors, each with their own idea of the current
position. They'll each start writing at offset 0, and
overwrite each other's data.
5. Like (4), but in each case open the file for appending.
That atomically positions each write at the end of the
file.
It's possible we may still get sheared writes between
the two processes, but this is already the case when
writing to stdout. It's not a problem in practice
because the test harness generally waits for snippets to
finish before writing the TAP output.
We can ignore buffering issues with tee, because POSIX
mandates that it does not buffer. Likewise, POSIX
specifies "tee -a", so it should be available
everywhere.
This patch implements option (5), which seems to work well
in practice.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-21 18:48:00 +08:00
|
|
|
|
|
|
|
|
# Make this filename available to the sub-process in case it is using
|
|
|
|
|
# --verbose-log.
|
|
|
|
|
GIT_TEST_TEE_OUTPUT_FILE=$BASE.out
|
|
|
|
|
export GIT_TEST_TEE_OUTPUT_FILE
|
|
|
|
|
|
|
|
|
|
# Truncate before calling "tee -a" to get rid of the results
|
|
|
|
|
# from any previous runs.
|
|
|
|
|
>"$GIT_TEST_TEE_OUTPUT_FILE"
|
|
|
|
|
|
2012-09-22 12:55:10 +08:00
|
|
|
(GIT_TEST_TEE_STARTED=done ${SHELL_PATH} "$0" "$@" 2>&1;
|
test-lib: add --verbose-log option
The "--verbose" option redirects output from arbitrary
test commands to stdout. This is useful for examining the
output manually, like:
./t5547-push-quarantine.sh -v | less
But it also means that the output is intermingled with the
TAP directives, which can confuse a TAP parser like "prove".
This has always been a potential problem, but became an
issue recently when one test happened to output the word
"ok" on a line by itself, which prove interprets as a test
success:
$ prove t5547-push-quarantine.sh :: -v
t5547-push-quarantine.sh .. 1/? To dest.git
* [new branch] HEAD -> master
To dest.git
! [remote rejected] reject -> reject (pre-receive hook declined)
error: failed to push some refs to 'dest.git'
fatal: git cat-file d08c8eba97f4e683ece08654c7c8d2ba0c03b129: bad file
t5547-push-quarantine.sh .. Failed -1/4 subtests
Test Summary Report
-------------------
t5547-push-quarantine.sh (Wstat: 0 Tests: 5 Failed: 0)
Parse errors: Tests out of sequence. Found (2) but expected (3)
Tests out of sequence. Found (3) but expected (4)
Tests out of sequence. Found (4) but expected (5)
Bad plan. You planned 4 tests but ran 5.
Files=1, Tests=5, 0 wallclock secs ( 0.01 usr + 0.01 sys = 0.02 CPU)
Result: FAIL
One answer is "if it hurts, don't do it", but that's not
quite the whole story. The Travis tests use "--verbose
--tee" so that they can get the benefit of prove's parallel
options, along with a verbose log in case there is a
failure. We just need the verbose output to go to the log,
but keep stdout clean.
Getting this right turns out to be surprisingly difficult.
Here's the progression of alternatives I considered:
1. Add an option to write verbose output to stderr. This is
hard to capture, though, because we want each test to
have its own log (because they're all run in parallel
and the jumbled output would be useless).
2. Add an option to write verbose output to a file in
test-results. This works, but the log is missing all of
the non-verbose output, which gives context.
3. Like (2), but teach say_color() to additionally output
to the log. This mostly works, but misses any output
that happens outside of the say() functions (which isn't
a lot, but is a potential maintenance headache).
4. Like (2), but make the log file the same as the "--tee"
file. That almost works, but now we have two processes
opening the same file. That gives us two separate
descriptors, each with their own idea of the current
position. They'll each start writing at offset 0, and
overwrite each other's data.
5. Like (4), but in each case open the file for appending.
That atomically positions each write at the end of the
file.
It's possible we may still get sheared writes between
the two processes, but this is already the case when
writing to stdout. It's not a problem in practice
because the test harness generally waits for snippets to
finish before writing the TAP output.
We can ignore buffering issues with tee, because POSIX
mandates that it does not buffer. Likewise, POSIX
specifies "tee -a", so it should be available
everywhere.
This patch implements option (5), which seems to work well
in practice.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-21 18:48:00 +08:00
|
|
|
echo $? >"$BASE.exit") | tee -a "$GIT_TEST_TEE_OUTPUT_FILE"
|
2016-10-21 18:42:10 +08:00
|
|
|
test "$(cat "$BASE.exit")" = 0
|
2012-09-22 12:55:10 +08:00
|
|
|
exit
|
|
|
|
|
;;
|
|
|
|
|
esac
|
|
|
|
|
|
2005-05-14 13:50:32 +08:00
|
|
|
# For repeatability, reset the environment to known value.
|
2015-06-18 05:11:21 +08:00
|
|
|
# TERM is sanitized below, after saving color control sequences.
|
2005-05-14 13:50:32 +08:00
|
|
|
LANG=C
|
2005-10-10 12:58:02 +08:00
|
|
|
LC_ALL=C
|
2005-08-11 10:10:01 +08:00
|
|
|
PAGER=cat
|
2005-05-14 13:50:32 +08:00
|
|
|
TZ=UTC
|
2015-06-18 05:11:21 +08:00
|
|
|
export LANG LC_ALL PAGER TZ
|
2006-07-12 03:01:54 +08:00
|
|
|
EDITOR=:
|
2012-03-03 02:48:36 +08:00
|
|
|
# A call to "unset" with no arguments causes at least Solaris 10
|
|
|
|
|
# /usr/xpg4/bin/sh and /bin/ksh to bail out. So keep the unsets
|
|
|
|
|
# deriving from the command substitution clustered with the other
|
|
|
|
|
# ones.
|
2012-06-25 13:01:35 +08:00
|
|
|
unset VISUAL EMAIL LANGUAGE COLUMNS $("$PERL_PATH" -e '
|
2011-03-15 18:10:45 +08:00
|
|
|
my @env = keys %ENV;
|
2011-03-29 03:16:09 +08:00
|
|
|
my $ok = join("|", qw(
|
|
|
|
|
TRACE
|
|
|
|
|
DEBUG
|
|
|
|
|
USE_LOOKUP
|
|
|
|
|
TEST
|
|
|
|
|
.*_TEST
|
|
|
|
|
PROVE
|
|
|
|
|
VALGRIND
|
2013-01-07 01:47:57 +08:00
|
|
|
UNZIP
|
2013-01-15 21:50:56 +08:00
|
|
|
PERF_
|
2014-05-22 17:28:50 +08:00
|
|
|
CURL_VERBOSE
|
2016-09-05 18:24:42 +08:00
|
|
|
TRACE_CURL
|
2011-03-29 03:16:09 +08:00
|
|
|
));
|
|
|
|
|
my @vars = grep(/^GIT_/ && !/^GIT_($ok)/o, @env);
|
2011-03-15 18:10:45 +08:00
|
|
|
print join("\n", @vars);
|
|
|
|
|
')
|
2012-07-24 19:53:05 +08:00
|
|
|
unset XDG_CONFIG_HOME
|
2013-07-05 04:38:55 +08:00
|
|
|
unset GITPERLLIB
|
2006-02-11 11:11:23 +08:00
|
|
|
GIT_AUTHOR_EMAIL=author@example.com
|
|
|
|
|
GIT_AUTHOR_NAME='A U Thor'
|
|
|
|
|
GIT_COMMITTER_EMAIL=committer@example.com
|
|
|
|
|
GIT_COMMITTER_NAME='C O Mitter'
|
2007-02-04 13:45:47 +08:00
|
|
|
GIT_MERGE_VERBOSITY=5
|
2012-01-11 14:44:45 +08:00
|
|
|
GIT_MERGE_AUTOEDIT=no
|
|
|
|
|
export GIT_MERGE_VERBOSITY GIT_MERGE_AUTOEDIT
|
2006-02-11 11:11:23 +08:00
|
|
|
export GIT_AUTHOR_EMAIL GIT_AUTHOR_NAME
|
|
|
|
|
export GIT_COMMITTER_EMAIL GIT_COMMITTER_NAME
|
Do not use VISUAL editor on dumb terminals
Refuse to use $VISUAL and fall back to $EDITOR if TERM is unset
or set to "dumb". Traditionally, VISUAL is set to a screen
editor and EDITOR to a line-based editor, which should be more
useful in that situation.
vim, for example, is happy to assume a terminal supports ANSI
sequences even if TERM is dumb (e.g., when running from a text
editor like Acme). git already refuses to fall back to vi on a
dumb terminal if GIT_EDITOR, core.editor, VISUAL, and EDITOR are
unset, but without this patch, that check is suppressed by
VISUAL=vi.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-11-12 07:56:07 +08:00
|
|
|
export EDITOR
|
2005-05-14 13:50:32 +08:00
|
|
|
|
2014-07-12 08:03:01 +08:00
|
|
|
# Tests using GIT_TRACE typically don't want <timestamp> <file>:<line> output
|
|
|
|
|
GIT_TRACE_BARE=1
|
|
|
|
|
export GIT_TRACE_BARE
|
|
|
|
|
|
2014-02-24 04:49:58 +08:00
|
|
|
if test -n "${TEST_GIT_INDEX_VERSION:+isset}"
|
|
|
|
|
then
|
|
|
|
|
GIT_INDEX_VERSION="$TEST_GIT_INDEX_VERSION"
|
|
|
|
|
export GIT_INDEX_VERSION
|
|
|
|
|
fi
|
|
|
|
|
|
Add MALLOC_CHECK_ and MALLOC_PERTURB_ libc env to the test suite for detecting heap corruption
Recent versions of Linux libc (later than 5.4.23) and glibc (2.x)
include a malloc() implementation which is tunable via environment
variables. When MALLOC_CHECK_ is set, a special (less efficient)
implementation is used which is designed to be tolerant against
simple errors, such as double calls of free() with the same argument,
or overruns of a single byte (off-by-one bugs). When MALLOC_CHECK_
is set to 3, a diagnostic message is printed on stderr
and the program is aborted.
Setting the MALLOC_PERTURB_ environment variable causes the malloc
functions in libc to return memory which has been wiped and clear
memory when it is returned.
Of course this does not affect calloc which always does clear the memory.
The reason for this exercise is, of course, to find code which uses
memory returned by malloc without initializing it and code which uses
code after it is freed. valgrind can do this but it's costly to run.
The MALLOC_PERTURB_ exchanges the ability to detect problems in 100%
of the cases with speed.
The byte value used to initialize values returned by malloc is the byte
value of the environment value. The value used to clear memory is the
bitwise inverse. Setting MALLOC_PERTURB_ to zero disables the feature.
This technique can find hard to detect bugs.
It is therefore suggested to always use this flag (at least temporarily)
when testing out code or a new distribution.
But the test suite can use also valgrind(memcheck) via 'make valgrind'
or 'make GIT_TEST_OPTS="--valgrind"'.
Memcheck wraps client calls to malloc(), and puts a "red zone" on
each end of each block in order to detect access overruns.
Memcheck already detects double free() (up to the limit of the buffer
which remembers pending free()). Thus memcheck subsumes all the
documented coverage of MALLOC_CHECK_.
If MALLOC_CHECK_ is set non-zero when running memcheck, then the
overruns that might be detected by MALLOC_CHECK_ would be overruns
on the wrapped blocks which include the red zones. Thus MALLOC_CHECK_
would be checking memcheck, and not the client. This is not useful,
and actually is wasteful. The only possible [documented] advantage
of using MALLOC_CHECK_ and memcheck together, would be if MALLOC_CHECK_
detected duplicate free() in more cases than memcheck because memcheck's
buffer is too small.
Therefore we don't use MALLOC_CHECK_ and valgrind(memcheck) at the
same time.
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-09-15 00:54:22 +08:00
|
|
|
# Add libc MALLOC and MALLOC_PERTURB test
|
|
|
|
|
# only if we are not executing the test with valgrind
|
2012-09-15 11:38:24 +08:00
|
|
|
if expr " $GIT_TEST_OPTS " : ".* --valgrind " >/dev/null ||
|
2012-09-27 04:16:39 +08:00
|
|
|
test -n "$TEST_NO_MALLOC_CHECK"
|
2012-09-15 11:38:24 +08:00
|
|
|
then
|
|
|
|
|
setup_malloc_check () {
|
|
|
|
|
: nothing
|
|
|
|
|
}
|
|
|
|
|
teardown_malloc_check () {
|
|
|
|
|
: nothing
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
setup_malloc_check () {
|
|
|
|
|
MALLOC_CHECK_=3 MALLOC_PERTURB_=165
|
|
|
|
|
export MALLOC_CHECK_ MALLOC_PERTURB_
|
|
|
|
|
}
|
|
|
|
|
teardown_malloc_check () {
|
|
|
|
|
unset MALLOC_CHECK_ MALLOC_PERTURB_
|
|
|
|
|
}
|
|
|
|
|
fi
|
Add MALLOC_CHECK_ and MALLOC_PERTURB_ libc env to the test suite for detecting heap corruption
Recent versions of Linux libc (later than 5.4.23) and glibc (2.x)
include a malloc() implementation which is tunable via environment
variables. When MALLOC_CHECK_ is set, a special (less efficient)
implementation is used which is designed to be tolerant against
simple errors, such as double calls of free() with the same argument,
or overruns of a single byte (off-by-one bugs). When MALLOC_CHECK_
is set to 3, a diagnostic message is printed on stderr
and the program is aborted.
Setting the MALLOC_PERTURB_ environment variable causes the malloc
functions in libc to return memory which has been wiped and clear
memory when it is returned.
Of course this does not affect calloc which always does clear the memory.
The reason for this exercise is, of course, to find code which uses
memory returned by malloc without initializing it and code which uses
code after it is freed. valgrind can do this but it's costly to run.
The MALLOC_PERTURB_ exchanges the ability to detect problems in 100%
of the cases with speed.
The byte value used to initialize values returned by malloc is the byte
value of the environment value. The value used to clear memory is the
bitwise inverse. Setting MALLOC_PERTURB_ to zero disables the feature.
This technique can find hard to detect bugs.
It is therefore suggested to always use this flag (at least temporarily)
when testing out code or a new distribution.
But the test suite can use also valgrind(memcheck) via 'make valgrind'
or 'make GIT_TEST_OPTS="--valgrind"'.
Memcheck wraps client calls to malloc(), and puts a "red zone" on
each end of each block in order to detect access overruns.
Memcheck already detects double free() (up to the limit of the buffer
which remembers pending free()). Thus memcheck subsumes all the
documented coverage of MALLOC_CHECK_.
If MALLOC_CHECK_ is set non-zero when running memcheck, then the
overruns that might be detected by MALLOC_CHECK_ would be overruns
on the wrapped blocks which include the red zones. Thus MALLOC_CHECK_
would be checking memcheck, and not the client. This is not useful,
and actually is wasteful. The only possible [documented] advantage
of using MALLOC_CHECK_ and memcheck together, would be if MALLOC_CHECK_
detected duplicate free() in more cases than memcheck because memcheck's
buffer is too small.
Therefore we don't use MALLOC_CHECK_ and valgrind(memcheck) at the
same time.
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-09-15 00:54:22 +08:00
|
|
|
|
t: support clang/gcc AddressSanitizer
When git is compiled with "-fsanitize=address" (using clang
or gcc >= 4.8), all invocations of git will check for buffer
overflows. This is similar to running with valgrind, except
that it is more thorough (because of the compiler support,
function-local buffers can be checked, too) and runs much
faster (making it much less painful to run the whole test
suite with the checks turned on).
Unlike valgrind, the magic happens at compile-time, so we
don't need the same infrastructure in the test suite that we
did to support --valgrind. But there are two things we can
help with:
1. On some platforms, the leak-detector is on by default,
and causes every invocation of "git init" (and thus
every test script) to fail. Since running git with
the leak detector is pointless, let's shut it off
automatically in the tests, unless the user has already
configured it.
2. When apache runs a CGI, it clears the environment of
unknown variables. This means that the $ASAN_OPTIONS
config doesn't make it to git-http-backend, and it
dies due to the leak detector. Let's mark the variable
as OK for apache to pass.
With these two changes, running
make CC=clang CFLAGS=-fsanitize=address test
works out of the box.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-12-08 15:47:06 +08:00
|
|
|
: ${ASAN_OPTIONS=detect_leaks=0}
|
|
|
|
|
export ASAN_OPTIONS
|
|
|
|
|
|
2007-04-25 02:21:47 +08:00
|
|
|
# Protect ourselves from common misconfiguration to export
|
|
|
|
|
# CDPATH into the environment
|
|
|
|
|
unset CDPATH
|
|
|
|
|
|
2009-11-19 00:15:19 +08:00
|
|
|
unset GREP_OPTIONS
|
2013-01-07 01:47:57 +08:00
|
|
|
unset UNZIP
|
2009-11-19 00:15:19 +08:00
|
|
|
|
2006-09-23 06:35:20 +08:00
|
|
|
case $(echo $GIT_TRACE |tr "[A-Z]" "[a-z]") in
|
2012-09-02 02:11:49 +08:00
|
|
|
1|2|true)
|
2015-03-13 12:50:56 +08:00
|
|
|
GIT_TRACE=4
|
2012-09-02 02:11:49 +08:00
|
|
|
;;
|
2006-09-03 00:23:48 +08:00
|
|
|
esac
|
|
|
|
|
|
2009-09-21 02:10:14 +08:00
|
|
|
# Convenience
|
|
|
|
|
#
|
|
|
|
|
# A regexp to match 5 and 40 hexdigits
|
|
|
|
|
_x05='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
|
|
|
|
|
_x40="$_x05$_x05$_x05$_x05$_x05$_x05$_x05$_x05"
|
|
|
|
|
|
2011-04-24 13:34:13 +08:00
|
|
|
# Zero SHA-1
|
|
|
|
|
_z40=0000000000000000000000000000000000000000
|
|
|
|
|
|
2016-07-16 13:06:24 +08:00
|
|
|
EMPTY_TREE=4b825dc642cb6eb9a060e54bf8d69288fbee4904
|
2016-07-16 13:06:25 +08:00
|
|
|
EMPTY_BLOB=e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
|
2016-07-16 13:06:24 +08:00
|
|
|
|
2011-08-09 02:51:00 +08:00
|
|
|
# Line feed
|
|
|
|
|
LF='
|
|
|
|
|
'
|
|
|
|
|
|
2014-12-16 07:15:20 +08:00
|
|
|
# UTF-8 ZERO WIDTH NON-JOINER, which HFS+ ignores
|
|
|
|
|
# when case-folding filenames
|
|
|
|
|
u200c=$(printf '\342\200\214')
|
|
|
|
|
|
2016-07-16 13:06:25 +08:00
|
|
|
export _x05 _x40 _z40 LF u200c EMPTY_TREE EMPTY_BLOB
|
2012-02-17 18:25:09 +08:00
|
|
|
|
2005-05-14 13:50:32 +08:00
|
|
|
# Each test should start with something like this, after copyright notices:
|
|
|
|
|
#
|
|
|
|
|
# test_description='Description of this test...
|
|
|
|
|
# This test checks if command xyzzy does the right thing...
|
|
|
|
|
# '
|
|
|
|
|
# . ./test-lib.sh
|
2015-06-18 05:11:21 +08:00
|
|
|
test "x$TERM" != "xdumb" && (
|
2015-06-18 03:06:25 +08:00
|
|
|
test -t 1 &&
|
|
|
|
|
tput bold >/dev/null 2>&1 &&
|
|
|
|
|
tput setaf 1 >/dev/null 2>&1 &&
|
|
|
|
|
tput sgr0 >/dev/null 2>&1
|
|
|
|
|
) &&
|
|
|
|
|
color=t
|
2005-05-14 13:50:32 +08:00
|
|
|
|
|
|
|
|
while test "$#" -ne 0
|
|
|
|
|
do
|
|
|
|
|
case "$1" in
|
|
|
|
|
-d|--d|--de|--deb|--debu|--debug)
|
|
|
|
|
debug=t; shift ;;
|
2005-05-14 15:24:27 +08:00
|
|
|
-i|--i|--im|--imm|--imme|--immed|--immedi|--immedia|--immediat|--immediate)
|
|
|
|
|
immediate=t; shift ;;
|
2008-06-17 09:29:02 +08:00
|
|
|
-l|--l|--lo|--lon|--long|--long-|--long-t|--long-te|--long-tes|--long-test|--long-tests)
|
2009-02-19 03:17:27 +08:00
|
|
|
GIT_TEST_LONG=t; export GIT_TEST_LONG; shift ;;
|
2014-04-30 17:50:44 +08:00
|
|
|
-r)
|
|
|
|
|
shift; test "$#" -ne 0 || {
|
|
|
|
|
echo 'error: -r requires an argument' >&2;
|
|
|
|
|
exit 1;
|
|
|
|
|
}
|
|
|
|
|
run_list=$1; shift ;;
|
|
|
|
|
--run=*)
|
2016-04-23 04:32:21 +08:00
|
|
|
run_list=${1#--*=}; shift ;;
|
2005-05-14 13:50:32 +08:00
|
|
|
-h|--h|--he|--hel|--help)
|
2007-11-10 22:17:25 +08:00
|
|
|
help=t; shift ;;
|
2005-05-14 13:50:32 +08:00
|
|
|
-v|--v|--ve|--ver|--verb|--verbo|--verbos|--verbose)
|
|
|
|
|
verbose=t; shift ;;
|
2013-06-24 02:12:56 +08:00
|
|
|
--verbose-only=*)
|
2016-04-23 04:32:21 +08:00
|
|
|
verbose_only=${1#--*=}
|
2013-06-24 02:12:56 +08:00
|
|
|
shift ;;
|
2007-10-25 04:03:39 +08:00
|
|
|
-q|--q|--qu|--qui|--quie|--quiet)
|
2010-08-01 00:40:05 +08:00
|
|
|
# Ignore --quiet under a TAP::Harness. Saying how many tests
|
|
|
|
|
# passed without the ok/not ok details is always an error.
|
|
|
|
|
test -z "$HARNESS_ACTIVE" && quiet=t; shift ;;
|
2009-12-03 13:14:06 +08:00
|
|
|
--with-dashes)
|
|
|
|
|
with_dashes=t; shift ;;
|
2007-10-25 04:03:38 +08:00
|
|
|
--no-color)
|
2008-02-28 03:28:45 +08:00
|
|
|
color=; shift ;;
|
2009-02-04 07:25:59 +08:00
|
|
|
--va|--val|--valg|--valgr|--valgri|--valgrin|--valgrind)
|
2013-03-31 16:00:16 +08:00
|
|
|
valgrind=memcheck
|
|
|
|
|
shift ;;
|
|
|
|
|
--valgrind=*)
|
2016-04-23 04:32:21 +08:00
|
|
|
valgrind=${1#--*=}
|
2013-03-31 16:00:16 +08:00
|
|
|
shift ;;
|
2013-06-24 02:12:57 +08:00
|
|
|
--valgrind-only=*)
|
2016-04-23 04:32:21 +08:00
|
|
|
valgrind_only=${1#--*=}
|
2013-06-24 02:12:57 +08:00
|
|
|
shift ;;
|
test-lib.sh: optionally output to test-results/$TEST.out, too
When tests are run in parallel and a few tests fail, it does not help
that the output of the terminal is totally confusing, as you rarely know
which test which line came from.
So introduce the option '--tee' which triggers that the output of the
tests will be written to t/test-results/$TEST.out in addition to the
terminal, where $TEST is the basename of the script.
Unfortunately, there seems to be no way to redirect a given file
descriptor to a specified subprocess in POSIX shell, only redirection
to a file is supported via 'exec > $FILE'.
At least with bash, one might think that 'exec >($COMMAND)' would work
as intended, but it does not.
The common way to work around the lack of proper tools support is to
work with named pipes, alas, one of our most beloved platforms does not
really support named pipes. Besides, we would need a pipe for every
script, as the whole point of this patch is to allow parallel execution.
Therefore, we handle the redirection in the following way: when '--tee'
was passed to the test script, the variable GIT_TEST_TEE_STARTED is set
(to avoid triggering that code path again) and the script is started
_again_, in a subshell, redirected to the command "tee".
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-02-04 07:26:12 +08:00
|
|
|
--tee)
|
|
|
|
|
shift ;; # was handled already
|
2009-08-09 16:39:45 +08:00
|
|
|
--root=*)
|
2016-04-23 04:32:21 +08:00
|
|
|
root=${1#--*=}
|
2009-08-09 16:39:45 +08:00
|
|
|
shift ;;
|
t/test-lib: introduce --chain-lint option
It's easy to miss an "&&"-chain in a test script, like:
test_expect_success 'check something important' '
cmd1 &&
cmd2
cmd3
'
The test harness will notice if cmd3 fails, but a failure of
cmd1 or cmd2 will go unnoticed, as their exit status is lost
after cmd3 runs.
The toy example above is easy to spot because the "cmds" are
all the same length, but real code is much more complicated.
It's also difficult to detect these situations by statically
analyzing the shell code with regexps (like the
check-non-portable-shell script does); there's too much
context required to know whether a &&-chain is appropriate
on a given line or not.
This patch instead lets the shell check each test by
sticking a command with a specific and unusual return code
at the top of each test, like:
(exit 117) &&
cmd1 &&
cmd2
cmd3
In a well-formed test, the non-zero exit from the first
command prevents any of the rest from being run, and the
test's exit code is 117. In a bad test (like the one above),
the 117 is lost, and cmd3 is run.
When we encounter a failure of this check, we abort the test
script entirely. For one thing, we have no clue which subset
of the commands in the test snippet were actually run.
Running further tests would be pointless, because we're now
in an unknown state. And two, this is not a "test failure"
in the traditional sense. The test script is buggy, not the
code it is testing. We should be able to fix these problems
in the script once, and not have them come back later as a
regression in git's code.
After checking a test snippet for --chain-lint, we do still
run the test itself. We could actually have a pure-lint
mode which just checks each test, but there are a few
reasons not to. One, because the tests are executing
arbitrary code, which could impact the later environment
(e.g., that could impact which set of tests we run at all).
And two, because a pure-lint mode would still be expensive
to run, because a significant amount of code runs outside of
the test_expect_* blocks. Instead, this option is designed
to be used as part of a normal test suite run, where it adds
very little overhead.
Turning on this option detects quite a few problems in
existing tests, which will be fixed in subsequent patches.
However, there are a number of places it cannot reach:
- it cannot find a failure to break out of loops on error,
like:
cmd1 &&
for i in a b c; do
cmd2 $i
done &&
cmd3
which will not notice failures of "cmd2 a" or "cmd b"
- it cannot find a missing &&-chain inside a block or
subfunction, like:
foo () {
cmd1
cmd2
}
foo &&
bar
which will not notice a failure of cmd1.
- it only checks tests that you run; every platform will
have some tests skipped due to missing prequisites,
so it's impossible to say from one run that the test
suite is free of broken &&-chains. However, all tests get
run by _somebody_, so eventually we will notice problems.
- it does not operate on test_when_finished or prerequisite
blocks. It could, but these tends to be much shorter and
less of a problem, so I punted on them in this patch.
This patch was inspired by an earlier patch by Jonathan
Nieder:
http://article.gmane.org/gmane.comp.version-control.git/235913
This implementation and all bugs are mine.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-20 18:05:48 +08:00
|
|
|
--chain-lint)
|
|
|
|
|
GIT_TEST_CHAIN_LINT=1
|
|
|
|
|
shift ;;
|
|
|
|
|
--no-chain-lint)
|
|
|
|
|
GIT_TEST_CHAIN_LINT=0
|
|
|
|
|
shift ;;
|
test-lib.sh: support -x option for shell-tracing
Usually running a test under "-v" makes it clear which
command is failing. However, sometimes it can be useful to
also see a complete trace of the shell commands being run in
the test. You can do so without any support from the test
suite by running "sh -x tXXXX-foo.sh". However, this
produces quite a large bit of output, as we see a trace of
the entire test suite.
This patch instead introduces a "-x" option to the test
scripts (i.e., "./tXXXX-foo.sh -x"). When enabled, this
turns on "set -x" only for the tests themselves. This can
still be a bit verbose, but should keep things to a more
manageable level. You can even use "--verbose-only" to see
the trace only for a specific test.
The implementation is a little invasive. We turn on the "set
-x" inside the "eval" of the test code. This lets the eval
itself avoid being reported in the trace (which would be
long, and redundant with the verbose listing we already
showed). And then after the eval runs, we do some trickery
with stderr to avoid showing the "set +x" to the user.
We also show traces for test_cleanup functions (since they
can impact the test outcome, too). However, we do avoid
running the noop ":" cleanup (the default if the test does
not use test_cleanup at all), as it creates unnecessary
noise in the "set -x" output.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-10 14:47:27 +08:00
|
|
|
-x)
|
|
|
|
|
trace=t
|
|
|
|
|
verbose=t
|
|
|
|
|
shift ;;
|
test-lib: add --verbose-log option
The "--verbose" option redirects output from arbitrary
test commands to stdout. This is useful for examining the
output manually, like:
./t5547-push-quarantine.sh -v | less
But it also means that the output is intermingled with the
TAP directives, which can confuse a TAP parser like "prove".
This has always been a potential problem, but became an
issue recently when one test happened to output the word
"ok" on a line by itself, which prove interprets as a test
success:
$ prove t5547-push-quarantine.sh :: -v
t5547-push-quarantine.sh .. 1/? To dest.git
* [new branch] HEAD -> master
To dest.git
! [remote rejected] reject -> reject (pre-receive hook declined)
error: failed to push some refs to 'dest.git'
fatal: git cat-file d08c8eba97f4e683ece08654c7c8d2ba0c03b129: bad file
t5547-push-quarantine.sh .. Failed -1/4 subtests
Test Summary Report
-------------------
t5547-push-quarantine.sh (Wstat: 0 Tests: 5 Failed: 0)
Parse errors: Tests out of sequence. Found (2) but expected (3)
Tests out of sequence. Found (3) but expected (4)
Tests out of sequence. Found (4) but expected (5)
Bad plan. You planned 4 tests but ran 5.
Files=1, Tests=5, 0 wallclock secs ( 0.01 usr + 0.01 sys = 0.02 CPU)
Result: FAIL
One answer is "if it hurts, don't do it", but that's not
quite the whole story. The Travis tests use "--verbose
--tee" so that they can get the benefit of prove's parallel
options, along with a verbose log in case there is a
failure. We just need the verbose output to go to the log,
but keep stdout clean.
Getting this right turns out to be surprisingly difficult.
Here's the progression of alternatives I considered:
1. Add an option to write verbose output to stderr. This is
hard to capture, though, because we want each test to
have its own log (because they're all run in parallel
and the jumbled output would be useless).
2. Add an option to write verbose output to a file in
test-results. This works, but the log is missing all of
the non-verbose output, which gives context.
3. Like (2), but teach say_color() to additionally output
to the log. This mostly works, but misses any output
that happens outside of the say() functions (which isn't
a lot, but is a potential maintenance headache).
4. Like (2), but make the log file the same as the "--tee"
file. That almost works, but now we have two processes
opening the same file. That gives us two separate
descriptors, each with their own idea of the current
position. They'll each start writing at offset 0, and
overwrite each other's data.
5. Like (4), but in each case open the file for appending.
That atomically positions each write at the end of the
file.
It's possible we may still get sheared writes between
the two processes, but this is already the case when
writing to stdout. It's not a problem in practice
because the test harness generally waits for snippets to
finish before writing the TAP output.
We can ignore buffering issues with tee, because POSIX
mandates that it does not buffer. Likewise, POSIX
specifies "tee -a", so it should be available
everywhere.
This patch implements option (5), which seems to work well
in practice.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-21 18:48:00 +08:00
|
|
|
--verbose-log)
|
|
|
|
|
verbose_log=t
|
|
|
|
|
shift ;;
|
2005-05-14 13:50:32 +08:00
|
|
|
*)
|
2009-06-01 20:14:40 +08:00
|
|
|
echo "error: unknown test option '$1'" >&2; exit 1 ;;
|
2005-05-14 13:50:32 +08:00
|
|
|
esac
|
|
|
|
|
done
|
|
|
|
|
|
2013-10-20 05:06:07 +08:00
|
|
|
if test -n "$valgrind_only"
|
2013-06-24 02:12:57 +08:00
|
|
|
then
|
|
|
|
|
test -z "$valgrind" && valgrind=memcheck
|
|
|
|
|
test -z "$verbose" && verbose_only="$valgrind_only"
|
|
|
|
|
elif test -n "$valgrind"
|
|
|
|
|
then
|
|
|
|
|
verbose=t
|
|
|
|
|
fi
|
2013-03-31 16:00:16 +08:00
|
|
|
|
2015-06-18 03:06:25 +08:00
|
|
|
if test -n "$color"
|
|
|
|
|
then
|
2015-06-18 05:11:21 +08:00
|
|
|
# Save the color control sequences now rather than run tput
|
|
|
|
|
# each time say_color() is called. This is done for two
|
|
|
|
|
# reasons:
|
|
|
|
|
# * TERM will be changed to dumb
|
|
|
|
|
# * HOME will be changed to a temporary directory and tput
|
|
|
|
|
# might need to read ~/.terminfo from the original HOME
|
|
|
|
|
# directory to get the control sequences
|
|
|
|
|
# Note: This approach assumes the control sequences don't end
|
|
|
|
|
# in a newline for any terminal of interest (command
|
|
|
|
|
# substitutions strip trailing newlines). Given that most
|
|
|
|
|
# (all?) terminals in common use are related to ECMA-48, this
|
|
|
|
|
# shouldn't be a problem.
|
|
|
|
|
say_color_error=$(tput bold; tput setaf 1) # bold red
|
|
|
|
|
say_color_skip=$(tput setaf 4) # blue
|
|
|
|
|
say_color_warn=$(tput setaf 3) # brown/yellow
|
|
|
|
|
say_color_pass=$(tput setaf 2) # green
|
|
|
|
|
say_color_info=$(tput setaf 6) # cyan
|
|
|
|
|
say_color_reset=$(tput sgr0)
|
|
|
|
|
say_color_="" # no formatting for normal text
|
2015-06-18 03:06:25 +08:00
|
|
|
say_color () {
|
2015-06-18 05:11:21 +08:00
|
|
|
test -z "$1" && test -n "$quiet" && return
|
|
|
|
|
eval "say_color_color=\$say_color_$1"
|
2015-06-18 03:06:25 +08:00
|
|
|
shift
|
2015-06-18 05:11:21 +08:00
|
|
|
printf "%s\\n" "$say_color_color$*$say_color_reset"
|
2015-06-18 03:06:25 +08:00
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
say_color() {
|
|
|
|
|
test -z "$1" && test -n "$quiet" && return
|
|
|
|
|
shift
|
|
|
|
|
printf "%s\n" "$*"
|
|
|
|
|
}
|
|
|
|
|
fi
|
|
|
|
|
|
2015-06-18 05:11:21 +08:00
|
|
|
TERM=dumb
|
|
|
|
|
export TERM
|
|
|
|
|
|
2007-10-25 04:03:38 +08:00
|
|
|
error () {
|
|
|
|
|
say_color error "error: $*"
|
2009-06-01 20:14:41 +08:00
|
|
|
GIT_EXIT_OK=t
|
2007-10-25 04:03:38 +08:00
|
|
|
exit 1
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
say () {
|
|
|
|
|
say_color info "$*"
|
|
|
|
|
}
|
|
|
|
|
|
test-lib: bail out when "-v" used under "prove"
When there is a TAP harness consuming the output of our test
scripts, the "--verbose" breaks the output by mingling
test command output with TAP. Because the TAP::Harness
module used by "prove" is fairly lenient, this _usually_
works, but it violates the spec, and things get very
confusing if the commands happen to output a line that looks
like TAP (e.g., the word "ok" on its own line).
Let's detect this situation and complain. Just calling
error() isn't great, though; prove will tell us that the
script failed, but the message doesn't make it through to
the user. Instead, we can use the special TAP signal "Bail
out!". This not only shows the message to the user, but
instructs the harness to stop running the tests entirely.
This is exactly what we want here, as the problem is in the
command-line options, and every test script would produce
the same error.
The result looks like this (the first "Bailout called" line
is in red if prove uses color on your terminal):
$ make GIT_TEST_OPTS='--verbose --tee'
rm -f -r 'test-results'
*** prove ***
Bailout called. Further testing stopped: verbose mode forbidden under TAP harness; try --verbose-log
FAILED--Further testing stopped: verbose mode forbidden under TAP harness; try --verbose-log
Makefile:39: recipe for target 'prove' failed
make: *** [prove] Error 255
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-22 12:45:06 +08:00
|
|
|
if test -n "$HARNESS_ACTIVE"
|
|
|
|
|
then
|
|
|
|
|
if test "$verbose" = t || test -n "$verbose_only"
|
|
|
|
|
then
|
|
|
|
|
printf 'Bail out! %s\n' \
|
|
|
|
|
'verbose mode forbidden under TAP harness; try --verbose-log'
|
|
|
|
|
exit 1
|
|
|
|
|
fi
|
|
|
|
|
fi
|
|
|
|
|
|
2007-11-10 22:17:25 +08:00
|
|
|
test "${test_description}" != "" ||
|
|
|
|
|
error "Test script did not set test_description."
|
|
|
|
|
|
|
|
|
|
if test "$help" = "t"
|
|
|
|
|
then
|
2014-03-18 08:14:11 +08:00
|
|
|
printf '%s\n' "$test_description"
|
2007-11-10 22:17:25 +08:00
|
|
|
exit 0
|
|
|
|
|
fi
|
|
|
|
|
|
2005-08-11 11:56:21 +08:00
|
|
|
exec 5>&1
|
test-lib: redirect stdin of tests
We want to run tests in a predictable, sterile environment
so we can get repeatable results. They should take as
little input as possible from the environment outside the
test script. We already sanitize environment variables, but
leave stdin untouched. This means that scripts can
accidentally be impacted by content on stdin, or whether
stdin isatty().
Furthermore, scripts reading from stdin can be annoying to
outer loops which care about their stdin offset, like:
while read sha1; do
make test
done
A test which accidentally reads stdin would soak up all of
the rest of the input intended for the outer shell loop.
Let's redirect stdin from /dev/null, which solves both
of these problems. It won't detect tests accidentally
reading from stdin, but since doing so now gives a
deterministic result, we don't need to consider that an
error.
We'll also leave file descriptor 6 as a link to the original
stdin. Tests shouldn't need to look at this, but it can be
convenient for inserting interactive commands while
debugging tests (e.g., you could insert "bash <&6 >&3 2>&4"
to run interactive commands in the environment of the test
script).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-12-15 14:55:29 +08:00
|
|
|
exec 6<&0
|
tests: create an interactive gdb session with the 'debug' helper
The 'debug' test helper is supposed to facilitate debugging by running
a command of the test suite under gdb. Unfortunately, its usefulness
is severely limited, because that gdb session is not interactive,
since the test's, and thus gdb's standard input is redirected from
/dev/null (for a good reason, see 781f76b15 (test-lib: redirect stdin
of tests, 2011-12-15)).
Redirect gdb's standard file descriptors from/to the test
environment's stdin, stdout and stderr in the 'debug' helper, thus
creating an interactive gdb session (even in non-verbose mode), which
is much, much more useful.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-19 00:13:59 +08:00
|
|
|
exec 7>&2
|
test-lib: add --verbose-log option
The "--verbose" option redirects output from arbitrary
test commands to stdout. This is useful for examining the
output manually, like:
./t5547-push-quarantine.sh -v | less
But it also means that the output is intermingled with the
TAP directives, which can confuse a TAP parser like "prove".
This has always been a potential problem, but became an
issue recently when one test happened to output the word
"ok" on a line by itself, which prove interprets as a test
success:
$ prove t5547-push-quarantine.sh :: -v
t5547-push-quarantine.sh .. 1/? To dest.git
* [new branch] HEAD -> master
To dest.git
! [remote rejected] reject -> reject (pre-receive hook declined)
error: failed to push some refs to 'dest.git'
fatal: git cat-file d08c8eba97f4e683ece08654c7c8d2ba0c03b129: bad file
t5547-push-quarantine.sh .. Failed -1/4 subtests
Test Summary Report
-------------------
t5547-push-quarantine.sh (Wstat: 0 Tests: 5 Failed: 0)
Parse errors: Tests out of sequence. Found (2) but expected (3)
Tests out of sequence. Found (3) but expected (4)
Tests out of sequence. Found (4) but expected (5)
Bad plan. You planned 4 tests but ran 5.
Files=1, Tests=5, 0 wallclock secs ( 0.01 usr + 0.01 sys = 0.02 CPU)
Result: FAIL
One answer is "if it hurts, don't do it", but that's not
quite the whole story. The Travis tests use "--verbose
--tee" so that they can get the benefit of prove's parallel
options, along with a verbose log in case there is a
failure. We just need the verbose output to go to the log,
but keep stdout clean.
Getting this right turns out to be surprisingly difficult.
Here's the progression of alternatives I considered:
1. Add an option to write verbose output to stderr. This is
hard to capture, though, because we want each test to
have its own log (because they're all run in parallel
and the jumbled output would be useless).
2. Add an option to write verbose output to a file in
test-results. This works, but the log is missing all of
the non-verbose output, which gives context.
3. Like (2), but teach say_color() to additionally output
to the log. This mostly works, but misses any output
that happens outside of the say() functions (which isn't
a lot, but is a potential maintenance headache).
4. Like (2), but make the log file the same as the "--tee"
file. That almost works, but now we have two processes
opening the same file. That gives us two separate
descriptors, each with their own idea of the current
position. They'll each start writing at offset 0, and
overwrite each other's data.
5. Like (4), but in each case open the file for appending.
That atomically positions each write at the end of the
file.
It's possible we may still get sheared writes between
the two processes, but this is already the case when
writing to stdout. It's not a problem in practice
because the test harness generally waits for snippets to
finish before writing the TAP output.
We can ignore buffering issues with tee, because POSIX
mandates that it does not buffer. Likewise, POSIX
specifies "tee -a", so it should be available
everywhere.
This patch implements option (5), which seems to work well
in practice.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-21 18:48:00 +08:00
|
|
|
if test "$verbose_log" = "t"
|
|
|
|
|
then
|
|
|
|
|
exec 3>>"$GIT_TEST_TEE_OUTPUT_FILE" 4>&3
|
|
|
|
|
elif test "$verbose" = "t"
|
2005-05-14 13:50:32 +08:00
|
|
|
then
|
|
|
|
|
exec 4>&2 3>&1
|
|
|
|
|
else
|
|
|
|
|
exec 4>/dev/null 3>/dev/null
|
|
|
|
|
fi
|
|
|
|
|
|
test-lib: set BASH_XTRACEFD automatically
Passing "-x" to a test script enables the shell's "set -x"
tracing, which can help with tracking down the command that
is causing a failure. Unfortunately, it can also _cause_
failures in some tests that redirect the stderr of a shell
function. Inside the function the shell continues to
respect "set -x", and the trace output is collected along
with whatever stderr is generated normally by the function.
You can see an example of this by running:
./t0040-parse-options.sh -x -i
which will fail immediately in the first test, as it
expects:
test_must_fail some-cmd 2>output.err
to leave output.err empty (but with "-x" it has our trace
output).
Unfortunately there isn't a portable or scalable solution to
this. We could teach test_must_fail to disable "set -x", but
that doesn't help any of the other functions or subshells.
However, we can work around it by pointing the "set -x"
output to our descriptor 4, which always points to the
original stderr of the test script. Unfortunately this only
works for bash, but it's better than nothing (and other
shells will just ignore the BASH_XTRACEFD variable).
The patch itself is a simple one-liner, but note the caveats
in the accompanying comments.
Automatic tests for our "-x" option may be a bit too meta
(and a pain, because they are bash-specific), but I did
confirm that it works correctly both with regular "-x" and
with "--verbose-only=1". This works because the latter flips
"set -x" off and on for particular tests (if it didn't, we
would get tracing for all tests, as going to descriptor 4
effectively circumvents the verbose flag).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-11 21:44:04 +08:00
|
|
|
# Send any "-x" output directly to stderr to avoid polluting tests
|
|
|
|
|
# which capture stderr. We can do this unconditionally since it
|
|
|
|
|
# has no effect if tracing isn't turned on.
|
|
|
|
|
#
|
|
|
|
|
# Note that this sets up the trace fd as soon as we assign the variable, so it
|
|
|
|
|
# must come after the creation of descriptor 4 above. Likewise, we must never
|
|
|
|
|
# unset this, as it has the side effect of closing descriptor 4, which we
|
|
|
|
|
# use to show verbose tests to the user.
|
|
|
|
|
#
|
|
|
|
|
# Note also that we don't need or want to export it. The tracing is local to
|
|
|
|
|
# this shell, and we would not want to influence any shells we exec.
|
|
|
|
|
BASH_XTRACEFD=4
|
|
|
|
|
|
2005-05-14 13:50:32 +08:00
|
|
|
test_failure=0
|
|
|
|
|
test_count=0
|
2008-02-01 17:50:53 +08:00
|
|
|
test_fixed=0
|
|
|
|
|
test_broken=0
|
2008-06-08 22:04:33 +08:00
|
|
|
test_success=0
|
2005-05-14 13:50:32 +08:00
|
|
|
|
2010-06-25 01:44:46 +08:00
|
|
|
test_external_has_tap=0
|
|
|
|
|
|
2008-02-28 03:28:45 +08:00
|
|
|
die () {
|
2009-06-01 20:14:41 +08:00
|
|
|
code=$?
|
|
|
|
|
if test -n "$GIT_EXIT_OK"
|
|
|
|
|
then
|
|
|
|
|
exit $code
|
|
|
|
|
else
|
|
|
|
|
echo >&5 "FATAL: Unexpected exit with code $code"
|
|
|
|
|
exit 1
|
|
|
|
|
fi
|
2008-02-28 03:28:45 +08:00
|
|
|
}
|
|
|
|
|
|
2009-06-01 20:14:41 +08:00
|
|
|
GIT_EXIT_OK=
|
2009-01-20 07:43:26 +08:00
|
|
|
trap 'die' EXIT
|
2015-03-13 12:48:48 +08:00
|
|
|
trap 'exit $?' INT
|
2005-08-12 00:00:40 +08:00
|
|
|
|
2012-02-17 18:25:08 +08:00
|
|
|
# The user-facing functions are loaded from a separate file so that
|
|
|
|
|
# test_perf subshells can have them too
|
2012-06-25 13:01:35 +08:00
|
|
|
. "$TEST_DIRECTORY/test-lib-functions.sh"
|
2010-10-17 02:36:58 +08:00
|
|
|
|
2005-05-14 15:24:27 +08:00
|
|
|
# You are not expected to call test_ok_ and test_failure_ directly, use
|
2013-10-27 17:56:33 +08:00
|
|
|
# the test_expect_* functions instead.
|
2005-05-14 15:24:27 +08:00
|
|
|
|
|
|
|
|
test_ok_ () {
|
2009-02-06 03:59:27 +08:00
|
|
|
test_success=$(($test_success + 1))
|
2013-10-20 05:06:08 +08:00
|
|
|
say_color "" "ok $test_count - $@"
|
2005-05-14 13:50:32 +08:00
|
|
|
}
|
|
|
|
|
|
2005-05-14 15:24:27 +08:00
|
|
|
test_failure_ () {
|
2009-02-06 03:59:27 +08:00
|
|
|
test_failure=$(($test_failure + 1))
|
2013-10-20 05:06:08 +08:00
|
|
|
say_color error "not ok $test_count - $1"
|
2005-07-23 10:09:34 +08:00
|
|
|
shift
|
2014-03-18 08:14:11 +08:00
|
|
|
printf '%s\n' "$*" | sed -e 's/^/# /'
|
2009-06-01 20:14:41 +08:00
|
|
|
test "$immediate" = "" || { GIT_EXIT_OK=t; exit 1; }
|
2005-05-14 15:24:27 +08:00
|
|
|
}
|
|
|
|
|
|
2008-02-01 17:50:53 +08:00
|
|
|
test_known_broken_ok_ () {
|
|
|
|
|
test_fixed=$(($test_fixed+1))
|
2013-10-20 05:06:08 +08:00
|
|
|
say_color error "ok $test_count - $@ # TODO known breakage vanished"
|
2008-02-01 17:50:53 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
test_known_broken_failure_ () {
|
|
|
|
|
test_broken=$(($test_broken+1))
|
2013-10-20 05:06:08 +08:00
|
|
|
say_color warn "not ok $test_count - $@ # TODO known breakage"
|
2008-02-01 17:50:53 +08:00
|
|
|
}
|
2005-05-14 15:24:27 +08:00
|
|
|
|
|
|
|
|
test_debug () {
|
2005-08-11 13:53:27 +08:00
|
|
|
test "$debug" = "" || eval "$1"
|
2005-05-14 13:50:32 +08:00
|
|
|
}
|
|
|
|
|
|
2013-06-18 20:25:58 +08:00
|
|
|
match_pattern_list () {
|
|
|
|
|
arg="$1"
|
|
|
|
|
shift
|
|
|
|
|
test -z "$*" && return 1
|
|
|
|
|
for pattern_
|
|
|
|
|
do
|
|
|
|
|
case "$arg" in
|
|
|
|
|
$pattern_)
|
|
|
|
|
return 0
|
|
|
|
|
esac
|
|
|
|
|
done
|
|
|
|
|
return 1
|
|
|
|
|
}
|
|
|
|
|
|
2014-04-30 17:50:44 +08:00
|
|
|
match_test_selector_list () {
|
|
|
|
|
title="$1"
|
|
|
|
|
shift
|
|
|
|
|
arg="$1"
|
|
|
|
|
shift
|
|
|
|
|
test -z "$1" && return 0
|
|
|
|
|
|
|
|
|
|
# Both commas and whitespace are accepted as separators.
|
|
|
|
|
OLDIFS=$IFS
|
|
|
|
|
IFS=' ,'
|
|
|
|
|
set -- $1
|
|
|
|
|
IFS=$OLDIFS
|
|
|
|
|
|
|
|
|
|
# If the first selector is negative we include by default.
|
|
|
|
|
include=
|
|
|
|
|
case "$1" in
|
|
|
|
|
!*) include=t ;;
|
|
|
|
|
esac
|
|
|
|
|
|
|
|
|
|
for selector
|
|
|
|
|
do
|
|
|
|
|
orig_selector=$selector
|
|
|
|
|
|
|
|
|
|
positive=t
|
|
|
|
|
case "$selector" in
|
|
|
|
|
!*)
|
|
|
|
|
positive=
|
|
|
|
|
selector=${selector##?}
|
|
|
|
|
;;
|
|
|
|
|
esac
|
|
|
|
|
|
|
|
|
|
test -z "$selector" && continue
|
|
|
|
|
|
|
|
|
|
case "$selector" in
|
|
|
|
|
*-*)
|
|
|
|
|
if expr "z${selector%%-*}" : "z[0-9]*[^0-9]" >/dev/null
|
|
|
|
|
then
|
|
|
|
|
echo "error: $title: invalid non-numeric in range" \
|
|
|
|
|
"start: '$orig_selector'" >&2
|
|
|
|
|
exit 1
|
|
|
|
|
fi
|
|
|
|
|
if expr "z${selector#*-}" : "z[0-9]*[^0-9]" >/dev/null
|
|
|
|
|
then
|
|
|
|
|
echo "error: $title: invalid non-numeric in range" \
|
|
|
|
|
"end: '$orig_selector'" >&2
|
|
|
|
|
exit 1
|
|
|
|
|
fi
|
|
|
|
|
;;
|
|
|
|
|
*)
|
|
|
|
|
if expr "z$selector" : "z[0-9]*[^0-9]" >/dev/null
|
|
|
|
|
then
|
|
|
|
|
echo "error: $title: invalid non-numeric in test" \
|
|
|
|
|
"selector: '$orig_selector'" >&2
|
|
|
|
|
exit 1
|
|
|
|
|
fi
|
|
|
|
|
esac
|
|
|
|
|
|
|
|
|
|
# Short cut for "obvious" cases
|
|
|
|
|
test -z "$include" && test -z "$positive" && continue
|
|
|
|
|
test -n "$include" && test -n "$positive" && continue
|
|
|
|
|
|
|
|
|
|
case "$selector" in
|
|
|
|
|
-*)
|
|
|
|
|
if test $arg -le ${selector#-}
|
|
|
|
|
then
|
|
|
|
|
include=$positive
|
|
|
|
|
fi
|
|
|
|
|
;;
|
|
|
|
|
*-)
|
|
|
|
|
if test $arg -ge ${selector%-}
|
|
|
|
|
then
|
|
|
|
|
include=$positive
|
|
|
|
|
fi
|
|
|
|
|
;;
|
|
|
|
|
*-*)
|
|
|
|
|
if test ${selector%%-*} -le $arg \
|
|
|
|
|
&& test $arg -le ${selector#*-}
|
|
|
|
|
then
|
|
|
|
|
include=$positive
|
|
|
|
|
fi
|
|
|
|
|
;;
|
|
|
|
|
*)
|
|
|
|
|
if test $arg -eq $selector
|
|
|
|
|
then
|
|
|
|
|
include=$positive
|
|
|
|
|
fi
|
|
|
|
|
;;
|
|
|
|
|
esac
|
|
|
|
|
done
|
|
|
|
|
|
|
|
|
|
test -n "$include"
|
|
|
|
|
}
|
|
|
|
|
|
2013-06-24 02:12:56 +08:00
|
|
|
maybe_teardown_verbose () {
|
|
|
|
|
test -z "$verbose_only" && return
|
|
|
|
|
exec 4>/dev/null 3>/dev/null
|
|
|
|
|
verbose=
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
last_verbose=t
|
|
|
|
|
maybe_setup_verbose () {
|
|
|
|
|
test -z "$verbose_only" && return
|
2013-10-20 05:06:07 +08:00
|
|
|
if match_pattern_list $test_count $verbose_only
|
2013-06-24 02:12:56 +08:00
|
|
|
then
|
|
|
|
|
exec 4>&2 3>&1
|
|
|
|
|
# Emit a delimiting blank line when going from
|
|
|
|
|
# non-verbose to verbose. Within verbose mode the
|
|
|
|
|
# delimiter is printed by test_expect_*. The choice
|
|
|
|
|
# of the initial $last_verbose is such that before
|
|
|
|
|
# test 1, we do not print it.
|
|
|
|
|
test -z "$last_verbose" && echo >&3 ""
|
|
|
|
|
verbose=t
|
|
|
|
|
else
|
|
|
|
|
exec 4>/dev/null 3>/dev/null
|
|
|
|
|
verbose=
|
|
|
|
|
fi
|
|
|
|
|
last_verbose=$verbose
|
|
|
|
|
}
|
|
|
|
|
|
2013-06-24 02:12:57 +08:00
|
|
|
maybe_teardown_valgrind () {
|
|
|
|
|
test -z "$GIT_VALGRIND" && return
|
|
|
|
|
GIT_VALGRIND_ENABLED=
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
maybe_setup_valgrind () {
|
|
|
|
|
test -z "$GIT_VALGRIND" && return
|
2013-10-20 05:06:07 +08:00
|
|
|
if test -z "$valgrind_only"
|
2013-06-24 02:12:57 +08:00
|
|
|
then
|
|
|
|
|
GIT_VALGRIND_ENABLED=t
|
|
|
|
|
return
|
|
|
|
|
fi
|
|
|
|
|
GIT_VALGRIND_ENABLED=
|
|
|
|
|
if match_pattern_list $test_count $valgrind_only
|
|
|
|
|
then
|
|
|
|
|
GIT_VALGRIND_ENABLED=t
|
|
|
|
|
fi
|
|
|
|
|
}
|
|
|
|
|
|
test-lib: disable trace when test is not verbose
The "-x" test-script option turns on the shell's "-x"
tracing, which can help show why a particular test is
failing. Unfortunately, this can create false negatives in
some tests if they invoke a shell function with its stderr
redirected. t5512.10 is such a test, as it does:
test_must_fail git ls-remote refs*master >actual 2>&1 &&
test_cmp exp actual
The "actual" file gets the "-x" trace for the test_must_fail
function, which prevents it from matching the expected
output.
There's no way to avoid this without managing the
trace flag inside each sub-function, which isn't really a
workable solution. But unless you specifically care about
t5512.10, we can work around it by enabling tracing only for
the specific tests we want.
You can already do:
./t5512-ls-remote.sh -x --verbose-only=16
to see the trace only for a specific test. But that doesn't
_disable_ the tracing in the other tests; it just sends it
to /dev/null. However, there's no point in generating a
trace that the user won't see, so we can simply disable
tracing whenever it doesn't have a matching verbose flag.
The normal case of just "./t5512-ls-remote.sh -x" stays the
same, as "-x" already implies "--verbose" (and
"--verbose-only" overrides "--verbose", which is why this
works at all). And for our test, we need only check
$verbose, as maybe_setup_verbose will have already
set that flag based on the $verbose_only list).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-06 13:33:57 +08:00
|
|
|
want_trace () {
|
|
|
|
|
test "$trace" = t && test "$verbose" = t
|
|
|
|
|
}
|
|
|
|
|
|
test-lib.sh: support -x option for shell-tracing
Usually running a test under "-v" makes it clear which
command is failing. However, sometimes it can be useful to
also see a complete trace of the shell commands being run in
the test. You can do so without any support from the test
suite by running "sh -x tXXXX-foo.sh". However, this
produces quite a large bit of output, as we see a trace of
the entire test suite.
This patch instead introduces a "-x" option to the test
scripts (i.e., "./tXXXX-foo.sh -x"). When enabled, this
turns on "set -x" only for the tests themselves. This can
still be a bit verbose, but should keep things to a more
manageable level. You can even use "--verbose-only" to see
the trace only for a specific test.
The implementation is a little invasive. We turn on the "set
-x" inside the "eval" of the test code. This lets the eval
itself avoid being reported in the trace (which would be
long, and redundant with the verbose listing we already
showed). And then after the eval runs, we do some trickery
with stderr to avoid showing the "set +x" to the user.
We also show traces for test_cleanup functions (since they
can impact the test outcome, too). However, we do avoid
running the noop ":" cleanup (the default if the test does
not use test_cleanup at all), as it creates unnecessary
noise in the "set -x" output.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-10 14:47:27 +08:00
|
|
|
# This is a separate function because some tests use
|
|
|
|
|
# "return" to end a test_expect_success block early
|
|
|
|
|
# (and we want to make sure we run any cleanup like
|
|
|
|
|
# "set +x").
|
|
|
|
|
test_eval_inner_ () {
|
|
|
|
|
# Do not add anything extra (including LF) after '$*'
|
|
|
|
|
eval "
|
test-lib: disable trace when test is not verbose
The "-x" test-script option turns on the shell's "-x"
tracing, which can help show why a particular test is
failing. Unfortunately, this can create false negatives in
some tests if they invoke a shell function with its stderr
redirected. t5512.10 is such a test, as it does:
test_must_fail git ls-remote refs*master >actual 2>&1 &&
test_cmp exp actual
The "actual" file gets the "-x" trace for the test_must_fail
function, which prevents it from matching the expected
output.
There's no way to avoid this without managing the
trace flag inside each sub-function, which isn't really a
workable solution. But unless you specifically care about
t5512.10, we can work around it by enabling tracing only for
the specific tests we want.
You can already do:
./t5512-ls-remote.sh -x --verbose-only=16
to see the trace only for a specific test. But that doesn't
_disable_ the tracing in the other tests; it just sends it
to /dev/null. However, there's no point in generating a
trace that the user won't see, so we can simply disable
tracing whenever it doesn't have a matching verbose flag.
The normal case of just "./t5512-ls-remote.sh -x" stays the
same, as "-x" already implies "--verbose" (and
"--verbose-only" overrides "--verbose", which is why this
works at all). And for our test, we need only check
$verbose, as maybe_setup_verbose will have already
set that flag based on the $verbose_only list).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-06 13:33:57 +08:00
|
|
|
want_trace && set -x
|
test-lib.sh: support -x option for shell-tracing
Usually running a test under "-v" makes it clear which
command is failing. However, sometimes it can be useful to
also see a complete trace of the shell commands being run in
the test. You can do so without any support from the test
suite by running "sh -x tXXXX-foo.sh". However, this
produces quite a large bit of output, as we see a trace of
the entire test suite.
This patch instead introduces a "-x" option to the test
scripts (i.e., "./tXXXX-foo.sh -x"). When enabled, this
turns on "set -x" only for the tests themselves. This can
still be a bit verbose, but should keep things to a more
manageable level. You can even use "--verbose-only" to see
the trace only for a specific test.
The implementation is a little invasive. We turn on the "set
-x" inside the "eval" of the test code. This lets the eval
itself avoid being reported in the trace (which would be
long, and redundant with the verbose listing we already
showed). And then after the eval runs, we do some trickery
with stderr to avoid showing the "set +x" to the user.
We also show traces for test_cleanup functions (since they
can impact the test outcome, too). However, we do avoid
running the noop ":" cleanup (the default if the test does
not use test_cleanup at all), as it creates unnecessary
noise in the "set -x" output.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-10 14:47:27 +08:00
|
|
|
$*"
|
|
|
|
|
}
|
|
|
|
|
|
test: cope better with use of return for errors
In olden times, tests would quietly exit the script when they failed
at an inconvenient moment, which was a little disconcerting.
Therefore v0.99.5~24^2~4 (Trapping exit in tests, using return for
errors, 2005-08-10) switched to an idiom of using "return" instead,
wrapping evaluation of test code in a function to make that safe:
test_run_ () {
eval >&3 2>&4 "$1"
eval_ret="$?"
return 0
}
Years later, the implementation of test_when_finished (v1.7.1.1~95,
2010-05-02) and v1.7.2-rc2~1^2~13 (test-lib: output a newline before
"ok" under a TAP harness, 2010-06-24) took advantage of test_run_ as a
place to put code shared by all test assertion functions, without
paying attention to the function's former purpose:
test_run_ () {
...
eval >&3 2>&4 "$1"
eval_ret=$?
if should run cleanup
then
eval >&3 2>&4 "$test_cleanup"
fi
if TAP format requires a newline here
then
echo
fi
return 0
}
That means cleanup commands and the newline to put TAP output at
column 0 are skipped when tests use "return" to fail early. Fix it by
introducing a test_eval_ function to catch the "return", with a
comment explaining the new function's purpose for the next person who
might touch this code.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-08-08 09:17:09 +08:00
|
|
|
test_eval_ () {
|
test-lib.sh: support -x option for shell-tracing
Usually running a test under "-v" makes it clear which
command is failing. However, sometimes it can be useful to
also see a complete trace of the shell commands being run in
the test. You can do so without any support from the test
suite by running "sh -x tXXXX-foo.sh". However, this
produces quite a large bit of output, as we see a trace of
the entire test suite.
This patch instead introduces a "-x" option to the test
scripts (i.e., "./tXXXX-foo.sh -x"). When enabled, this
turns on "set -x" only for the tests themselves. This can
still be a bit verbose, but should keep things to a more
manageable level. You can even use "--verbose-only" to see
the trace only for a specific test.
The implementation is a little invasive. We turn on the "set
-x" inside the "eval" of the test code. This lets the eval
itself avoid being reported in the trace (which would be
long, and redundant with the verbose listing we already
showed). And then after the eval runs, we do some trickery
with stderr to avoid showing the "set +x" to the user.
We also show traces for test_cleanup functions (since they
can impact the test outcome, too). However, we do avoid
running the noop ":" cleanup (the default if the test does
not use test_cleanup at all), as it creates unnecessary
noise in the "set -x" output.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-10 14:47:27 +08:00
|
|
|
# We run this block with stderr redirected to avoid extra cruft
|
|
|
|
|
# during a "-x" trace. Once in "set -x" mode, we cannot prevent
|
|
|
|
|
# the shell from printing the "set +x" to turn it off (nor the saving
|
|
|
|
|
# of $? before that). But we can make sure that the output goes to
|
|
|
|
|
# /dev/null.
|
|
|
|
|
#
|
|
|
|
|
# The test itself is run with stderr put back to &4 (so either to
|
|
|
|
|
# /dev/null, or to the original stderr if --verbose was used).
|
|
|
|
|
{
|
|
|
|
|
test_eval_inner_ "$@" </dev/null >&3 2>&4
|
|
|
|
|
test_eval_ret_=$?
|
test-lib: disable trace when test is not verbose
The "-x" test-script option turns on the shell's "-x"
tracing, which can help show why a particular test is
failing. Unfortunately, this can create false negatives in
some tests if they invoke a shell function with its stderr
redirected. t5512.10 is such a test, as it does:
test_must_fail git ls-remote refs*master >actual 2>&1 &&
test_cmp exp actual
The "actual" file gets the "-x" trace for the test_must_fail
function, which prevents it from matching the expected
output.
There's no way to avoid this without managing the
trace flag inside each sub-function, which isn't really a
workable solution. But unless you specifically care about
t5512.10, we can work around it by enabling tracing only for
the specific tests we want.
You can already do:
./t5512-ls-remote.sh -x --verbose-only=16
to see the trace only for a specific test. But that doesn't
_disable_ the tracing in the other tests; it just sends it
to /dev/null. However, there's no point in generating a
trace that the user won't see, so we can simply disable
tracing whenever it doesn't have a matching verbose flag.
The normal case of just "./t5512-ls-remote.sh -x" stays the
same, as "-x" already implies "--verbose" (and
"--verbose-only" overrides "--verbose", which is why this
works at all). And for our test, we need only check
$verbose, as maybe_setup_verbose will have already
set that flag based on the $verbose_only list).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-06 13:33:57 +08:00
|
|
|
if want_trace
|
test-lib.sh: support -x option for shell-tracing
Usually running a test under "-v" makes it clear which
command is failing. However, sometimes it can be useful to
also see a complete trace of the shell commands being run in
the test. You can do so without any support from the test
suite by running "sh -x tXXXX-foo.sh". However, this
produces quite a large bit of output, as we see a trace of
the entire test suite.
This patch instead introduces a "-x" option to the test
scripts (i.e., "./tXXXX-foo.sh -x"). When enabled, this
turns on "set -x" only for the tests themselves. This can
still be a bit verbose, but should keep things to a more
manageable level. You can even use "--verbose-only" to see
the trace only for a specific test.
The implementation is a little invasive. We turn on the "set
-x" inside the "eval" of the test code. This lets the eval
itself avoid being reported in the trace (which would be
long, and redundant with the verbose listing we already
showed). And then after the eval runs, we do some trickery
with stderr to avoid showing the "set +x" to the user.
We also show traces for test_cleanup functions (since they
can impact the test outcome, too). However, we do avoid
running the noop ":" cleanup (the default if the test does
not use test_cleanup at all), as it creates unnecessary
noise in the "set -x" output.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-10 14:47:27 +08:00
|
|
|
then
|
|
|
|
|
set +x
|
|
|
|
|
if test "$test_eval_ret_" != 0
|
|
|
|
|
then
|
|
|
|
|
say_color error >&4 "error: last command exited with \$?=$test_eval_ret_"
|
|
|
|
|
fi
|
|
|
|
|
fi
|
|
|
|
|
} 2>/dev/null
|
|
|
|
|
return $test_eval_ret_
|
test: cope better with use of return for errors
In olden times, tests would quietly exit the script when they failed
at an inconvenient moment, which was a little disconcerting.
Therefore v0.99.5~24^2~4 (Trapping exit in tests, using return for
errors, 2005-08-10) switched to an idiom of using "return" instead,
wrapping evaluation of test code in a function to make that safe:
test_run_ () {
eval >&3 2>&4 "$1"
eval_ret="$?"
return 0
}
Years later, the implementation of test_when_finished (v1.7.1.1~95,
2010-05-02) and v1.7.2-rc2~1^2~13 (test-lib: output a newline before
"ok" under a TAP harness, 2010-06-24) took advantage of test_run_ as a
place to put code shared by all test assertion functions, without
paying attention to the function's former purpose:
test_run_ () {
...
eval >&3 2>&4 "$1"
eval_ret=$?
if should run cleanup
then
eval >&3 2>&4 "$test_cleanup"
fi
if TAP format requires a newline here
then
echo
fi
return 0
}
That means cleanup commands and the newline to put TAP output at
column 0 are skipped when tests use "return" to fail early. Fix it by
introducing a test_eval_ function to catch the "return", with a
comment explaining the new function's purpose for the next person who
might touch this code.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-08-08 09:17:09 +08:00
|
|
|
}
|
|
|
|
|
|
2005-08-11 11:56:21 +08:00
|
|
|
test_run_ () {
|
2010-05-06 16:41:10 +08:00
|
|
|
test_cleanup=:
|
2011-06-28 02:02:22 +08:00
|
|
|
expecting_failure=$2
|
t/test-lib: introduce --chain-lint option
It's easy to miss an "&&"-chain in a test script, like:
test_expect_success 'check something important' '
cmd1 &&
cmd2
cmd3
'
The test harness will notice if cmd3 fails, but a failure of
cmd1 or cmd2 will go unnoticed, as their exit status is lost
after cmd3 runs.
The toy example above is easy to spot because the "cmds" are
all the same length, but real code is much more complicated.
It's also difficult to detect these situations by statically
analyzing the shell code with regexps (like the
check-non-portable-shell script does); there's too much
context required to know whether a &&-chain is appropriate
on a given line or not.
This patch instead lets the shell check each test by
sticking a command with a specific and unusual return code
at the top of each test, like:
(exit 117) &&
cmd1 &&
cmd2
cmd3
In a well-formed test, the non-zero exit from the first
command prevents any of the rest from being run, and the
test's exit code is 117. In a bad test (like the one above),
the 117 is lost, and cmd3 is run.
When we encounter a failure of this check, we abort the test
script entirely. For one thing, we have no clue which subset
of the commands in the test snippet were actually run.
Running further tests would be pointless, because we're now
in an unknown state. And two, this is not a "test failure"
in the traditional sense. The test script is buggy, not the
code it is testing. We should be able to fix these problems
in the script once, and not have them come back later as a
regression in git's code.
After checking a test snippet for --chain-lint, we do still
run the test itself. We could actually have a pure-lint
mode which just checks each test, but there are a few
reasons not to. One, because the tests are executing
arbitrary code, which could impact the later environment
(e.g., that could impact which set of tests we run at all).
And two, because a pure-lint mode would still be expensive
to run, because a significant amount of code runs outside of
the test_expect_* blocks. Instead, this option is designed
to be used as part of a normal test suite run, where it adds
very little overhead.
Turning on this option detects quite a few problems in
existing tests, which will be fixed in subsequent patches.
However, there are a number of places it cannot reach:
- it cannot find a failure to break out of loops on error,
like:
cmd1 &&
for i in a b c; do
cmd2 $i
done &&
cmd3
which will not notice failures of "cmd2 a" or "cmd b"
- it cannot find a missing &&-chain inside a block or
subfunction, like:
foo () {
cmd1
cmd2
}
foo &&
bar
which will not notice a failure of cmd1.
- it only checks tests that you run; every platform will
have some tests skipped due to missing prequisites,
so it's impossible to say from one run that the test
suite is free of broken &&-chains. However, all tests get
run by _somebody_, so eventually we will notice problems.
- it does not operate on test_when_finished or prerequisite
blocks. It could, but these tends to be much shorter and
less of a problem, so I punted on them in this patch.
This patch was inspired by an earlier patch by Jonathan
Nieder:
http://article.gmane.org/gmane.comp.version-control.git/235913
This implementation and all bugs are mine.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-20 18:05:48 +08:00
|
|
|
|
2015-04-23 04:09:57 +08:00
|
|
|
if test "${GIT_TEST_CHAIN_LINT:-1}" != 0; then
|
2015-08-06 13:31:47 +08:00
|
|
|
# turn off tracing for this test-eval, as it simply creates
|
|
|
|
|
# confusing noise in the "-x" output
|
|
|
|
|
trace_tmp=$trace
|
|
|
|
|
trace=
|
t/test-lib: introduce --chain-lint option
It's easy to miss an "&&"-chain in a test script, like:
test_expect_success 'check something important' '
cmd1 &&
cmd2
cmd3
'
The test harness will notice if cmd3 fails, but a failure of
cmd1 or cmd2 will go unnoticed, as their exit status is lost
after cmd3 runs.
The toy example above is easy to spot because the "cmds" are
all the same length, but real code is much more complicated.
It's also difficult to detect these situations by statically
analyzing the shell code with regexps (like the
check-non-portable-shell script does); there's too much
context required to know whether a &&-chain is appropriate
on a given line or not.
This patch instead lets the shell check each test by
sticking a command with a specific and unusual return code
at the top of each test, like:
(exit 117) &&
cmd1 &&
cmd2
cmd3
In a well-formed test, the non-zero exit from the first
command prevents any of the rest from being run, and the
test's exit code is 117. In a bad test (like the one above),
the 117 is lost, and cmd3 is run.
When we encounter a failure of this check, we abort the test
script entirely. For one thing, we have no clue which subset
of the commands in the test snippet were actually run.
Running further tests would be pointless, because we're now
in an unknown state. And two, this is not a "test failure"
in the traditional sense. The test script is buggy, not the
code it is testing. We should be able to fix these problems
in the script once, and not have them come back later as a
regression in git's code.
After checking a test snippet for --chain-lint, we do still
run the test itself. We could actually have a pure-lint
mode which just checks each test, but there are a few
reasons not to. One, because the tests are executing
arbitrary code, which could impact the later environment
(e.g., that could impact which set of tests we run at all).
And two, because a pure-lint mode would still be expensive
to run, because a significant amount of code runs outside of
the test_expect_* blocks. Instead, this option is designed
to be used as part of a normal test suite run, where it adds
very little overhead.
Turning on this option detects quite a few problems in
existing tests, which will be fixed in subsequent patches.
However, there are a number of places it cannot reach:
- it cannot find a failure to break out of loops on error,
like:
cmd1 &&
for i in a b c; do
cmd2 $i
done &&
cmd3
which will not notice failures of "cmd2 a" or "cmd b"
- it cannot find a missing &&-chain inside a block or
subfunction, like:
foo () {
cmd1
cmd2
}
foo &&
bar
which will not notice a failure of cmd1.
- it only checks tests that you run; every platform will
have some tests skipped due to missing prequisites,
so it's impossible to say from one run that the test
suite is free of broken &&-chains. However, all tests get
run by _somebody_, so eventually we will notice problems.
- it does not operate on test_when_finished or prerequisite
blocks. It could, but these tends to be much shorter and
less of a problem, so I punted on them in this patch.
This patch was inspired by an earlier patch by Jonathan
Nieder:
http://article.gmane.org/gmane.comp.version-control.git/235913
This implementation and all bugs are mine.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-20 18:05:48 +08:00
|
|
|
# 117 is magic because it is unlikely to match the exit
|
|
|
|
|
# code of other programs
|
2017-03-23 13:43:18 +08:00
|
|
|
if test "OK-117" != "$(test_eval_ "(exit 117) && $1${LF}${LF}echo OK-\$?" 3>&1)"
|
|
|
|
|
then
|
|
|
|
|
error "bug in the test script: broken &&-chain or run-away HERE-DOC: $1"
|
t/test-lib: introduce --chain-lint option
It's easy to miss an "&&"-chain in a test script, like:
test_expect_success 'check something important' '
cmd1 &&
cmd2
cmd3
'
The test harness will notice if cmd3 fails, but a failure of
cmd1 or cmd2 will go unnoticed, as their exit status is lost
after cmd3 runs.
The toy example above is easy to spot because the "cmds" are
all the same length, but real code is much more complicated.
It's also difficult to detect these situations by statically
analyzing the shell code with regexps (like the
check-non-portable-shell script does); there's too much
context required to know whether a &&-chain is appropriate
on a given line or not.
This patch instead lets the shell check each test by
sticking a command with a specific and unusual return code
at the top of each test, like:
(exit 117) &&
cmd1 &&
cmd2
cmd3
In a well-formed test, the non-zero exit from the first
command prevents any of the rest from being run, and the
test's exit code is 117. In a bad test (like the one above),
the 117 is lost, and cmd3 is run.
When we encounter a failure of this check, we abort the test
script entirely. For one thing, we have no clue which subset
of the commands in the test snippet were actually run.
Running further tests would be pointless, because we're now
in an unknown state. And two, this is not a "test failure"
in the traditional sense. The test script is buggy, not the
code it is testing. We should be able to fix these problems
in the script once, and not have them come back later as a
regression in git's code.
After checking a test snippet for --chain-lint, we do still
run the test itself. We could actually have a pure-lint
mode which just checks each test, but there are a few
reasons not to. One, because the tests are executing
arbitrary code, which could impact the later environment
(e.g., that could impact which set of tests we run at all).
And two, because a pure-lint mode would still be expensive
to run, because a significant amount of code runs outside of
the test_expect_* blocks. Instead, this option is designed
to be used as part of a normal test suite run, where it adds
very little overhead.
Turning on this option detects quite a few problems in
existing tests, which will be fixed in subsequent patches.
However, there are a number of places it cannot reach:
- it cannot find a failure to break out of loops on error,
like:
cmd1 &&
for i in a b c; do
cmd2 $i
done &&
cmd3
which will not notice failures of "cmd2 a" or "cmd b"
- it cannot find a missing &&-chain inside a block or
subfunction, like:
foo () {
cmd1
cmd2
}
foo &&
bar
which will not notice a failure of cmd1.
- it only checks tests that you run; every platform will
have some tests skipped due to missing prequisites,
so it's impossible to say from one run that the test
suite is free of broken &&-chains. However, all tests get
run by _somebody_, so eventually we will notice problems.
- it does not operate on test_when_finished or prerequisite
blocks. It could, but these tends to be much shorter and
less of a problem, so I punted on them in this patch.
This patch was inspired by an earlier patch by Jonathan
Nieder:
http://article.gmane.org/gmane.comp.version-control.git/235913
This implementation and all bugs are mine.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-20 18:05:48 +08:00
|
|
|
fi
|
2015-08-06 13:31:47 +08:00
|
|
|
trace=$trace_tmp
|
t/test-lib: introduce --chain-lint option
It's easy to miss an "&&"-chain in a test script, like:
test_expect_success 'check something important' '
cmd1 &&
cmd2
cmd3
'
The test harness will notice if cmd3 fails, but a failure of
cmd1 or cmd2 will go unnoticed, as their exit status is lost
after cmd3 runs.
The toy example above is easy to spot because the "cmds" are
all the same length, but real code is much more complicated.
It's also difficult to detect these situations by statically
analyzing the shell code with regexps (like the
check-non-portable-shell script does); there's too much
context required to know whether a &&-chain is appropriate
on a given line or not.
This patch instead lets the shell check each test by
sticking a command with a specific and unusual return code
at the top of each test, like:
(exit 117) &&
cmd1 &&
cmd2
cmd3
In a well-formed test, the non-zero exit from the first
command prevents any of the rest from being run, and the
test's exit code is 117. In a bad test (like the one above),
the 117 is lost, and cmd3 is run.
When we encounter a failure of this check, we abort the test
script entirely. For one thing, we have no clue which subset
of the commands in the test snippet were actually run.
Running further tests would be pointless, because we're now
in an unknown state. And two, this is not a "test failure"
in the traditional sense. The test script is buggy, not the
code it is testing. We should be able to fix these problems
in the script once, and not have them come back later as a
regression in git's code.
After checking a test snippet for --chain-lint, we do still
run the test itself. We could actually have a pure-lint
mode which just checks each test, but there are a few
reasons not to. One, because the tests are executing
arbitrary code, which could impact the later environment
(e.g., that could impact which set of tests we run at all).
And two, because a pure-lint mode would still be expensive
to run, because a significant amount of code runs outside of
the test_expect_* blocks. Instead, this option is designed
to be used as part of a normal test suite run, where it adds
very little overhead.
Turning on this option detects quite a few problems in
existing tests, which will be fixed in subsequent patches.
However, there are a number of places it cannot reach:
- it cannot find a failure to break out of loops on error,
like:
cmd1 &&
for i in a b c; do
cmd2 $i
done &&
cmd3
which will not notice failures of "cmd2 a" or "cmd b"
- it cannot find a missing &&-chain inside a block or
subfunction, like:
foo () {
cmd1
cmd2
}
foo &&
bar
which will not notice a failure of cmd1.
- it only checks tests that you run; every platform will
have some tests skipped due to missing prequisites,
so it's impossible to say from one run that the test
suite is free of broken &&-chains. However, all tests get
run by _somebody_, so eventually we will notice problems.
- it does not operate on test_when_finished or prerequisite
blocks. It could, but these tends to be much shorter and
less of a problem, so I punted on them in this patch.
This patch was inspired by an earlier patch by Jonathan
Nieder:
http://article.gmane.org/gmane.comp.version-control.git/235913
This implementation and all bugs are mine.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-20 18:05:48 +08:00
|
|
|
fi
|
|
|
|
|
|
2013-06-17 17:18:46 +08:00
|
|
|
setup_malloc_check
|
test: cope better with use of return for errors
In olden times, tests would quietly exit the script when they failed
at an inconvenient moment, which was a little disconcerting.
Therefore v0.99.5~24^2~4 (Trapping exit in tests, using return for
errors, 2005-08-10) switched to an idiom of using "return" instead,
wrapping evaluation of test code in a function to make that safe:
test_run_ () {
eval >&3 2>&4 "$1"
eval_ret="$?"
return 0
}
Years later, the implementation of test_when_finished (v1.7.1.1~95,
2010-05-02) and v1.7.2-rc2~1^2~13 (test-lib: output a newline before
"ok" under a TAP harness, 2010-06-24) took advantage of test_run_ as a
place to put code shared by all test assertion functions, without
paying attention to the function's former purpose:
test_run_ () {
...
eval >&3 2>&4 "$1"
eval_ret=$?
if should run cleanup
then
eval >&3 2>&4 "$test_cleanup"
fi
if TAP format requires a newline here
then
echo
fi
return 0
}
That means cleanup commands and the newline to put TAP output at
column 0 are skipped when tests use "return" to fail early. Fix it by
introducing a test_eval_ function to catch the "return", with a
comment explaining the new function's purpose for the next person who
might touch this code.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-08-08 09:17:09 +08:00
|
|
|
test_eval_ "$1"
|
2010-05-06 16:41:10 +08:00
|
|
|
eval_ret=$?
|
2013-06-17 17:18:46 +08:00
|
|
|
teardown_malloc_check
|
2011-06-28 02:02:22 +08:00
|
|
|
|
test-lib.sh: support -x option for shell-tracing
Usually running a test under "-v" makes it clear which
command is failing. However, sometimes it can be useful to
also see a complete trace of the shell commands being run in
the test. You can do so without any support from the test
suite by running "sh -x tXXXX-foo.sh". However, this
produces quite a large bit of output, as we see a trace of
the entire test suite.
This patch instead introduces a "-x" option to the test
scripts (i.e., "./tXXXX-foo.sh -x"). When enabled, this
turns on "set -x" only for the tests themselves. This can
still be a bit verbose, but should keep things to a more
manageable level. You can even use "--verbose-only" to see
the trace only for a specific test.
The implementation is a little invasive. We turn on the "set
-x" inside the "eval" of the test code. This lets the eval
itself avoid being reported in the trace (which would be
long, and redundant with the verbose listing we already
showed). And then after the eval runs, we do some trickery
with stderr to avoid showing the "set +x" to the user.
We also show traces for test_cleanup functions (since they
can impact the test outcome, too). However, we do avoid
running the noop ":" cleanup (the default if the test does
not use test_cleanup at all), as it creates unnecessary
noise in the "set -x" output.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-10 14:47:27 +08:00
|
|
|
if test -z "$immediate" || test $eval_ret = 0 ||
|
|
|
|
|
test -n "$expecting_failure" && test "$test_cleanup" != ":"
|
2011-06-28 02:02:22 +08:00
|
|
|
then
|
2012-09-15 11:38:24 +08:00
|
|
|
setup_malloc_check
|
test: cope better with use of return for errors
In olden times, tests would quietly exit the script when they failed
at an inconvenient moment, which was a little disconcerting.
Therefore v0.99.5~24^2~4 (Trapping exit in tests, using return for
errors, 2005-08-10) switched to an idiom of using "return" instead,
wrapping evaluation of test code in a function to make that safe:
test_run_ () {
eval >&3 2>&4 "$1"
eval_ret="$?"
return 0
}
Years later, the implementation of test_when_finished (v1.7.1.1~95,
2010-05-02) and v1.7.2-rc2~1^2~13 (test-lib: output a newline before
"ok" under a TAP harness, 2010-06-24) took advantage of test_run_ as a
place to put code shared by all test assertion functions, without
paying attention to the function's former purpose:
test_run_ () {
...
eval >&3 2>&4 "$1"
eval_ret=$?
if should run cleanup
then
eval >&3 2>&4 "$test_cleanup"
fi
if TAP format requires a newline here
then
echo
fi
return 0
}
That means cleanup commands and the newline to put TAP output at
column 0 are skipped when tests use "return" to fail early. Fix it by
introducing a test_eval_ function to catch the "return", with a
comment explaining the new function's purpose for the next person who
might touch this code.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-08-08 09:17:09 +08:00
|
|
|
test_eval_ "$test_cleanup"
|
2012-09-15 11:38:24 +08:00
|
|
|
teardown_malloc_check
|
2011-06-28 02:02:22 +08:00
|
|
|
fi
|
2012-09-02 02:11:49 +08:00
|
|
|
if test "$verbose" = "t" && test -n "$HARNESS_ACTIVE"
|
|
|
|
|
then
|
test-lib: output a newline before "ok" under a TAP harness
Some tests in the testsuite will emit a line that doesn't end with a
newline, right before we're about to output "ok" or "not ok". This
breaks the TAP output with "Tests out of sequence" errors since a TAP
harness can't understand this:
ok 1 - A test
[some output here]ok 2 - Another test
ok 3 - Yet another test
Work around it by emitting an empty line before we're about to say
"ok" or "not ok", but only if we're running under --verbose and
HARNESS_ACTIVE=1 is set, which'll only be the case when running under
a harnesses like prove(1).
I think it's better to do this than fix each tests by adding `&& echo'
everywhere. More tests might be added that break TAP in the future,
and a human isn't going to look at the extra whitespace, since
HARNESS_ACTIVE=1 always means a harness is reading it.
The tests that had issues were:
t1007, t3410, t3413, t3409, t3414, t3415, t3416, t3412, t3404,
t5407, t7402, t7003, t9001
With this workaround the entire test suite runs without errors under:
prove -j 10 ./t[0-9]*.sh :: --verbose
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-06-25 01:44:47 +08:00
|
|
|
echo ""
|
|
|
|
|
fi
|
2011-08-08 09:15:34 +08:00
|
|
|
return "$eval_ret"
|
2005-08-11 11:56:21 +08:00
|
|
|
}
|
|
|
|
|
|
2013-06-18 20:25:59 +08:00
|
|
|
test_start_ () {
|
2009-02-06 04:20:56 +08:00
|
|
|
test_count=$(($test_count+1))
|
2013-06-24 02:12:56 +08:00
|
|
|
maybe_setup_verbose
|
2013-06-24 02:12:57 +08:00
|
|
|
maybe_setup_valgrind
|
2013-06-18 20:25:59 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
test_finish_ () {
|
|
|
|
|
echo >&3 ""
|
2013-06-24 02:12:57 +08:00
|
|
|
maybe_teardown_valgrind
|
2013-06-24 02:12:56 +08:00
|
|
|
maybe_teardown_verbose
|
2013-06-18 20:25:59 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
test_skip () {
|
2006-12-29 09:58:00 +08:00
|
|
|
to_skip=
|
2014-04-30 17:50:43 +08:00
|
|
|
skipped_reason=
|
2013-06-18 20:25:58 +08:00
|
|
|
if match_pattern_list $this_test.$test_count $GIT_SKIP_TESTS
|
|
|
|
|
then
|
|
|
|
|
to_skip=t
|
2014-04-30 17:50:43 +08:00
|
|
|
skipped_reason="GIT_SKIP_TESTS"
|
2013-06-18 20:25:58 +08:00
|
|
|
fi
|
2010-10-17 02:36:58 +08:00
|
|
|
if test -z "$to_skip" && test -n "$test_prereq" &&
|
|
|
|
|
! test_have_prereq "$test_prereq"
|
2009-03-02 04:04:46 +08:00
|
|
|
then
|
|
|
|
|
to_skip=t
|
2014-04-30 17:50:43 +08:00
|
|
|
|
2010-08-24 15:34:10 +08:00
|
|
|
of_prereq=
|
2010-10-17 02:36:58 +08:00
|
|
|
if test "$missing_prereq" != "$test_prereq"
|
2010-08-24 15:34:10 +08:00
|
|
|
then
|
2010-10-17 02:36:58 +08:00
|
|
|
of_prereq=" of $test_prereq"
|
2010-08-24 15:34:10 +08:00
|
|
|
fi
|
2014-04-30 17:50:43 +08:00
|
|
|
skipped_reason="missing $missing_prereq${of_prereq}"
|
|
|
|
|
fi
|
2014-04-30 17:50:44 +08:00
|
|
|
if test -z "$to_skip" && test -n "$run_list" &&
|
|
|
|
|
! match_test_selector_list '--run' $test_count "$run_list"
|
|
|
|
|
then
|
|
|
|
|
to_skip=t
|
|
|
|
|
skipped_reason="--run"
|
|
|
|
|
fi
|
2010-08-24 15:34:10 +08:00
|
|
|
|
2014-04-30 17:50:43 +08:00
|
|
|
case "$to_skip" in
|
|
|
|
|
t)
|
2013-10-20 05:06:08 +08:00
|
|
|
say_color skip >&3 "skipping test: $@"
|
2014-04-30 17:50:43 +08:00
|
|
|
say_color skip "ok $test_count # skip $1 ($skipped_reason)"
|
2006-12-29 09:58:00 +08:00
|
|
|
: true
|
|
|
|
|
;;
|
|
|
|
|
*)
|
|
|
|
|
false
|
|
|
|
|
;;
|
|
|
|
|
esac
|
|
|
|
|
}
|
|
|
|
|
|
2012-02-17 18:25:09 +08:00
|
|
|
# stub; perf-lib overrides it
|
|
|
|
|
test_at_end_hook_ () {
|
|
|
|
|
:
|
|
|
|
|
}
|
|
|
|
|
|
2005-05-14 13:50:32 +08:00
|
|
|
test_done () {
|
2009-06-01 20:14:41 +08:00
|
|
|
GIT_EXIT_OK=t
|
2008-06-08 22:04:33 +08:00
|
|
|
|
2012-09-02 02:11:49 +08:00
|
|
|
if test -z "$HARNESS_ACTIVE"
|
|
|
|
|
then
|
2012-02-17 18:25:09 +08:00
|
|
|
test_results_dir="$TEST_OUTPUT_DIRECTORY/test-results"
|
2010-08-12 03:37:31 +08:00
|
|
|
mkdir -p "$test_results_dir"
|
2012-11-04 10:13:33 +08:00
|
|
|
base=${0##*/}
|
test-lib: drop PID from test-results/*.count
Each test run generates a "count" file in t/test-results
that stores the number of successful, failed, etc tests.
If you run "t1234-foo.sh", that file is named as
"t/test-results/t1234-foo-$$.count"
The addition of the PID there is serving no purpose, and
makes analysis of the count files harder.
The presence of the PID dates back to 2d84e9f (Modify
test-lib.sh to output stats to t/test-results/*,
2008-06-08), but no reasoning is given there. Looking at the
current code, we can see that other files we write to
test-results (like *.exit and *.out) do _not_ have the PID
included. So the presence of the PID does not meaningfully
allow one to store the results from multiple runs anyway.
Moreover, anybody wishing to read the *.count files to
aggregate results has to deal with the presence of multiple
files for a given test (and figure out which one is the most
recent based on their timestamps!). The only consumer of
these files is the aggregate.sh script, which arguably gets
this wrong. If a test is run multiple times, its counts will
appear multiple times in the total (I say arguably only
because the desired semantics aren't documented anywhere,
but I have trouble seeing how this behavior could be
useful).
So let's just drop the PID, which fixes aggregate.sh, and
will make new features based around the count files easier
to write.
Note that since the count-file may already exist (when
re-running a test), we also switch the "cat" from appending
to truncating. The use of append here was pointless in the
first place, as we expected to always write to a unique file.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-30 16:43:57 +08:00
|
|
|
test_results_path="$test_results_dir/${base%.sh}.counts"
|
2010-08-12 03:37:31 +08:00
|
|
|
|
test-lib: drop PID from test-results/*.count
Each test run generates a "count" file in t/test-results
that stores the number of successful, failed, etc tests.
If you run "t1234-foo.sh", that file is named as
"t/test-results/t1234-foo-$$.count"
The addition of the PID there is serving no purpose, and
makes analysis of the count files harder.
The presence of the PID dates back to 2d84e9f (Modify
test-lib.sh to output stats to t/test-results/*,
2008-06-08), but no reasoning is given there. Looking at the
current code, we can see that other files we write to
test-results (like *.exit and *.out) do _not_ have the PID
included. So the presence of the PID does not meaningfully
allow one to store the results from multiple runs anyway.
Moreover, anybody wishing to read the *.count files to
aggregate results has to deal with the presence of multiple
files for a given test (and figure out which one is the most
recent based on their timestamps!). The only consumer of
these files is the aggregate.sh script, which arguably gets
this wrong. If a test is run multiple times, its counts will
appear multiple times in the total (I say arguably only
because the desired semantics aren't documented anywhere,
but I have trouble seeing how this behavior could be
useful).
So let's just drop the PID, which fixes aggregate.sh, and
will make new features based around the count files easier
to write.
Note that since the count-file may already exist (when
re-running a test), we also switch the "cat" from appending
to truncating. The use of append here was pointless in the
first place, as we expected to always write to a unique file.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-30 16:43:57 +08:00
|
|
|
cat >"$test_results_path" <<-EOF
|
2011-04-29 20:30:30 +08:00
|
|
|
total $test_count
|
|
|
|
|
success $test_success
|
|
|
|
|
fixed $test_fixed
|
|
|
|
|
broken $test_broken
|
|
|
|
|
failed $test_failure
|
|
|
|
|
|
|
|
|
|
EOF
|
2010-08-12 03:37:31 +08:00
|
|
|
fi
|
2008-02-01 17:50:53 +08:00
|
|
|
|
|
|
|
|
if test "$test_fixed" != 0
|
|
|
|
|
then
|
2013-10-20 05:06:08 +08:00
|
|
|
say_color error "# $test_fixed known breakage(s) vanished; please update test(s)"
|
2008-02-01 17:50:53 +08:00
|
|
|
fi
|
|
|
|
|
if test "$test_broken" != 0
|
|
|
|
|
then
|
2013-10-20 05:06:08 +08:00
|
|
|
say_color warn "# still have $test_broken known breakage(s)"
|
2012-12-17 02:28:15 +08:00
|
|
|
fi
|
|
|
|
|
if test "$test_broken" != 0 || test "$test_fixed" != 0
|
|
|
|
|
then
|
|
|
|
|
test_remaining=$(( $test_count - $test_broken - $test_fixed ))
|
|
|
|
|
msg="remaining $test_remaining test(s)"
|
2008-02-03 16:23:02 +08:00
|
|
|
else
|
2012-12-17 02:28:15 +08:00
|
|
|
test_remaining=$test_count
|
2008-02-03 16:23:02 +08:00
|
|
|
msg="$test_count test(s)"
|
2008-02-01 17:50:53 +08:00
|
|
|
fi
|
2005-05-14 13:50:32 +08:00
|
|
|
case "$test_failure" in
|
2005-12-10 09:32:18 +08:00
|
|
|
0)
|
test-lib: Adjust output to be valid TAP format
TAP, the Test Anything Protocol, is a simple text-based interface
between testing modules in a test harness. test-lib.sh's output was
already very close to being valid TAP. This change brings it all the
way there. Before:
$ ./t0005-signals.sh
* ok 1: sigchain works
* passed all 1 test(s)
And after:
$ ./t0005-signals.sh
ok 1 - sigchain works
# passed all 1 test(s)
1..1
The advantage of using TAP is that any program that reads the format
(a "test harness") can run the tests. The most popular of these is the
prove(1) utility that comes with Perl. It can run tests in parallel,
display colored output, format the output to console, file, HTML etc.,
and much more. An example:
$ prove ./t0005-signals.sh
./t0005-signals.sh .. ok
All tests successful.
Files=1, Tests=1, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.01 cusr 0.02 csys = 0.06 CPU)
Result: PASS
prove(1) gives you human readable output without being too
verbose. Running the test suite in parallel with `make test -j15`
produces a flood of text. Running them with `prove -j 15 ./t[0-9]*.sh`
makes it easy to follow what's going on.
All this patch does is re-arrange the output a bit so that it conforms
with the TAP spec, everything that the test suite did before continues
to work. That includes aggregating results in t/test-results/, the
--verbose, --debug and other options for tests, and the test color
output.
TAP harnesses ignore everything that they don't know about, so running
the tests with --verbose works:
$ prove ./t0005-signals.sh :: --verbose --debug
./t0005-signals.sh .. Terminated
./t0005-signals.sh .. ok
All tests successful.
Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.01 sys + 0.01 cusr 0.01 csys = 0.05 CPU)
Result: PASS
Just supply the -v option to prove itself to get all the verbose
output that it suppresses:
$ prove -v ./t0005-signals.sh :: --verbose --debug
./t0005-signals.sh ..
Initialized empty Git repository in /home/avar/g/git/t/trash directory.t0005-signals/.git/
expecting success:
test-sigchain >actual
case "$?" in
143) true ;; # POSIX w/ SIGTERM=15
3) true ;; # Windows
*) false ;;
esac &&
test_cmp expect actual
Terminated
ok 1 - sigchain works
# passed all 1 test(s)
1..1
ok
All tests successful.
Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.01 cusr 0.01 csys = 0.04 CPU)
Result: PASS
As a further example, consider this test script that uses a lot of
test-lib.sh features by Jakub Narebski:
#!/bin/sh
test_description='this is a sample test.
This test is here to see various test outputs.'
. ./test-lib.sh
say 'diagnostic message'
test_expect_success 'true test' 'true'
test_expect_success 'false test' 'false'
test_expect_failure 'true test (todo)' 'true'
test_expect_failure 'false test (todo)' 'false'
test_debug 'echo "debug message"'
test_done
The output of that was previously:
* diagnostic message # yellow
* ok 1: true test
* FAIL 2: false test # bold red
false
* FIXED 3: true test (todo)
* still broken 4: false test (todo) # bold green
* fixed 1 known breakage(s) # green
* still have 1 known breakage(s) # bold red
* failed 1 among remaining 3 test(s) # bold red
But is now:
diagnostic message # yellow
ok 1 - true test
not ok - 2 false test # bold red
# false
ok 3 - true test (todo) # TODO known breakage
not ok 4 - false test (todo) # TODO known breakage # bold green
# fixed 1 known breakage(s) # green
# still have 1 known breakage(s) # bold red
# failed 1 among remaining 3 test(s) # bold red
1..4
All the coloring is preserved when the test is run manually. Under
prove(1) the test performs as expected, even with --debug and
--verbose options:
$ prove ./example.sh :: --debug --verbose
./example.sh .. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/4 subtests
(1 TODO test unexpectedly succeeded)
Test Summary Report
-------------------
./example.sh (Wstat: 256 Tests: 4 Failed: 1)
Failed test: 2
TODO passed: 3
Non-zero exit status: 1
Files=1, Tests=4, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.00 cusr 0.01 csys = 0.03 CPU)
Result: FAIL
The TAP harness itself doesn't get confused by the color output, they
aren't used by test-lib.sh stdout isn't open to a terminal (test -t 1).
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-06-25 05:52:12 +08:00
|
|
|
# Maybe print SKIP message
|
2012-09-02 02:13:19 +08:00
|
|
|
if test -n "$skip_all" && test $test_count -gt 0
|
|
|
|
|
then
|
|
|
|
|
error "Can't use skip_all after running some tests"
|
|
|
|
|
fi
|
2015-01-07 06:57:50 +08:00
|
|
|
test -z "$skip_all" || skip_all=" # SKIP $skip_all"
|
test-lib: Adjust output to be valid TAP format
TAP, the Test Anything Protocol, is a simple text-based interface
between testing modules in a test harness. test-lib.sh's output was
already very close to being valid TAP. This change brings it all the
way there. Before:
$ ./t0005-signals.sh
* ok 1: sigchain works
* passed all 1 test(s)
And after:
$ ./t0005-signals.sh
ok 1 - sigchain works
# passed all 1 test(s)
1..1
The advantage of using TAP is that any program that reads the format
(a "test harness") can run the tests. The most popular of these is the
prove(1) utility that comes with Perl. It can run tests in parallel,
display colored output, format the output to console, file, HTML etc.,
and much more. An example:
$ prove ./t0005-signals.sh
./t0005-signals.sh .. ok
All tests successful.
Files=1, Tests=1, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.01 cusr 0.02 csys = 0.06 CPU)
Result: PASS
prove(1) gives you human readable output without being too
verbose. Running the test suite in parallel with `make test -j15`
produces a flood of text. Running them with `prove -j 15 ./t[0-9]*.sh`
makes it easy to follow what's going on.
All this patch does is re-arrange the output a bit so that it conforms
with the TAP spec, everything that the test suite did before continues
to work. That includes aggregating results in t/test-results/, the
--verbose, --debug and other options for tests, and the test color
output.
TAP harnesses ignore everything that they don't know about, so running
the tests with --verbose works:
$ prove ./t0005-signals.sh :: --verbose --debug
./t0005-signals.sh .. Terminated
./t0005-signals.sh .. ok
All tests successful.
Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.01 sys + 0.01 cusr 0.01 csys = 0.05 CPU)
Result: PASS
Just supply the -v option to prove itself to get all the verbose
output that it suppresses:
$ prove -v ./t0005-signals.sh :: --verbose --debug
./t0005-signals.sh ..
Initialized empty Git repository in /home/avar/g/git/t/trash directory.t0005-signals/.git/
expecting success:
test-sigchain >actual
case "$?" in
143) true ;; # POSIX w/ SIGTERM=15
3) true ;; # Windows
*) false ;;
esac &&
test_cmp expect actual
Terminated
ok 1 - sigchain works
# passed all 1 test(s)
1..1
ok
All tests successful.
Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.01 cusr 0.01 csys = 0.04 CPU)
Result: PASS
As a further example, consider this test script that uses a lot of
test-lib.sh features by Jakub Narebski:
#!/bin/sh
test_description='this is a sample test.
This test is here to see various test outputs.'
. ./test-lib.sh
say 'diagnostic message'
test_expect_success 'true test' 'true'
test_expect_success 'false test' 'false'
test_expect_failure 'true test (todo)' 'true'
test_expect_failure 'false test (todo)' 'false'
test_debug 'echo "debug message"'
test_done
The output of that was previously:
* diagnostic message # yellow
* ok 1: true test
* FAIL 2: false test # bold red
false
* FIXED 3: true test (todo)
* still broken 4: false test (todo) # bold green
* fixed 1 known breakage(s) # green
* still have 1 known breakage(s) # bold red
* failed 1 among remaining 3 test(s) # bold red
But is now:
diagnostic message # yellow
ok 1 - true test
not ok - 2 false test # bold red
# false
ok 3 - true test (todo) # TODO known breakage
not ok 4 - false test (todo) # TODO known breakage # bold green
# fixed 1 known breakage(s) # green
# still have 1 known breakage(s) # bold red
# failed 1 among remaining 3 test(s) # bold red
1..4
All the coloring is preserved when the test is run manually. Under
prove(1) the test performs as expected, even with --debug and
--verbose options:
$ prove ./example.sh :: --debug --verbose
./example.sh .. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/4 subtests
(1 TODO test unexpectedly succeeded)
Test Summary Report
-------------------
./example.sh (Wstat: 256 Tests: 4 Failed: 1)
Failed test: 2
TODO passed: 3
Non-zero exit status: 1
Files=1, Tests=4, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.00 cusr 0.01 csys = 0.03 CPU)
Result: FAIL
The TAP harness itself doesn't get confused by the color output, they
aren't used by test-lib.sh stdout isn't open to a terminal (test -t 1).
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-06-25 05:52:12 +08:00
|
|
|
|
2012-09-02 02:11:49 +08:00
|
|
|
if test $test_external_has_tap -eq 0
|
|
|
|
|
then
|
2012-12-17 02:28:15 +08:00
|
|
|
if test $test_remaining -gt 0
|
2012-09-02 02:26:21 +08:00
|
|
|
then
|
2013-10-20 05:06:08 +08:00
|
|
|
say_color pass "# passed all $msg"
|
2012-09-02 02:26:21 +08:00
|
|
|
fi
|
2013-10-20 05:06:08 +08:00
|
|
|
say "1..$test_count$skip_all"
|
2010-06-25 01:44:46 +08:00
|
|
|
fi
|
Enable parallel tests
On multiprocessor machines, or with I/O heavy tests (that leave the
CPU waiting a lot), it makes sense to parallelize the tests.
However, care has to be taken that the different jobs use different
trash directories.
This commit does so, by creating the trash directories with a suffix
that is unique with regard to the test, as it is the test's base name.
Further, the trash directory is removed in the test itself if
everything went fine, so that the trash directories do not
pile up only to be removed at the very end.
If a test failed, the trash directory is not removed. Chances are
that the exact error message is lost in the clutter, but you can still
see what test failed from the name of the trash directory, and repeat
the test (without -j).
If all was good, you will see the aggregated results.
Suggestions to simplify this commit came from Junio and René.
There still is an issue with tests that want to run a server process and
listen to a fixed port (http and svn) --- they cannot run in parallel but
this patch does not address this issue.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-08-08 19:08:37 +08:00
|
|
|
|
2017-04-24 08:15:09 +08:00
|
|
|
if test -z "$debug"
|
2017-04-25 14:39:47 +08:00
|
|
|
then
|
2017-04-24 08:15:09 +08:00
|
|
|
test -d "$TRASH_DIRECTORY" ||
|
2017-04-25 14:39:47 +08:00
|
|
|
error "Tests passed but trash directory already removed before test cleanup; aborting"
|
Enable parallel tests
On multiprocessor machines, or with I/O heavy tests (that leave the
CPU waiting a lot), it makes sense to parallelize the tests.
However, care has to be taken that the different jobs use different
trash directories.
This commit does so, by creating the trash directories with a suffix
that is unique with regard to the test, as it is the test's base name.
Further, the trash directory is removed in the test itself if
everything went fine, so that the trash directories do not
pile up only to be removed at the very end.
If a test failed, the trash directory is not removed. Chances are
that the exact error message is lost in the clutter, but you can still
see what test failed from the name of the trash directory, and repeat
the test (without -j).
If all was good, you will see the aggregated results.
Suggestions to simplify this commit came from Junio and René.
There still is an issue with tests that want to run a server process and
listen to a fixed port (http and svn) --- they cannot run in parallel but
this patch does not address this issue.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-08-08 19:08:37 +08:00
|
|
|
|
2017-04-24 08:15:09 +08:00
|
|
|
cd "$TRASH_DIRECTORY/.." &&
|
|
|
|
|
rm -fr "$TRASH_DIRECTORY" ||
|
2017-04-25 14:39:47 +08:00
|
|
|
error "Tests passed but test cleanup failed; aborting"
|
|
|
|
|
fi
|
2012-02-17 18:25:09 +08:00
|
|
|
test_at_end_hook_
|
|
|
|
|
|
2005-05-14 13:50:32 +08:00
|
|
|
exit 0 ;;
|
|
|
|
|
|
|
|
|
|
*)
|
2012-09-02 02:11:49 +08:00
|
|
|
if test $test_external_has_tap -eq 0
|
|
|
|
|
then
|
2013-10-20 05:06:08 +08:00
|
|
|
say_color error "# failed $test_failure among $msg"
|
|
|
|
|
say "1..$test_count"
|
2010-06-25 01:44:46 +08:00
|
|
|
fi
|
test-lib: Adjust output to be valid TAP format
TAP, the Test Anything Protocol, is a simple text-based interface
between testing modules in a test harness. test-lib.sh's output was
already very close to being valid TAP. This change brings it all the
way there. Before:
$ ./t0005-signals.sh
* ok 1: sigchain works
* passed all 1 test(s)
And after:
$ ./t0005-signals.sh
ok 1 - sigchain works
# passed all 1 test(s)
1..1
The advantage of using TAP is that any program that reads the format
(a "test harness") can run the tests. The most popular of these is the
prove(1) utility that comes with Perl. It can run tests in parallel,
display colored output, format the output to console, file, HTML etc.,
and much more. An example:
$ prove ./t0005-signals.sh
./t0005-signals.sh .. ok
All tests successful.
Files=1, Tests=1, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.01 cusr 0.02 csys = 0.06 CPU)
Result: PASS
prove(1) gives you human readable output without being too
verbose. Running the test suite in parallel with `make test -j15`
produces a flood of text. Running them with `prove -j 15 ./t[0-9]*.sh`
makes it easy to follow what's going on.
All this patch does is re-arrange the output a bit so that it conforms
with the TAP spec, everything that the test suite did before continues
to work. That includes aggregating results in t/test-results/, the
--verbose, --debug and other options for tests, and the test color
output.
TAP harnesses ignore everything that they don't know about, so running
the tests with --verbose works:
$ prove ./t0005-signals.sh :: --verbose --debug
./t0005-signals.sh .. Terminated
./t0005-signals.sh .. ok
All tests successful.
Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.01 sys + 0.01 cusr 0.01 csys = 0.05 CPU)
Result: PASS
Just supply the -v option to prove itself to get all the verbose
output that it suppresses:
$ prove -v ./t0005-signals.sh :: --verbose --debug
./t0005-signals.sh ..
Initialized empty Git repository in /home/avar/g/git/t/trash directory.t0005-signals/.git/
expecting success:
test-sigchain >actual
case "$?" in
143) true ;; # POSIX w/ SIGTERM=15
3) true ;; # Windows
*) false ;;
esac &&
test_cmp expect actual
Terminated
ok 1 - sigchain works
# passed all 1 test(s)
1..1
ok
All tests successful.
Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.01 cusr 0.01 csys = 0.04 CPU)
Result: PASS
As a further example, consider this test script that uses a lot of
test-lib.sh features by Jakub Narebski:
#!/bin/sh
test_description='this is a sample test.
This test is here to see various test outputs.'
. ./test-lib.sh
say 'diagnostic message'
test_expect_success 'true test' 'true'
test_expect_success 'false test' 'false'
test_expect_failure 'true test (todo)' 'true'
test_expect_failure 'false test (todo)' 'false'
test_debug 'echo "debug message"'
test_done
The output of that was previously:
* diagnostic message # yellow
* ok 1: true test
* FAIL 2: false test # bold red
false
* FIXED 3: true test (todo)
* still broken 4: false test (todo) # bold green
* fixed 1 known breakage(s) # green
* still have 1 known breakage(s) # bold red
* failed 1 among remaining 3 test(s) # bold red
But is now:
diagnostic message # yellow
ok 1 - true test
not ok - 2 false test # bold red
# false
ok 3 - true test (todo) # TODO known breakage
not ok 4 - false test (todo) # TODO known breakage # bold green
# fixed 1 known breakage(s) # green
# still have 1 known breakage(s) # bold red
# failed 1 among remaining 3 test(s) # bold red
1..4
All the coloring is preserved when the test is run manually. Under
prove(1) the test performs as expected, even with --debug and
--verbose options:
$ prove ./example.sh :: --debug --verbose
./example.sh .. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/4 subtests
(1 TODO test unexpectedly succeeded)
Test Summary Report
-------------------
./example.sh (Wstat: 256 Tests: 4 Failed: 1)
Failed test: 2
TODO passed: 3
Non-zero exit status: 1
Files=1, Tests=4, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.00 cusr 0.01 csys = 0.03 CPU)
Result: FAIL
The TAP harness itself doesn't get confused by the color output, they
aren't used by test-lib.sh stdout isn't open to a terminal (test -t 1).
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-06-25 05:52:12 +08:00
|
|
|
|
2005-05-14 13:50:32 +08:00
|
|
|
exit 1 ;;
|
|
|
|
|
|
|
|
|
|
esac
|
|
|
|
|
}
|
|
|
|
|
|
2009-12-03 13:14:06 +08:00
|
|
|
if test -n "$valgrind"
|
2009-02-04 07:25:59 +08:00
|
|
|
then
|
|
|
|
|
make_symlink () {
|
|
|
|
|
test -h "$2" &&
|
|
|
|
|
test "$1" = "$(readlink "$2")" || {
|
|
|
|
|
# be super paranoid
|
|
|
|
|
if mkdir "$2".lock
|
|
|
|
|
then
|
|
|
|
|
rm -f "$2" &&
|
|
|
|
|
ln -s "$1" "$2" &&
|
|
|
|
|
rm -r "$2".lock
|
|
|
|
|
else
|
|
|
|
|
while test -d "$2".lock
|
|
|
|
|
do
|
|
|
|
|
say "Waiting for lock on $2."
|
|
|
|
|
sleep 1
|
|
|
|
|
done
|
|
|
|
|
fi
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
make_valgrind_symlink () {
|
2011-06-18 04:36:32 +08:00
|
|
|
# handle only executables, unless they are shell libraries that
|
2013-11-26 05:03:52 +08:00
|
|
|
# need to be in the exec-path.
|
2011-06-18 04:36:32 +08:00
|
|
|
test -x "$1" ||
|
2013-11-26 05:03:52 +08:00
|
|
|
test "# " = "$(head -c 2 <"$1")" ||
|
2011-06-18 04:36:32 +08:00
|
|
|
return;
|
2009-02-04 07:25:59 +08:00
|
|
|
|
|
|
|
|
base=$(basename "$1")
|
2016-10-28 06:14:00 +08:00
|
|
|
case "$base" in
|
|
|
|
|
test-*)
|
|
|
|
|
symlink_target="$GIT_BUILD_DIR/t/helper/$base"
|
|
|
|
|
;;
|
|
|
|
|
*)
|
|
|
|
|
symlink_target="$GIT_BUILD_DIR/$base"
|
|
|
|
|
;;
|
|
|
|
|
esac
|
2009-02-04 07:25:59 +08:00
|
|
|
# do not override scripts
|
|
|
|
|
if test -x "$symlink_target" &&
|
|
|
|
|
test ! -d "$symlink_target" &&
|
|
|
|
|
test "#!" != "$(head -c 2 < "$symlink_target")"
|
|
|
|
|
then
|
|
|
|
|
symlink_target=../valgrind.sh
|
|
|
|
|
fi
|
2009-02-04 07:26:08 +08:00
|
|
|
case "$base" in
|
|
|
|
|
*.sh|*.perl)
|
|
|
|
|
symlink_target=../unprocessed-script
|
|
|
|
|
esac
|
2009-02-04 07:25:59 +08:00
|
|
|
# create the link, or replace it if it is out of date
|
|
|
|
|
make_symlink "$symlink_target" "$GIT_VALGRIND/bin/$base" || exit
|
|
|
|
|
}
|
|
|
|
|
|
2013-10-20 05:06:07 +08:00
|
|
|
# override all git executables in TEST_DIRECTORY/..
|
|
|
|
|
GIT_VALGRIND=$TEST_DIRECTORY/valgrind
|
|
|
|
|
mkdir -p "$GIT_VALGRIND"/bin
|
2016-07-11 19:45:08 +08:00
|
|
|
for file in $GIT_BUILD_DIR/git* $GIT_BUILD_DIR/t/helper/test-*
|
2013-10-20 05:06:07 +08:00
|
|
|
do
|
|
|
|
|
make_valgrind_symlink $file
|
|
|
|
|
done
|
|
|
|
|
# special-case the mergetools loadables
|
|
|
|
|
make_symlink "$GIT_BUILD_DIR"/mergetools "$GIT_VALGRIND/bin/mergetools"
|
|
|
|
|
OLDIFS=$IFS
|
|
|
|
|
IFS=:
|
|
|
|
|
for path in $PATH
|
|
|
|
|
do
|
|
|
|
|
ls "$path"/git-* 2> /dev/null |
|
|
|
|
|
while read file
|
test-lib: support running tests under valgrind in parallel
With the new --valgrind-parallel=<n> option, we support running the
tests in a single test script under valgrind in parallel using 'n'
processes.
This really follows the dumbest approach possible, as follows:
* We spawn the test script 'n' times, using a throw-away
TEST_OUTPUT_DIRECTORY. Each of the instances is given options that
ensures that it only runs every n-th test under valgrind, but
together they cover the entire range.
* We add up the numbers from the individual tests, and provide the
usual output.
This is really a gross hack at this point, and should be improved. In
particular we should keep the actual outputs somewhere more easily
discoverable, and summarize them to the user.
Nevertheless, this is already workable and gives a speedup of more
than 2 on a dual-core (hyperthreaded) machine, using n=4. This is
expected since the overhead of valgrind is so big (on the order of 20x
under good conditions, and a large startup overhead at every git
invocation) that redundantly running the non-valgrind tests in between
is not that expensive.
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-24 02:12:59 +08:00
|
|
|
do
|
2013-10-20 05:06:07 +08:00
|
|
|
make_valgrind_symlink "$file"
|
test-lib: support running tests under valgrind in parallel
With the new --valgrind-parallel=<n> option, we support running the
tests in a single test script under valgrind in parallel using 'n'
processes.
This really follows the dumbest approach possible, as follows:
* We spawn the test script 'n' times, using a throw-away
TEST_OUTPUT_DIRECTORY. Each of the instances is given options that
ensures that it only runs every n-th test under valgrind, but
together they cover the entire range.
* We add up the numbers from the individual tests, and provide the
usual output.
This is really a gross hack at this point, and should be improved. In
particular we should keep the actual outputs somewhere more easily
discoverable, and summarize them to the user.
Nevertheless, this is already workable and gives a speedup of more
than 2 on a dual-core (hyperthreaded) machine, using n=4. This is
expected since the overhead of valgrind is so big (on the order of 20x
under good conditions, and a large startup overhead at every git
invocation) that redundantly running the non-valgrind tests in between
is not that expensive.
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-24 02:12:59 +08:00
|
|
|
done
|
2013-10-20 05:06:07 +08:00
|
|
|
done
|
|
|
|
|
IFS=$OLDIFS
|
2009-02-04 07:25:59 +08:00
|
|
|
PATH=$GIT_VALGRIND/bin:$PATH
|
|
|
|
|
GIT_EXEC_PATH=$GIT_VALGRIND/bin
|
|
|
|
|
export GIT_VALGRIND
|
2013-03-31 16:00:16 +08:00
|
|
|
GIT_VALGRIND_MODE="$valgrind"
|
|
|
|
|
export GIT_VALGRIND_MODE
|
2013-06-24 02:12:57 +08:00
|
|
|
GIT_VALGRIND_ENABLED=t
|
2013-10-20 05:06:07 +08:00
|
|
|
test -n "$valgrind_only" && GIT_VALGRIND_ENABLED=
|
2013-06-24 02:12:57 +08:00
|
|
|
export GIT_VALGRIND_ENABLED
|
2012-09-02 02:11:49 +08:00
|
|
|
elif test -n "$GIT_TEST_INSTALLED"
|
|
|
|
|
then
|
2009-12-03 13:14:06 +08:00
|
|
|
GIT_EXEC_PATH=$($GIT_TEST_INSTALLED/git --exec-path) ||
|
|
|
|
|
error "Cannot run git from $GIT_TEST_INSTALLED."
|
2010-08-20 00:08:10 +08:00
|
|
|
PATH=$GIT_TEST_INSTALLED:$GIT_BUILD_DIR:$PATH
|
2009-12-03 13:14:06 +08:00
|
|
|
GIT_EXEC_PATH=${GIT_TEST_EXEC_PATH:-$GIT_EXEC_PATH}
|
|
|
|
|
else # normal case, use ../bin-wrappers only unless $with_dashes:
|
2010-08-20 00:08:10 +08:00
|
|
|
git_bin_dir="$GIT_BUILD_DIR/bin-wrappers"
|
2012-09-02 02:11:49 +08:00
|
|
|
if ! test -x "$git_bin_dir/git"
|
|
|
|
|
then
|
|
|
|
|
if test -z "$with_dashes"
|
|
|
|
|
then
|
2009-12-03 13:14:06 +08:00
|
|
|
say "$git_bin_dir/git is not executable; using GIT_EXEC_PATH"
|
|
|
|
|
fi
|
|
|
|
|
with_dashes=t
|
|
|
|
|
fi
|
|
|
|
|
PATH="$git_bin_dir:$PATH"
|
2010-08-20 00:08:10 +08:00
|
|
|
GIT_EXEC_PATH=$GIT_BUILD_DIR
|
2012-09-02 02:11:49 +08:00
|
|
|
if test -n "$with_dashes"
|
|
|
|
|
then
|
2010-08-20 00:08:10 +08:00
|
|
|
PATH="$GIT_BUILD_DIR:$PATH"
|
2009-12-03 13:14:06 +08:00
|
|
|
fi
|
2009-02-04 07:25:59 +08:00
|
|
|
fi
|
2010-08-20 00:08:10 +08:00
|
|
|
GIT_TEMPLATE_DIR="$GIT_BUILD_DIR"/templates/blt
|
2008-02-06 18:11:53 +08:00
|
|
|
GIT_CONFIG_NOSYSTEM=1
|
2011-03-15 17:05:10 +08:00
|
|
|
GIT_ATTR_NOSYSTEM=1
|
2011-03-15 17:04:49 +08:00
|
|
|
export PATH GIT_EXEC_PATH GIT_TEMPLATE_DIR GIT_CONFIG_NOSYSTEM GIT_ATTR_NOSYSTEM
|
2005-12-08 13:52:28 +08:00
|
|
|
|
2010-06-12 00:40:25 +08:00
|
|
|
if test -z "$GIT_TEST_CMP"
|
|
|
|
|
then
|
|
|
|
|
if test -n "$GIT_TEST_CMP_USE_COPIED_CONTEXT"
|
|
|
|
|
then
|
|
|
|
|
GIT_TEST_CMP="$DIFF -c"
|
|
|
|
|
else
|
|
|
|
|
GIT_TEST_CMP="$DIFF -u"
|
|
|
|
|
fi
|
|
|
|
|
fi
|
|
|
|
|
|
2010-08-20 00:08:10 +08:00
|
|
|
GITPERLLIB="$GIT_BUILD_DIR"/perl/blib/lib:"$GIT_BUILD_DIR"/perl/blib/arch/auto/Git
|
2006-07-04 05:16:32 +08:00
|
|
|
export GITPERLLIB
|
2010-08-20 00:08:10 +08:00
|
|
|
test -d "$GIT_BUILD_DIR"/templates/blt || {
|
2005-12-11 12:55:32 +08:00
|
|
|
error "You haven't built things yet, have you?"
|
|
|
|
|
}
|
2005-05-14 13:50:32 +08:00
|
|
|
|
2016-04-13 21:22:42 +08:00
|
|
|
if ! test -x "$GIT_BUILD_DIR"/t/helper/test-chmtime
|
2012-09-02 02:11:49 +08:00
|
|
|
then
|
2007-02-25 08:59:52 +08:00
|
|
|
echo >&2 'You need to build test-chmtime:'
|
2016-04-13 21:22:42 +08:00
|
|
|
echo >&2 'Run "make t/helper/test-chmtime" in the source (toplevel) directory'
|
2007-02-25 08:59:52 +08:00
|
|
|
exit 1
|
|
|
|
|
fi
|
|
|
|
|
|
2005-05-14 13:50:32 +08:00
|
|
|
# Test repository
|
2013-04-15 03:38:21 +08:00
|
|
|
TRASH_DIRECTORY="trash directory.$(basename "$0" .sh)"
|
|
|
|
|
test -n "$root" && TRASH_DIRECTORY="$root/$TRASH_DIRECTORY"
|
|
|
|
|
case "$TRASH_DIRECTORY" in
|
|
|
|
|
/*) ;; # absolute path is good
|
|
|
|
|
*) TRASH_DIRECTORY="$TEST_OUTPUT_DIRECTORY/$TRASH_DIRECTORY" ;;
|
2009-08-09 16:39:45 +08:00
|
|
|
esac
|
2013-04-15 00:34:56 +08:00
|
|
|
rm -fr "$TRASH_DIRECTORY" || {
|
2009-06-01 20:14:41 +08:00
|
|
|
GIT_EXIT_OK=t
|
2008-03-19 12:58:01 +08:00
|
|
|
echo >&5 "FATAL: Cannot prepare test area"
|
|
|
|
|
exit 1
|
|
|
|
|
}
|
|
|
|
|
|
2011-03-27 02:46:34 +08:00
|
|
|
HOME="$TRASH_DIRECTORY"
|
2014-09-16 05:59:00 +08:00
|
|
|
GNUPGHOME="$HOME/gnupg-home-not-used"
|
|
|
|
|
export HOME GNUPGHOME
|
2011-03-27 02:46:34 +08:00
|
|
|
|
2012-09-02 02:11:49 +08:00
|
|
|
if test -z "$TEST_NO_CREATE_REPO"
|
|
|
|
|
then
|
2013-04-15 00:34:56 +08:00
|
|
|
test_create_repo "$TRASH_DIRECTORY"
|
2012-02-17 18:25:09 +08:00
|
|
|
else
|
2013-04-15 00:34:56 +08:00
|
|
|
mkdir -p "$TRASH_DIRECTORY"
|
2012-02-17 18:25:09 +08:00
|
|
|
fi
|
2008-06-01 05:11:21 +08:00
|
|
|
# Use -P to resolve symlinks in our working directory so that the cwd
|
|
|
|
|
# in subprocesses like git equals our $PWD (for pathname comparisons).
|
2013-04-15 00:34:56 +08:00
|
|
|
cd -P "$TRASH_DIRECTORY" || exit 1
|
2006-12-29 09:58:00 +08:00
|
|
|
|
2009-02-06 03:59:27 +08:00
|
|
|
this_test=${0##*/}
|
|
|
|
|
this_test=${this_test%%-*}
|
2013-06-18 20:25:58 +08:00
|
|
|
if match_pattern_list "$this_test" $GIT_SKIP_TESTS
|
|
|
|
|
then
|
|
|
|
|
say_color info >&3 "skipping test $this_test altogether"
|
|
|
|
|
skip_all="skip all tests in $this_test"
|
|
|
|
|
test_done
|
|
|
|
|
fi
|
2009-03-12 04:17:26 +08:00
|
|
|
|
2009-08-29 06:32:41 +08:00
|
|
|
# Provide an implementation of the 'yes' utility
|
|
|
|
|
yes () {
|
|
|
|
|
if test $# = 0
|
|
|
|
|
then
|
|
|
|
|
y=y
|
|
|
|
|
else
|
|
|
|
|
y="$*"
|
|
|
|
|
fi
|
|
|
|
|
|
2016-02-03 02:15:53 +08:00
|
|
|
i=0
|
|
|
|
|
while test $i -lt 99
|
2009-08-29 06:32:41 +08:00
|
|
|
do
|
2016-02-03 02:15:53 +08:00
|
|
|
echo "$y"
|
|
|
|
|
i=$(($i+1))
|
2009-08-29 06:32:41 +08:00
|
|
|
done
|
|
|
|
|
}
|
|
|
|
|
|
2009-03-12 04:17:26 +08:00
|
|
|
# Fix some commands on Windows
|
2016-07-22 00:02:54 +08:00
|
|
|
uname_s=$(uname -s)
|
|
|
|
|
case $uname_s in
|
2009-03-12 04:17:26 +08:00
|
|
|
*MINGW*)
|
|
|
|
|
# Windows has its own (incompatible) sort and find
|
|
|
|
|
sort () {
|
|
|
|
|
/usr/bin/sort "$@"
|
|
|
|
|
}
|
|
|
|
|
find () {
|
|
|
|
|
/usr/bin/find "$@"
|
|
|
|
|
}
|
2009-03-15 05:21:27 +08:00
|
|
|
sum () {
|
|
|
|
|
md5sum "$@"
|
|
|
|
|
}
|
2009-03-14 06:35:24 +08:00
|
|
|
# git sees Windows-style pwd
|
|
|
|
|
pwd () {
|
|
|
|
|
builtin pwd -W
|
|
|
|
|
}
|
2009-03-14 05:55:27 +08:00
|
|
|
# no POSIX permissions
|
2009-03-14 06:00:15 +08:00
|
|
|
# backslashes in pathspec are converted to '/'
|
2009-03-25 20:21:15 +08:00
|
|
|
# exec does not inherit the PID
|
2010-09-12 17:37:24 +08:00
|
|
|
test_set_prereq MINGW
|
2014-08-31 05:39:07 +08:00
|
|
|
test_set_prereq NATIVE_CRLF
|
2010-12-15 02:32:12 +08:00
|
|
|
test_set_prereq SED_STRIPS_CR
|
2013-07-19 05:44:57 +08:00
|
|
|
test_set_prereq GREP_STRIPS_CR
|
2013-10-27 03:17:15 +08:00
|
|
|
GIT_TEST_CMP=mingw_test_cmp
|
2010-12-15 02:32:12 +08:00
|
|
|
;;
|
|
|
|
|
*CYGWIN*)
|
|
|
|
|
test_set_prereq POSIXPERM
|
|
|
|
|
test_set_prereq EXECKEEPSPID
|
2013-01-27 11:11:11 +08:00
|
|
|
test_set_prereq CYGWIN
|
2010-12-15 02:32:12 +08:00
|
|
|
test_set_prereq SED_STRIPS_CR
|
2013-07-19 05:44:57 +08:00
|
|
|
test_set_prereq GREP_STRIPS_CR
|
2009-03-14 05:55:27 +08:00
|
|
|
;;
|
|
|
|
|
*)
|
|
|
|
|
test_set_prereq POSIXPERM
|
2009-03-14 06:00:15 +08:00
|
|
|
test_set_prereq BSLASHPSPEC
|
2009-03-25 20:21:15 +08:00
|
|
|
test_set_prereq EXECKEEPSPID
|
2009-03-12 04:17:26 +08:00
|
|
|
;;
|
|
|
|
|
esac
|
2009-03-05 05:38:24 +08:00
|
|
|
|
2012-04-27 17:25:25 +08:00
|
|
|
( COLUMNS=1 && test $COLUMNS = 1 ) && test_set_prereq COLUMNS_CAN_BE_1
|
2009-04-04 03:33:59 +08:00
|
|
|
test -z "$NO_PERL" && test_set_prereq PERL
|
2009-11-18 09:42:31 +08:00
|
|
|
test -z "$NO_PYTHON" && test_set_prereq PYTHON
|
2011-05-10 05:52:07 +08:00
|
|
|
test -n "$USE_LIBPCRE" && test_set_prereq LIBPCRE
|
i18n: add infrastructure for translating Git with gettext
Change the skeleton implementation of i18n in Git to one that can show
localized strings to users for our C, Shell and Perl programs using
either GNU libintl or the Solaris gettext implementation.
This new internationalization support is enabled by default. If
gettext isn't available, or if Git is compiled with
NO_GETTEXT=YesPlease, Git falls back on its current behavior of
showing interface messages in English. When using the autoconf script
we'll auto-detect if the gettext libraries are installed and act
appropriately.
This change is somewhat large because as well as adding a C, Shell and
Perl i18n interface we're adding a lot of tests for them, and for
those tests to work we need a skeleton PO file to actually test
translations. A minimal Icelandic translation is included for this
purpose. Icelandic includes multi-byte characters which makes it easy
to test various edge cases, and it's a language I happen to
understand.
The rest of the commit message goes into detail about various
sub-parts of this commit.
= Installation
Gettext .mo files will be installed and looked for in the standard
$(prefix)/share/locale path. GIT_TEXTDOMAINDIR can also be set to
override that, but that's only intended to be used to test Git itself.
= Perl
Perl code that's to be localized should use the new Git::I18n
module. It imports a __ function into the caller's package by default.
Instead of using the high level Locale::TextDomain interface I've
opted to use the low-level (equivalent to the C interface)
Locale::Messages module, which Locale::TextDomain itself uses.
Locale::TextDomain does a lot of redundant work we don't need, and
some of it would potentially introduce bugs. It tries to set the
$TEXTDOMAIN based on package of the caller, and has its own
hardcoded paths where it'll search for messages.
I found it easier just to completely avoid it rather than try to
circumvent its behavior. In any case, this is an issue wholly
internal Git::I18N. Its guts can be changed later if that's deemed
necessary.
See <AANLkTilYD_NyIZMyj9dHtVk-ylVBfvyxpCC7982LWnVd@mail.gmail.com> for
a further elaboration on this topic.
= Shell
Shell code that's to be localized should use the git-sh-i18n
library. It's basically just a wrapper for the system's gettext.sh.
If gettext.sh isn't available we'll fall back on gettext(1) if it's
available. The latter is available without the former on Solaris,
which has its own non-GNU gettext implementation. We also need to
emulate eval_gettext() there.
If neither are present we'll use a dumb printf(1) fall-through
wrapper.
= About libcharset.h and langinfo.h
We use libcharset to query the character set of the current locale if
it's available. I.e. we'll use it instead of nl_langinfo if
HAVE_LIBCHARSET_H is set.
The GNU gettext manual recommends using langinfo.h's
nl_langinfo(CODESET) to acquire the current character set, but on
systems that have libcharset.h's locale_charset() using the latter is
either saner, or the only option on those systems.
GNU and Solaris have a nl_langinfo(CODESET), FreeBSD can use either,
but MinGW and some others need to use libcharset.h's locale_charset()
instead.
=Credits
This patch is based on work by Jeff Epler <jepler@unpythonic.net> who
did the initial Makefile / C work, and a lot of comments from the Git
mailing list, including Jonathan Nieder, Jakub Narebski, Johannes
Sixt, Erik Faye-Lund, Peter Krefting, Junio C Hamano, Thomas Rast and
others.
[jc: squashed a small Makefile fix from Ramsay]
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-11-18 07:14:42 +08:00
|
|
|
test -z "$NO_GETTEXT" && test_set_prereq GETTEXT
|
2009-04-04 03:33:59 +08:00
|
|
|
|
2011-02-23 07:41:21 +08:00
|
|
|
# Can we rely on git's output in the C locale?
|
2011-02-23 07:41:22 +08:00
|
|
|
if test -n "$GETTEXT_POISON"
|
|
|
|
|
then
|
|
|
|
|
GIT_GETTEXT_POISON=YesPlease
|
|
|
|
|
export GIT_GETTEXT_POISON
|
i18n: add infrastructure for translating Git with gettext
Change the skeleton implementation of i18n in Git to one that can show
localized strings to users for our C, Shell and Perl programs using
either GNU libintl or the Solaris gettext implementation.
This new internationalization support is enabled by default. If
gettext isn't available, or if Git is compiled with
NO_GETTEXT=YesPlease, Git falls back on its current behavior of
showing interface messages in English. When using the autoconf script
we'll auto-detect if the gettext libraries are installed and act
appropriately.
This change is somewhat large because as well as adding a C, Shell and
Perl i18n interface we're adding a lot of tests for them, and for
those tests to work we need a skeleton PO file to actually test
translations. A minimal Icelandic translation is included for this
purpose. Icelandic includes multi-byte characters which makes it easy
to test various edge cases, and it's a language I happen to
understand.
The rest of the commit message goes into detail about various
sub-parts of this commit.
= Installation
Gettext .mo files will be installed and looked for in the standard
$(prefix)/share/locale path. GIT_TEXTDOMAINDIR can also be set to
override that, but that's only intended to be used to test Git itself.
= Perl
Perl code that's to be localized should use the new Git::I18n
module. It imports a __ function into the caller's package by default.
Instead of using the high level Locale::TextDomain interface I've
opted to use the low-level (equivalent to the C interface)
Locale::Messages module, which Locale::TextDomain itself uses.
Locale::TextDomain does a lot of redundant work we don't need, and
some of it would potentially introduce bugs. It tries to set the
$TEXTDOMAIN based on package of the caller, and has its own
hardcoded paths where it'll search for messages.
I found it easier just to completely avoid it rather than try to
circumvent its behavior. In any case, this is an issue wholly
internal Git::I18N. Its guts can be changed later if that's deemed
necessary.
See <AANLkTilYD_NyIZMyj9dHtVk-ylVBfvyxpCC7982LWnVd@mail.gmail.com> for
a further elaboration on this topic.
= Shell
Shell code that's to be localized should use the git-sh-i18n
library. It's basically just a wrapper for the system's gettext.sh.
If gettext.sh isn't available we'll fall back on gettext(1) if it's
available. The latter is available without the former on Solaris,
which has its own non-GNU gettext implementation. We also need to
emulate eval_gettext() there.
If neither are present we'll use a dumb printf(1) fall-through
wrapper.
= About libcharset.h and langinfo.h
We use libcharset to query the character set of the current locale if
it's available. I.e. we'll use it instead of nl_langinfo if
HAVE_LIBCHARSET_H is set.
The GNU gettext manual recommends using langinfo.h's
nl_langinfo(CODESET) to acquire the current character set, but on
systems that have libcharset.h's locale_charset() using the latter is
either saner, or the only option on those systems.
GNU and Solaris have a nl_langinfo(CODESET), FreeBSD can use either,
but MinGW and some others need to use libcharset.h's locale_charset()
instead.
=Credits
This patch is based on work by Jeff Epler <jepler@unpythonic.net> who
did the initial Makefile / C work, and a lot of comments from the Git
mailing list, including Jonathan Nieder, Jakub Narebski, Johannes
Sixt, Erik Faye-Lund, Peter Krefting, Junio C Hamano, Thomas Rast and
others.
[jc: squashed a small Makefile fix from Ramsay]
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-11-18 07:14:42 +08:00
|
|
|
test_set_prereq GETTEXT_POISON
|
2011-02-23 07:41:22 +08:00
|
|
|
else
|
|
|
|
|
test_set_prereq C_LOCALE_OUTPUT
|
|
|
|
|
fi
|
2011-02-23 07:41:21 +08:00
|
|
|
|
2011-04-13 02:23:23 +08:00
|
|
|
# Use this instead of test_cmp to compare files that contain expected and
|
|
|
|
|
# actual output from git commands that can be translated. When running
|
|
|
|
|
# under GETTEXT_POISON this pretends that the command produced expected
|
|
|
|
|
# results.
|
|
|
|
|
test_i18ncmp () {
|
|
|
|
|
test -n "$GETTEXT_POISON" || test_cmp "$@"
|
|
|
|
|
}
|
|
|
|
|
|
2011-04-13 06:57:08 +08:00
|
|
|
# Use this instead of "grep expected-string actual" to see if the
|
|
|
|
|
# output from a git command that can be translated either contains an
|
|
|
|
|
# expected string, or does not contain an unwanted one. When running
|
|
|
|
|
# under GETTEXT_POISON this pretends that the command produced expected
|
|
|
|
|
# results.
|
|
|
|
|
test_i18ngrep () {
|
|
|
|
|
if test -n "$GETTEXT_POISON"
|
|
|
|
|
then
|
|
|
|
|
: # pretend success
|
|
|
|
|
elif test "x!" = "x$1"
|
|
|
|
|
then
|
|
|
|
|
shift
|
|
|
|
|
! grep "$@"
|
|
|
|
|
else
|
|
|
|
|
grep "$@"
|
|
|
|
|
fi
|
|
|
|
|
}
|
|
|
|
|
|
2013-04-11 10:07:04 +08:00
|
|
|
test_lazy_prereq PIPE '
|
|
|
|
|
# test whether the filesystem supports FIFOs
|
2013-07-05 06:04:30 +08:00
|
|
|
case $(uname -s) in
|
2016-01-28 00:19:43 +08:00
|
|
|
CYGWIN*|MINGW*)
|
2013-07-05 06:04:30 +08:00
|
|
|
false
|
|
|
|
|
;;
|
|
|
|
|
*)
|
|
|
|
|
rm -f testfifo && mkfifo testfifo
|
|
|
|
|
;;
|
|
|
|
|
esac
|
2013-04-11 10:07:04 +08:00
|
|
|
'
|
|
|
|
|
|
2012-07-27 06:50:45 +08:00
|
|
|
test_lazy_prereq SYMLINKS '
|
|
|
|
|
# test whether the filesystem supports symbolic links
|
|
|
|
|
ln -s x y && test -h y
|
|
|
|
|
'
|
2010-08-07 06:09:09 +08:00
|
|
|
|
2013-11-26 05:02:16 +08:00
|
|
|
test_lazy_prereq FILEMODE '
|
|
|
|
|
test "$(git config --bool core.filemode)" = true
|
|
|
|
|
'
|
|
|
|
|
|
2012-07-26 21:39:53 +08:00
|
|
|
test_lazy_prereq CASE_INSENSITIVE_FS '
|
|
|
|
|
echo good >CamelCase &&
|
|
|
|
|
echo bad >camelcase &&
|
|
|
|
|
test "$(cat CamelCase)" != good
|
|
|
|
|
'
|
|
|
|
|
|
2012-07-26 21:39:56 +08:00
|
|
|
test_lazy_prereq UTF8_NFD_TO_NFC '
|
|
|
|
|
# check whether FS converts nfd unicode to nfc
|
|
|
|
|
auml=$(printf "\303\244")
|
|
|
|
|
aumlcdiar=$(printf "\141\314\210")
|
|
|
|
|
>"$auml" &&
|
|
|
|
|
case "$(echo *)" in
|
|
|
|
|
"$aumlcdiar")
|
|
|
|
|
true ;;
|
|
|
|
|
*)
|
|
|
|
|
false ;;
|
|
|
|
|
esac
|
|
|
|
|
'
|
|
|
|
|
|
2012-11-15 08:33:40 +08:00
|
|
|
test_lazy_prereq AUTOIDENT '
|
|
|
|
|
sane_unset GIT_AUTHOR_NAME &&
|
|
|
|
|
sane_unset GIT_AUTHOR_EMAIL &&
|
|
|
|
|
git var GIT_AUTHOR_IDENT
|
|
|
|
|
'
|
|
|
|
|
|
2014-06-10 03:23:30 +08:00
|
|
|
test_lazy_prereq EXPENSIVE '
|
|
|
|
|
test -n "$GIT_TEST_LONG"
|
|
|
|
|
'
|
|
|
|
|
|
2014-06-10 04:54:25 +08:00
|
|
|
test_lazy_prereq USR_BIN_TIME '
|
|
|
|
|
test -x /usr/bin/time
|
|
|
|
|
'
|
|
|
|
|
|
2015-01-16 17:16:49 +08:00
|
|
|
test_lazy_prereq NOT_ROOT '
|
|
|
|
|
uid=$(id -u) &&
|
|
|
|
|
test "$uid" != 0
|
|
|
|
|
'
|
|
|
|
|
|
2016-09-10 01:36:28 +08:00
|
|
|
test_lazy_prereq JGIT '
|
|
|
|
|
type jgit
|
|
|
|
|
'
|
|
|
|
|
|
2016-01-20 06:09:37 +08:00
|
|
|
# SANITY is about "can you correctly predict what the filesystem would
|
|
|
|
|
# do by only looking at the permission bits of the files and
|
|
|
|
|
# directories?" A typical example of !SANITY is running the test
|
|
|
|
|
# suite as root, where a test may expect "chmod -r file && cat file"
|
|
|
|
|
# to fail because file is supposed to be unreadable after a successful
|
|
|
|
|
# chmod. In an environment (i.e. combination of what filesystem is
|
|
|
|
|
# being used and who is running the tests) that lacks SANITY, you may
|
|
|
|
|
# be able to delete or create a file when the containing directory
|
|
|
|
|
# doesn't have write permissions, or access a file even if the
|
|
|
|
|
# containing directory doesn't have read or execute permissions.
|
|
|
|
|
|
2015-01-27 23:39:01 +08:00
|
|
|
test_lazy_prereq SANITY '
|
|
|
|
|
mkdir SANETESTD.1 SANETESTD.2 &&
|
|
|
|
|
|
|
|
|
|
chmod +w SANETESTD.1 SANETESTD.2 &&
|
|
|
|
|
>SANETESTD.1/x 2>SANETESTD.2/x &&
|
|
|
|
|
chmod -w SANETESTD.1 &&
|
2016-01-20 06:09:37 +08:00
|
|
|
chmod -r SANETESTD.1/x &&
|
2015-01-27 23:39:01 +08:00
|
|
|
chmod -rx SANETESTD.2 ||
|
|
|
|
|
error "bug in test sript: cannot prepare SANETESTD"
|
|
|
|
|
|
2016-01-20 06:09:37 +08:00
|
|
|
! test -r SANETESTD.1/x &&
|
2015-01-27 23:39:01 +08:00
|
|
|
! rm SANETESTD.1/x && ! test -f SANETESTD.2/x
|
|
|
|
|
status=$?
|
|
|
|
|
|
|
|
|
|
chmod +rwx SANETESTD.1 SANETESTD.2 &&
|
|
|
|
|
rm -rf SANETESTD.1 SANETESTD.2 ||
|
|
|
|
|
error "bug in test sript: cannot clean SANETESTD"
|
|
|
|
|
return $status
|
|
|
|
|
'
|
2013-03-11 09:31:47 +08:00
|
|
|
|
2016-07-22 00:02:54 +08:00
|
|
|
test FreeBSD != $uname_s || GIT_UNZIP=${GIT_UNZIP:-/usr/local/bin/unzip}
|
2013-03-11 09:31:47 +08:00
|
|
|
GIT_UNZIP=${GIT_UNZIP:-unzip}
|
|
|
|
|
test_lazy_prereq UNZIP '
|
|
|
|
|
"$GIT_UNZIP" -v
|
|
|
|
|
test $? -ne 127
|
|
|
|
|
'
|
2015-03-13 12:53:07 +08:00
|
|
|
|
|
|
|
|
run_with_limited_cmdline () {
|
|
|
|
|
(ulimit -s 128 && "$@")
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
test_lazy_prereq CMDLINE_LIMIT 'run_with_limited_cmdline true'
|
2016-07-12 07:54:18 +08:00
|
|
|
|
|
|
|
|
build_option () {
|
|
|
|
|
git version --build-options |
|
|
|
|
|
sed -ne "s/^$1: //p"
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
test_lazy_prereq LONG_IS_64BIT '
|
|
|
|
|
test 8 -le "$(build_option sizeof-long)"
|
|
|
|
|
'
|
t0006 & t5000: prepare for 64-bit timestamps
Git's source code refers to timestamps as unsigned longs. On 32-bit
platforms, as well as on Windows, unsigned long is not large enough to
capture dates that are "absurdly far in the future".
It is perfectly valid by the C standard, of course, for the `long` data
type to refer to 32-bit integers. That is why the `time_t` data type
exists: so that it can be 64-bit even if `long` is 32-bit. Git's source
code simply uses an incorrect data type for timestamps, is all.
The earlier quick fix 6b9c38e14cd (t0006: skip "far in the future" test
when unsigned long is not long enough, 2016-07-11) papered over this
issue simply by skipping the respective test cases on platforms where
they would fail due to the data type in use.
This quick fix, however, tests for *long* to be 64-bit or not. What we
need, though, is a test that says whether *whatever data type we use for
timestamps* is 64-bit or not.
The same quick fix was used to handle the similar problem where Git's
source code uses `unsigned long` to represent size, instead of `size_t`,
conflating the two issues.
So let's just add another prerequisite to test specifically whether
timestamps are represented by a 64-bit data type or not. Later, after we
switch to a larger data type, we can flip that prerequisite to test
`time_t` instead of `long`.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-21 04:52:13 +08:00
|
|
|
|
|
|
|
|
test_lazy_prereq TIME_IS_64BIT 'test-date is64bit'
|
2017-04-21 04:58:21 +08:00
|
|
|
test_lazy_prereq TIME_T_IS_64BIT 'test-date time_t-is64bit'
|
