Commit Graph

95 Commits

Author SHA1 Message Date
Junio C Hamano
5002702e48 Merge branch 'js/test-unset-prereq'
Test debugging aid.

* js/test-unset-prereq:
  tests: introduce test_unset_prereq, for debugging
2018-05-23 14:38:17 +09:00
Johannes Schindelin
7d0ee47c11 tests: introduce test_unset_prereq, for debugging
While working on the --convert-graft-file test, I missed that I was
relying on the GPG prereq, by using output of test cases that were only
run under that prereq.

For debugging, it was really convenient to force that prereq to be
unmet, but there was no easy way to do that. So I came up with a way,
and this patch reflects the cleaned-up version of that way.

For convenience, the following two methods are now supported ways to
pretend that a prereq is not met:

	test_set_prereq !GPG

and

	test_unset_prereq GPG

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-30 09:38:51 +09:00
Elijah Newren
842436466a Make running git under other debugger-like programs easy
This allows us to run git, when using the script from bin-wrappers, under
other programs.  A few examples for usage within testsuite scripts:

   debug git checkout master
   debug --debugger=nemiver git $ARGS
   debug -d "valgrind --tool-memcheck --track-origins=yes" git $ARGS

Or, if someone has bin-wrappers/ in their $PATH and is executing git
outside the testsuite:

   GIT_DEBUGGER="gdb --args" git $ARGS
   GIT_DEBUGGER=nemiver git $ARGS
   GIT_DEBUGGER="valgrind --tool=memcheck --track-origins=yes" git $ARGS

There is also a handy shortcut of GIT_DEBUGGER=1 meaning the same as
GIT_DEBUGGER="gdb --args"

Original-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-25 10:47:22 +09:00
Junio C Hamano
5ff42d42da Merge branch 'jc/test-must-be-empty'
Test helper update.

* jc/test-must-be-empty:
  test_must_be_empty: simplify file existence check
2018-04-11 13:09:54 +09:00
SZEDER Gábor
9eb2308019 test_must_be_empty: simplify file existence check
Commit 11395a3b4b (test_must_be_empty: make sure the file exists, not
just empty, 2018-02-27) basically duplicated the 'test_path_is_file'
helper function in 'test_must_be_empty'.

Just call 'test_path_is_file' to avoid this code duplication.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-27 17:08:26 -07:00
Junio C Hamano
571e472dc4 Merge branch 'sg/test-x'
Running test scripts under -x option of the shell is often not a
useful way to debug them, because the error messages from the
commands tests try to capture and inspect are contaminated by the
tracing output by the shell.  An earlier work done to make it more
pleasant to run tests under -x with recent versions of bash is
extended to cover posix shells that do not support BASH_XTRACEFD.

* sg/test-x:
  travis-ci: run tests with '-x' tracing
  t/README: add a note about don't saving stderr of compound commands
  t1510-repo-setup: mark as untraceable with '-x'
  t9903-bash-prompt: don't check the stderr of __git_ps1()
  t5570-git-daemon: don't check the stderr of a subshell
  t5526: use $TRASH_DIRECTORY to specify the path of GIT_TRACE log file
  t5500-fetch-pack: don't check the stderr of a subshell
  t3030-merge-recursive: don't check the stderr of a subshell
  t1507-rev-parse-upstream: don't check the stderr of a shell function
  t: add means to disable '-x' tracing for individual test scripts
  t: prevent '-x' tracing from interfering with test helpers' stderr
2018-03-14 12:01:03 -07:00
Junio C Hamano
a8d45dcfc0 Merge branch 'jc/test-must-be-empty'
Test framework tweak to catch developer thinko.

* jc/test-must-be-empty:
  test_must_be_empty: make sure the file exists, not just empty
2018-03-08 12:36:28 -08:00
Junio C Hamano
148bce96e5 Merge branch 'jk/test-helper-v-output-fix'
Test framework update.

* jk/test-helper-v-output-fix:
  t: send verbose test-helper output to fd 4
2018-03-06 14:54:05 -08:00
Junio C Hamano
11395a3b4b test_must_be_empty: make sure the file exists, not just empty
The helper function test_must_be_empty is meant to make sure the
given file is empty, but its implementation is:

	if test -s "$1"
	then
		... not empty, we detected a failure ...
	fi

Surely, the file having non-zero size is a sign that the condition
"the file must be empty" is violated, but it misses the case where
the file does not even exist.  It is an accident waiting to happen
with a buggy test like this:

	git frotz 2>error-message &&
	test_must_be_empty errro-message

that won't get caught until you deliberately break 'git frotz' and
notice why the test does not fail.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-27 13:58:43 -08:00
SZEDER Gábor
a5bf824f3b t: prevent '-x' tracing from interfering with test helpers' stderr
Running a test script with '-x' turns on 'set -x' tracing, the output
of which is normally sent to stderr.  This causes a lot of
test failures, because many tests redirect and verify the stderr
of shell functions, most frequently that of 'test_must_fail'.
These issues were worked around somewhat in d88785e424 (test-lib: set
BASH_XTRACEFD automatically, 2016-05-11), so at least we could
reliably run tests with '-x' tracing under a Bash version supporting
BASH_XTRACEFD, i.e. v4.1 and later.

Futhermore, redirecting the stderr of test helper functions like
'test_must_fail' or 'test_expect_code' is the cause of a different
issue as well.  If these functions detect something unexpected, they
will write their error messages intended to the user to thier stderr.
However, if their stderr is redirected in order to save and verify the
stderr of the tested git command invoked in the function, then the
function's error messages will be redirected as well.  Consequently,
those messages won't reach the user, making the test's verbose output
less useful.

This patch makes it safe to redirect and verify the stderr of those
test helper functions which are meant to run the tested command given
as argument, even when running tests with '-x' and /bin/sh.  This is
achieved through a couple of file descriptor redirections:

  - Duplicate stderr of the tested command executed in the test helper
    function from the function's fd 7 (see next point), to ensure that
    the tested command's error messages go to a different fd than the
    '-x' trace of the commands executed in the function or the
    function's error messages.

  - Duplicate the test helper function's fd 7 from the function's
    original stderr, meaning that, after taking a detour through fd 7,
    the error messages of the tested command do end up on the
    function's original stderr.

  - Duplicate stderr of the test helper function from fd 4, i.e. the
    fd connected to the test script's original stderr and the fd used
    for BASH_XTRACEFD.  This ensures that the '-x' trace of the
    commands executed in the function

      - doesn't go to the function's original stderr, so it won't mess
	with callers who want to save and verify the tested command's
	stderr.

      - does go to the same fd independently from the shell running
        the test script, be it /bin/sh, an older Bash without
        BASH_XTRACEFD, or a more recent Bash already supporting
        BASH_XTRACEFD.

    Furthermore, this also makes sure that the function's error
    messages go to this fd 4, meaning that the user will be able to
    see them even if the function's stderr is redirected in the test.

  - Specify the latter two redirections above in the test helper
    function's definition, so they are performed every time the
    function is invoked, without the need to modify the callsites of
    the function.

Perform these redirections in those test helper functions which can be
expected to have their stderr redirected, i.e. in the functions
'test_must_fail', 'test_might_fail', 'test_expect_code', 'test_env',
'nongit', 'test_terminal' and 'perl'.  Note that 'test_might_fail',
'test_env', and 'nongit' are not involved in any test failures when
running tests with '-x' and /bin/sh.

The other test helper functions are left unchanged, because they
either don't run commands specified as their arguments, or redirecting
their stderr wouldn't make sense, or both.

With this change the number of failures when running the test suite
with '-x' tracing and /bin/sh goes down from 340 failed tests in 43
test scripts to 22 failed tests in 6 scripts (or 23 in 7, if the
system (OSX) uses an older Bash version without BASH_XTRACEFD to run
't9903-bash-prompt.sh').

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-27 12:43:13 -08:00
Junio C Hamano
e3eb405027 Merge branch 'sg/doc-test-must-fail-args'
Devdoc update.

* sg/doc-test-must-fail-args:
  t: document 'test_must_fail ok=<signal-name>'
2018-02-27 10:33:58 -08:00
Jeff King
03aa3783f2 t: send verbose test-helper output to fd 4
Test helper functions like test_must_fail may produce
messages to stderr when they see a problem. When the tests
are run with "--verbose", this ends up on the test script's
stderr, and the user can read it.

But there's a problem. Some tests record stderr as part of
the test, like:

  test_must_fail git foo 2>output &&
  test_i18ngrep expected.message output

In this case the error text goes into "output". This makes
the --verbose output less useful (it also means we might
accidentally match it in the second, though in practice we
tend to produce these messages only on error, so we'd abort
the test when the first command fails).

Let's instead send this user-facing output directly to
descriptor 4, which always points to the original stderr (or
/dev/null in non-verbose mode). And it's already forbidden
to redirect descriptor 4, since we use it for BASH_XTRACEFD,
as explained in 9be795fbce (t5615: avoid re-using descriptor
4, 2017-12-08).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-22 12:17:39 -08:00
Junio C Hamano
66023bbd78 Merge branch 'sg/test-i18ngrep'
Test fixes.

* sg/test-i18ngrep:
  t: make 'test_i18ngrep' more informative on failure
  t: validate 'test_i18ngrep's parameters
  t: move 'test_i18ncmp' and 'test_i18ngrep' to 'test-lib-functions.sh'
  t5536: let 'test_i18ngrep' read the file without redirection
  t5510: consolidate 'grep' and 'test_i18ngrep' patterns
  t4001: don't run 'git status' upstream of a pipe
  t6022: don't run 'git merge' upstream of a pipe
  t5812: add 'test_i18ngrep's missing filename parameter
  t5541: add 'test_i18ngrep's missing filename parameter
2018-02-21 12:45:05 -08:00
SZEDER Gábor
12e31a6b12 t: document 'test_must_fail ok=<signal-name>'
Since 'test_might_fail' is implemented as a thin wrapper around
'test_must_fail', it also accepts the same options.  Mention this in
the docs as well.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-12 11:00:38 -08:00
SZEDER Gábor
63b1a175ee t: make 'test_i18ngrep' more informative on failure
When 'test_i18ngrep' can't find the expected pattern, it exits
completely silently; when its negated form does find the pattern that
shouldn't be there, it prints the matching line(s) but otherwise exits
without any error message.  This leaves the developer puzzled about
what could have gone wrong.

Make 'test_i18ngrep' more informative on failure by printing an error
message including the invoked 'grep' command and the contents of the
file it had to scan through.

Note that this "dump the scanned file" part is not quite perfect, as
it dumps only the file specified as the function's last positional
parameter, thus assuming that there is only a single file parameter.
I think that's a reasonable assumption to make, one that holds true in
the current code base.  And even if someone were to scan multiple
files at once in the future, the worst thing that could happen is that
the verbose error message won't include the contents of all those
files, only the last one.  Alas, we can't really do any better than
this, because checking whether the other positional parameters match a
filename can result in false positives: 't3400-rebase.sh' and
't3404-rebase-interactive.sh' contain one test each, where the
'test_i18ngrep's pattern verbatimly matches a file in the trash
directory.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-08 10:54:27 -08:00
SZEDER Gábor
fd29d7b9d7 t: validate 'test_i18ngrep's parameters
Some of the previous patches in this series fixed bogus
'test_i18ngrep' invocations:

  - Two invocations where the tested git command's standard output is
    directly piped into 'test_i18ngrep'.  While convenient, this is an
    antipattern, because the pipe hides the git command's exit code,
    and the test could continue even if the command exited with error.

  - Two invocations that had neither a filename parameter nor anything
    piped into their standard input, yet both managed to remain
    unnoticed for years.  A third similarly bogus invocation is
    currently lurking in 'pu' for a couple of weeks now.

Prevent similar mistakes in the future by validating 'test_i18ngrep's
parameters requiring that

  - The last parameter names an existing file to be read, effectively
    forbidding piping into 'test_i18ngrep'.

    Note that this change will also forbid cases where 'test_i18ngrep'
    would legitimately read its standard input, e.g. when its standard
    input is redirected from a file, or when a git command's standard
    output is first written to an intermediate file, which is then
    preprocessed by a non-git command before the results are piped
    into 'test_i18ngrep'.  See two of the previous patches for the
    only such cases we had in our test suite.  However, reliably
    preventing the piping antipattern is arguably more important than
    supporting these cases, which can be easily worked around by
    opening the file directly or using an intermediate file anyway.

  - There are at least two parameters, not including the optional '!'
    to negate the pattern.  This ought to catch corner cases when
    'test_i18ngrep' looks for the name of an existing file on its
    standard input; the above check would miss this case becase the
    filename as pattern would be the last parameter.

    Note that this is not quite perfect, as it doesn't account for any
    'grep --options' given as parameters.  However, doing so would be
    far too complicated, considering that patterns can start with
    dashes as well, and in the majority of the cases we don't use any
    such options anyway.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-08 10:54:27 -08:00
SZEDER Gábor
0f59128f7b t: move 'test_i18ncmp' and 'test_i18ngrep' to 'test-lib-functions.sh'
Both 'test_i18ncmp' and 'test_i18ngrep' helper functions are supposed
to be called from our test scripts, so they should be in
'test-lib-functions.sh'.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-08 10:54:27 -08:00
Jeff King
4414a15002 t/lib-git-daemon: add network-protocol helpers
All of our git-protocol tests rely on invoking the client
and having it make a request of a server. That gives a nice
real-world test of how the two behave together, but it
doesn't leave any room for testing how a server might react
to _other_ clients.

Let's add a few test helper functions which can be used to
manually conduct a git-protocol conversation with a remote
git-daemon:

  1. To connect to a remote git-daemon, we need something
     like "netcat". But not everybody will have netcat. And
     even if they do, the behavior with respect to
     half-duplex shutdowns is not portable (openbsd netcat
     has "-N", with others you must rely on "-q 1", which is
     racy).

     Here we provide a "fake_nc" that is capable of doing
     a client-side netcat, with sane half-duplex semantics.
     It relies on perl's IO::Socket::INET. That's been in
     the base distribution since 5.6.0, so it's probably
     available everywhere. But just to be on the safe side,
     we'll add a prereq.

  2. To help tests speak and read pktline, this patch adds
     packetize() and depacketize() functions.

I've put fake_nc() into lib-git-daemon.sh, since that's
really the only server where we'd need to use a network
socket.  Whereas the pktline helpers may be of more general
use, so I've added them to test-lib-functions.sh. Programs
like upload-pack speak pktline, but can talk directly over
stdio without a network socket.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-25 13:50:17 -08:00
Junio C Hamano
447f80f508 Merge branch 'jk/ref-filter-colors' into maint
"%C(color name)" in the pretty print format always produced ANSI
color escape codes, which was an early design mistake.  They now
honor the configuration (e.g. "color.ui = never") and also tty-ness
of the output medium.

* jk/ref-filter-colors:
  ref-filter: consult want_color() before emitting colors
  pretty: respect color settings for %C placeholders
  rev-list: pass diffopt->use_colors through to pretty-print
  for-each-ref: load config earlier
  color: check color.ui in git_default_config()
  ref-filter: pass ref_format struct to atom parsers
  ref-filter: factor out the parsing of sorting atoms
  ref-filter: make parse_ref_filter_atom a private function
  ref-filter: provide a function for parsing sort options
  ref-filter: move need_color_reset_at_eol into ref_format
  ref-filter: abstract ref format into its own struct
  ref-filter: simplify automatic color reset
  t: use test_decode_color rather than literal ANSI codes
  docs/for-each-ref: update pointer to color syntax
  check return value of verify_ref_format()
2017-08-23 14:33:42 -07:00
Junio C Hamano
15595ce438 Merge branch 'jk/ref-filter-colors'
"%C(color name)" in the pretty print format always produced ANSI
color escape codes, which was an early design mistake.  They now
honor the configuration (e.g. "color.ui = never") and also tty-ness
of the output medium.

* jk/ref-filter-colors:
  ref-filter: consult want_color() before emitting colors
  pretty: respect color settings for %C placeholders
  rev-list: pass diffopt->use_colors through to pretty-print
  for-each-ref: load config earlier
  color: check color.ui in git_default_config()
  ref-filter: pass ref_format struct to atom parsers
  ref-filter: factor out the parsing of sorting atoms
  ref-filter: make parse_ref_filter_atom a private function
  ref-filter: provide a function for parsing sort options
  ref-filter: move need_color_reset_at_eol into ref_format
  ref-filter: abstract ref format into its own struct
  ref-filter: simplify automatic color reset
  t: use test_decode_color rather than literal ANSI codes
  docs/for-each-ref: update pointer to color syntax
  check return value of verify_ref_format()
2017-08-11 13:26:58 -07:00
Jeff King
f7f6dc340e t: handle EOF in test_copy_bytes()
The test_copy_bytes() function claims to read up to N bytes,
or until it gets EOF. But we never handle EOF in our loop,
and a short input will cause perl to go into an infinite
loop of read() getting zero bytes.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-17 14:55:43 -07:00
Jeff King
097b681baa t: use test_decode_color rather than literal ANSI codes
When we put literal ANSI terminal codes into our test
scripts, it makes diffs on those scripts hard to read (the
colors may be indistinguishable from diff coloring, or in
the case of a reset, may not be visible at all).

Some scripts get around this by including human-readable
names and converting to literal codes with a git-config
hack. This makes the actual code diffs look OK, but test_cmp
output suffers from the same problem.

Let's use test_decode_color instead, which turns the codes
into obvious text tags.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-13 12:42:50 -07:00
Christian Couder
73de1c93ad t1301: move modebits() to test-lib-functions.sh
As the modebits() function can be useful outside t1301,
let's move it into test-lib-functions.sh, and while at
it let's rename it test_modebits().

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-25 10:42:52 -07:00
SZEDER Gábor
59210dd56c tests: make the 'test_pause' helper work in non-verbose mode
When the 'test_pause' helper function invokes the shell mid-test, it
explicitly redirects the shell's stdout and stderr to file descriptors
3 and 4, which are the stdout and stderr of the tests (i.e. where they
would be connected anyway without those redirections).  These file
descriptors are only attached to the terminal in verbose mode, hence
the restriction of 'test_pause' to work only with '-v'.

Redirect the shell's stdout and stderr to the test environment's
original stdout and stderr, allowing it to work properly even in
non-verbose mode, and the restriction can be lifted.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-18 10:18:22 -07:00
SZEDER Gábor
4ecae3c8c1 tests: create an interactive gdb session with the 'debug' helper
The 'debug' test helper is supposed to facilitate debugging by running
a command of the test suite under gdb.  Unfortunately, its usefulness
is severely limited, because that gdb session is not interactive,
since the test's, and thus gdb's standard input is redirected from
/dev/null (for a good reason, see 781f76b15 (test-lib: redirect stdin
of tests, 2011-12-15)).

Redirect gdb's standard file descriptors from/to the test
environment's stdin, stdout and stderr in the 'debug' helper, thus
creating an interactive gdb session (even in non-verbose mode), which
is much, much more useful.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-18 10:18:20 -07:00
Junio C Hamano
da2b74eeec Merge branch 'sb/submodule-embed-gitdir'
A new submodule helper "git submodule embedgitdirs" to make it
easier to move embedded .git/ directory for submodules in a
superproject to .git/modules/ (and point the latter with the former
that is turned into a "gitdir:" file) has been added.

* sb/submodule-embed-gitdir:
  worktree: initialize return value for submodule_uses_worktrees
  submodule: add absorb-git-dir function
  move connect_work_tree_and_git_dir to dir.h
  worktree: check if a submodule uses worktrees
  test-lib-functions.sh: teach test_commit -C <dir>
  submodule helper: support super prefix
  submodule: use absolute path for computing relative path connecting
2017-01-10 15:24:28 -08:00
Jeff King
de95302a4c t5000: extract nongit function to test-lib-functions.sh
This function abstracts the idea of running a command
outside of any repository (which is slightly awkward to do
because even if you make a non-repo directory, git may keep
walking up outside of the trash directory). There are
several scripts that use the same technique, so let's make
the function available for everyone.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-16 09:29:16 -08:00
Stefan Beller
6f94351b0a test-lib-functions.sh: teach test_commit -C <dir>
Specifically when setting up submodule tests, it comes in handy if
we can create commits in repositories that are not at the root of
the tested trash dir. Add "-C <dir>" similar to gits -C parameter
that will perform the operation in the given directory.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-09 14:52:57 -08:00
Jeff Hostetler
b249e39f99 test-lib-functions.sh: add lf_to_nul helper
Add lf_to_nul helper function to test-lib-functions.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-11 11:16:13 -07:00
Junio C Hamano
39cadeec0d Merge branch 'jk/test-match-signal'
The test framework learned a new helper test_match_signal to
check an exit code from getting killed by an expected signal.

* jk/test-match-signal:
  t/lib-git-daemon: use test_match_signal
  test_must_fail: use test_match_signal
  t0005: use test_match_signal as appropriate
  tests: factor portable signal check out of t0005
2016-07-19 13:22:20 -07:00
Jeff King
2472448c88 test_must_fail: use test_match_signal
In 8bf4bec (add "ok=sigpipe" to test_must_fail and use it to
fix flaky tests, 2015-11-27), test_must_fail learned to
recognize "141" as a sigpipe failure. However, testing for
a signal is more complicated than that; we should use
test_match_signal to implement more portable checking.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-06 07:44:25 -07:00
Jeff King
9b67c9942e tests: factor portable signal check out of t0005
In POSIX shells, a program which exits due to a signal
generally has an exit code of 128 plus the signal number.
However, ksh uses 256 plus the signal number.  We've
accounted for that in t0005, but not in other tests.  Let's
pull out the logic so we can use it elsewhere.

It would be nice for debugging if this additionally printed
errors to stderr, like our other test_* helpers. But we're
going to need to use it in other places besides the innards
of a test_expect block. So let's leave it as generic as
possible.

Note that we also leave the magic "3" for Windows out of the
generic helper. This is an artifact of the way we use
raise() to kill ourselves in test-sigchain.c, and will not
necessarily apply to all programs. So it's better to keep it
out of the helper, to reduce the chance of confusing it with
a real call to exit(3).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-06 07:43:29 -07:00
Jeff King
48860819e8 t9300: factor out portable "head -c" replacement
It is sometimes useful to be able to read exactly N bytes from a
pipe. Doing this portably turns out to be surprisingly difficult
in shell scripts.

We want a solution that:

  - is portable

  - never reads more than N bytes due to buffering (which
    would mean those bytes are not available to the next
    program to read from the same pipe)

  - handles partial reads by looping until N bytes are read
    (or we see EOF)

  - is resilient to stray signals giving us EINTR while
    trying to read (even though we don't send them, things
    like SIGWINCH could cause apparently-random failures)

Some possible solutions are:

  - "head -c" is not portable, and implementations may
    buffer (though GNU head does not)

  - "read -N" is a bash-ism, and thus not portable

  - "dd bs=$n count=1" does not handle partial reads. GNU dd
    has iflags=fullblock, but that is not portable

  - "dd bs=1 count=$n" fixes the partial read problem (all
    reads are 1-byte, so there can be no partial response).
    It does make a lot of write() calls, but for our tests
    that's unlikely to matter.  It's fairly portable. We
    already use it in our tests, and it's unlikely that
    implementations would screw up any of our criteria. The
    most unknown one would be signal handling.

  - perl can do a sysread() loop pretty easily. On my Linux
    system, at least, it seems to restart the read() call
    automatically. If that turns out not to be portable,
    though, it would be easy for us to handle it.

That makes the perl solution the least bad (because we
conveniently omitted "length of code" as a criterion).
It's also what t9300 is currently using, so we can just pull
the implementation from there.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-01 10:17:39 -07:00
Jeff King
d2554c7207 test-lib: add in-shell "env" replacement
The one-shot environment variable syntax:

  FOO=BAR some-program

is unportable when some-program is actually a shell
function, like test_must_fail (on some shells FOO remains
set after the function returns, and on others it does not).

We sometimes get around this by using env, like:

  test_must_fail env FOO=BAR some-program

But that only works because test_must_fail's arguments are
themselves a command which can be run. You can't run:

  env FOO=BAR test_must_fail some-program

because env does not know about our shell functions. So
there is no equivalent for test_commit, for example, and one
must resort to:

  (
    FOO=BAR
    export FOO
    test_commit
  )

which is a bit verbose.  Let's add a version of "env" that
works _inside_ the shell, by creating a subshell, exporting
variables from its argument list, and running the command.

Its use is demonstrated on a currently-unportable case in
t4014.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-01 08:04:08 -07:00
Junio C Hamano
6de6aba9f2 Merge branch 'jc/test-seq' into maint
Test fix.

* jc/test-seq:
  test-lib-functions.sh: rewrite test_seq without Perl
  test-lib-functions.sh: remove misleading comment on test_seq
2016-05-26 13:17:16 -07:00
Junio C Hamano
4df4313532 test-lib-functions.sh: rewrite test_seq without Perl
Rewrite the 'seq' imitation using only commands and features that
are typically found built into modern POSIX shells, instead of
relying on Perl to run a single-liner script.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-09 14:21:57 -07:00
Junio C Hamano
55672a39b4 test-lib-functions.sh: remove misleading comment on test_seq
We never used the "letters" form since we came up with "test_seq" to
replace use of non-portable "seq" in our test script, which we
introduced it at d17cf5f3 (tests: Introduce test_seq, 2012-08-04).

We use this helper to either iterate for N times (i.e. the values on
the lines do not even matter), or just to get N distinct strings
(i.e. the values on the lines themselves do not really matter, but
we care that they are different from each other and reproducible).

Stop promising that we may allow using "letters"; this would open an
easier reimplementation that does not rely on $PERL, if somebody
later wants to.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-09 12:32:42 -07:00
Jeff King
f3ed0b372d test_must_fail: report number of unexpected signal
If a command is marked as test_must_fail but dies with a
signal, we consider that a problem and report the error to
stderr. However, we don't say _which_ signal; knowing that
can make debugging easier. Let's share as much as we know.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-25 13:51:47 -08:00
Lars Schneider
8bf4becf0c add "ok=sigpipe" to test_must_fail and use it to fix flaky tests
t5516 "75 - deny fetch unreachable SHA1, allowtipsha1inwant=true" is
flaky in the following case:
1. remote upload-pack finds out "not our ref"
2. remote sends a response and closes the pipe
3. fetch-pack still tries to write commands to the remote upload-pack
4. write call in wrapper.c dies with SIGPIPE

The test is flaky because the sending fetch-pack may or may
not have finished writing its output by step (3). If it did,
then we see a closed pipe on the next read() call. If it
didn't, then we get the SIGPIPE from step (4) above. Both
are fine, but the latter fools test_must_fail.

t5504 "9 - push with transfer.fsckobjects" is flaky, too, and returns
SIGPIPE once in a while. I had to remove the final "To dst..." output
check because there is no output if the process dies with SIGPIPE.

Accept such a death-with-sigpipe also as OK when we are expecting a
failure.

Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
2015-11-28 12:06:14 -05:00
Lars Schneider
bbfe5302d5 implement test_might_fail using a refactored test_must_fail
Add an (optional) first parameter "ok=<special case>" to test_must_fail
and return success for "<special case>". Add "success" as
"<special case>" and use it to implement "test_might_fail". This removes
redundancies in test-lib-function.sh.

You can pass multiple <special case> arguments divided by comma (e.g.
"test_must_fail ok=success,something")

Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Jeff King <peff@peff.net>
2015-11-28 12:04:28 -05:00
Johannes Schindelin
6a94088cc3 test: facilitate debugging Git executables in tests with gdb
When prefixing a Git call in the test suite with 'debug ', it will
now be run with GDB, allowing the developer to debug test failures
more conveniently.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-10-30 14:02:38 -07:00
John Keeping
0968f12a99 test-lib-functions: detect test_when_finished in subshell
test_when_finished does nothing in a subshell because the change to
test_cleanup does not affect the parent.

There is no POSIX way to detect that we are in a subshell ($$ and $PPID
are specified to remain unchanged), but we can detect it on Bash and
fall back to ignoring the bug on other shells.

Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-08 10:35:05 -07:00
John Keeping
5fafc07fca test-lib-functions: support "test_config -C <dir> ..."
If used in a subshell, test_config cannot unset variables at the end of
a test.  This is a problem when testing submodules because we do not
want to "cd" at to top level of a test script in order to run the
command inside the submodule.

Add a "-C" option to test_config (and test_unconfig) so that test_config
can be kept outside subshells and still affect subrepositories.

Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-08 10:35:01 -07:00
Junio C Hamano
072f391c53 Merge branch 'jc/test-prereq-validate'
Help us to find broken test script that splits the body part of the
test by mistaken use of wrong kind of quotes.

* jc/test-prereq-validate:
  test: validate prerequistes syntax
2015-05-19 13:17:55 -07:00
Junio C Hamano
21b56b9259 Merge branch 'ep/fix-test-lib-functions-report' into maint
* ep/fix-test-lib-functions-report:
  test-lib-functions.sh: fix the second argument to some helper functions
2015-05-13 14:05:53 -07:00
Junio C Hamano
64c9e02765 Merge branch 'ep/fix-test-lib-functions-report'
* ep/fix-test-lib-functions-report:
  test-lib-functions.sh: fix the second argument to some helper functions
2015-05-05 21:00:36 -07:00
Junio C Hamano
d93d5d51e3 test: validate prerequistes syntax
Brian Carson noticed that a test piece in t5601 had a pair of single
quotes in the body, which made it into 4 parameter call to
test_expect_success, as if its test title were a prerequisite.

As the prerequisites have a specific syntax (i.e. comma separated
tokens spelled in capital letters, possibly prefixed with ! for
negation), validate them to catch such a mistake in the future.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-28 15:20:33 -07:00
Elia Pinto
de248e92c1 test-lib-functions.sh: fix the second argument to some helper functions
The second argument to test_path_is_file and test_path_is_dir
must be $2 and not $*, which instead would repeat the file
name in the error message.

Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
Reviewed-by: Matthieu Moy <Matthieu.Moy@grenoble-inp.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16 13:31:35 -07:00
Junio C Hamano
bb8f6de064 Merge branch 'jc/diff-test-updates' into maint
Test clean-up.

* jc/diff-test-updates:
  test_ln_s_add: refresh stat info of fake symbolic links
  t4008: modernise style
  t/diff-lib: check exact object names in compare_diff_raw
  tests: do not borrow from COPYING and README from the real source
  t4010: correct expected object names
  t9300: correct expected object names
  t4008: correct stale comments
2015-03-13 22:56:10 -07:00
Junio C Hamano
fa8baa4b2a Merge branch 'jc/diff-test-updates'
Test clean-up.

* jc/diff-test-updates:
  test_ln_s_add: refresh stat info of fake symbolic links
  t4008: modernise style
  t/diff-lib: check exact object names in compare_diff_raw
  tests: do not borrow from COPYING and README from the real source
  t4010: correct expected object names
  t9300: correct expected object names
  t4008: correct stale comments
2015-03-05 12:45:43 -08:00