Tests that test the test-lib.sh itself need to be executed in the
dynamically created trash directory, so we can't assume
$TEST_DIRECTORY is ../ for those.
As a side benefit this change also makes it easy for us to move the
t/*.sh tests into subdirectories if we ever want to do that.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change code that used $TEST_DIRECTORY/.. to use $GIT_BUILD_DIR
instead, the two are equivalent, but the latter is easier to read.
This required moving the assignment od GIT_BUILD_DIR to earlier in the
test-lib.sh file.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the redundant calls to $(pwd) to use $TEST_DIRECTORY
instead. None of these were being executed after we cd'd somewhere
else so they weren't actually needed.
This also makes it easier to add support for overriding the test
library location and run tests in a different directory than t/.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Introduce a new variable $GIT_BUILD_DIR which can be used to locate
data that resides under the build directory, and use that instead.
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some tests in t96xx series (cvsimport) want to write into the control area
(CVSROOT) of their test CVS repositories, but this does not work well when
the source area is made read-only (test trash directories are moved via
--root=else/where option).
Copy the supplied test CVS repository to a scratch place at the beginning
of these tests.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change this test to declare a PERL prerequisite. These tests use the
-p switch, so they implicitly depend on Perl code, but nothing was
declaring this.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change this test to declare a PERL prerequisite. These tests use the
-p switch, so they implicitly depend on Perl code, but nothing was
declaring this.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change this test to declare a PERL prerequisite. These tests use the
-p switch, so they implicitly depend on Perl code, but nothing was
declaring this.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the setup code in t/t9001-send-email.sh to use
test_expect_success. This way it isn't needlessly run in environments
where the test prerequisites aren't met.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change this test to skip test with test prerequisites, and to do setup
work in tests. This improves the skipped statistics on platforms where
the test isn't run.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Remove the PROG=* assignment from t9001-send-email.sh. It's been there
since v1.4.0-rc1~30 when the test was originally added, but only tests
that source annotate-tests.sh need it, it was seemingly introduced to
this test via copy/paste coding.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change this test to skip test with test prerequisites, and to do setup
work in tests. This improves the skipped statistics on platforms where
the test isn't run.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change this test to skip test with test prerequisites, and to do setup
work in tests. This improves the skipped statistics on platforms where
the test isn't run.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change this test to skip test with test prerequisites, and to do setup
work in tests. This improves the skipped statistics on platforms where
the test isn't run.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the five tests that were all checking "git config --bool
core.filemode" to use a new FILEMODE prerequisite in
lib-prereq-FILEMODE.sh.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the smoke target to create a test-results directory. This was
done implicitly by the test-lib before my "test-lib: Don't write
test-results when HARNESS_ACTIVE" patch, but after that smoking from
the pu branch hasn't worked.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The git-notes expensive timing test is only expensive because it
either did 10,100,1k and 10k iterations or nothing.
Change it to do 10 by default, with an option to run the expensive
version with the old GIT_NOTES_TIMING_TESTS=ZomgYesPlease variable.
Since nobody was ostensibly running this test under TAP the code had
bitrotted so that it emitted invalid TAP. This change fixes that.
The old version would also mysteriously fail on systems without
/usr/bin/time, there's now a check for that using the multiple test
prerequisite facility.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change this test to skip test with test prerequisites, and to do setup
work in tests. This improves the skipped statistics on platforms where
the test isn't run.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change this test to skip test with test prerequisites, and to do setup
work in tests. This improves the skipped statistics on platforms where
the test isn't run.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change this test to skip test with test prerequisites, and to do setup
work in tests. This improves the skipped statistics on platforms where
the test isn't run.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change this test to skip test with test prerequisites, and to do setup
work in tests. This improves the skipped statistics on platforms where
the test isn't run.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change this test to skip test with test prerequisites, and to do setup
work in tests. This improves the skipped statistics on platforms where
the test isn't run.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change this test to skip test with test prerequisites, and to do setup
work in tests. This improves the skipped statistics on platforms where
the test isn't run.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change this test to skip test with test prerequisites, and to do setup
work in tests. This improves the skipped statistics on platforms where
the test isn't run.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The example I initially added to "Skipping tests" wasn't very
good. We'd rather skip tests using the three-arg prereq form to the
test_* functions, not bail out with a skip message.
Change the documentation to reflect that, but retain the bailout
example under a disclaimer which explains that it's probably not a
good idea to use it.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change t/t7800-difftool.sh to to skip with the the three-arg prereq
form of test_expect_success instead of bailing out.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the t/t5800-remote-helpers.sh test to skip with the the
three-arg prereq form of test_expect_success instead of bailing out.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the tests that skipped due to unavailable SYMLINKS support to
use the three-arg prereq form of test_expect_success.
This is like the "tests: implicitly skip SYMLINKS tests using
<prereq>" change, but I needed to create an additional test for some
setup code. It's in a separate change as suggested by Jonathan Nieder
for ease of reviewing.
Date: Tue, 27 Jul 2010 16:17:37 -0500
From: Jonathan Nieder <jrnieder@gmail.com>
Message-ID: <20100727211737.GA11768@burratino>
In-Reply-To: <1280265254-19642-2-git-send-email-avarab@gmail.com>
Subject: Re: [PATCH 1/4] tests: implicitly skip SYMLINKS tests using <prereq>
Ævar Arnfjörð Bjarmason wrote:
> +++ b/t/t4004-diff-rename-symlink.sh
> @@ -40,8 +34,9 @@ test_expect_success \
> # rezrov and nitfol are rename/copy of frotz and bozbar should be
> # a new creation.
>
> -GIT_DIFF_OPTS=--unified=0 git diff-index -M -p $tree >current
> -cat >expected <<\EOF
> +test_expect_success SYMLINKS 'setup diff output' "
> + GIT_DIFF_OPTS=--unified=0 git diff-index -M -p $tree >current
> + cat >expected <<\EOF
> diff --git a/bozbar b/bozbar
> new file mode 120000
> --- /dev/null
Probably belongs in a separate patch. More importantly, it is missing
&&-chaining (not a regression, but it is best to set a good example).
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the tests that skipped due to unavailable SYMLINKS support to
use the three-arg prereq form of test_expect_success.
Now we get an indication of how many tests that need symlinks are
being skipped on platforms that don't support them.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If you customize CC to use a different version of gcc, most likely you
also need to use a different version of gcov. Make it configurable.
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Having no coverage at all is almost always a bad sign, but trying to
attain 100% coverage everywhere is usually a waste of time. Add a
paragraph to explain this to future test writers.
Inspired-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Document how test writers can generate coverage reports, to ensure
that their tests are really testing the code they think they're
testing.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a target to generate a detailed HTML report for the entire Git
codebase using Devel::Cover's cover(1) tool. Output it in
cover_db_html instead of the default cover_db, so that it isn't mixed
up with our raw report files.
The target depends on the coverage-report-cover-db target, it may be
run redundantly if it was previously run. But the HTML output won't be
affected by running gcov2perl twice, so I didn't try to avoid that
small redundancy.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a target to convert the *.gcov files to a Devel::Cover
database. That database can subsequently be formatted by the cover(1)
tool which is included with Devel::Cover.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the coverage-report target so that it doesn't generate the
coverage-untested-functions file by default. I'm adding more targets
for doing various things with the gcov files, and they shouldn't all
run by default.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We generate profiling files in all the $(OBJECTS) dirs. Aggregate
results from there, and add them to the corresponding clean target.
Also expand the gcov arguments. Generate reports for things like "x()
|| y()" using --all-blocks, and add --preserve-paths since we're
profiling in subdirectories now.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The "make coverage" support added by Thomas Rast in 901c369af5 didn't
contain a corresponding patch to patch .gitignore.
Change gitignore to ignore the *.gcda, *.gcno and *.gcov files
generated by GCC and our coverage invocations.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The smoke server supports a free form text field with comments about a
report, and a comma delimited list of tags. Change the smoke_report
target to expose this functionality. Now smokers can send more data
that explains and categorizes the reports they're submitting.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the smoke testing portion of t/Makefile not to include
GIT-BUILD-OPTIONS. It's a shellscript, not a Makefile snippet, so it
had the nasty side-effect of sneaking e.g. SHELL_PATH = '/bin/sh'
(with quotes) everywhere.
Just add our own PERL_PATH variable as a workaround. The t/Makefile
already has e.g. an equivalent SHELL_PATH and TAR option which
duplicate the definitions in GIT-BUILD-OPTIONS.
Reported-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Git now has a smoke testing service at http://smoke.git.nix.is that
anyone can send reports to. Change the t/README file to mention this.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add the capability to send smoke reports from the Git test suite.
Currently we only notice bugs in the test suite when it's run
manually. Bugs in Git that only occur on obscure platforms or setups
that the core developers aren't using can thus go unnoticed.
This series aims to change that. With it, anyone that's interested in
avoiding bitrot in Git can volunteer to run a smoke tester. A smoke
tester periodically compiles the latest version of Git, runs the test
suite, and submits a report to a central server indicating how the
test run went.
A smoke tester might run something like this in cron:
#!/bin/sh
cd ~/g/git
git fetch
for branch in maint master next pu; do
git checkout origin/$i &&
make clean all &&
cd t &&
make smoke_report
done
The smoker might want to compile git with non-default flags, include
bisecting functionality or run the tests under valgrind. Doing that is
outside the scope of this patch, this just adds a report submission
mechanism. But including a canonical smoke runner is something we'll
want to include eventually.
What this does now is add smoke and smoke_report targets to t/Makefile
(this example only uses a few tests for demonstration):
$ make clean smoke
rm -f -r 'trash directory'.* test-results
rm -f t????/cvsroot/CVSROOT/?*
rm -f -r valgrind/bin
rm -f .prove
perl ./harness --git-version="1.7.2.1.173.gc9b40" \
--no-verbose \
--archive="test-results/git-smoke.tar.gz" \
t0000-basic.sh t0001-init.sh t0002-gitfile.sh t0003-attributes.sh t0004-unwritable.sh t0005-signals.sh t0006-date.sh
t0000-basic.sh ....... ok
t0001-init.sh ........ ok
t0002-gitfile.sh ..... ok
t0003-attributes.sh .. ok
t0004-unwritable.sh .. ok
t0005-signals.sh ..... ok
t0006-date.sh ........ ok
All tests successful.
Test Summary Report
-------------------
t0000-basic.sh (Wstat: 0 Tests: 46 Failed: 0)
TODO passed: 5
Files=7, Tests=134, 3 wallclock secs ( 0.06 usr 0.05 sys + 0.23 cusr 1.33 csys = 1.67 CPU)
Result: PASS
TAP Archive created at /home/avar/g/git/t/test-results/git-smoke.tar.gz
The smoke target uses TAP::Harness::Archive to aggregate the test
results into a tarball. The tarball contains two things, the output of
every test file that was run, and a metadata file:
Tarball contents:
$ tar xzvf git-smoke.tar.gz
t0004-unwritable.sh
t0001-init.sh
t0002-gitfile.sh
t0005-signals.sh
t0000-basic.sh
t0003-attributes.sh
t0006-date.sh
meta.yml
A test report:
$ cat t0005-signals.sh
ok 1 - sigchain works
# passed all 1 test(s)
1..1
A metadata file:
---
extra_properties:
file_attributes:
-
description: t0000-basic.sh
end_time: 1280437324.61398
start_time: 1280437324.22186
-
description: t0001-init.sh
end_time: 1280437325.12346
start_time: 1280437324.62393
-
description: t0002-gitfile.sh
end_time: 1280437325.29428
start_time: 1280437325.13646
-
description: t0003-attributes.sh
end_time: 1280437325.59678
start_time: 1280437325.30565
-
description: t0004-unwritable.sh
end_time: 1280437325.77376
start_time: 1280437325.61003
-
description: t0005-signals.sh
end_time: 1280437325.85426
start_time: 1280437325.78727
-
description: t0006-date.sh
end_time: 1280437326.2362
start_time: 1280437325.86768
file_order:
- t0000-basic.sh
- t0001-init.sh
- t0002-gitfile.sh
- t0003-attributes.sh
- t0004-unwritable.sh
- t0005-signals.sh
- t0006-date.sh
start_time: 1280437324
stop_time: 1280437326
The "extra_properties" hash is where we'll stick Git-specific info,
like whether Git was compiled with gettext or the fallback regex
engine, and what branch we're compiling. Currently no metadata like
this is included.
The entire tarball is then submitted to a central smokebox at
smoke.git.nix.is. This is done with curl(1) via the "smoke_report"
target:
$ make smoke_report
curl \
-H "Expect: " \
-F project=Git \
-F architecture=x86_64 \
-F platform=Linux \
-F revision="1.7.2.1.173.gc9b40" \
-F report_file=@test-results/git-smoke.tar.gz \
http://smoke.git.nix.is/app/projects/process_add_report/1 \
| grep -v ^Redirecting
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 117k 100 63 100 117k 3 6430 0:00:21 0:00:18 0:00:03 0
Reported #8 added.
Reports are then made available on the smokebox via a web interface:
http://smoke.git.nix.is/app/projects/smoke_reports/1
The smoke reports are also mirrored to a Git repository hosted on
GitHub:
http://github.com/gitsmoke/smoke-reports
The Smolder SQLite database that contains metadata about the reports
is also made available:
http://github.com/gitsmoke/smoke-database
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The support for multiple test prerequisites added by me in "test-lib:
Add support for multiple test prerequisites" was broken.
The for iterated over each prerequisite and returned true/false within
a case statement, but since it missed a return statement only the last
prerequisite in the list of prerequisites was ever considered, the
rest were ignored.
Fix that by changing the test_have_prereq code to something less
clever that keeps a count of the total prereqs and the ones we have
and compares the count at the end.
This comes with the added advantage that it's easy to list the missing
prerequisites in the test output, implement that while I'm at it.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some tests depend on not being able to write to files after chmod
-w. This doesn't work when running the tests as root.
Change test-lib.sh to test if this works, and if so it sets a new
SANITY test prerequisite. The tests that use this previously failed
when run under root.
There was already a test for this in t3600-rm.sh, added by Junio C
Hamano in 2283645 in 2006. That check now uses the new SANITY
prerequisite.
Some of this was resurrected from the "Tests in Cygwin" thread in May
2009:
http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118385
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The README for the test library suggested that you grep the
test-lib.sh for test_set_prereq to see what the preset prerequisites
were.
Remove that bit, and write a section explaining all the preset
prerequisites. Most of the text was lifted from from Junio C Hamano
and Johannes Sixt, See the "Tests in Cygwin" thread in May 2009 for
the originals:
http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118385http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118434
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the test output to print needed prerequisites as part of the
TAP. This makes it easy to see at a glance why a test was
skipped. Before:
ok 7 # skip <message>
ok 9 # skip <message>
After:
ok 7 # skip <message> (prereqs: DONTHAVEIT)
ok 9 # skip <message> (prereqs: HAVEIT,DONTHAVEIT)
This'll also be useful for smoke testing output, where the developer
reading the output may not be familiar with the system where tests are
being skipped.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the test_have_prereq function in test-lib.sh to support a
comma-separated list of prerequisites. This is useful for tests that
need e.g. both POSIXPERM and SANITY.
The implementation was stolen from Junio C Hamano and Johannes Sixt,
the tests and documentation were not. See the "Tests in Cygwin" thread
in May 2009 for the originals:
http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118385http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118434
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
TAP harnesses don't need to read test-results/*, since they keep track
of the number of passing/failing tests internally. Skip the generation
of these files when HARNESS_ACTIVE is set.
It's now possible to run the Git test suite without writing anything
to the t/ directory at all if you use a TAP harness and the --root
switch:
cd t
sudo mount -t tmpfs none /tmp/memory -o size=300m
prove -j9 ./t[0-9]*.sh :: --root=/tmp/memory
The I/O that the ~500 test-results/* files contributed was very
minimal, but I thought this was worth mentioning.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
