mirrors/git
0
0
Fork 0
mirror of https://github.com/git/git.git synced 2024-11-28 04:23:30 +08:00
Code Issues Packages Projects Releases Wiki Activity
84d79009d9
git/t/t0091-bugreport.sh

151 lines
4.1 KiB
Bash
Raw Normal View History

bugreport: add tool to generate debugging info Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository. If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor. Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by. Signed-off-by: Emily Shaffer <emilyshaffer@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-17 05:18:04 +08:00
#!/bin/sh
test_description='git bugreport'
tests: add a test mode for SANITIZE=leak, run it in CI While git can be compiled with SANITIZE=leak, we have not run regression tests under that mode. Memory leaks have only been fixed as one-offs without structured regression testing. This change adds CI testing for it. We'll now build and small set of whitelisted t00*.sh tests under Linux with a new job called "linux-leaks". The CI target uses a new GIT_TEST_PASSING_SANITIZE_LEAK=true test mode. When running in that mode, we'll assert that we were compiled with SANITIZE=leak. We'll then skip all tests, except those that we've opted-in by setting "TEST_PASSES_SANITIZE_LEAK=true". A test setting "TEST_PASSES_SANITIZE_LEAK=true" setting can in turn make use of the "SANITIZE_LEAK" prerequisite, should they wish to selectively skip tests even under "GIT_TEST_PASSING_SANITIZE_LEAK=true". In the preceding commit we started doing this in "t0004-unwritable.sh" under SANITIZE=leak, now it'll combine nicely with "GIT_TEST_PASSING_SANITIZE_LEAK=true". This is how tests that don't set "TEST_PASSES_SANITIZE_LEAK=true" will be skipped under GIT_TEST_PASSING_SANITIZE_LEAK=true: $ GIT_TEST_PASSING_SANITIZE_LEAK=true ./t0001-init.sh 1..0 # SKIP skip all tests in t0001 under SANITIZE=leak, TEST_PASSES_SANITIZE_LEAK not set The intent is to add more TEST_PASSES_SANITIZE_LEAK=true annotations as follow-up change, but let's start small to begin with. In ci/run-build-and-tests.sh we make use of the default "*" case to run "make test" without any GIT_TEST_* modes. SANITIZE=leak is known to fail in combination with GIT_TEST_SPLIT_INDEX=true in t0016-oidmap.sh, and we're likely to have other such failures in various GIT_TEST_* modes. Let's focus on getting the base tests passing, we can expand coverage to GIT_TEST_* modes later. It would also be possible to implement a more lightweight version of this by only relying on setting "LSAN_OPTIONS". See <YS9OT/pn5rRK9cGB@coredump.intra.peff.net>[1] and <YS9ZIDpANfsh7N+S@coredump.intra.peff.net>[2] for a discussion of that. I've opted for this approach of adding a GIT_TEST_* mode instead because it's consistent with how we handle other special test modes. Being able to add a "!SANITIZE_LEAK" prerequisite and calling "test_done" early if it isn't satisfied also means that we can more incrementally add regression tests without being forced to fix widespread and hard-to-fix leaks at the same time. We have tests that do simple checking of some tool we're interested in, but later on in the script might be stressing trace2, or common sources of leaks like "git log" in combination with the tool (e.g. the commit-graph tests). To be clear having a prerequisite could also be accomplished by using "LSAN_OPTIONS" directly. On the topic of "LSAN_OPTIONS": It would be nice to have a mode to aggregate all failures in our various scripts, see [2] for a start at doing that which sets "log_path" in "LSAN_OPTIONS". I've punted on that for now, it can be added later. As of writing this we've got major regressions between master..seen, i.e. the t000*.sh tests and more fixed since 31f9acf9ce2 (Merge branch 'ah/plugleaks', 2021-08-04) have regressed recently. See the discussion at <87czsv2idy.fsf@evledraar.gmail.com>[3] about the lack of this sort of test mode, and 0e5bba53af (add UNLEAK annotation for reducing leak false positives, 2017-09-08) for the initial addition of SANITIZE=leak. See also 09595ab381 (Merge branch 'jk/leak-checkers', 2017-09-19), 7782066f67 (Merge branch 'jk/apache-lsan', 2019-05-19) and the recent 936e58851a (Merge branch 'ah/plugleaks', 2021-05-07) for some of the past history of "one-off" SANITIZE=leak (and more) fixes. As noted in [5] we can't support this on OSX yet until Clang 14 is released, at that point we'll probably want to resurrect that "osx-leaks" job. 1. https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer 2. https://lore.kernel.org/git/YS9OT%2Fpn5rRK9cGB@coredump.intra.peff.net/ 3. https://lore.kernel.org/git/87czsv2idy.fsf@evledraar.gmail.com/ 4. https://lore.kernel.org/git/YS9ZIDpANfsh7N+S@coredump.intra.peff.net/ 5. https://lore.kernel.org/git/20210916035603.76369-1-carenas@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-23 17:20:46 +08:00
TEST_PASSES_SANITIZE_LEAK=true
bugreport: add tool to generate debugging info Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository. If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor. Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by. Signed-off-by: Emily Shaffer <emilyshaffer@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-17 05:18:04 +08:00
. ./test-lib.sh
t0091-bugreport.sh: actually verify some content of report In the first test in this script, 'creates a report with content in the right places', we generate a report and pipe it into our helper `check_all_headers_populated()`. The idea of the helper is to find all lines that look like headers ("[Some Header Here]") and to check that the next line is non-empty. This is supposed to catch erroneous outputs such as the following: [A Header] something more here [Another Header] [Too Early Header] contents However, we provide the lines of the bug report as filenames to grep, meaning we mostly end up spewing errors: grep: : No such file or directory grep: [System Info]: No such file or directory grep: git version:: No such file or directory grep: git version 2.41.0.2.gfb7d80edca: No such file or directory This doesn't disturb the test, which tugs along and reports success, not really having verified the contents of the report at all. Note that after 788a776069 ("bugreport: collect list of populated hooks", 2020-05-07), the bug report, which is created in our hook-less test repo, contains an empty section with the enabled hooks. Thus, even the intention of our helper is a bit misguided: there is nothing inherently wrong with having an empty section in the bug report. Let's instead split this test into three: first verify that we generate a report at all, then check that the introductory blurb looks the way it should, then verify that the "[System Info]" seems to contain the right things. (The "[Enabled Hooks]" section is tested later in the script.) Reported-by: SZEDER Gábor <szeder.dev@gmail.com> Helped-by: Phillip Wood <phillip.wood123@gmail.com> Signed-off-by: Martin Ågren <martin.agren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-07-06 02:35:27 +08:00
test_expect_success 'create a report' '
git bugreport -s format &&
test_file_not_empty git-bugreport-format.txt
'
test_expect_success 'report contains wanted template (before first section)' '
sed -ne "/^\[/q;p" git-bugreport-format.txt >actual &&
cat >expect <<-\EOF &&
Thank you for filling out a Git bug report!
Please answer the following questions to help us understand your issue.
What did you do before the bug happened? (Steps to reproduce your issue)
What did you expect to happen? (Expected behavior)
What happened instead? (Actual behavior)
What'\''s different between what you expected and what actually happened?
Anything else you want to add:
Please review the rest of the bug report below.
You can delete any lines you don'\''t wish to share.
EOF
test_cmp expect actual
'
test_expect_success 'sanity check "System Info" section' '
test_when_finished rm -f git-bugreport-format.txt &&
sed -ne "/^\[System Info\]$/,/^$/p" <git-bugreport-format.txt >system &&
# The beginning should match "git version --build-info" verbatim,
# but rather than checking bit-for-bit equality, just test some basics.
grep "git version [0-9]." system &&
grep "shell-path: ." system &&
# After the version, there should be some more info.
# This is bound to differ from environment to environment,
# so we just do some rather high-level checks.
grep "uname: ." system &&
grep "compiler info: ." system
bugreport: add tool to generate debugging info Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository. If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor. Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by. Signed-off-by: Emily Shaffer <emilyshaffer@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-17 05:18:04 +08:00
'
test_expect_success 'dies if file with same name as report already exists' '
test_when_finished rm git-bugreport-duplicate.txt &&
>>git-bugreport-duplicate.txt &&
test_must_fail git bugreport --suffix duplicate
'
test_expect_success '--output-directory puts the report in the provided dir' '
test_when_finished rm -fr foo/ &&
git bugreport -o foo/ &&
test_path_is_file foo/git-bugreport-*
'
test_expect_success 'incorrect arguments abort with usage' '
test_must_fail git bugreport --false 2>output &&
test_i18ngrep usage output &&
test_path_is_missing git-bugreport-*
'
test_expect_success 'runs outside of a git dir' '
test_when_finished rm non-repo/git-bugreport-* &&
nongit git bugreport
'
test_expect_success 'can create leading directories outside of a git dir' '
test_when_finished rm -fr foo/bar/baz &&
nongit git bugreport -o foo/bar/baz
'
bugreport: collect list of populated hooks Occasionally a failure a user is seeing may be related to a specific hook which is being run, perhaps without the user realizing. While the contents of hooks can be sensitive - containing user data or process information specific to the user's organization - simply knowing that a hook is being run at a certain stage can help us to understand whether something is going wrong. Without a definitive list of hook names within the code, we compile our own list from the documentation. This is likely prone to bitrot, but designing a single source of truth for acceptable hooks is too much overhead for this small change to the bugreport tool. Signed-off-by: Emily Shaffer <emilyshaffer@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-08 08:53:57 +08:00
test_expect_success 'indicates populated hooks' '
test_when_finished rm git-bugreport-hooks.txt &&
bugreport tests: tighten up "git bugreport -s hooks" test Amend a test added in 788a776069b (bugreport: collect list of populated hooks, 2020-05-07) to "test_cmp" for the expected output, instead of selectively using "grep" to check for specific things we either expect or don't expect in the output. As noted in a preceding commit our .git/hooks directory already contains *.sample hooks, so we have no need to clobber the prepare-commit-msg.sample hook in particular. Instead we should assert that those *.sample hooks are not included in the output, and for good measure let's add a new "unknown-hook", to check that we only look through our own known hooks. See cfe853e66be (hook-list.h: add a generated list of hooks, like config-list.h, 2021-09-26) for how we generate that data. We're intentionally not piping the "actual" output through "sort" or similar, we'd also like to check that our reported hooks are sorted. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-17 18:13:10 +08:00
test_hook applypatch-msg <<-\EOF &&
true
EOF
test_hook unknown-hook <<-\EOF &&
true
EOF
bugreport: collect list of populated hooks Occasionally a failure a user is seeing may be related to a specific hook which is being run, perhaps without the user realizing. While the contents of hooks can be sensitive - containing user data or process information specific to the user's organization - simply knowing that a hook is being run at a certain stage can help us to understand whether something is going wrong. Without a definitive list of hook names within the code, we compile our own list from the documentation. This is likely prone to bitrot, but designing a single source of truth for acceptable hooks is too much overhead for this small change to the bugreport tool. Signed-off-by: Emily Shaffer <emilyshaffer@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-08 08:53:57 +08:00
git bugreport -s hooks &&
bugreport tests: tighten up "git bugreport -s hooks" test Amend a test added in 788a776069b (bugreport: collect list of populated hooks, 2020-05-07) to "test_cmp" for the expected output, instead of selectively using "grep" to check for specific things we either expect or don't expect in the output. As noted in a preceding commit our .git/hooks directory already contains *.sample hooks, so we have no need to clobber the prepare-commit-msg.sample hook in particular. Instead we should assert that those *.sample hooks are not included in the output, and for good measure let's add a new "unknown-hook", to check that we only look through our own known hooks. See cfe853e66be (hook-list.h: add a generated list of hooks, like config-list.h, 2021-09-26) for how we generate that data. We're intentionally not piping the "actual" output through "sort" or similar, we'd also like to check that our reported hooks are sorted. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-17 18:13:10 +08:00
sort >expect <<-\EOF &&
[Enabled Hooks]
applypatch-msg
EOF
sed -ne "/^\[Enabled Hooks\]$/,/^$/p" <git-bugreport-hooks.txt >actual &&
test_cmp expect actual
bugreport: collect list of populated hooks Occasionally a failure a user is seeing may be related to a specific hook which is being run, perhaps without the user realizing. While the contents of hooks can be sensitive - containing user data or process information specific to the user's organization - simply knowing that a hook is being run at a certain stage can help us to understand whether something is going wrong. Without a definitive list of hook names within the code, we compile our own list from the documentation. This is likely prone to bitrot, but designing a single source of truth for acceptable hooks is too much overhead for this small change to the bugreport tool. Signed-off-by: Emily Shaffer <emilyshaffer@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-08 08:53:57 +08:00
'
bugreport: add tool to generate debugging info Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository. If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor. Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by. Signed-off-by: Emily Shaffer <emilyshaffer@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-17 05:18:04 +08:00
builtin/bugreport.c: create '--diagnose' option Create a '--diagnose' option for 'git bugreport' to collect additional information about the repository and write it to a zipped archive. The '--diagnose' option behaves effectively as an alias for simultaneously running 'git bugreport' and 'git diagnose'. In the documentation, users are explicitly recommended to attach the diagnostics alongside a bug report to provide additional context to readers, ideally reducing some back-and-forth between reporters and those debugging the issue. Note that '--diagnose' may take an optional string arg (either 'stats' or 'all'). If specified without the arg, the behavior corresponds to running 'git diagnose' without '--mode'. As with 'git diagnose', this default is intended to help reduce unintentional leaking of sensitive information). Users can also explicitly specify '--diagnose=(stats|all)' to generate the respective archive created by 'git diagnose --mode=(stats|all)'. Suggested-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Helped-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-13 04:10:17 +08:00
test_expect_success UNZIP '--diagnose creates diagnostics zip archive' '
test_when_finished rm -rf report &&
git bugreport --diagnose -o report -s test >out &&
zip_path=report/git-diagnostics-test.zip &&
grep "Available space" out &&
test_path_is_file "$zip_path" &&
# Check zipped archive content
"$GIT_UNZIP" -p "$zip_path" diagnostics.log >out &&
test_file_not_empty out &&
"$GIT_UNZIP" -p "$zip_path" packs-local.txt >out &&
grep ".git/objects" out &&
"$GIT_UNZIP" -p "$zip_path" objects-local.txt >out &&
grep "^Total: [0-9][0-9]*" out &&
# Should not include .git directory contents by default
! "$GIT_UNZIP" -l "$zip_path" | grep ".git/"
'
test_expect_success UNZIP '--diagnose=stats excludes .git dir contents' '
test_when_finished rm -rf report &&
git bugreport --diagnose=stats -o report -s test >out &&
# Includes pack quantity/size info
"$GIT_UNZIP" -p "$zip_path" packs-local.txt >out &&
grep ".git/objects" out &&
# Does not include .git directory contents
! "$GIT_UNZIP" -l "$zip_path" | grep ".git/"
'
test_expect_success UNZIP '--diagnose=all includes .git dir contents' '
test_when_finished rm -rf report &&
git bugreport --diagnose=all -o report -s test >out &&
# Includes .git directory contents
"$GIT_UNZIP" -l "$zip_path" | grep ".git/" &&
"$GIT_UNZIP" -p "$zip_path" .git/HEAD >out &&
test_file_not_empty out
'
bugreport: add tool to generate debugging info Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository. If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor. Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by. Signed-off-by: Emily Shaffer <emilyshaffer@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-17 05:18:04 +08:00
test_done
Reference in New Issue Copy Permalink