linux/scripts
Masahiro Yamada 4cb3025465 scripts: handle BrokenPipeError for python scripts
[ Upstream commit 87c7ee67de ]

In the follow-up of commit fb3041d61f ("kbuild: fix SIGPIPE error
message for AR=gcc-ar and AR=llvm-ar"), Kees Cook pointed out that
tools should _not_ catch their own SIGPIPEs [1] [2].

Based on his feedback, LLVM was fixed [3].

However, Python's default behavior is to show noisy bracktrace when
SIGPIPE is sent. So, scripts written in Python are basically in the
same situation as the buggy llvm tools.

Example:

  $ make -s allnoconfig
  $ make -s allmodconfig
  $ scripts/diffconfig .config.old .config | head -n1
  -ALIX n
  Traceback (most recent call last):
    File "/home/masahiro/linux/scripts/diffconfig", line 132, in <module>
      main()
    File "/home/masahiro/linux/scripts/diffconfig", line 130, in main
      print_config("+", config, None, b[config])
    File "/home/masahiro/linux/scripts/diffconfig", line 64, in print_config
      print("+%s %s" % (config, new_value))
  BrokenPipeError: [Errno 32] Broken pipe

Python documentation [4] notes how to make scripts die immediately and
silently:

  """
  Piping output of your program to tools like head(1) will cause a
  SIGPIPE signal to be sent to your process when the receiver of its
  standard output closes early. This results in an exception like
  BrokenPipeError: [Errno 32] Broken pipe. To handle this case,
  wrap your entry point to catch this exception as follows:

    import os
    import sys

    def main():
        try:
            # simulate large output (your code replaces this loop)
            for x in range(10000):
                print("y")
            # flush output here to force SIGPIPE to be triggered
            # while inside this try block.
            sys.stdout.flush()
        except BrokenPipeError:
            # Python flushes standard streams on exit; redirect remaining output
            # to devnull to avoid another BrokenPipeError at shutdown
            devnull = os.open(os.devnull, os.O_WRONLY)
            os.dup2(devnull, sys.stdout.fileno())
            sys.exit(1)  # Python exits with error code 1 on EPIPE

    if __name__ == '__main__':
        main()

  Do not set SIGPIPE’s disposition to SIG_DFL in order to avoid
  BrokenPipeError. Doing that would cause your program to exit
  unexpectedly whenever any socket connection is interrupted while
  your program is still writing to it.
  """

Currently, tools/perf/scripts/python/intel-pt-events.py seems to be the
only script that fixes the issue that way.

tools/perf/scripts/python/compaction-times.py uses another approach
signal.signal(signal.SIGPIPE, signal.SIG_DFL) but the Python
documentation clearly says "Don't do it".

I cannot fix all Python scripts since there are so many.
I fixed some in the scripts/ directory.

[1]: https://lore.kernel.org/all/202211161056.1B9611A@keescook/
[2]: https://github.com/llvm/llvm-project/issues/59037
[3]: 4787efa380
[4]: https://docs.python.org/3/library/signal.html#note-on-sigpipe

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2023-03-17 08:49:02 +01:00
..
atomic atomics: Fix atomic64_{read_acquire,set_release} fallbacks 2022-04-08 14:23:57 +02:00
basic .gitignore: prefix local generated files with a slash 2021-05-02 00:43:35 +09:00
clang-tools scripts: handle BrokenPipeError for python scripts 2023-03-17 08:49:02 +01:00
coccinelle scripts: coccinelle: allow list_entry_is_head() to use pos 2021-08-08 22:00:49 +02:00
dtc kbuild: Allow kernel installation packaging to override pkg-config 2023-02-01 08:27:30 +01:00
dummy-tools kbuild: dummy-tools: avoid tmpdir leak in dummy gcc 2022-08-25 11:40:14 +02:00
gcc-plugins gcc-plugins: latent_entropy: use /dev/urandom 2022-04-20 09:34:18 +02:00
gdb scripts/gdb: fix 'lx-dmesg' on 32 bits arch 2022-08-17 14:23:56 +02:00
genksyms .gitignore: prefix local generated files with a slash 2021-05-02 00:43:35 +09:00
kconfig kbuild: Allow kernel installation packaging to override pkg-config 2023-02-01 08:27:30 +01:00
ksymoops
mod modpost: fix section mismatch check for exported init/exit sections 2022-06-29 09:03:31 +02:00
package builddeb: clean generated package content 2023-03-10 09:39:44 +01:00
selinux selinux: use "grep -E" instead of "egrep" 2022-10-26 12:34:35 +02:00
tracing ftrace/scripts: Update the instructions for ftrace-bisect.sh 2023-02-01 08:27:23 +01:00
.gitignore .gitignore: prefix local generated files with a slash 2021-05-02 00:43:35 +09:00
adjust_autoksyms.sh kbuild: redo fake deps at include/ksym/*.h 2021-09-03 08:17:21 +09:00
as-version.sh kbuild: Switch to 'f' variants of integrated assembler flag 2021-09-03 08:17:20 +09:00
asn1_compiler.c
bin2c.c
bloat-o-meter scripts: switch explicitly to Python 3 2021-01-22 06:34:44 +09:00
bootgraph.pl
bpf_doc.py bpf: Introduce bpf timers. 2021-07-15 22:31:10 +02:00
cc-can-link.sh
cc-version.sh kbuild: collect minimum tool versions into scripts/min-tool-version.sh 2021-04-25 05:14:26 +09:00
check_extable.sh scripts: check_extable: fix typo in user error message 2021-09-08 11:50:28 -07:00
check-sysctl-docs
checkdeclares.pl scripts: make some scripts executable 2021-08-10 09:13:25 +09:00
checkincludes.pl
checkkconfigsymbols.py scripts: handle BrokenPipeError for python scripts 2023-03-17 08:49:02 +01:00
checkpatch.pl Compiler Attributes: add __alloc_size() for better bounds checking 2022-07-12 16:35:01 +02:00
checkstack.pl scripts/checkstack.pl: fix arm sp regex 2020-05-26 00:03:16 +09:00
checksyscalls.sh checksyscalls: Unconditionally ignore fstat{,at}64 2021-10-07 17:16:28 -07:00
checkversion.pl scripts: checkversion: modernize linux/version.h search strings 2021-08-05 20:55:39 +09:00
cleanfile
cleanpatch
coccicheck scripts: coccicheck: fix troubles on non-English builds 2021-05-18 11:09:59 +02:00
config kconfig: config script: add a little user help 2021-01-04 10:38:11 +09:00
const_structs.checkpatch const_structs.checkpatch: add pinctrl_ops and pinmux_ops 2020-10-16 11:11:21 -07:00
decode_stacktrace.sh scripts/decode_stacktrace.sh: indicate 'auto' can be used for base path 2021-07-08 11:48:22 -07:00
decodecode scripts/decodecode: add the capability to supply the program counter 2020-10-13 18:38:26 -07:00
depmod.sh depmod: handle the case of /sbin/depmod without /sbin in PATH 2021-01-01 12:26:39 -08:00
dev-needs.sh scripts/dev-needs: Add script to list device dependencies 2020-09-04 18:19:37 +02:00
diffconfig scripts: handle BrokenPipeError for python scripts 2023-03-17 08:49:02 +01:00
documentation-file-ref-check documentation-file-ref-check: Make git check work for multiple working directories 2021-06-04 11:20:52 -06:00
export_report.pl modpost: move the namespace field in Module.symvers last 2020-03-17 08:59:03 +09:00
extract_xc3028.pl
extract-cert.c cert host tools: Stop complaining about deprecated OpenSSL functions 2022-11-16 09:58:28 +01:00
extract-ikconfig
extract-module-sig.pl
extract-sys-certs.pl
extract-vmlinux
faddr2line scripts/faddr2line: Fix regression in name resolution on ppc64le 2022-12-08 11:28:38 +01:00
file-size.sh
find-unused-docs.sh
gcc-goto.sh
gcc-ld
gcc-x86_32-has-stack-protector.sh x86/stackprotector/32: Make the canary into a regular percpu variable 2021-03-08 13:19:05 +01:00
gcc-x86_64-has-stack-protector.sh
gen_autoksyms.sh kbuild: Fix TRIM_UNUSED_KSYMS with LTO_CLANG 2021-09-03 08:12:39 +09:00
gen_ksymdeps.sh kbuild: redo fake deps at include/ksym/*.h 2021-09-03 08:17:21 +09:00
generate_initcall_order.pl init: lto: ensure initcall ordering 2021-01-14 08:21:09 -08:00
get_abi.pl scripts: get_abi: ignore code blocks for cross-references 2021-03-31 13:53:16 -06:00
get_dvb_firmware
get_feat.pl scripts: get_feat.pl: reduce table width for all features output 2020-12-04 14:34:27 -07:00
get_maintainer.pl get_maintainer: exclude MAINTAINERS file(s) from --git-fallback 2020-10-16 11:11:19 -07:00
gfp-translate
headerdep.pl
headers_check.pl
headers_install.sh Merge branch 'work.fdpic' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2020-08-07 13:29:39 -07:00
insert-sys-cert.c
jobserver-exec scripts/jobserver-exec: Fix a typo ("envirnoment") 2021-05-17 12:10:03 +09:00
kallsyms.c kallsyms: fix nonconverging kallsyms table with lld 2021-02-05 17:53:28 +09:00
Kbuild.include kbuild: remove the target in signal traps when interrupted 2022-10-26 12:35:27 +02:00
Kconfig.include kbuild: check the minimum assembler version in Kconfig 2021-04-25 05:14:41 +09:00
kernel-doc stddef: Introduce DECLARE_FLEX_ARRAY() helper 2022-07-12 16:35:03 +02:00
ld-version.sh kbuild: collect minimum tool versions into scripts/min-tool-version.sh 2021-04-25 05:14:26 +09:00
leaking_addresses.pl leaking_addresses: Always print a trailing newline 2021-11-18 19:16:16 +01:00
Lindent
link-vmlinux.sh kbuild: Unify options for BTF generation for vmlinux and modules 2022-09-08 12:28:08 +02:00
Makefile kbuild: Allow kernel installation packaging to override pkg-config 2023-02-01 08:27:30 +01:00
Makefile.asm-generic kbuild: prefix $(srctree)/ to some included Makefiles 2021-03-15 19:20:48 +09:00
Makefile.build x86/retbleed: Add fine grained Kconfig knobs 2022-07-23 12:54:10 +02:00
Makefile.clang kbuild: Add -Werror=ignored-optimization-argument to CLANG_FLAGS 2021-09-19 10:55:18 +09:00
Makefile.clean kbuild: prefix $(srctree)/ to some included Makefiles 2021-03-15 19:20:48 +09:00
Makefile.compiler kbuild: remove TMPO from try-run 2021-04-25 05:25:56 +09:00
Makefile.dtbinst kbuild: prefix $(srctree)/ to some included Makefiles 2021-03-15 19:20:48 +09:00
Makefile.extrawarn Makefile.extrawarn: Move -Wcast-function-type-strict to W=1 2022-10-12 09:53:26 +02:00
Makefile.gcc-plugins gcc-plugins: Undefine LATENT_ENTROPY_PLUGIN when plugin disabled for a file 2022-08-25 11:40:27 +02:00
Makefile.headersinst kbuild: prefix $(srctree)/ to some included Makefiles 2021-03-15 19:20:48 +09:00
Makefile.host kbuild: sort hostprogs before passing it to ifneq 2020-08-10 01:32:59 +09:00
Makefile.kasan kasan: always respect CONFIG_KASAN_STACK 2021-09-24 16:13:35 -07:00
Makefile.kcov kbuild: include scripts/Makefile.* only when relevant CONFIG is enabled 2020-08-10 01:32:59 +09:00
Makefile.kcsan Kbuild updates for v5.10 2020-10-22 13:13:57 -07:00
Makefile.lib kbuild: move objtool_args back to scripts/Makefile.build 2022-05-15 20:18:50 +02:00
Makefile.modfinal kbuild: Unify options for BTF generation for vmlinux and modules 2022-09-08 12:28:08 +02:00
Makefile.modinst kbuild: add support for zstd compressed modules 2021-04-25 05:25:06 +09:00
Makefile.modpost kbuild: Fix include path in scripts/Makefile.modpost 2022-09-05 10:30:06 +02:00
Makefile.package kbuild: deb-pkg: change the source package name to linux-upstream 2021-04-25 05:30:45 +09:00
Makefile.ubsan ubsan: remove CONFIG_UBSAN_OBJECT_SIZE 2022-04-13 20:59:27 +02:00
Makefile.userprogs kbuild: add infrastructure to build userspace programs 2020-05-17 18:52:01 +09:00
makelst
markup_oops.pl
min-tool-version.sh Documentation: raise minimum supported version of GCC to 5.1 2021-09-13 10:18:28 -07:00
mkcompile_h kbuild: mkcompile_h: consider timestamp if KBUILD_BUILD_TIMESTAMP is set 2021-06-17 10:01:12 +09:00
mksysmap mksysmap: Fix the mismatch of 'L0' symbols in System.map 2022-09-23 14:15:51 +02:00
mkuboot.sh
module.lds.S modules: Ensure natural alignment for .altinstructions and __bug_table sections 2022-08-25 11:40:41 +02:00
modules-check.sh kbuild: check module name conflict for external modules as well 2021-04-25 05:22:42 +09:00
nsdeps kbuild: replace LANG=C with LC_ALL=C 2021-05-02 00:43:35 +09:00
objdiff
pahole-flags.sh scripts/pahole-flags.sh: Use pahole-version.sh 2023-02-25 12:06:46 +01:00
pahole-version.sh kbuild: Add CONFIG_PAHOLE_VERSION 2023-02-25 12:06:45 +01:00
parse-maintainers.pl parse-maintainers: Do not sort section content by default 2020-03-26 15:08:27 -07:00
patch-kernel
profile2linkerlist.pl
prune-kernel
recordmcount.c ftrace: Have recordmcount use w8 to read relp->r_info in arm64_is_fake_mcount 2021-03-02 17:27:18 -05:00
recordmcount.h recordmcount: Correct st_shndx handling 2021-06-18 09:09:17 -04:00
recordmcount.pl recordmcount.pl: fix typo in s390 mcount regex 2022-01-05 12:42:33 +01:00
remove-stale-files kbuild: add a script to remove stale generated files 2021-05-02 00:43:35 +09:00
setlocalversion scripts/setlocalversion: fix a bug when LOCALVERSION is empty 2021-07-18 23:48:14 +09:00
show_delta tweewide: Fix most Shebang lines 2020-12-08 23:30:04 +09:00
sign-file.c cert host tools: Stop complaining about deprecated OpenSSL functions 2022-11-16 09:58:28 +01:00
sorttable.c x86/extable: Rework the exception table mechanics 2022-07-29 17:25:25 +02:00
sorttable.h
spdxcheck-test.sh scripts/spdxcheck-test.sh: Drop python2 2021-07-21 15:59:32 +02:00
spdxcheck.py scripts/spdxcheck.py: Strictly read license files in utf-8 2021-07-12 09:56:50 -06:00
spelling.txt scripts/spelling.txt: add more spellings to spelling.txt 2021-06-29 10:53:45 -07:00
sphinx-pre-install scripts: sphinx-pre-install: Fix ctex support on Debian 2022-01-27 11:05:44 +01:00
split-man.pl tweewide: Fix most Shebang lines 2020-12-08 23:30:04 +09:00
stackdelta
stackusage
subarch.include parisc: Fix compile failure when building 64-bit kernel natively 2021-09-01 21:52:02 +02:00
syscallhdr.sh scripts: check duplicated syscall number in syscall table 2021-07-09 04:00:39 +09:00
syscallnr.sh scripts: make some scripts executable 2021-08-10 09:13:25 +09:00
syscalltbl.sh scripts: check duplicated syscall number in syscall table 2021-07-09 04:00:39 +09:00
tags.sh scripts/tags.sh: fix incompatibility with PCRE2 2023-03-03 11:45:53 +01:00
tools-support-relr.sh Makefile: fix GDB warning with CONFIG_RELR 2021-06-08 13:09:34 +01:00
unifdef.c
ver_linux Removed the oprofiled version option 2021-05-03 17:23:06 -06:00
xen-hypercalls.sh scripts: make some scripts executable 2021-08-10 09:13:25 +09:00
xz_wrap.sh kbuild: add variables for compression tools 2020-06-06 23:42:01 +09:00