2
0
mirror of https://github.com/edk2-porting/linux-next.git synced 2024-12-24 05:04:00 +08:00
Commit Graph

153 Commits

Author SHA1 Message Date
Masahiro Yamada
392885ee82 kbuild: let fixdep directly write to .*.cmd files
Currently, fixdep writes dependencies to .*.tmp, which is renamed to
.*.cmd after everything succeeds. This is a very safe way to avoid
corrupted .*.cmd files. The if_changed_dep has carried this safety
mechanism since it was added in 2002.

If fixdep fails for some reasons or a user terminates the build while
fixdep is running, the incomplete output from the fixdep could be
troublesome.

This is my insight about some bad scenarios:

[1] If the compiler succeeds to generate *.o file, but fixdep fails
    to write necessary dependencies to .*.cmd file, Make will miss
    to rebuild the object when headers or CONFIG options are changed.
    In this case, fixdep should not generate .*.cmd file at all so
    that 'arg-check' will surely trigger the rebuild of the object.

[2] A partially constructed .*.cmd file may not be a syntactically
    correct makefile. The next time Make runs, it would include it,
    then fail to parse it. Once this happens, 'make clean' is be the
    only way to fix it.

In fact, [1] is no longer a problem since commit 9c2af1c737 ("kbuild:
add .DELETE_ON_ERROR special target"). Make deletes a target file on
any failure in its recipe. Because fixdep is a part of the recipe of
*.o target, if it fails, the *.o is deleted anyway. However, I am a
bit worried about the slight possibility of [2].

So, here is a solution. Let fixdep directly write to a .*.cmd file,
but allow makefiles to include it only when its corresponding target
exists.

This effectively reverts commit 2982c95357 ("kbuild: remove redundant
$(wildcard ...) for cmd_files calculation"), and commit 00d78ab2ba
("kbuild: remove dead code in cmd_files calculation in top Makefile")
because now we must check the presence of targets.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-12-01 23:13:14 +09:00
Linus Torvalds
ac747c0715 Kbuild updates for v4.20
- optimize kallsyms slightly
 
 - remove check for old CFLAGS usage
 
 - add some compiler flags unconditionally instead of evaluating
   $(call cc-option,...)
 
 - fix variable shadowing in host tools
 
 - refactor scripts/mkmakefile
 
 - refactor various makefiles
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQIcBAABAgAGBQJb1eyEAAoJED2LAQed4NsGUjwP/0/W0nLP+nKCJ0NpsSD151Ea
 Vrbm+RMxl8uPuQ0+Bh59rdO2yWR8v7jOX8CPbX1DqW0XwW4vTNRZm9j6A83rJzHv
 dPpinUObq6vXGJCYvMLoumhOkM1lRZie1AeBeLgKF0G0jprxUspGvakSyM/5SOo6
 3gIhpPcczijC980KHIJbmwiGqRFVs3/zwqcKjQaRD3C6f4HdJL3i9Zr7kuZF0g1x
 dxCdN4Shv5x0igggp58z8646bbKlN7hItYaPq2csDop55/jWKzUkBkFkeaDjiUdB
 B6ysQfwkuTb6sbKXE6euYLUTyc6epx9v9pey0FpOx5tjXT+QmgK1ddLQEwdFH2+/
 Fd5VR70h8uKTNvmqFJ+iebpR/aC71sUqYAzsoiTVZibR/F6+QAuhHJPZJDlSSwLC
 QH7j0fwJ3X5p9PYe3JBWyWzOd9BDvtV+HGE67Kx17iRNW0FDyKSvn7tezRVtqvHr
 KENrMT4n+lNQzB+c4lfjOE9ANpOf+PP4+ODhbrtvKItyb5QfF4F+5CDVif3wv3Kt
 6dvh13hvR06gQCpbuxcFgD17mT1T/lVieuzOnWjtEh4HiMI1H0abYfkRWbQshrti
 u7TfYxwAyWKtisiPiT/1THYsW3ux7QmTSIo2iOznQxDlbmzENpQx+f22HsLaC0XU
 AUgvKGBaN0WOar2f2gyq
 =jM4E
 -----END PGP SIGNATURE-----

Merge tag 'kbuild-v4.20' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild

Pull Kbuild updates from Masahiro Yamada:

 - optimize kallsyms slightly

 - remove check for old CFLAGS usage

 - add some compiler flags unconditionally instead of evaluating
   $(call cc-option,...)

 - fix variable shadowing in host tools

 - refactor scripts/mkmakefile

 - refactor various makefiles

* tag 'kbuild-v4.20' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild:
  modpost: Create macro to avoid variable shadowing
  ASN.1: Remove unnecessary shadowed local variable
  kbuild: use 'else ifeq' for checksrc to improve readability
  kbuild: remove unneeded link_multi_deps
  kbuild: add -Wno-unused-but-set-variable flag unconditionally
  kbuild: add -Wdeclaration-after-statement flag unconditionally
  kbuild: add -Wno-pointer-sign flag unconditionally
  modpost: remove leftover symbol prefix handling for module device table
  kbuild: simplify command line creation in scripts/mkmakefile
  kbuild: do not pass $(objtree) to scripts/mkmakefile
  kbuild: remove user ID check in scripts/mkmakefile
  kbuild: remove VERSION and PATCHLEVEL from $(objtree)/Makefile
  kbuild: add --include-dir flag only for out-of-tree build
  kbuild: remove dead code in cmd_files calculation in top Makefile
  kbuild: hide most of targets when running config or mixed targets
  kbuild: remove old check for CFLAGS use
  kbuild: prefix Makefile.dtbinst path with $(srctree) unconditionally
  kallsyms: remove left-over Blackfin code
  kallsyms: reduce size a little on 64-bit
2018-10-28 13:22:35 -07:00
Masahiro Yamada
7d0ea25242 kbuild: use 'else ifeq' for checksrc to improve readability
'ifeq ... else ifeq ... endif' notation is supported by GNU Make 3.81
or later, which is the requirement for building the kernel since
commit 37d69ee308 ("docs: bump minimal GNU Make version to 3.81").

Use it to improve the readability.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-10-19 09:48:47 +09:00
Masahiro Yamada
69ea912fda kbuild: remove unneeded link_multi_deps
Since commit c8589d1e9e ("kbuild: handle multi-objs dependency
appropriately"), $^ really represents all the prerequisite of the
composite object being built.

Hence, $(filter %.o,$^) contains all the objects to link together,
which is much simpler than link_multi_deps calculation.

Please note $(filter-out FORCE,$^) does not work here. When a single
object module is turned into a multi object module, $^ will contain
header files that were previously included for building the single
object, and recorded in the .*.cmd file. To filter out such headers,
$(filter %.o,$^) should be used here.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-10-19 09:47:46 +09:00
Joel Stanley
5a4630aadb ftrace: Build with CPPFLAGS to get -Qunused-arguments
When building to record the mcount locations the kernel uses
KBUILD_CFLAGS but not KBUILD_CPPFLAGS. This means it lacks
-Qunused-arguments when building with clang, resulting in a lot of
noisy warnings.

Signed-off-by: Joel Stanley <joel@jms.id.au>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-09-19 23:53:54 +09:00
Masahiro Yamada
0d91bf584f kbuild: remove old check for CFLAGS use
This check has been here for more than a decade since
commit 0c53c8e6eb ("kbuild: check for wrong use of CFLAGS").

Enough time for migration has passed.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-09-13 00:01:16 +09:00
Masahiro Yamada
113fc08357 objtool: Remove workaround for unreachable warnings from old GCC
Commit cafa0010cd ("Raise the minimum required gcc version to 4.6")
bumped the minimum GCC version to 4.6 for all architectures.

This effectively reverts commit da541b2002 ("objtool: Skip unreachable
warnings for GCC 4.4 and older"), which was a workaround for GCC 4.4 or
older.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Michal Marek <michal.lkml@markovi.net>
Cc: linux-kbuild@vger.kernel.org
Link: https://lkml.kernel.org/r/1535341183-19994-1-git-send-email-yamada.masahiro@socionext.com
2018-08-30 12:56:40 +02:00
Masahiro Yamada
d503ac531a kbuild: rename LDFLAGS to KBUILD_LDFLAGS
Commit a0f97e06a4 ("kbuild: enable 'make CFLAGS=...' to add
additional options to CC") renamed CFLAGS to KBUILD_CFLAGS.

Commit 222d394d30 ("kbuild: enable 'make AFLAGS=...' to add
additional options to AS") renamed AFLAGS to KBUILD_AFLAGS.

Commit 06c5040cdb ("kbuild: enable 'make CPPFLAGS=...' to add
additional options to CPP") renamed CPPFLAGS to KBUILD_CPPFLAGS.

For some reason, LDFLAGS was not renamed.

Using a well-known variable like LDFLAGS may result in accidental
override of the variable.

Kbuild generally uses KBUILD_ prefixed variables for the internally
appended options, so here is one more conversion to sanitize the
naming convention.

I did not touch Makefiles under tools/ since the tools build system
is a different world.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Reviewed-by: Palmer Dabbelt <palmer@sifive.com>
2018-08-24 08:22:08 +09:00
Masahiro Yamada
87a32e6240 kbuild: pass LDFLAGS to recordmcount.pl
Since commit 0fbe9a245c ("microblaze: add endianness options to
LDFLAGS instead of LD"), you cannot build the kernel for microblaze
with CONFIG_DYNAMIC_FTRACE.

Fixes: 0fbe9a245c ("microblaze: add endianness options to LDFLAGS instead of LD")
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-08-24 08:22:08 +09:00
Linus Torvalds
7140ad3898 Updates for v4.19:
- Restructure of lockdep and latency tracers
 
    This is the biggest change. Joel Fernandes restructured the hooks
    from irqs and preemption disabling and enabling. He got rid of
    a lot of the preprocessor #ifdef mess that they caused.
 
    He turned both lockdep and the latency tracers to use trace events
    inserted in the preempt/irqs disabling paths. But unfortunately,
    these started to cause issues in corner cases. Thus, parts of the
    code was reverted back to where lockde and the latency tracers
    just get called directly (without using the trace events).
    But because the original change cleaned up the code very nicely
    we kept that, as well as the trace events for preempt and irqs
    disabling, but they are limited to not being called in NMIs.
 
  - Have trace events use SRCU for "rcu idle" calls. This was required
    for the preempt/irqs off trace events. But it also had to not
    allow them to be called in NMI context. Waiting till Paul makes
    an NMI safe SRCU API.
 
  - New notrace SRCU API to allow trace events to use SRCU.
 
  - Addition of mcount-nop option support
 
  - SPDX headers replacing GPL templates.
 
  - Various other fixes and clean ups.
 
  - Some fixes are marked for stable, but were not fully tested
    before the merge window opened.
 -----BEGIN PGP SIGNATURE-----
 
 iIoEABYIADIWIQRRSw7ePDh/lE+zeZMp5XQQmuv6qgUCW3ruhRQccm9zdGVkdEBn
 b29kbWlzLm9yZwAKCRAp5XQQmuv6qiM7AP47NhYdSnCFCRUJfrt6PovXmQtuCHt3
 c3QMoGGdvzh9YAEAqcSXwh7uLhpHUp1LjMAPkXdZVwNddf4zJQ1zyxQ+EAU=
 =vgEr
 -----END PGP SIGNATURE-----

Merge tag 'trace-v4.19' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace

Pull tracing updates from Steven Rostedt:

 - Restructure of lockdep and latency tracers

   This is the biggest change. Joel Fernandes restructured the hooks
   from irqs and preemption disabling and enabling. He got rid of a lot
   of the preprocessor #ifdef mess that they caused.

   He turned both lockdep and the latency tracers to use trace events
   inserted in the preempt/irqs disabling paths. But unfortunately,
   these started to cause issues in corner cases. Thus, parts of the
   code was reverted back to where lockdep and the latency tracers just
   get called directly (without using the trace events). But because the
   original change cleaned up the code very nicely we kept that, as well
   as the trace events for preempt and irqs disabling, but they are
   limited to not being called in NMIs.

 - Have trace events use SRCU for "rcu idle" calls. This was required
   for the preempt/irqs off trace events. But it also had to not allow
   them to be called in NMI context. Waiting till Paul makes an NMI safe
   SRCU API.

 - New notrace SRCU API to allow trace events to use SRCU.

 - Addition of mcount-nop option support

 - SPDX headers replacing GPL templates.

 - Various other fixes and clean ups.

 - Some fixes are marked for stable, but were not fully tested before
   the merge window opened.

* tag 'trace-v4.19' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace: (44 commits)
  tracing: Fix SPDX format headers to use C++ style comments
  tracing: Add SPDX License format tags to tracing files
  tracing: Add SPDX License format to bpf_trace.c
  blktrace: Add SPDX License format header
  s390/ftrace: Add -mfentry and -mnop-mcount support
  tracing: Add -mcount-nop option support
  tracing: Avoid calling cc-option -mrecord-mcount for every Makefile
  tracing: Handle CC_FLAGS_FTRACE more accurately
  Uprobe: Additional argument arch_uprobe to uprobe_write_opcode()
  Uprobes: Simplify uprobe_register() body
  tracepoints: Free early tracepoints after RCU is initialized
  uprobes: Use synchronize_rcu() not synchronize_sched()
  tracing: Fix synchronizing to event changes with tracepoint_synchronize_unregister()
  ftrace: Remove unused pointer ftrace_swapper_pid
  tracing: More reverting of "tracing: Centralize preemptirq tracepoints and unify their usage"
  tracing/irqsoff: Handle preempt_count for different configs
  tracing: Partial revert of "tracing: Centralize preemptirq tracepoints and unify their usage"
  tracing: irqsoff: Account for additional preempt_disable
  trace: Use rcu_dereference_raw for hooks from trace-event subsystem
  tracing/kprobes: Fix within_notrace_func() to check only notrace functions
  ...
2018-08-20 18:32:00 -07:00
Vasily Gorbik
07d0408120 tracing: Avoid calling cc-option -mrecord-mcount for every Makefile
Currently if CONFIG_FTRACE_MCOUNT_RECORD is enabled -mrecord-mcount
compiler flag support is tested for every Makefile.

Top 4 cc-option usages:
    511 -mrecord-mcount
     11  -fno-stack-protector
      9 -Wno-override-init
      2 -fsched-pressure

To address that move cc-option from scripts/Makefile.build to top Makefile
and export CC_USING_RECORD_MCOUNT to be used in original place.

While doing that also add -mrecord-mcount to CC_FLAGS_FTRACE (if gcc
actually supports it).

Link: http://lkml.kernel.org/r/patch-2.thread-aa7b8d.git-de935bace15a.your-ad-here.call-01533557518-ext-9465@work.hours

Acked-by: Andi Kleen <ak@linux.intel.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2018-08-15 22:37:22 -04:00
Masahiro Yamada
d6c6ab93e1 kbuild: remove deprecated host-progs variable
The host-progs has been kept as an alias of hostprogs-y for a long time
(at least since the beginning of Git era), with the clear prompt:
  Usage of host-progs is deprecated. Please replace with hostprogs-y!

Enough time for the migration has passed.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Max Filippov <jcmvbkbc@gmail.com>
2018-08-09 21:51:17 +09:00
Masahiro Yamada
6916162c73 kbuild: remove duplicated comments about PHONY
The comment is the same as in the top-level Makefile.

Also, the comments contain typos:
  - the .PHONY variable  ->  the PHONY variable
  - se we can ...        ->  so we can ...

Instead of fixing the typos, just remove the duplicated comments.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-07-06 22:04:03 +09:00
Greg Thelen
ed7d40bc67 tracing: Fix SKIP_STACK_VALIDATION=1 build due to bad merge with -mrecord-mcount
Non gcc-5 builds with CONFIG_STACK_VALIDATION=y and
SKIP_STACK_VALIDATION=1 fail.
Example output:
  /bin/sh: init/.tmp_main.o: Permission denied

commit 96f60dfa58 ("trace: Use -mcount-record for dynamic ftrace"),
added a mismatched endif.  This causes cmd_objtool to get mistakenly
set.

Relocate endif to balance the newly added -record-mcount check.

Link: http://lkml.kernel.org/r/20180608214746.136554-1-gthelen@google.com

Fixes: 96f60dfa58 ("trace: Use -mcount-record for dynamic ftrace")
Acked-by: Andi Kleen <ak@linux.intel.com>
Tested-by: David Rientjes <rientjes@google.com>
Signed-off-by: Greg Thelen <gthelen@google.com>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2018-06-21 15:12:56 -04:00
Linus Torvalds
5eb6eed7e0 One new feature was added to ftrace, which is the trace_marker now supports
triggers. For example:
 
   # cd /sys/kernel/debug/tracing
   # echo 'snapshot' > events/ftrace/print/trigger
   # echo 'cause snapshot' > trace_marker
 
 The rest of the changes are various clean ups and also one stable fix that
 was added late in the cycle.
 -----BEGIN PGP SIGNATURE-----
 
 iIoEABYIADIWIQRRSw7ePDh/lE+zeZMp5XQQmuv6qgUCWxftjBQccm9zdGVkdEBn
 b29kbWlzLm9yZwAKCRAp5XQQmuv6qosyAQCgmhb+/HgRYIbsKa8Q70E2pMx5A/za
 4rQ37Q9PhDUJlAD/aWXfrqcMm+buAU0yY4K+eb+f2feKXWXxdZ/fuh7+nA4=
 =lpXM
 -----END PGP SIGNATURE-----

Merge tag 'trace-v4.18' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace

Pull tracing updates from Steven Rostedt:
 "One new feature was added to ftrace, which is the trace_marker now
  supports triggers. For example:

    # cd /sys/kernel/debug/tracing
    # echo 'snapshot' > events/ftrace/print/trigger
    # echo 'cause snapshot' > trace_marker

  The rest of the changes are various clean ups and also one stable fix
  that was added late in the cycle"

* tag 'trace-v4.18' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace: (21 commits)
  tracing: Use match_string() instead of open coding it in trace_set_options()
  branch-check: fix long->int truncation when profiling branches
  ring-buffer: Fix typo in comment
  ring-buffer: Fix a bunch of typos in comments
  tracing/selftest: Add test to test simple snapshot trigger for trace_marker
  tracing/selftest: Add test to test hist trigger between kernel event and trace_marker
  tracing/selftest: Add selftests to test trace_marker histogram triggers
  ftrace/selftest: Fix reset_trigger() to handle triggers with filters
  ftrace/selftest: Have the reset_trigger code be a bit more careful
  tracing: Document trace_marker triggers
  tracing: Allow histogram triggers to access ftrace internal events
  tracing: Prevent further users of zero size static arrays in trace events
  tracing: Have zero size length in filter logic be full string
  tracing: Add trigger file for trace_markers tracefs/ftrace/print
  tracing: Do not show filter file for ftrace internal events
  tracing: Add brackets in ftrace event dynamic arrays
  tracing: Have event_trace_init() called by trace_init_tracefs()
  tracing: Add __find_event_file() to find event files without restrictions
  tracing: Do not reference event data in post call triggers
  tracepoints: Fix the descriptions of tracepoint_probe_register{_prio}
  ...
2018-06-06 16:39:18 -07:00
Andi Kleen
96f60dfa58 trace: Use -mcount-record for dynamic ftrace
gcc 5 supports a new -mcount-record option to generate ftrace
tables directly. This avoids the need to run record_mcount
manually.

Use this option when available.

So far doesn't use -mcount-nop, which also exists now.

This is needed to make ftrace work with LTO because the
normal record-mcount script doesn't run over the link
time output.

It should also improve build times slightly in the general
case.
Link: http://lkml.kernel.org/r/20171127213423.27218-12-andi@firstfloor.org

Signed-off-by: Andi Kleen <ak@linux.intel.com>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2018-05-28 12:49:51 -04:00
Masahiro Yamada
704db5433f kbuild: remove CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIX
CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIX was selected by BLACKFIN, METAG.
They were removed by commit 4ba66a9760 ("arch: remove blackfin port"),
commit bb6fb6dfcc ("metag: Remove arch/metag/"), respectively.

No more architecture enables CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIX.
Clean up the rest of scripts, and remove the Kconfig entry.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Sam Ravnborg <sam@ravnborg.org>
2018-05-17 22:44:57 +09:00
Masahiro Yamada
74d9317161 genksyms: remove symbol prefix support
CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIX was selected by BLACKFIN, METAG.
They were removed by commit 4ba66a9760 ("arch: remove blackfin port"),
commit bb6fb6dfcc ("metag: Remove arch/metag/"), respectively.

No more architecture enables CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIX,
hence the -s (--symbol-prefix) option is unnecessary.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Sam Ravnborg <sam@ravnborg.org>
2018-05-17 22:43:35 +09:00
Masahiro Yamada
54a702f705 kbuild: mark $(targets) as .SECONDARY and remove .PRECIOUS markers
GNU Make automatically deletes intermediate files that are updated
in a chain of pattern rules.

Example 1) %.dtb.o <- %.dtb.S <- %.dtb <- %.dts
Example 2) %.o <- %.c <- %.c_shipped

A couple of makefiles mark such targets as .PRECIOUS to prevent Make
from deleting them, but the correct way is to use .SECONDARY.

  .SECONDARY
    Prerequisites of this special target are treated as intermediate
    files but are never automatically deleted.

  .PRECIOUS
    When make is interrupted during execution, it may delete the target
    file it is updating if the file was modified since make started.
    If you mark the file as precious, make will never delete the file
    if interrupted.

Both can avoid deletion of intermediate files, but the difference is
the behavior when Make is interrupted; .SECONDARY deletes the target,
but .PRECIOUS does not.

The use of .PRECIOUS is relatively rare since we do not want to keep
partially constructed (possibly corrupted) targets.

Another difference is that .PRECIOUS works with pattern rules whereas
.SECONDARY does not.

  .PRECIOUS: $(obj)/%.lex.c

works, but

  .SECONDARY: $(obj)/%.lex.c

has no effect.  However, for the reason above, I do not want to use
.PRECIOUS which could cause obscure build breakage.

The targets specified as .SECONDARY must be explicit.  $(targets)
contains all targets that need to include .*.cmd files.  So, the
intermediates you want to keep are mostly in there.  Therefore, mark
$(targets) as .SECONDARY.  It means primary targets are also marked
as .SECONDARY, but I do not see any drawback for this.

I replaced some .SECONDARY / .PRECIOUS markers with 'targets'.  This
will make Kbuild search for non-existing .*.cmd files, but this is
not a noticeable performance issue.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Frank Rowand <frowand.list@gmail.com>
Acked-by: Ingo Molnar <mingo@kernel.org>
2018-04-07 19:04:02 +09:00
Masahiro Yamada
4fa8bc949d kbuild: rename *-asn1.[ch] to *.asn1.[ch]
Our convention is to distinguish file types by suffixes with a period
as a separator.

*-asn1.[ch] is a different pattern from other generated sources such
as *.lex.c, *.tab.[ch], *.dtb.S, etc.  More confusing, files with
'-asn1.[ch]' are generated files, but '_asn1.[ch]' are checked-in
files:
  net/netfilter/nf_conntrack_h323_asn1.c
  include/linux/netfilter/nf_conntrack_h323_asn1.h
  include/linux/sunrpc/gss_asn1.h

Rename generated files to *.asn1.[ch] for consistency.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-04-07 19:04:02 +09:00
Masahiro Yamada
a7f9241909 kbuild: add %.dtb.S and %.dtb to 'targets' automatically
Another common pattern that consists of chained commands is to compile
a DTB as binary data into the kernel image or a module.  It is used in
several places in the source tree.  Support it in the core Makefile.

$(call if_changed,dt_S_dtb) is more suitable than $(call cmd,dt_S_dtb)
in case cmd_dt_S_dtb is changed in the future.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Frank Rowand <frowand.list@gmail.com>
2018-04-07 19:04:02 +09:00
Masahiro Yamada
b23d1a241f kbuild: add %.lex.c and %.tab.[ch] to 'targets' automatically
Files generated by if_changed* must be added to 'targets' to include
*.cmd files.  Otherwise, they would be regenerated every time.

The build system automatically adds objects to 'targets' where
appropriate, such as obj-y, extra-y, etc. but does nothing for
intermediate files.  So, each Makefile needs to add them by itself.

There are some common cases where objects are generated by chained
rules.  Lexers and parsers are compiled like follows:

   %.lex.o <- %.lex.c <- %.l
   %.tab.o <- %.tab.c <- %.y

They are common patterns, so it is reasonable to take care of them
in the core Makefile instead of requiring each Makefile to do so.

At this moment, you cannot delete 'target += zconf.lex.c' in the
Kconfig Makefile because zconf.lex.c is included from zconf.tab.c
instead of being compiled separately.  It should be deleted after
Kconfig is more refactored.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Frank Rowand <frowand.list@gmail.com>
2018-04-07 19:04:02 +09:00
Masahiro Yamada
127668cf76 kbuild: clean up link rule of composite modules
cmd_link_multi-link is used only for cmd_link_multi-m.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-03-26 02:01:29 +09:00
Masahiro Yamada
5e18f0290f kbuild: clean up archive rule of built-in.a
With the incremental linking entirely dropped, we can simplify
the Makefile.

While I am here, I renamed cmd_link_o_target to cmd_ar_builtin.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-03-26 02:01:28 +09:00
Masahiro Yamada
7657f60e8f kbuild: remove partial section mismatch detection for built-in.a
When built-in.o was incrementally linked with 'ld -r', the section
mismatch analysis for the individual built-in.o was possible when
CONFIG_DEBUG_SECTION_MISMATCH was enabled.

With the migration to the thin archive, built-in.a (former, built-in.o)
is no longer an ELF file.  So, the modpost does nothing useful.
scripts/mod/modpost.c just checks the header to bail out, as follows:

        /* Is this a valid ELF file? */
        if ((hdr->e_ident[EI_MAG0] != ELFMAG0) ||
            (hdr->e_ident[EI_MAG1] != ELFMAG1) ||
            (hdr->e_ident[EI_MAG2] != ELFMAG2) ||
            (hdr->e_ident[EI_MAG3] != ELFMAG3)) {
                /* Not an ELF file - silently ignore it */
                return 0;
        }

We have the full analysis in the final link stage anyway, so we would
not miss the section mismatching.

I do not see a good reason to require extra linking only for the
purpose of the per-directory analysis.  Just get rid of this part.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-03-26 02:01:28 +09:00
Masahiro Yamada
f98fe47ce5 kbuild: link $(real-obj-y) instead of $(obj-y) into built-in.a
In Kbuild, Makefiles can add the same object to obj-y multiple
times.  So,

   obj-y += foo.o
   obj-y += foo.o

is fine.

However, this is not true when the same object is added multiple
times via composite objects.  For example,

   obj-y    += foo.o bar.o
   foo-objs := foo-bar-common.o foo-only.o
   bar-objs := foo-bar-common.o bar-only.o

causes build error because two instances of foo-bar-common.o are
linked into the vmlinux.

Makefiles tend to invent ugly work-around, for example
  - lib/zstd/Makefile
  - drivers/net/ethernet/cavium/liquidio/Makefile

The technique used in Kbuild to avoid the multiple definition error
is to use $(filter $(obj-y), $^).  Here, $^ lists the names of all
the prerequisites with duplicated names removed.

By replacing it with $(filter $(real-obj-y), $^) we can do likewise
for composite objects.  For built-in objects, we do not need to keep
the composite object structure.  We can simply expand them, and link
$(real-obj-y) to built-in.a.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-03-26 02:01:27 +09:00
Masahiro Yamada
f5f336812c kbuild: rename real-objs-y/m to real-obj-y/m
When I was refactoring Makefiles, I stupidly mistook 'real-obj-y' for
'real-objs-y' over and over again.  Finally, I decide to rename it to
'real-obj-y'.  This is consistent with 'obj-y', 'subdir-obj-y'.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-03-26 02:01:26 +09:00
Masahiro Yamada
c0152e9a6b kbuild: move modname and modname-multi close to modname_flags
Just a cosmetic change to put related code close together.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Cao jin <caoj.fnst@cn.fujitsu.com>
2018-03-26 02:01:26 +09:00
Masahiro Yamada
fe852ac200 kbuild: simplify modname calculation
modname can be calculated much more simply.  If modname-multi is
empty, it is a single-used object.  So, modname = $(basetarget).
Otherwise, modname = $(modname-multi).

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Cao jin <caoj.fnst@cn.fujitsu.com>
2018-03-26 02:01:25 +09:00
Nicholas Piggin
f49821ee32 kbuild: rename built-in.o to built-in.a
Incremental linking is gone, so rename built-in.o to built-in.a, which
is the usual extension for archive files.

This patch does two things, first is a simple search/replace:

git grep -l 'built-in\.o' | xargs sed -i 's/built-in\.o/built-in\.a/g'

The second is to invert nesting of nested text manipulations to avoid
filtering built-in.a out from libs-y2:

-libs-y2 := $(filter-out %.a, $(patsubst %/, %/built-in.a, $(libs-y)))
+libs-y2 := $(patsubst %/, %/built-in.a, $(filter-out %.a, $(libs-y)))

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-03-26 02:01:19 +09:00
Nicholas Piggin
6358d6e8b9 kbuild: remove incremental linking option
This removes the old `ld -r` incremental link option, which has not
been selected by any architecture since June 2017.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-03-26 02:01:19 +09:00
Michael Forney
1fe7d2bb24 kbuild: Improve portability of some sed invocations
* Use BREs where EREs aren't necessary.
* Pass -E instead of -r to use EREs. This will be standardized in the
  next POSIX revision[0]. GNU sed supports this since 4.2 (May 2009),
  and busybox since 1.22.0 (Jan 2014).
* Use the [:space:] character class instead of ` \t` in bracket
  expressions. In bracket expressions, POSIX says that <backslash> loses
  its special meaning, so a conforming implementation cannot expand \t
  to <tab>[1].
* In BREs, use interval expressions (\{n,m\}) instead of non-standard
  features like \+ and \?.
* Use a loop instead of -s flag.

There are still plenty of other cases of non-standard sed invocations
(use of ERE features in BREs, in-place editing), but this fixes some
core ones.

[0] http://austingroupbugs.net/view.php?id=528
[1] http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09_03_05

Signed-off-by: Michael Forney <forney@google.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-03-26 02:01:18 +09:00
Peter Zijlstra
d5028ba8ee objtool, retpolines: Integrate objtool with retpoline support more closely
Disable retpoline validation in objtool if your compiler sucks, and otherwise
select the validation stuff for CONFIG_RETPOLINE=y (most builds would already
have it set due to ORC).

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2018-02-21 16:54:32 +01:00
Peter Zijlstra
ca41b97ed9 objtool: Add module specific retpoline rules
David allowed retpolines in .init.text, except for modules, which will
trip up objtool retpoline validation, fix that.

Requested-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2018-02-21 09:05:05 +01:00
Peter Zijlstra
b5bc2231b8 objtool: Add retpoline validation
David requested a objtool validation pass for CONFIG_RETPOLINE=y enabled
builds, where it validates no unannotated indirect  jumps or calls are
left.

Add an additional .discard.retpoline_safe section to allow annotating
the few indirect sites that are required and safe.

Requested-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: David Woodhouse <dwmw@amazon.co.uk>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2018-02-21 09:05:04 +01:00
Linus Torvalds
88dc7fca18 Merge branch 'x86-pti-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull x86 pti bits and fixes from Thomas Gleixner:
 "This last update contains:

   - An objtool fix to prevent a segfault with the gold linker by
     changing the invocation order. That's not just for gold, it's a
     general robustness improvement.

   - An improved error message for objtool which spares tearing hairs.

   - Make KASAN fail loudly if there is not enough memory instead of
     oopsing at some random place later

   - RSB fill on context switch to prevent RSB underflow and speculation
     through other units.

   - Make the retpoline/RSB functionality work reliably for both Intel
     and AMD

   - Add retpoline to the module version magic so mismatch can be
     detected

   - A small (non-fix) update for cpufeatures which prevents cpu feature
     clashing for the upcoming extra mitigation bits to ease
     backporting"

* 'x86-pti-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  module: Add retpoline tag to VERMAGIC
  x86/cpufeature: Move processor tracing out of scattered features
  objtool: Improve error message for bad file argument
  objtool: Fix seg fault with gold linker
  x86/retpoline: Add LFENCE to the retpoline/RSB filling RSB macros
  x86/retpoline: Fill RSB on context switch for affected CPUs
  x86/kasan: Panic if there is not enough memory to boot
2018-01-17 11:54:56 -08:00
Josh Poimboeuf
2a0098d706 objtool: Fix seg fault with gold linker
Objtool segfaults when the gold linker is used with
CONFIG_MODVERSIONS=y and CONFIG_UNWINDER_ORC=y.

With CONFIG_MODVERSIONS=y, the .o file gets passed to the linker before
being passed to objtool.  The gold linker seems to strip unused ELF
symbols by default, which confuses objtool and causes the seg fault when
it's trying to generate ORC metadata.

Objtool should really be running immediately after GCC anyway, without a
linker call in between.  Change the makefile ordering so that objtool is
called before the linker.

Reported-and-tested-by: Markus <M4rkusXXL@web.de>
Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Fixes: ee9f8fce99 ("x86/unwind: Add the ORC unwinder")
Link: http://lkml.kernel.org/r/355f04da33581f4a3bf82e5b512973624a1e23a2.1516025651.git.jpoimboe@redhat.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2018-01-16 01:27:27 +01:00
Ingo Molnar
0fd2e9c53d Merge commit 'upstream-x86-entry' into WIP.x86/mm
Pull in a minimal set of v4.15 entry code changes, for a base for the MM isolation patches.

Signed-off-by: Ingo Molnar <mingo@kernel.org>
2017-12-17 12:58:53 +01:00
Linus Torvalds
5e2fda4776 Kbuild updates for v4.15 (2nd)
- Use pwd instead of /bin/pwd for portability
 
 - Clean up Makefiles
 
 - Fix ld-option for clang
 
 - Fix malloc'ed data size in Kconfig
 
 - Fix parallel building along with coccicheck
 
 - Fix a minor issue of package building
 
 - Prompt to use "rpm-pkg" instead of "rpm"
 
 - Clean up *.i and *.lst patterns by "make clean"
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQIcBAABAgAGBQJaGEuYAAoJED2LAQed4NsG1H8P/1cKSZk7G9tA1L7DndvoTNLz
 oc+wM7dUQ3NBndBsyymGe3/cvmlPoB6tam5otAyHaDjBsiGL6pQAeCi1RjJ24aMO
 BHb3vUWLh7GiGnIpe9uMNYIghuRpHdK53rkh9uPaKXmLZBSHtOdd81wHlZGsUdyY
 6U2HHUM6c6KiueTYuULiP2wXE6u0YhREjb5MUL6SThAoME0tDaf7JnWguo7sKJAv
 CUlw9196LoDtj/2VRN3Rd9baGefqDw05PYqUZ9RWfCcl/h8gm7Xo9jW8THRLTIj5
 hRbqNRDHEBjN0QY1FFghrfQf8Iud7zyZC2bzD+0PeUHtIMsb2zA+0zNzdOLN8y9f
 VC34LcQjpG4k9jgrRabAt5xJKcKPLA3rBfkZdB36NKnyj0EFndGoR5YaYsINFBrd
 Asg58Es6OQVINvLdinInpp2GAZAz30zNuF1VivjKgDIUpMw1yU7x5h+Sea3+b5n2
 hO439hlQj4Hb5yNrKeDVaOUJ8eY4yIUtGAUm1vFXODdC4y5FcH7aNf5GiY5bxC7n
 warleTm/DxCV9CI2MANOJmJgC2XgdE7/JNMNvWQnjcmOvhFywyYE65c26QMLIQSM
 k2L0b26FtlES3aCUUKaAmBvAGEZznturBg0fmpnuIULHvYWf2aLndzWDv6C5LhiQ
 mC3FAJc1rzDfQR7UsPrs
 =a2Td
 -----END PGP SIGNATURE-----

Merge tag 'kbuild-v4.15-2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild

Pull more Kbuild updates from Masahiro Yamada:

 - use 'pwd' instead of '/bin/pwd' for portability

 - clean up Makefiles

 - fix ld-option for clang

 - fix malloc'ed data size in Kconfig

 - fix parallel building along with coccicheck

 - fix a minor issue of package building

 - prompt to use "rpm-pkg" instead of "rpm"

 - clean up *.i and *.lst patterns by "make clean"

* tag 'kbuild-v4.15-2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild:
  kbuild: drop $(extra-y) from real-objs-y
  kbuild: clean up *.i and *.lst patterns by make clean
  kbuild: rpm: prompt to use "rpm-pkg" if "rpm" target is used
  kbuild: pkg: use --transform option to prefix paths in tar
  coccinelle: fix parallel build with CHECK=scripts/coccicheck
  kconfig/symbol.c: use correct pointer type argument for sizeof
  kbuild: Set KBUILD_CFLAGS before incl. arch Makefile
  kbuild: remove all dummy assignments to obj-
  kbuild: create built-in.o automatically if parent directory wants it
  kbuild: /bin/pwd -> pwd
2017-11-25 08:06:30 -10:00
Linus Torvalds
1d3bc6363a A few late-arriving docs updates that have no real reason to wait. There's
a new "Co-Developed-by" tag described by Greg, and a build enhancement from
 Willy to generate docs warnings during a kernel build (but only when
 additional warnings have been requested in general).
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABAgAGBQJaFaXjAAoJEI3ONVYwIuV6AXAQAMDinEQQBxNJmO/PkQrIxT4t
 sgbLfI+Sd9zD/lEb9aC5e99XYqw+SH0H9xtEcOrhwx+fjuzkKa6NKLPWjopGzVxH
 CM+N7lCE3AFuzbBCmcQvQyEelQg+p7rsY+2MGLYxMZINnaHFgTa1lvamcE/wKu5d
 CSXs1K2TwFQEpippYlzTmiWU1Rt3gWKGwtBFgLtikSBqiS3HVr4yn7dgm1WTEpVo
 IZPQYoglHjb8vL/vnVDDsfu1PW6Q1uE1aBSgTFBgFIv3UXJBBSlfWQJ7MQsD12Ww
 ZkkAxssFm6TRa87mtgd68Du0Ebg4wZQJG9fizCSy6yIh1ExYxvG0rUmqGrZ1rRYu
 4F+hukXINn7OK5L2laKNQT8ZWCPP+RoN6YUQpz2dhXC3nULZbd5GI9y8pQTdZjmK
 p39SIovicQltlw8ap9MkzTKxm4mvLo/wjFWhQT4qH2QENKO8uSqA9BpLt5a1gJU5
 dSLKDElph5EhliQAQfN/wXdPnTzSaGovele23zTTOLu2vr2JUSnBWWwiaOwuTHRQ
 OEdzxQceoINnc/iqC1qt8F/57E8BT76YedAlmsn77umTXq3mq28wkd1RCRFcjqz5
 KeqrvC5WMSbWZXZDxow7Pr+CPONUy4WpqVbHbwcQ4V8zlFfGjtQdR6zKN88Mb364
 4oJpUwUMz36HchZpi1Tx
 =ydAe
 -----END PGP SIGNATURE-----

Merge tag 'docs-4.15-2' of git://git.lwn.net/linux

Pull documentation updates from Jonathan Corbet:
 "A few late-arriving docs updates that have no real reason to wait.

  There's a new "Co-Developed-by" tag described by Greg, and a build
  enhancement from Willy to generate docs warnings during a kernel build
  (but only when additional warnings have been requested in general)"

* tag 'docs-4.15-2' of git://git.lwn.net/linux:
  Add optional check for bad kernel-doc comments
  Documentation: fix profile= options in kernel-parameters.txt
  documentation/svga.txt: update outdated file
  kokr/memory-barriers.txt: Fix typo in paring example
  kokr/memory-barriers/txt: Replace uses of "transitive"
  Documentation/process: add Co-Developed-by: tag for patches with multiple authors
2017-11-23 21:01:32 -10:00
Matthew Wilcox
3a025e1d1c Add optional check for bad kernel-doc comments
Implement a '-none' output mode for kernel-doc which will only output
warning messages, and suppresses the warning message about there being
no kernel-doc in the file.

If the build has requested additional warnings, automatically check all
.c files.  This patch does not check .h files.  Enabling the warning
by default would add about 1300 warnings, so it's default off for now.
People who care can use this to check they didn't break the docs and
maybe we'll get all the warnings fixed and be able to enable this check
by default in the future.

Signed-off-by: Matthew Wilcox <mawilcox@microsoft.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-11-20 12:09:11 -07:00
Masahiro Yamada
f7adc3124d kbuild: create built-in.o automatically if parent directory wants it
"obj-y += foo/" syntax requires Kbuild to visit the "foo" subdirectory
and link built-in.o from that directory.  This means foo/Makefile is
responsible for creating built-in.o even if there is no object to
link (in this case, built-in.o is an empty archive).

We have had several fixups like commit 4b024242e8 ("kbuild: Fix
linking error built-in.o no such file or directory"), then ended up
with a complex condition as follows:

  ifneq ($(strip $(obj-y) $(obj-m) $(obj-) $(subdir-m) $(lib-target)),)
  builtin-target := $(obj)/built-in.o
  endif

We still have more cases not covered by the above, so we need to add
  obj- := dummy.o
in several places just for creating empty built-in.o.

A key point is, the parent Makefile knows whether built-in.o is needed
or not.  If a subdirectory needs to create built-in.o, its parent can
tell the fact when descending.

If non-empty $(need-builtin) flag is passed from the parent, built-in.o
should be created.  $(obj-y) should be still checked to support the
single target "%/".  All of ugly tricks will go away.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Sam Ravnborg <sam@ravnborg.org>
2017-11-18 11:38:58 +09:00
Linus Torvalds
09bd7c75e5 Kbuild updates for v4.15
One of the most remarkable improvements in this cycle is, Kbuild is
 now able to cache the result of shell commands.  Some variables are
 expensive to compute, for example, $(call cc-option,...) invokes the
 compiler.  It is not efficient to redo this computation every time,
 even when we are not actually building anything.  Kbuild creates a
 hidden file ".cache.mk" that contains invoked shell commands and
 their results.  The speed-up should be noticeable.
 
 Summary:
 
 - Fix arch build issues (hexagon, sh)
 
 - Clean up various Makefiles and scripts
 
 - Fix wrong usage of {CFLAGS,LDFLAGS}_MODULE in arch Makefiles
 
 - Cache variables that are expensive to compute
 
 - Improve cc-ldopton and ld-option for Clang
 
 - Optimize output directory creation
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQIcBAABAgAGBQJaDxaLAAoJED2LAQed4NsGIHQP/isMxxaIxIAWU56+ZcII74k7
 639VgrKi9n5y25d1dBRTQg+vReHE6E2JbkCqpVOu11t7m0LT7yUK8v3WwyLf1qTN
 GxnqZ/WMQU5/AYVqIWo8jN4FGHpivHJ6qbeiNJM9qN4RAkzG0sZUq746VaFZYmIR
 Lu0Gf4m4qjifkkhXsQdWT5i7yNTidPqaL6GNb+FcFkEHlVre8jma0kJlgfHxru84
 WmETpjQXvHAZ/R61vY6ekAWpqFhw3ecJY96A9npnx+SQVQdSNAdpaU0SK29jB0ON
 /SAfpHg9oa/gD0LFOKV6zkjnAkd4TEjrJEiHHhz5gjT/SbS3T1llBIGZ1oV4X7Y0
 Vlh9KWlm1FJJI4SIzc9qUaQMp6JtLfEfHKJCc45xVaN3fNrDnR8jl80x5+95ELga
 dCkZgnq5u82MtTysCbHBESwDYQaVPyIrh7In+mduglaCqhqj9KoDjoLoiGfCg7SA
 3tPflYVd629w5l5GrazJ40jWn1+ggMtgMOVooJNJ+dINCP+GxsUpH84Ww2Pdic+/
 qLdud6TeqxrZDGzWXqKNLu8alM8NGgSr101l9gIf1oqSyy63duBpMrxGDoIJS3FU
 rFDoFFUhlfkAXNbQHtVGNzKtcpCjURh992j9Fa1+NfMwSce5IHkMwTvPmNSRowi8
 0llLjXhD/bxK6FpdvlV8
 =zIdO
 -----END PGP SIGNATURE-----

Merge tag 'kbuild-v4.15' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild

Pull Kbuild updates from Masahiro Yamada:
 "One of the most remarkable improvements in this cycle is, Kbuild is
  now able to cache the result of shell commands. Some variables are
  expensive to compute, for example, $(call cc-option,...) invokes the
  compiler. It is not efficient to redo this computation every time,
  even when we are not actually building anything. Kbuild creates a
  hidden file ".cache.mk" that contains invoked shell commands and their
  results. The speed-up should be noticeable.

  Summary:

   - Fix arch build issues (hexagon, sh)

   - Clean up various Makefiles and scripts

   - Fix wrong usage of {CFLAGS,LDFLAGS}_MODULE in arch Makefiles

   - Cache variables that are expensive to compute

   - Improve cc-ldopton and ld-option for Clang

   - Optimize output directory creation"

* tag 'kbuild-v4.15' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (30 commits)
  kbuild: move coccicheck help from scripts/Makefile.help to top Makefile
  sh: decompressor: add shipped files to .gitignore
  frv: .gitignore: ignore vmlinux.lds
  selinux: remove unnecessary assignment to subdir-
  kbuild: specify FORCE in Makefile.headersinst as .PHONY target
  kbuild: remove redundant mkdir from ./Kbuild
  kbuild: optimize object directory creation for incremental build
  kbuild: create object directories simpler and faster
  kbuild: filter-out PHONY targets from "targets"
  kbuild: remove redundant $(wildcard ...) for cmd_files calculation
  kbuild: create directory for make cache only when necessary
  sh: select KBUILD_DEFCONFIG depending on ARCH
  kbuild: fix linker feature test macros when cross compiling with Clang
  kbuild: shrink .cache.mk when it exceeds 1000 lines
  kbuild: do not call cc-option before KBUILD_CFLAGS initialization
  kbuild: Cache a few more calls to the compiler
  kbuild: Add a cache for generated variables
  kbuild: add forward declaration of default target to Makefile.asm-generic
  kbuild: remove KBUILD_SUBDIR_ASFLAGS and KBUILD_SUBDIR_CCFLAGS
  hexagon/kbuild: replace CFLAGS_MODULE with KBUILD_CFLAGS_MODULE
  ...
2017-11-17 17:45:29 -08:00
Masahiro Yamada
c4da7ed0e7 kbuild: optimize object directory creation for incremental build
The previous commit largely optimized the object directory creation.
We can optimize it more for incremental build.

There are already *.cmd files in the output directory.  The existing
*.cmd files have been picked up by $(wildcard ...).  Obviously,
directories containing them exist too, so we can skip "mkdir -p".

With this, Kbuild runs almost zero "mkdir -p" in incremental building.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-11-16 09:07:36 +09:00
Masahiro Yamada
8a78756eb5 kbuild: create object directories simpler and faster
For the out-of-tree build, scripts/Makefile.build creates output
directories, but this operation is not efficient.

scripts/Makefile.lib calculates obj-dirs as follows:

  obj-dirs := $(dir $(multi-objs) $(obj-y))

Please notice $(sort ...) is not used here.  Usually the result is
as many "./" as objects here.

For a lot of duplicated paths, the following command is invoked.

  _dummy := $(foreach d,$(obj-dirs), $(shell [ -d $(d) ] || mkdir -p $(d)))

Then, the costly shell command is run over and over again.

I see many points for optimization:

[1] Use $(sort ...) to cut down duplicated paths before passing them
    to system call
[2] Use single $(shell ...) instead of repeating it with $(foreach ...)
    This will reduce forking.
[3] We can calculate obj-dirs more simply.  Most of objects are already
    accumulated in $(targets).  So, $(dir $(targets)) is fine and more
    comprehensive.

I also removed ugly code in arch/x86/entry/vdso/Makefile.  This is now
really unnecessary.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Ingo Molnar <mingo@kernel.org>
Tested-by: Douglas Anderson <dianders@chromium.org>
2017-11-16 09:07:35 +09:00
Masahiro Yamada
591f668997 kbuild: filter-out PHONY targets from "targets"
The variable "targets" contains object paths for which existing .*.cmd
files should be included.

scripts/Makefile.build automatically adds $(MAKECMDGOALS) to "targets"
as follows:

  targets += $(extra-y) $(MAKECMDGOALS) $(always)

The $(MAKECMDGOALS) is a PHONY target in several places.  PHONY targets
never create .*.cmd files.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-11-16 09:07:35 +09:00
Masahiro Yamada
2982c95357 kbuild: remove redundant $(wildcard ...) for cmd_files calculation
I do not see any reason why $(wildcard ...) needs to be called twice
for computing cmd_files.  Remove the first one.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-11-16 09:07:34 +09:00
Ingo Molnar
b3d9a13681 Merge branch 'linus' into x86/asm, to pick up fixes and resolve conflicts
Conflicts:
	arch/x86/kernel/cpu/Makefile

Signed-off-by: Ingo Molnar <mingo@kernel.org>
2017-11-07 10:53:06 +01:00
Linus Torvalds
5cb0512c02 Kbuild: don't pass "-C" to preprocessor when processing linker scripts
For some odd historical reason, we preprocessed the linker scripts with
"-C", which keeps comments around.  That makes no sense, since the
comments are not meaningful for the build anyway.

And it actually breaks things, since linker scripts can't have C++ style
"//" comments in them, so keeping comments after preprocessing now
limits us in odd and surprising ways in our header files for no good
reason.

The -C option goes back to pre-git and pre-bitkeeper times, but seems to
have been historically used (along with "-traditional") for some
odd-ball architectures (ia64, MIPS and SH).  It probably didn't matter
back then either, but might possibly have been used to minimize the
difference between the original file and the pre-processed result.

The reason for this may be lost in time, but let's not perpetuate it
only because we can't remember why we did this crazy thing.

This was triggered by the recent addition of SPDX lines to the source
tree, where people apparently were confused about why header files
couldn't use the C++ comment format.

Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Greg KH <gregkh@linuxfoundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2017-11-02 14:10:37 -07:00
Greg Kroah-Hartman
b24413180f License cleanup: add SPDX GPL-2.0 license identifier to files with no license
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.

By default all files without license information are under the default
license of the kernel, which is GPL version 2.

Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier.  The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.

This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.

How this work was done:

Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
 - file had no licensing information it it.
 - file was a */uapi/* one with no licensing information in it,
 - file was a */uapi/* one with existing licensing information,

Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.

The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne.  Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.

The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed.  Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.

Criteria used to select files for SPDX license identifier tagging was:
 - Files considered eligible had to be source code files.
 - Make and config files were included as candidates if they contained >5
   lines of source
 - File already had some variant of a license header in it (even if <5
   lines).

All documentation files were explicitly excluded.

The following heuristics were used to determine which SPDX license
identifiers to apply.

 - when both scanners couldn't find any license traces, file was
   considered to have no license information in it, and the top level
   COPYING file license applied.

   For non */uapi/* files that summary was:

   SPDX license identifier                            # files
   ---------------------------------------------------|-------
   GPL-2.0                                              11139

   and resulted in the first patch in this series.

   If that file was a */uapi/* path one, it was "GPL-2.0 WITH
   Linux-syscall-note" otherwise it was "GPL-2.0".  Results of that was:

   SPDX license identifier                            # files
   ---------------------------------------------------|-------
   GPL-2.0 WITH Linux-syscall-note                        930

   and resulted in the second patch in this series.

 - if a file had some form of licensing information in it, and was one
   of the */uapi/* ones, it was denoted with the Linux-syscall-note if
   any GPL family license was found in the file or had no licensing in
   it (per prior point).  Results summary:

   SPDX license identifier                            # files
   ---------------------------------------------------|------
   GPL-2.0 WITH Linux-syscall-note                       270
   GPL-2.0+ WITH Linux-syscall-note                      169
   ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause)    21
   ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause)    17
   LGPL-2.1+ WITH Linux-syscall-note                      15
   GPL-1.0+ WITH Linux-syscall-note                       14
   ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause)    5
   LGPL-2.0+ WITH Linux-syscall-note                       4
   LGPL-2.1 WITH Linux-syscall-note                        3
   ((GPL-2.0 WITH Linux-syscall-note) OR MIT)              3
   ((GPL-2.0 WITH Linux-syscall-note) AND MIT)             1

   and that resulted in the third patch in this series.

 - when the two scanners agreed on the detected license(s), that became
   the concluded license(s).

 - when there was disagreement between the two scanners (one detected a
   license but the other didn't, or they both detected different
   licenses) a manual inspection of the file occurred.

 - In most cases a manual inspection of the information in the file
   resulted in a clear resolution of the license that should apply (and
   which scanner probably needed to revisit its heuristics).

 - When it was not immediately clear, the license identifier was
   confirmed with lawyers working with the Linux Foundation.

 - If there was any question as to the appropriate license identifier,
   the file was flagged for further research and to be revisited later
   in time.

In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.

Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights.  The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.

Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.

In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.

Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
 - a full scancode scan run, collecting the matched texts, detected
   license ids and scores
 - reviewing anything where there was a license detected (about 500+
   files) to ensure that the applied SPDX license was correct
 - reviewing anything where there was no detection but the patch license
   was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
   SPDX license was correct

This produced a worksheet with 20 files needing minor correction.  This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.

These .csv files were then reviewed by Greg.  Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected.  This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.)  Finally Greg ran the script using the .csv files to
generate the patches.

Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-11-02 11:10:55 +01:00