Commit Graph

64 Commits

Author SHA1 Message Date
Greg Kroah-Hartman
b24413180f License cleanup: add SPDX GPL-2.0 license identifier to files with no license
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.

By default all files without license information are under the default
license of the kernel, which is GPL version 2.

Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier.  The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.

This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.

How this work was done:

Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
 - file had no licensing information it it.
 - file was a */uapi/* one with no licensing information in it,
 - file was a */uapi/* one with existing licensing information,

Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.

The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne.  Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.

The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed.  Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.

Criteria used to select files for SPDX license identifier tagging was:
 - Files considered eligible had to be source code files.
 - Make and config files were included as candidates if they contained >5
   lines of source
 - File already had some variant of a license header in it (even if <5
   lines).

All documentation files were explicitly excluded.

The following heuristics were used to determine which SPDX license
identifiers to apply.

 - when both scanners couldn't find any license traces, file was
   considered to have no license information in it, and the top level
   COPYING file license applied.

   For non */uapi/* files that summary was:

   SPDX license identifier                            # files
   ---------------------------------------------------|-------
   GPL-2.0                                              11139

   and resulted in the first patch in this series.

   If that file was a */uapi/* path one, it was "GPL-2.0 WITH
   Linux-syscall-note" otherwise it was "GPL-2.0".  Results of that was:

   SPDX license identifier                            # files
   ---------------------------------------------------|-------
   GPL-2.0 WITH Linux-syscall-note                        930

   and resulted in the second patch in this series.

 - if a file had some form of licensing information in it, and was one
   of the */uapi/* ones, it was denoted with the Linux-syscall-note if
   any GPL family license was found in the file or had no licensing in
   it (per prior point).  Results summary:

   SPDX license identifier                            # files
   ---------------------------------------------------|------
   GPL-2.0 WITH Linux-syscall-note                       270
   GPL-2.0+ WITH Linux-syscall-note                      169
   ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause)    21
   ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause)    17
   LGPL-2.1+ WITH Linux-syscall-note                      15
   GPL-1.0+ WITH Linux-syscall-note                       14
   ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause)    5
   LGPL-2.0+ WITH Linux-syscall-note                       4
   LGPL-2.1 WITH Linux-syscall-note                        3
   ((GPL-2.0 WITH Linux-syscall-note) OR MIT)              3
   ((GPL-2.0 WITH Linux-syscall-note) AND MIT)             1

   and that resulted in the third patch in this series.

 - when the two scanners agreed on the detected license(s), that became
   the concluded license(s).

 - when there was disagreement between the two scanners (one detected a
   license but the other didn't, or they both detected different
   licenses) a manual inspection of the file occurred.

 - In most cases a manual inspection of the information in the file
   resulted in a clear resolution of the license that should apply (and
   which scanner probably needed to revisit its heuristics).

 - When it was not immediately clear, the license identifier was
   confirmed with lawyers working with the Linux Foundation.

 - If there was any question as to the appropriate license identifier,
   the file was flagged for further research and to be revisited later
   in time.

In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.

Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights.  The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.

Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.

In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.

Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
 - a full scancode scan run, collecting the matched texts, detected
   license ids and scores
 - reviewing anything where there was a license detected (about 500+
   files) to ensure that the applied SPDX license was correct
 - reviewing anything where there was no detection but the patch license
   was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
   SPDX license was correct

This produced a worksheet with 20 files needing minor correction.  This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.

These .csv files were then reviewed by Greg.  Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected.  This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.)  Finally Greg ran the script using the .csv files to
generate the patches.

Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-11-02 11:10:55 +01:00
Masahiro Yamada
8e9b466799 kbuild: use $(abspath ...) instead of $(shell cd ... && /bin/pwd)
Kbuild conventionally uses $(shell cd ... && /bin/pwd) idiom to get
the absolute path of the directory because GNU Make 3.80, the minimal
supported version at that time, did not support $(abspath ...) or
$(realpath ...).

Commit 37d69ee308 ("docs: bump minimal GNU Make version to 3.81")
dropped the GNU Make 3.80 support, so we are now allowed to use those
make-builtin helpers.

This conversion will provide better portability without relying on
the pwd command or its location /bin/pwd.

I am intentionally using $(realpath ...) instead $(abspath ...) in
some places.  The difference between the two is $(realpath ...)
returns an empty string if the given path does not exist.  It is
convenient in places where we need to error-out if the makefile fails
to create an output directory.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Thierry Reding <treding@nvidia.com>
2017-09-01 08:50:32 +09:00
Leonard Crestez
46d10a0943 scripts/gdb: lx-dmesg: use explicit encoding=utf8 errors=replace
Use errors=replace because it is never desirable for lx-dmesg to fail on
string decoding errors, not even if the log buffer is corrupt and we
show incorrect info.

The kernel will sometimes print utf8, for example the copyright symbol
from jffs2.  In order to make this work specify 'utf8' everywhere
because python2 otherwise defaults to 'ascii'.

In theory the second errors='replace' is not be required because
everything that can be decoded as utf8 should also be encodable back to
utf8.  But it's better to be extra safe here.  It's worth noting that
this is definitely not true for encoding='ascii', unknown characters are
replaced with U+FFFD REPLACEMENT CHARACTER and they fail to encode back
to ascii.

Link: http://lkml.kernel.org/r/acee067f3345954ed41efb77b80eebdc038619c6.1498481469.git.leonard.crestez@nxp.com
Signed-off-by: Leonard Crestez <leonard.crestez@nxp.com>
Acked-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Jason Wessel <jason.wessel@windriver.com>
Cc: Kieran Bingham <kieran@ksquared.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2017-07-12 16:26:01 -07:00
Leonard Crestez
c454756f47 scripts/gdb: lx-dmesg: cast log_buf to void* for addr fetch
In some cases it is possible for the str() conversion here to throw
encoding errors because log_buf might not point to valid ascii.  For
example:

  (gdb) python print str(gdb.parse_and_eval("log_buf"))
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
  UnicodeEncodeError: 'ascii' codec can't encode character u'\u0303' in
  	position 24: ordinal not in range(128)

Avoid this by explicitly casting to (void *) inside the gdb expression.

Link: http://lkml.kernel.org/r/ba6f85dbb02ca980ebd0e2399b0649423399b565.1498481469.git.leonard.crestez@nxp.com
Signed-off-by: Leonard Crestez <leonard.crestez@nxp.com>
Reviewed-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Jason Wessel <jason.wessel@windriver.com>
Cc: Kieran Bingham <kieran@ksquared.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2017-07-12 16:26:01 -07:00
Peter Griffin
821f74402a scripts/gdb: add lx-fdtdump command
lx-fdtdump dumps the flattened device tree passed to the kernel from the
bootloader to the filename specified as the command argument.  If no
argument is provided it defaults to fdtdump.dtb.  This then allows
further post processing on the machine running GDB.  The fdt header is
also also printed in the GDB console.  For example:

  (gdb) lx-fdtdump
  fdt_magic:         0xD00DFEED
  fdt_totalsize:     0xC108
  off_dt_struct:     0x38
  off_dt_strings:    0x3804
  off_mem_rsvmap:    0x28
  version:           17
  last_comp_version: 16
  Dumped fdt to fdtdump.dtb

  >fdtdump fdtdump.dtb | less

This command is useful as the bootloader can often re-write parts of the
device tree, and this can sometimes cause the kernel to not boot.

Link: http://lkml.kernel.org/r/1481280065-5336-2-git-send-email-kbingham@kernel.org
Signed-off-by: Peter Griffin <peter.griffin@linaro.org>
Signed-off-by: Kieran Bingham <kbingham@kernel.org>
Cc: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2017-07-12 16:26:00 -07:00
André Draszik
d6c9708737 scripts/gdb: make lx-dmesg command work (reliably)
lx-dmesg needs access to the log_buf symbol from printk.c.
Unfortunately, the symbol log_buf also exists in BPF's verifier.c and
hence gdb can pick one or the other.  If it happens to pick BPF's
log_buf, lx-dmesg doesn't work:

  (gdb) lx-dmesg
  Python Exception <class 'gdb.MemoryError'> Cannot access memory at address 0x0:
  Error occurred in Python command: Cannot access memory at address 0x0
  (gdb) p log_buf
  $15 = 0x0

Luckily, GDB has a way to deal with this, see
  https://sourceware.org/gdb/onlinedocs/gdb/Symbols.html

  (gdb) info variables ^log_buf$
  All variables matching regular expression "^log_buf$":

  File <linux.git>/kernel/bpf/verifier.c:
  static char *log_buf;

  File <linux.git>/kernel/printk/printk.c:
  static char *log_buf;
  (gdb) p 'verifier.c'::log_buf
  $1 = 0x0
  (gdb) p 'printk.c'::log_buf
  $2 = 0x811a6aa0 <__log_buf> ""
  (gdb) p &log_buf
  $3 = (char **) 0x8120fe40 <log_buf>
  (gdb) p &'verifier.c'::log_buf
  $4 = (char **) 0x8120fe40 <log_buf>
  (gdb) p &'printk.c'::log_buf
  $5 = (char **) 0x8048b7d0 <log_buf>

By being explicit about the location of the symbol, we can make lx-dmesg
work again.  While at it, do the same for the other symbols we need from
printk.c

Link: http://lkml.kernel.org/r/20170526112222.3414-1-git@andred.net
Signed-off-by: André Draszik <git@andred.net>
Tested-by: Kieran Bingham <kieran@bingham.xyz>
Acked-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2017-06-02 15:07:38 -07:00
Kieran Bingham
b447e02548 Revert "scripts/gdb: add a Radix Tree Parser"
This reverts commit e127a73d41 ("scripts/gdb: add a Radix Tree
Parser")

The python implementation of radix-tree was merged at the same time as
the radix-tree system was heavily reworked from commit e9256efcc8
("radix-tree: introduce radix_tree_empty") to 3bcadd6fa6 ("radix-tree:
free up the bottom bit of exceptional entries for reuse") and no longer
functions, but also prevents other gdb scripts from loading.

This functionality has not yet hit a release, so simply remove it for
now

Link: http://lkml.kernel.org/r/1467127337-11135-6-git-send-email-kieran@bingham.xyz
Signed-off-by: Kieran Bingham <kieran@bingham.xyz>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-07-15 14:54:27 +09:00
Nikolay Borisov
552ab2a3ea scripts/gdb: Perform path expansion to lx-symbol's arguments
Python doesn't do automatic expansion of paths.  In case one passes path
of the from ~/foo/bar the gdb scripts won't automatically expand that
and as a result the symbols files won't be loaded.

Fix this by explicitly expanding all paths which begin with "~"

Link: http://lkml.kernel.org/r/1467127337-11135-5-git-send-email-kieran@bingham.xyz
Signed-off-by: Nikolay Borisov <n.borisov.lkml@gmail.com>
Signed-off-by: Kieran Bingham <kieran@bingham.xyz>
Reviewed-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-07-15 14:54:27 +09:00
Omar Sandoval
e2aa2f8fac scripts/gdb: add constants.py to .gitignore
Since scripts/gdb/linux/constants.py is autogenerated, this should have
been added to .gitignore when it was introduced.

Fixes: f197d75fca ("scripts/gdb: provide linux constants")
Link: http://lkml.kernel.org/r/1467127337-11135-4-git-send-email-kieran@bingham.xyz
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: Kieran Bingham <kieran@bingham.xyz>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-07-15 14:54:27 +09:00
Kieran Bingham
834a35296a scripts/gdb: rebuild constants.py on dependancy change
The autogenerated constants.py file was only being built on the initial
call, and if the constants.py.in file changed.  As we are utilising the
CPP hooks, we can successfully use the call if_changed_dep rules to
determine when to rebuild the file based on it's inclusions.

Link: http://lkml.kernel.org/r/1467127337-11135-3-git-send-email-kieran@bingham.xyz
Signed-off-by: Kieran Bingham <kieran@bingham.xyz>
Reported-by: Jan Kiszka <jan.kiszka@web.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-07-15 14:54:27 +09:00
Kieran Bingham
abb035b482 scripts/gdb: silence 'nothing to do' message
The constants.py generation, involves a rule to link into the main
makefile.  This rule has no command and generates a spurious warning
message in the build logs when CONFIG_SCRIPTS_GDB is enabled.

Fix simply by giving a no-op action

Link: http://lkml.kernel.org/r/1467127337-11135-2-git-send-email-kieran@bingham.xyz
Signed-off-by: Kieran Bingham <kieran@bingham.xyz>
Reported-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-07-15 14:54:27 +09:00
Kieran Bingham
b3b0842985 scripts/gdb: decode bytestream on dmesg for Python3
The recent fixes to lx-dmesg, now allow the command to print
successfully on Python3, however the python interpreter wraps the bytes
for each line with a b'<text>' marker.

To remove this, we need to decode the line, where .decode() will default
to 'UTF-8'

Link: http://lkml.kernel.org/r/d67ccf93f2479c94cb3399262b9b796e0dbefcf2.1462865983.git.jan.kiszka@siemens.com
Signed-off-by: Kieran Bingham <kieran@bingham.xyz>
Acked-by: Dom Cote <buzdelabuz2@gmail.com>
Tested-by: Dom Cote <buzdelabuz2@gmail.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-05-23 17:04:14 -07:00
Dom Cote
d21d5b9eb0 scripts/gdb: fix issue with dmesg.py and python 3.X
When built against Python 3, GDB differs in the return type for its
read_memory function, causing the lx-dmesg command to fail.

Now that we have an improved read_16() we can use the new
read_memoryview() abstraction to make lx-dmesg return valid data on both
current Python APIs

Tested with python 3.4 and 2.7
Tested with gdb 7.7

Link: http://lkml.kernel.org/r/28477b727ff7fe3101fd4e426060e8a68317a639.1462865983.git.jan.kiszka@siemens.com
Signed-off-by: Dom Cote <buzdelabuz2+git@gmail.com>
[kieran@bingham.xyz: Adjusted commit log to better reflect code changes]
Tested-by: Kieran Bingham <kieran@bingham.xyz> (Py2.7,Py3.4,GDB10)
Signed-off-by: Kieran Bingham <kieran@bingham.xyz>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-05-23 17:04:14 -07:00
Dom Cote
321958d971 scripts/gdb: improve types abstraction for gdb python scripts
Change the read_u16 function so it accepts both 'str' and 'byte' as type
for the arguments.

When calling read_memory() from gdb API, depending on if it was built
with 2.7 or 3.X, the format used to return the data will differ ( 'str'
for 2.7, and 'byte' for 3.X ).

Add a function read_memoryview() to be able to get a 'memoryview' object
back from read_memory() both with python 2.7 and 3.X .

Tested with python 3.4 and 2.7
Tested with gdb 7.7

Link: http://lkml.kernel.org/r/73621f564503137a002a639d174e4fb35f73f462.1462865983.git.jan.kiszka@siemens.com
Signed-off-by: Dom Cote <buzdelabuz2+git@gmail.com>
Tested-by: Kieran Bingham <kieran@bingham.xyz> (Py2.7,Py3.4,GDB10)
Signed-off-by: Kieran Bingham <kieran@bingham.xyz>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-05-23 17:04:14 -07:00
Kieran Bingham
9f66dee720 scripts/gdb: add lx_thread_info_by_pid helper
The tasks module already provides helpers to find the task struct by
pid, and the thread_info by task struct; however this is cumbersome to
utilise on the gdb commandline.

Wrap these two functionalities together in an extra single helper to
allow exploring the thread info, from a PID value

Link: http://lkml.kernel.org/r/dadc5667f053ec811eb3e3033d99d937fedbc93b.1462865983.git.jan.kiszka@siemens.com
Signed-off-by: Kieran Bingham <kieran.bingham@linaro.org>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-05-23 17:04:14 -07:00
Kieran Bingham
e127a73d41 scripts/gdb: add a Radix Tree Parser
Linux makes use of the Radix Tree data structure to store pointers
indexed by integer values.  This structure is utilised across many
structures in the kernel including the IRQ descriptor tables, and
several filesystems.

This module provides a method to lookup values from a structure given
its head node.

Usage:

The function lx_radix_tree_lookup, must be given a symbol of type struct
radix_tree_root, and an index into that tree.

The object returned is a generic integer value, and must be cast
correctly to the type based on the storage in the data structure.

For example, to print the irq descriptor in the sparse irq_desc_tree at
index 18, try the following:

 (gdb) print (struct irq_desc)$lx_radix_tree_lookup(irq_desc_tree, 18)

Link: http://lkml.kernel.org/r/d2028c55e50cf95a9b7f8ca0d11885174b0cc709.1462865983.git.jan.kiszka@siemens.com
Signed-off-by: Kieran Bingham <kieran.bingham@linaro.org>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-05-23 17:04:14 -07:00
Jan Kiszka
4bc393dbcf scripts/gdb: cast CPU numbers to integer
We won't see more than 2 billion CPUs any time soon, and having cpu_list
return long makes the output of lx-cpus a bit ugly.

Link: http://lkml.kernel.org/r/dcb45c3b0a59e0fd321fa56ff7aa398458c689b3.1462865983.git.jan.kiszka@siemens.com
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-05-23 17:04:14 -07:00
Kieran Bingham
b1503934a5 scripts/gdb: add cpu iterators
The linux kernel provides macro's for iterating against values from the
cpu_list masks.  By providing some commonly used masks, we can mirror
the kernels helper macros with easy to use generators.

Link: http://lkml.kernel.org/r/d045c6599771ada1999d49612ee30fd2f9acf17f.1462865983.git.jan.kiszka@siemens.com
Signed-off-by: Kieran Bingham <kieran.bingham@linaro.org>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-05-23 17:04:14 -07:00
Kieran Bingham
c1a153992e scripts/gdb: add mount point list command
lx-mounts will identify current mount points based on the 'init_task'
namespace by default, as we do not yet have a kernel thread list
implementation to select the current running thread.

Optionally, a user can specify a PID to list from that process'
namespace

Link: http://lkml.kernel.org/r/e614c7bc32d2350b4ff1627ec761a7148e65bfe6.1462865983.git.jan.kiszka@siemens.com
Signed-off-by: Kieran Bingham <kieran.bingham@linaro.org>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-05-23 17:04:14 -07:00
Kieran Bingham
e7165a2d7d scripts/gdb: add io resource readers
Provide iomem_resource and ioports_resource printers and command hooks

It can be quite interesting to halt the kernel as it's booting and check
to see this list as it is being populated.

It should be useful in the event that a kernel is not booting, you can
identify what memory resources have been registered

Link: http://lkml.kernel.org/r/f0a6b9fa9c92af4d7ed2e7343ccc84150e9c6fc5.1462865983.git.jan.kiszka@siemens.com
Signed-off-by: Kieran Bingham <kieran.bingham@linaro.org>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-05-23 17:04:14 -07:00
Kieran Bingham
74627cf2df scripts/gdb: provide a dentry_name VFS path helper
Walk the VFS entries, pre-pending the iname strings to generate a full
VFS path name from a dentry.

Link: http://lkml.kernel.org/r/4328fdb2d15ba7f1b21ad21c2eecc38d9cfc4d13.1462865983.git.jan.kiszka@siemens.com
Signed-off-by: Kieran Bingham <kieran.bingham@linaro.org>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-05-23 17:04:14 -07:00
Kieran Bingham
958ef8a09a scripts/gdb: support !CONFIG_MODULES gracefully
If CONFIG_MODULES is not enabled, lx-lsmod tries to find a non-existent
symbol and generates an unfriendly traceback:

  (gdb) lx-lsmod
  Address    Module                  Size  Used by
  Traceback (most recent call last):
    File "scripts/gdb/linux/modules.py", line 75, in invoke
      for module in module_list():
    File "scripts/gdb/linux/modules.py", line 24, in module_list
      module_ptr_type = module_type.get_type().pointer()
    File "scripts/gdb/linux/utils.py", line 28, in get_type
      self._type = gdb.lookup_type(self._name)
  gdb.error: No struct type named module.
  Error occurred in Python command: No struct type named module.

Catch the error and return an empty module_list() for a clean command
output as follows:

  (gdb) lx-lsmod
  Address    Module                  Size  Used by
  (gdb)

Link: http://lkml.kernel.org/r/94d533819437408b85ae5864f939dd7ca6fbfcd6.1462865983.git.jan.kiszka@siemens.com
Signed-off-by: Kieran Bingham <kieran.bingham@linaro.org>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-05-23 17:04:14 -07:00
Kieran Bingham
e78f3d70b3 scripts/gdb: provide exception catching parser
If we attempt to read a value that is not available to GDB, an exception
is raised.  Most of the time, this is a good thing; however on occasion
we will want to be able to determine if a symbol is available.

By catching the exception to simply return None, we can determine if we
tried to read an invalid value, without the exception taking our
execution context away from us

Link: http://lkml.kernel.org/r/c72b25c06fc66e1d68371154097e2cbb112555d8.1462865983.git.jan.kiszka@siemens.com
Signed-off-by: Kieran Bingham <kieran.bingham@linaro.org>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-05-23 17:04:14 -07:00
Kieran Bingham
619ccaf3e9 scripts/gdb: convert modules usage to lists functions
Simplify the module list functions with the new list_for_each_entry
abstractions

Link: http://lkml.kernel.org/r/ad0101c9391088608166fcec26af179868973d86.1462865983.git.jan.kiszka@siemens.com
Signed-off-by: Kieran Bingham <kieran.bingham@linaro.org>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-05-23 17:04:14 -07:00
Kieran Bingham
a84be61d0e scripts/gdb: provide kernel list item generators
Facilitate linked-list items by providing a generator to return the
dereferenced, and type-cast objects from a kernel linked list

Link: http://lkml.kernel.org/r/2b0998564e6e5abe53585d466f87e491331fd2a4.1462865983.git.jan.kiszka@siemens.com
Signed-off-by: Kieran Bingham <kieran.bingham@linaro.org>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-05-23 17:04:14 -07:00
Kieran Bingham
f197d75fca scripts/gdb: provide linux constants
Some macro's and defines are needed when parsing memory, and without
compiling the kernel as -g3 they are not available in the debug-symbols.

We use the pre-processor here to extract constants to a dedicated module
for the linux debugger extensions

Top level Kbuild is used to call in and generate the constants file,
while maintaining dependencies on autogenerated files in
include/generated

Link: http://lkml.kernel.org/r/bc3df9c25f57ea72177c066a51a446fc19e2c27f.1462865983.git.jan.kiszka@siemens.com
Signed-off-by: Kieran Bingham <kieran.bingham@linaro.org>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Michal Marek <mmarek@suse.cz>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-05-23 17:04:14 -07:00
Jan Kiszka
0c22fde8b0 scripts/gdb: Adjust module reference counter reported by lx-lsmod
This takes the MODULE_REF_BASE into account.

Link: http://lkml.kernel.org/r/d926d2d54caa034adb964b52215090cbdb875249.1462865983.git.jan.kiszka@siemens.com
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-05-23 17:04:14 -07:00
Jan Kiszka
ad4db3b24a scripts/gdb: account for changes in module data structure
Commit 7523e4dc50 ("module: use a structure to encapsulate layout.")
factored out the module_layout structure.  Adjust the symbol loader and
the lsmod command to this.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Reviewed-by: Kieran Bingham <kieran.bingham@linaro.org>
Tested-by: Kieran Bingham <kieran.bingham@linaro.org> (qemu-{ARM,x86})
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Jiri Kosina <jkosina@suse.cz>
Cc: Jason Wessel <jason.wessel@windriver.com>
Cc: <stable@vger.kernel.org>	[4.4+]
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-03-22 15:36:02 -07:00
Kieran Bingham
72bf92ec29 scripts/gdb: add cmdline reader command
lx-cmdline Report the Linux Commandline used in the current kernel

[jan.kiszka@siemens.com: remove blank line from help output and fix pep8 warning]
Signed-off-by: Kieran Bingham <kieran.bingham@linaro.org>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-03-22 15:36:02 -07:00
Kieran Bingham
2d061d9994 scripts/gdb: add version command
lx-version Report the Linux Version of the current kernel.

Add a command to identify the version specified by the banner in the
debugged kernel.

This lets the user identify the kernel of the running kernel, and will
let later scripts compare the banner of the attached kernel against the
banner in the vmlinux symbols files to verify that the files are
correct.

[jan.kiszka@siemens.com: remove blank line from help output and fix pep8 warning]
Signed-off-by: Kieran Bingham <kieran.bingham@linaro.org>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-03-22 15:36:02 -07:00
Thiébaud Weksteen
15f1d82781 scripts/gdb: remove useless global instruction
Signed-off-by: Thiébaud Weksteen <thiebaud@weksteen.fr>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-06-30 19:44:58 -07:00
Thiébaud Weksteen
a930850b62 scripts/gdb: add ps command
Signed-off-by: Thiébaud Weksteen <thiebaud@weksteen.fr>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-06-30 19:44:58 -07:00
Thiébaud Weksteen
6ad18b7331 scripts/gdb: fix PEP8 compliance
Signed-off-by: Thiébaud Weksteen <thiebaud@weksteen.fr>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-06-30 19:44:58 -07:00
Thiébaud Weksteen
a2e73c4804 scripts/gdb: fix typo in exception name
Signed-off-by: Thiébaud Weksteen <thiebaud@weksteen.fr>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-06-30 19:44:58 -07:00
Jan Kiszka
3328bc9ee4 scripts/gdb: enable completion for lx-list-check parameter
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Thiébaud Weksteen <thiebaud@weksteen.fr>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-06-30 19:44:58 -07:00
Jan Kiszka
433296b32b scripts/gdb: also allow list_head pointer as lx-list-check paramter
This makes the usage more flexible.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Thiébaud Weksteen <thiebaud@weksteen.fr>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-06-30 19:44:58 -07:00
Thiébaud Weksteen
084f6b1ec6 scripts/gdb: add command to check list consistency
Add a gdb script to verify the consistency of lists.

Signed-off-by: Thiébaud Weksteen <thiebaud@weksteen.fr>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-06-30 19:44:57 -07:00
Adrien Schildknecht
ca3f172c19 scripts/gdb: fix lx-lsmod refcnt
Commit 2f35c41f58 ("module: Replace module_ref with atomic_t refcnt")
changes the way refcnt is handled but did not update the gdb script to
use the new variable.

Since refcnt is not per-cpu anymore, we can directly read its value.

Signed-off-by: Adrien Schildknecht <adrien+dev@schischi.me>
Reviewed-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Pantelis Koukousoulas <pktoss@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-05-28 18:25:19 -07:00
Jan Kiszka
586a1a125e scripts/gdb: add empty package initialization script
This got lost during the initial merge process: Python requires an
__init__.py script, even if empty, in order to accept a directory as
package.  Add it, this time as a non-empty file.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-02-28 09:57:51 -08:00
Jan Kiszka
a9c5bcfa43 scripts/gdb: disable pagination while printing from breakpoint handler
While reporting the (refreshed) list of modules on automatic updates we
may hit the page boundary of the output console and cause a stop if
pagination is enabled.  However, gdb does not accept user input while
running over the breakpoint handler.  So we get stuck, and the user is
forced to interrupt gdb.

Resolve this by disabling pagination during automatic symbol updates.  We
restore the user's configuration once done.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Wessel <jason.wessel@windriver.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Ben Widawsky <ben@bwidawsk.net>
Cc: Borislav Petkov <bp@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-02-17 14:34:55 -08:00
Jan Kiszka
a77e15e8b4 scripts/gdb: convert CpuList to generator function
Yet another code simplification.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Wessel <jason.wessel@windriver.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Ben Widawsky <ben@bwidawsk.net>
Cc: Borislav Petkov <bp@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-02-17 14:34:55 -08:00
Jan Kiszka
fffb944c4e scripts/gdb: convert ModuleList to generator function
Analogously to the task list, convert the module list to a generator
function.  It noticeably simplifies the code.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Wessel <jason.wessel@windriver.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Ben Widawsky <ben@bwidawsk.net>
Cc: Borislav Petkov <bp@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-02-17 14:34:55 -08:00
Daniel Wagner
54e2289a34 scripts/gdb: use a generator instead of iterator for task list
The iterator does not return any task_struct from the thread_group list
because the first condition in the 'if not t or ...' will only be the
first time None.

Instead of keeping track of the state ourself in the next() function, we
fall back using Python's generator.

Signed-off-by: Daniel Wagner <daniel.wagner@bmw-carit.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Wessel <jason.wessel@windriver.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Ben Widawsky <ben@bwidawsk.net>
Cc: Borislav Petkov <bp@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-02-17 14:34:54 -08:00
Daniel Thompson
2478a8a15c scripts/gdb: ignore byte-compiled python files
Using the gdb scripts leaves byte-compiled python files in the scripts/
directory.  These should be ignored by git.

[jan.kiszka@siemens.com: drop redundant mrproper rule as suggested by Michal]
Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Michal Marek <mmarek@suse.cz>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Wessel <jason.wessel@windriver.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Ben Widawsky <ben@bwidawsk.net>
Cc: Borislav Petkov <bp@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-02-17 14:34:54 -08:00
Pantelis Koukousoulas
276d97d90a scripts/gdb: port to python3 / gdb7.7
I tried to use these scripts in an ubuntu 14.04 host (gdb 7.7 compiled
against python 3.3) but there were several errors.

I believe this patch fixes these issues so that the commands now work (I
tested lx-symbols, lx-dmesg, lx-lsmod).

Main issues that needed to be resolved:

  * In python 2 iterators have a "next()" method. In python 3 it is
    __next__() instead (so let's just add both).

  * In older python versions there was an implicit conversion
    in object.__format__() (used when an object is in string.format())
    where it was converting the object to str first and then
    calling str's __format__(). This has now been removed so
    we must explicitly convert to str the objects for which
    we need to keep this behavior.

  * In dmesg.py: in python 3 log_buf is now a "memoryview" object
    which needs to be converted to a string in order to use string
    methods like "splitlines()". Luckily memoryview exists in
    python 2.7.6 as well, so we can convert log_buf to memoryview
    and use the same code in both python 2 and python 3.

This version of the patch has now been tested with gdb 7.7 and both python
3.4 and python 2.7.6 (I think asking for at least python 2.7.6 is a
reasonable requirement instead of complicating the code with version
checks etc).

Signed-off-by: Pantelis Koukousoulas <pktoss@gmail.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Wessel <jason.wessel@windriver.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Ben Widawsky <ben@bwidawsk.net>
Cc: Borislav Petkov <bp@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-02-17 14:34:54 -08:00
Jan Kiszka
5403727f98 scripts/gdb: add lx-lsmod command
This adds a lsmod-like command to list all currently loaded modules of the
target.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Wessel <jason.wessel@windriver.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Ben Widawsky <ben@bwidawsk.net>
Cc: Borislav Petkov <bp@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-02-17 14:34:54 -08:00
Jan Kiszka
3d4cd9c941 scripts/gdb: add class to iterate over CPU masks
Will be used first to count module references.  It is optimized to read
the mask only once per stop.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Wessel <jason.wessel@windriver.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Ben Widawsky <ben@bwidawsk.net>
Cc: Borislav Petkov <bp@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-02-17 14:34:54 -08:00
Jan Kiszka
116b47b4da scripts/gdb: add lx_current convenience function
This is a shorthand for *$lx_per_cpu("current_task"), i.e.  a convenience
function to retrieve the currently running task of the active context.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Wessel <jason.wessel@windriver.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Ben Widawsky <ben@bwidawsk.net>
Cc: Borislav Petkov <bp@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-02-17 14:34:54 -08:00
Jan Kiszka
fe7f9ed98d scripts/gdb: add internal helper and convenience function for per-cpu lookup
This function allows to obtain a per-cpu variable, either of the current
or an explicitly specified CPU.

Note: sparc64 version is untested.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Wessel <jason.wessel@windriver.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Ben Widawsky <ben@bwidawsk.net>
Cc: Borislav Petkov <bp@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-02-17 14:34:54 -08:00
Jan Kiszka
a4d86792c7 scripts/gdb: add get_gdbserver_type helper
This helper probes the type of the gdb server.  Supported are QEMU and
KGDB so far.  Knowledge about the gdb server is required e.g.  to
retrieve the current CPU or current task.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Wessel <jason.wessel@windriver.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Ben Widawsky <ben@bwidawsk.net>
Cc: Borislav Petkov <bp@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-02-17 14:34:54 -08:00