linux/Documentation/ABI
Andrii Nakryiko 341dfcf8d7 btf: expose BTF info through sysfs
Make .BTF section allocated and expose its contents through sysfs.

/sys/kernel/btf directory is created to contain all the BTFs present
inside kernel. Currently there is only kernel's main BTF, represented as
/sys/kernel/btf/kernel file. Once kernel modules' BTFs are supported,
each module will expose its BTF as /sys/kernel/btf/<module-name> file.

Current approach relies on a few pieces coming together:
1. pahole is used to take almost final vmlinux image (modulo .BTF and
   kallsyms) and generate .BTF section by converting DWARF info into
   BTF. This section is not allocated and not mapped to any segment,
   though, so is not yet accessible from inside kernel at runtime.
2. objcopy dumps .BTF contents into binary file and subsequently
   convert binary file into linkable object file with automatically
   generated symbols _binary__btf_kernel_bin_start and
   _binary__btf_kernel_bin_end, pointing to start and end, respectively,
   of BTF raw data.
3. final vmlinux image is generated by linking this object file (and
   kallsyms, if necessary). sysfs_btf.c then creates
   /sys/kernel/btf/kernel file and exposes embedded BTF contents through
   it. This allows, e.g., libbpf and bpftool access BTF info at
   well-known location, without resorting to searching for vmlinux image
   on disk (location of which is not standardized and vmlinux image
   might not be even available in some scenarios, e.g., inside qemu
   during testing).

Alternative approach using .incbin assembler directive to embed BTF
contents directly was attempted but didn't work, because sysfs_proc.o is
not re-compiled during link-vmlinux.sh stage. This is required, though,
to update embedded BTF data (initially empty data is embedded, then
pahole generates BTF info and we need to regenerate sysfs_btf.o with
updated contents, but it's too late at that point).

If BTF couldn't be generated due to missing or too old pahole,
sysfs_btf.c handles that gracefully by detecting that
_binary__btf_kernel_bin_start (weak symbol) is 0 and not creating
/sys/kernel/btf at all.

v2->v3:
- added Documentation/ABI/testing/sysfs-kernel-btf (Greg K-H);
- created proper kobject (btf_kobj) for btf directory (Greg K-H);
- undo v2 change of reusing vmlinux, as it causes extra kallsyms pass
  due to initially missing  __binary__btf_kernel_bin_{start/end} symbols;

v1->v2:
- allow kallsyms stage to re-use vmlinux generated by gen_btf();

Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-08-13 16:14:15 +02:00
..
obsolete docs: gpio: add sysfs interface to the admin-guide 2019-07-15 11:03:03 -03:00
removed docs: driver-api: add a series of orphaned documents 2019-07-15 11:03:02 -03:00
stable docs conversion for v5.3-rc1 2019-07-16 12:21:41 -07:00
testing btf: expose BTF info through sysfs 2019-08-13 16:14:15 +02:00
README docs: fix locations of several documents that got moved 2016-10-24 08:12:35 -02:00

This directory attempts to document the ABI between the Linux kernel and
userspace, and the relative stability of these interfaces.  Due to the
everchanging nature of Linux, and the differing maturity levels, these
interfaces should be used by userspace programs in different ways.

We have four different levels of ABI stability, as shown by the four
different subdirectories in this location.  Interfaces may change levels
of stability according to the rules described below.

The different levels of stability are:

  stable/
	This directory documents the interfaces that the developer has
	defined to be stable.  Userspace programs are free to use these
	interfaces with no restrictions, and backward compatibility for
	them will be guaranteed for at least 2 years.  Most interfaces
	(like syscalls) are expected to never change and always be
	available.

  testing/
	This directory documents interfaces that are felt to be stable,
	as the main development of this interface has been completed.
	The interface can be changed to add new features, but the
	current interface will not break by doing this, unless grave
	errors or security problems are found in them.  Userspace
	programs can start to rely on these interfaces, but they must be
	aware of changes that can occur before these interfaces move to
	be marked stable.  Programs that use these interfaces are
	strongly encouraged to add their name to the description of
	these interfaces, so that the kernel developers can easily
	notify them if any changes occur (see the description of the
	layout of the files below for details on how to do this.)

  obsolete/
  	This directory documents interfaces that are still remaining in
	the kernel, but are marked to be removed at some later point in
	time.  The description of the interface will document the reason
	why it is obsolete and when it can be expected to be removed.

  removed/
	This directory contains a list of the old interfaces that have
	been removed from the kernel.

Every file in these directories will contain the following information:

What:		Short description of the interface
Date:		Date created
KernelVersion:	Kernel version this feature first showed up in.
Contact:	Primary contact for this interface (may be a mailing list)
Description:	Long description of the interface and how to use it.
Users:		All users of this interface who wish to be notified when
		it changes.  This is very important for interfaces in
		the "testing" stage, so that kernel developers can work
		with userspace developers to ensure that things do not
		break in ways that are unacceptable.  It is also
		important to get feedback for these interfaces to make
		sure they are working in a proper way and do not need to
		be changed further.


How things move between levels:

Interfaces in stable may move to obsolete, as long as the proper
notification is given.

Interfaces may be removed from obsolete and the kernel as long as the
documented amount of time has gone by.

Interfaces in the testing state can move to the stable state when the
developers feel they are finished.  They cannot be removed from the
kernel tree without going through the obsolete state first.

It's up to the developer to place their interfaces in the category they
wish for it to start out in.


Notable bits of non-ABI, which should not under any circumstances be considered
stable:

- Kconfig.  Userspace should not rely on the presence or absence of any
  particular Kconfig symbol, in /proc/config.gz, in the copy of .config
  commonly installed to /boot, or in any invocation of the kernel build
  process.

- Kernel-internal symbols.  Do not rely on the presence, absence, location, or
  type of any kernel symbol, either in System.map files or the kernel binary
  itself.  See Documentation/process/stable-api-nonsense.rst.