korg/linux
0
0
Fork 0
mirror of https://mirrors.bfsu.edu.cn/git/linux.git synced 2024-11-23 20:24:12 +08:00
Code Issues Packages Projects Releases Wiki Activity

Rust introduction for v6.1-rc1

Browse Source 
The initial support of Rust-for-Linux comes in roughly 4 areas:
 
 - Kernel internals (kallsyms expansion for Rust symbols, %pA format)
 
 - Kbuild infrastructure (Rust build rules and support scripts)
 
 - Rust crates and bindings for initial minimum viable build
 
 - Rust kernel documentation and samples
 
 Rust support has been in linux-next for a year and a half now, and the
 short log doesn't do justice to the number of people who have contributed
 both to the Linux kernel side but also to the upstream Rust side to
 support the kernel's needs. Thanks to these 173 people, and many more,
 who have been involved in all kinds of ways:
 
 Miguel Ojeda, Wedson Almeida Filho, Alex Gaynor, Boqun Feng, Gary Guo,
 Björn Roy Baron, Andreas Hindborg, Adam Bratschi-Kaye, Benno Lossin,
 Maciej Falkowski, Finn Behrens, Sven Van Asbroeck, Asahi Lina, FUJITA
 Tomonori, John Baublitz, Wei Liu, Geoffrey Thomas, Philip Herron,
 Arthur Cohen, David Faust, Antoni Boucher, Philip Li, Yujie Liu,
 Jonathan Corbet, Greg Kroah-Hartman, Paul E. McKenney, Josh Triplett,
 Kent Overstreet, David Gow, Alice Ryhl, Robin Randhawa, Kees Cook,
 Nick Desaulniers, Matthew Wilcox, Linus Walleij, Joe Perches, Michael
 Ellerman, Petr Mladek, Masahiro Yamada, Arnaldo Carvalho de Melo,
 Andrii Nakryiko, Konstantin Shelekhin, Rasmus Villemoes, Konstantin
 Ryabitsev, Stephen Rothwell, Andy Shevchenko, Sergey Senozhatsky, John
 Paul Adrian Glaubitz, David Laight, Nathan Chancellor, Jonathan
 Cameron, Daniel Latypov, Shuah Khan, Brendan Higgins, Julia Lawall,
 Laurent Pinchart, Geert Uytterhoeven, Akira Yokosawa, Pavel Machek,
 David S. Miller, John Hawley, James Bottomley, Arnd Bergmann,
 Christian Brauner, Dan Robertson, Nicholas Piggin, Zhouyi Zhou, Elena
 Zannoni, Jose E. Marchesi, Leon Romanovsky, Will Deacon, Richard
 Weinberger, Randy Dunlap, Paolo Bonzini, Roland Dreier, Mark Brown,
 Sasha Levin, Ted Ts'o, Steven Rostedt, Jarkko Sakkinen, Michal
 Kubecek, Marco Elver, Al Viro, Keith Busch, Johannes Berg, Jan Kara,
 David Sterba, Connor Kuehl, Andy Lutomirski, Andrew Lunn, Alexandre
 Belloni, Peter Zijlstra, Russell King, Eric W. Biederman, Willy
 Tarreau, Christoph Hellwig, Emilio Cobos Álvarez, Christian Poveda,
 Mark Rousskov, John Ericson, TennyZhuang, Xuanwo, Daniel Paoliello,
 Manish Goregaokar, comex, Josh Stone, Stephan Sokolow, Philipp Krones,
 Guillaume Gomez, Joshua Nelson, Mats Larsen, Marc Poulhiès, Samantha
 Miller, Esteban Blanc, Martin Schmidt, Martin Rodriguez Reboredo,
 Daniel Xu, Viresh Kumar, Bartosz Golaszewski, Vegard Nossum, Milan
 Landaverde, Dariusz Sosnowski, Yuki Okushi, Matthew Bakhtiari, Wu
 XiangCheng, Tiago Lam, Boris-Chengbiao Zhou, Sumera Priyadarsini,
 Viktor Garske, Niklas Mohrin, Nándor István Krácser, Morgan Bartlett,
 Miguel Cano, Léo Lanteri Thauvin, Julian Merkle, Andreas Reindl,
 Jiapeng Chong, Fox Chen, Douglas Su, Antonio Terceiro, SeongJae Park,
 Sergio González Collado, Ngo Iok Ui (Wu Yu Wei), Joshua Abraham,
 Milan, Daniel Kolsoi, ahomescu, Manas, Luis Gerhorst, Li Hongyu,
 Philipp Gesang, Russell Currey, Jalil David Salamé Messina, Jon Olson,
 Raghvender, Angelos, Kaviraj Kanagaraj, Paul Römer, Sladyn Nunes,
 Mauro Baladés, Hsiang-Cheng Yang, Abhik Jain, Hongyu Li, Sean Nash,
 Yuheng Su, Peng Hao, Anhad Singh, Roel Kluin, Sara Saa, Geert
 Stappers, Garrett LeSage, IFo Hancroft, and Linus Torvalds.
 -----BEGIN PGP SIGNATURE-----
 
 iQJKBAABCgA0FiEEpcP2jyKd1g9yPm4TiXL039xtwCYFAmM4WcIWHGtlZXNjb29r
 QGNocm9taXVtLm9yZwAKCRCJcvTf3G3AJlGrD/93HbmxjNi/hwdWF5UdWV1/W0kJ
 bSTh9JsNtN9atQGEUwxePBjrtxHE75lxSL0RJ+sWvaJ7vR3iv2qys+cEgU0ePrgX
 INZ3bvHAGgvPG1b0R6VxmakksHq1BdCDbCT3Ft5lSNxB0uQBi95KgjtR0lCH/NUl
 eoZnGJ0ZbKs5KpbzFqOjM2gmJ51geZppnfNFmbKOb3lSUpPQqhZLPDCzweE57GNo
 e2vcMoY4daVaSUxmo01TSEphrM5IjDxp5rs09+aeovfmpbeoiz33siyGiAxyM7CI
 +Ybxl+bBnyqXLadjbs9VvvtYzASFZgmrQdwIQbY8j/sqsw34jmZarOwa5iUVmo+Q
 2w1CDDNLMG3XpI/PdnUklFRIJg1uYCM+OXgZY2MFFqzbjoik/zFv2qFWTp1F5+XV
 DdLxoN9quBPDSVDFQjAZPsyCD/pSRfiJYh9s7BdlhUPL6rk9uLIgZyZuPqy3kWXn
 2Z02lWJpiHUtTaICdUDyNPFzTggDHEfY2DvmuedXpsyhlMkCdtFS5zoo/evl8pb6
 xUV7qdfpjyLyTLmLWjYEVRO6DJJuFQWMK5Qpqn6O0y3wch3XV+At5QDk2TE2WMvB
 cYwd9nCqcMs7J0HrdoDmtLwew1jrLd1xefqDgD0zd6B/+Dk9W4gFD69Stmtarg7d
 KGRvH0wnL0keMxy31w==
 =zz09
 -----END PGP SIGNATURE-----

Merge tag 'rust-v6.1-rc1' of https://github.com/Rust-for-Linux/linux

Pull Rust introductory support from Kees Cook:
 "The tree has a recent base, but has fundamentally been in linux-next
  for a year and a half[1]. It's been updated based on feedback from the
  Kernel Maintainer's Summit, and to gain recent Reviewed-by: tags.

  Miguel is the primary maintainer, with me helping where needed/wanted.
  Our plan is for the tree to switch to the standard non-rebasing
  practice once this initial infrastructure series lands.

  The contents are the absolute minimum to get Rust code building in the
  kernel, with many more interfaces[2] (and drivers - NVMe[3], 9p[4], M1
  GPU[5]) on the way.

  The initial support of Rust-for-Linux comes in roughly 4 areas:

   - Kernel internals (kallsyms expansion for Rust symbols, %pA format)

   - Kbuild infrastructure (Rust build rules and support scripts)

   - Rust crates and bindings for initial minimum viable build

   - Rust kernel documentation and samples

  Rust support has been in linux-next for a year and a half now, and the
  short log doesn't do justice to the number of people who have
  contributed both to the Linux kernel side but also to the upstream
  Rust side to support the kernel's needs. Thanks to these 173 people,
  and many more, who have been involved in all kinds of ways:

  Miguel Ojeda, Wedson Almeida Filho, Alex Gaynor, Boqun Feng, Gary Guo,
  Björn Roy Baron, Andreas Hindborg, Adam Bratschi-Kaye, Benno Lossin,
  Maciej Falkowski, Finn Behrens, Sven Van Asbroeck, Asahi Lina, FUJITA
  Tomonori, John Baublitz, Wei Liu, Geoffrey Thomas, Philip Herron,
  Arthur Cohen, David Faust, Antoni Boucher, Philip Li, Yujie Liu,
  Jonathan Corbet, Greg Kroah-Hartman, Paul E. McKenney, Josh Triplett,
  Kent Overstreet, David Gow, Alice Ryhl, Robin Randhawa, Kees Cook,
  Nick Desaulniers, Matthew Wilcox, Linus Walleij, Joe Perches, Michael
  Ellerman, Petr Mladek, Masahiro Yamada, Arnaldo Carvalho de Melo,
  Andrii Nakryiko, Konstantin Shelekhin, Rasmus Villemoes, Konstantin
  Ryabitsev, Stephen Rothwell, Andy Shevchenko, Sergey Senozhatsky, John
  Paul Adrian Glaubitz, David Laight, Nathan Chancellor, Jonathan
  Cameron, Daniel Latypov, Shuah Khan, Brendan Higgins, Julia Lawall,
  Laurent Pinchart, Geert Uytterhoeven, Akira Yokosawa, Pavel Machek,
  David S. Miller, John Hawley, James Bottomley, Arnd Bergmann,
  Christian Brauner, Dan Robertson, Nicholas Piggin, Zhouyi Zhou, Elena
  Zannoni, Jose E. Marchesi, Leon Romanovsky, Will Deacon, Richard
  Weinberger, Randy Dunlap, Paolo Bonzini, Roland Dreier, Mark Brown,
  Sasha Levin, Ted Ts'o, Steven Rostedt, Jarkko Sakkinen, Michal
  Kubecek, Marco Elver, Al Viro, Keith Busch, Johannes Berg, Jan Kara,
  David Sterba, Connor Kuehl, Andy Lutomirski, Andrew Lunn, Alexandre
  Belloni, Peter Zijlstra, Russell King, Eric W. Biederman, Willy
  Tarreau, Christoph Hellwig, Emilio Cobos Álvarez, Christian Poveda,
  Mark Rousskov, John Ericson, TennyZhuang, Xuanwo, Daniel Paoliello,
  Manish Goregaokar, comex, Josh Stone, Stephan Sokolow, Philipp Krones,
  Guillaume Gomez, Joshua Nelson, Mats Larsen, Marc Poulhiès, Samantha
  Miller, Esteban Blanc, Martin Schmidt, Martin Rodriguez Reboredo,
  Daniel Xu, Viresh Kumar, Bartosz Golaszewski, Vegard Nossum, Milan
  Landaverde, Dariusz Sosnowski, Yuki Okushi, Matthew Bakhtiari, Wu
  XiangCheng, Tiago Lam, Boris-Chengbiao Zhou, Sumera Priyadarsini,
  Viktor Garske, Niklas Mohrin, Nándor István Krácser, Morgan Bartlett,
  Miguel Cano, Léo Lanteri Thauvin, Julian Merkle, Andreas Reindl,
  Jiapeng Chong, Fox Chen, Douglas Su, Antonio Terceiro, SeongJae Park,
  Sergio González Collado, Ngo Iok Ui (Wu Yu Wei), Joshua Abraham,
  Milan, Daniel Kolsoi, ahomescu, Manas, Luis Gerhorst, Li Hongyu,
  Philipp Gesang, Russell Currey, Jalil David Salamé Messina, Jon Olson,
  Raghvender, Angelos, Kaviraj Kanagaraj, Paul Römer, Sladyn Nunes,
  Mauro Baladés, Hsiang-Cheng Yang, Abhik Jain, Hongyu Li, Sean Nash,
  Yuheng Su, Peng Hao, Anhad Singh, Roel Kluin, Sara Saa, Geert
  Stappers, Garrett LeSage, IFo Hancroft, and Linus Torvalds"

Link: https://lwn.net/Articles/849849/ [1]
Link: https://github.com/Rust-for-Linux/linux/commits/rust [2]
Link: d88c3744d6 [3]
Link: 9367032607 [4]
Link: https://github.com/AsahiLinux/linux/commits/gpu/rust-wip [5]

* tag 'rust-v6.1-rc1' of https://github.com/Rust-for-Linux/linux: (27 commits)
  MAINTAINERS: Rust
  samples: add first Rust examples
  x86: enable initial Rust support
  docs: add Rust documentation
  Kbuild: add Rust support
  rust: add `.rustfmt.toml`
  scripts: add `is_rust_module.sh`
  scripts: add `rust_is_available.sh`
  scripts: add `generate_rust_target.rs`
  scripts: add `generate_rust_analyzer.py`
  scripts: decode_stacktrace: demangle Rust symbols
  scripts: checkpatch: enable language-independent checks for Rust
  scripts: checkpatch: diagnose uses of `%pA` in the C side as errors
  vsprintf: add new `%pA` format specifier
  rust: export generated symbols
  rust: add `kernel` crate
  rust: add `bindings` crate
  rust: add `macros` crate
  rust: add `compiler_builtins` crate
  rust: adapt `alloc` crate to the kernel
  ...
This commit is contained in:
Linus Torvalds 2022-10-03 16:39:37 -07:00
commit 8aebac8293
89 changed files with 12552 additions and 51 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
.gitignore vendored
View File

@ -37,6 +37,8 @@
*.o
*.o.*
*.patch
*.rmeta
*.rsi
*.s
*.so
*.so.dbg
@ -97,6 +99,7 @@ modules.order
!.gitattributes
!.gitignore
!.mailmap
!.rustfmt.toml
#
# Generated include files
@ -162,3 +165,6 @@ x509.genkey
# Documentation toolchain
sphinx_*/
# Rust analyzer configuration
/rust-project.json

12
.rustfmt.toml Normal file
View File

@ -0,0 +1,12 @@
edition = "2021"
newline_style = "Unix"
# Unstable options that help catching some mistakes in formatting and that we may want to enable
# when they become stable.
#
# They are kept here since they are useful to run from time to time.
#format_code_in_doc_comments = true
#reorder_impl_items = true
#comment_width = 100
#wrap_comments = true
#normalize_comments = true

10
Documentation/core-api/printk-formats.rst
View File

@ -625,6 +625,16 @@ Examples::
%p4cc Y10 little-endian (0x20303159)
%p4cc NV12 big-endian (0xb231564e)
Rust
----
::
%pA
Only intended to be used from Rust code to format ``core::fmt::Arguments``.
Do *not* use it from C.
Thanks
======

3
Documentation/doc-guide/kernel-doc.rst
View File

@ -14,6 +14,9 @@ when it is embedded in source files.
reasons. The kernel source contains tens of thousands of kernel-doc
comments. Please stick to the style described here.
.. note:: kernel-doc does not cover Rust code: please see
Documentation/rust/general-information.rst instead.
The kernel-doc structure is extracted from the comments, and proper
`Sphinx C Domain`_ function and type descriptions with anchors are
generated from them. The descriptions are filtered for special kernel-doc

1
Documentation/index.rst
View File

@ -58,6 +58,7 @@ Various other manuals with useful information for all kernel developers.
trace/index
fault-injection/index
livepatch/index
rust/index
User-oriented documentation

17
Documentation/kbuild/kbuild.rst
View File

@ -48,6 +48,10 @@ KCFLAGS
-------
Additional options to the C compiler (for built-in and modules).
KRUSTFLAGS
----------
Additional options to the Rust compiler (for built-in and modules).
CFLAGS_KERNEL
-------------
Additional options for $(CC) when used to compile
@ -57,6 +61,15 @@ CFLAGS_MODULE
-------------
Additional module specific options to use for $(CC).
RUSTFLAGS_KERNEL
----------------
Additional options for $(RUSTC) when used to compile
code that is compiled as built-in.
RUSTFLAGS_MODULE
----------------
Additional module specific options to use for $(RUSTC).
LDFLAGS_MODULE
--------------
Additional options used for $(LD) when linking modules.
@ -69,6 +82,10 @@ HOSTCXXFLAGS
------------
Additional flags to be passed to $(HOSTCXX) when building host programs.
HOSTRUSTFLAGS
-------------
Additional flags to be passed to $(HOSTRUSTC) when building host programs.
HOSTLDFLAGS
-----------
Additional flags to be passed when linking host programs.

50
Documentation/kbuild/makefiles.rst
View File

@ -29,8 +29,9 @@ This document describes the Linux kernel Makefiles.
--- 4.1 Simple Host Program
--- 4.2 Composite Host Programs
--- 4.3 Using C++ for host programs
--- 4.4 Controlling compiler options for host programs
--- 4.5 When host programs are actually built
--- 4.4 Using Rust for host programs
--- 4.5 Controlling compiler options for host programs
--- 4.6 When host programs are actually built
=== 5 Userspace Program support
--- 5.1 Simple Userspace Program
@ -835,7 +836,24 @@ Both possibilities are described in the following.
qconf-cxxobjs := qconf.o
qconf-objs := check.o
4.4 Controlling compiler options for host programs
4.4 Using Rust for host programs
--------------------------------
Kbuild offers support for host programs written in Rust. However,
since a Rust toolchain is not mandatory for kernel compilation,
it may only be used in scenarios where Rust is required to be
available (e.g. when ``CONFIG_RUST`` is enabled).
Example::
hostprogs := target
target-rust := y
Kbuild will compile ``target`` using ``target.rs`` as the crate root,
located in the same directory as the ``Makefile``. The crate may
consist of several source files (see ``samples/rust/hostprogs``).
4.5 Controlling compiler options for host programs
--------------------------------------------------
When compiling host programs, it is possible to set specific flags.
@ -867,7 +885,7 @@ Both possibilities are described in the following.
When linking qconf, it will be passed the extra option
"-L$(QTDIR)/lib".
4.5 When host programs are actually built
4.6 When host programs are actually built
-----------------------------------------
Kbuild will only build host-programs when they are referenced
@ -1181,6 +1199,17 @@ When kbuild executes, the following steps are followed (roughly):
The first example utilises the trick that a config option expands
to 'y' when selected.
KBUILD_RUSTFLAGS
$(RUSTC) compiler flags
Default value - see top level Makefile
Append or modify as required per architecture.
Often, the KBUILD_RUSTFLAGS variable depends on the configuration.
Note that target specification file generation (for ``--target``)
is handled in ``scripts/generate_rust_target.rs``.
KBUILD_AFLAGS_KERNEL
Assembler options specific for built-in
@ -1208,6 +1237,19 @@ When kbuild executes, the following steps are followed (roughly):
are used for $(CC).
From commandline CFLAGS_MODULE shall be used (see kbuild.rst).
KBUILD_RUSTFLAGS_KERNEL
$(RUSTC) options specific for built-in
$(KBUILD_RUSTFLAGS_KERNEL) contains extra Rust compiler flags used to
compile resident kernel code.
KBUILD_RUSTFLAGS_MODULE
Options for $(RUSTC) when building modules
$(KBUILD_RUSTFLAGS_MODULE) is used to add arch-specific options that
are used for $(RUSTC).
From commandline RUSTFLAGS_MODULE shall be used (see kbuild.rst).
KBUILD_LDFLAGS_MODULE
Options for $(LD) when linking modules

41
Documentation/process/changes.rst
View File

@ -31,6 +31,8 @@ you probably needn't concern yourself with pcmciautils.
====================== =============== ========================================
GNU C 5.1 gcc --version
Clang/LLVM (optional) 11.0.0 clang --version
Rust (optional) 1.62.0 rustc --version
bindgen (optional) 0.56.0 bindgen --version
GNU make 3.81 make --version
bash 4.2 bash --version
binutils 2.23 ld -v
@ -80,6 +82,29 @@ kernels. Older releases aren't guaranteed to work, and we may drop workarounds
from the kernel that were used to support older versions. Please see additional
docs on :ref:`Building Linux with Clang/LLVM <kbuild_llvm>`.
Rust (optional)
---------------
A particular version of the Rust toolchain is required. Newer versions may or
may not work because the kernel depends on some unstable Rust features, for
the moment.
Each Rust toolchain comes with several "components", some of which are required
(like ``rustc``) and some that are optional. The ``rust-src`` component (which
is optional) needs to be installed to build the kernel. Other components are
useful for developing.
Please see Documentation/rust/quick-start.rst for instructions on how to
satisfy the build requirements of Rust support. In particular, the ``Makefile``
target ``rustavailable`` is useful to check why the Rust toolchain may not
be detected.
bindgen (optional)
------------------
``bindgen`` is used to generate the Rust bindings to the C side of the kernel.
It depends on ``libclang``.
Make
----
@ -348,6 +373,12 @@ Sphinx
Please see :ref:`sphinx_install` in :ref:`Documentation/doc-guide/sphinx.rst <sphinxdoc>`
for details about Sphinx requirements.
rustdoc
-------
``rustdoc`` is used to generate the documentation for Rust code. Please see
Documentation/rust/general-information.rst for more information.
Getting updated software
========================
@ -364,6 +395,16 @@ Clang/LLVM
- :ref:`Getting LLVM <getting_llvm>`.
Rust
----
- Documentation/rust/quick-start.rst.
bindgen
-------
- Documentation/rust/quick-start.rst.
Make
----

19
Documentation/rust/arch-support.rst Normal file
View File

@ -0,0 +1,19 @@
.. SPDX-License-Identifier: GPL-2.0
Arch Support
============
Currently, the Rust compiler (``rustc``) uses LLVM for code generation,
which limits the supported architectures that can be targeted. In addition,
support for building the kernel with LLVM/Clang varies (please see
Documentation/kbuild/llvm.rst). This support is needed for ``bindgen``
which uses ``libclang``.
Below is a general summary of architectures that currently work. Level of
support corresponds to ``S`` values in the ``MAINTAINERS`` file.
============ ================ ==============================================
Architecture Level of support Constraints
============ ================ ==============================================
``x86`` Maintained ``x86_64`` only.
============ ================ ==============================================

216
Documentation/rust/coding-guidelines.rst Normal file
View File

@ -0,0 +1,216 @@
.. SPDX-License-Identifier: GPL-2.0
Coding Guidelines
=================
This document describes how to write Rust code in the kernel.
Style & formatting
------------------
The code should be formatted using ``rustfmt``. In this way, a person
contributing from time to time to the kernel does not need to learn and
remember one more style guide. More importantly, reviewers and maintainers
do not need to spend time pointing out style issues anymore, and thus
less patch roundtrips may be needed to land a change.
.. note:: Conventions on comments and documentation are not checked by
``rustfmt``. Thus those are still needed to be taken care of.
The default settings of ``rustfmt`` are used. This means the idiomatic Rust
style is followed. For instance, 4 spaces are used for indentation rather
than tabs.
It is convenient to instruct editors/IDEs to format while typing,
when saving or at commit time. However, if for some reason reformatting
the entire kernel Rust sources is needed at some point, the following can be
run::
make LLVM=1 rustfmt
It is also possible to check if everything is formatted (printing a diff
otherwise), for instance for a CI, with::
make LLVM=1 rustfmtcheck
Like ``clang-format`` for the rest of the kernel, ``rustfmt`` works on
individual files, and does not require a kernel configuration. Sometimes it may
even work with broken code.
Comments
--------
"Normal" comments (i.e. ``//``, rather than code documentation which starts
with ``///`` or ``//!``) are written in Markdown the same way as documentation
comments are, even though they will not be rendered. This improves consistency,
simplifies the rules and allows to move content between the two kinds of
comments more easily. For instance:
.. code-block:: rust
// `object` is ready to be handled now.
f(object);
Furthermore, just like documentation, comments are capitalized at the beginning
of a sentence and ended with a period (even if it is a single sentence). This
includes ``// SAFETY:``, ``// TODO:`` and other "tagged" comments, e.g.:
.. code-block:: rust
// FIXME: The error should be handled properly.
Comments should not be used for documentation purposes: comments are intended
for implementation details, not users. This distinction is useful even if the
reader of the source file is both an implementor and a user of an API. In fact,
sometimes it is useful to use both comments and documentation at the same time.
For instance, for a ``TODO`` list or to comment on the documentation itself.
For the latter case, comments can be inserted in the middle; that is, closer to
the line of documentation to be commented. For any other case, comments are
written after the documentation, e.g.:
.. code-block:: rust
/// Returns a new [`Foo`].
///
/// # Examples
///
// TODO: Find a better example.
/// ```
/// let foo = f(42);
/// ```
// FIXME: Use fallible approach.
pub fn f(x: i32) -> Foo {
// ...
}
One special kind of comments are the ``// SAFETY:`` comments. These must appear
before every ``unsafe`` block, and they explain why the code inside the block is
correct/sound, i.e. why it cannot trigger undefined behavior in any case, e.g.:
.. code-block:: rust
// SAFETY: `p` is valid by the safety requirements.
unsafe { *p = 0; }
``// SAFETY:`` comments are not to be confused with the ``# Safety`` sections
in code documentation. ``# Safety`` sections specify the contract that callers
(for functions) or implementors (for traits) need to abide by. ``// SAFETY:``
comments show why a call (for functions) or implementation (for traits) actually
respects the preconditions stated in a ``# Safety`` section or the language
reference.
Code documentation
------------------
Rust kernel code is not documented like C kernel code (i.e. via kernel-doc).
Instead, the usual system for documenting Rust code is used: the ``rustdoc``
tool, which uses Markdown (a lightweight markup language).
To learn Markdown, there are many guides available out there. For instance,
the one at:
https://commonmark.org/help/
This is how a well-documented Rust function may look like:
.. code-block:: rust
/// Returns the contained [`Some`] value, consuming the `self` value,
/// without checking that the value is not [`None`].
///
/// # Safety
///
/// Calling this method on [`None`] is *[undefined behavior]*.
///
/// [undefined behavior]: https://doc.rust-lang.org/reference/behavior-considered-undefined.html
///
/// # Examples
///
/// ```
/// let x = Some("air");
/// assert_eq!(unsafe { x.unwrap_unchecked() }, "air");
/// ```
pub unsafe fn unwrap_unchecked(self) -> T {
match self {
Some(val) => val,
// SAFETY: The safety contract must be upheld by the caller.
None => unsafe { hint::unreachable_unchecked() },
}
}
This example showcases a few ``rustdoc`` features and some conventions followed
in the kernel:
- The first paragraph must be a single sentence briefly describing what
the documented item does. Further explanations must go in extra paragraphs.
- Unsafe functions must document their safety preconditions under
a ``# Safety`` section.
- While not shown here, if a function may panic, the conditions under which
that happens must be described under a ``# Panics`` section.
Please note that panicking should be very rare and used only with a good
reason. In almost all cases, a fallible approach should be used, typically
returning a ``Result``.
- If providing examples of usage would help readers, they must be written in
a section called ``# Examples``.
- Rust items (functions, types, constants...) must be linked appropriately
(``rustdoc`` will create a link automatically).
- Any ``unsafe`` block must be preceded by a ``// SAFETY:`` comment
describing why the code inside is sound.
While sometimes the reason might look trivial and therefore unneeded,
writing these comments is not just a good way of documenting what has been
taken into account, but most importantly, it provides a way to know that
there are no *extra* implicit constraints.
To learn more about how to write documentation for Rust and extra features,
please take a look at the ``rustdoc`` book at:
https://doc.rust-lang.org/rustdoc/how-to-write-documentation.html
Naming
------
Rust kernel code follows the usual Rust naming conventions:
https://rust-lang.github.io/api-guidelines/naming.html
When existing C concepts (e.g. macros, functions, objects...) are wrapped into
a Rust abstraction, a name as close as reasonably possible to the C side should
be used in order to avoid confusion and to improve readability when switching
back and forth between the C and Rust sides. For instance, macros such as
``pr_info`` from C are named the same in the Rust side.
Having said that, casing should be adjusted to follow the Rust naming
conventions, and namespacing introduced by modules and types should not be
repeated in the item names. For instance, when wrapping constants like:
.. code-block:: c
#define GPIO_LINE_DIRECTION_IN 0
#define GPIO_LINE_DIRECTION_OUT 1
The equivalent in Rust may look like (ignoring documentation):
.. code-block:: rust
pub mod gpio {
pub enum LineDirection {
In = bindings::GPIO_LINE_DIRECTION_IN as _,
Out = bindings::GPIO_LINE_DIRECTION_OUT as _,
}
}
That is, the equivalent of ``GPIO_LINE_DIRECTION_IN`` would be referred to as
``gpio::LineDirection::In``. In particular, it should not be named
``gpio::gpio_line_direction::GPIO_LINE_DIRECTION_IN``.

79
Documentation/rust/general-information.rst Normal file
View File

@ -0,0 +1,79 @@
.. SPDX-License-Identifier: GPL-2.0
General Information
===================
This document contains useful information to know when working with
the Rust support in the kernel.
Code documentation
------------------
Rust kernel code is documented using ``rustdoc``, its built-in documentation
generator.
The generated HTML docs include integrated search, linked items (e.g. types,
functions, constants), source code, etc. They may be read at (TODO: link when
in mainline and generated alongside the rest of the documentation):
http://kernel.org/
The docs can also be easily generated and read locally. This is quite fast
(same order as compiling the code itself) and no special tools or environment
are needed. This has the added advantage that they will be tailored to
the particular kernel configuration used. To generate them, use the ``rustdoc``
target with the same invocation used for compilation, e.g.::
make LLVM=1 rustdoc
To read the docs locally in your web browser, run e.g.::
xdg-open rust/doc/kernel/index.html
To learn about how to write the documentation, please see coding-guidelines.rst.
Extra lints
-----------
While ``rustc`` is a very helpful compiler, some extra lints and analyses are
available via ``clippy``, a Rust linter. To enable it, pass ``CLIPPY=1`` to
the same invocation used for compilation, e.g.::
make LLVM=1 CLIPPY=1
Please note that Clippy may change code generation, thus it should not be
enabled while building a production kernel.
Abstractions vs. bindings
-------------------------
Abstractions are Rust code wrapping kernel functionality from the C side.
In order to use functions and types from the C side, bindings are created.
Bindings are the declarations for Rust of those functions and types from
the C side.
For instance, one may write a ``Mutex`` abstraction in Rust which wraps
a ``struct mutex`` from the C side and calls its functions through the bindings.
Abstractions are not available for all the kernel internal APIs and concepts,
but it is intended that coverage is expanded as time goes on. "Leaf" modules
(e.g. drivers) should not use the C bindings directly. Instead, subsystems
should provide as-safe-as-possible abstractions as needed.
Conditional compilation
-----------------------
Rust code has access to conditional compilation based on the kernel
configuration:
.. code-block:: rust
#[cfg(CONFIG_X)] // Enabled (`y` or `m`)
#[cfg(CONFIG_X="y")] // Enabled as a built-in (`y`)
#[cfg(CONFIG_X="m")] // Enabled as a module (`m`)
#[cfg(not(CONFIG_X))] // Disabled

22
Documentation/rust/index.rst Normal file
View File

@ -0,0 +1,22 @@
.. SPDX-License-Identifier: GPL-2.0
Rust
====
Documentation related to Rust within the kernel. To start using Rust
in the kernel, please read the quick-start.rst guide.
.. toctree::
:maxdepth: 1
quick-start
general-information
coding-guidelines
arch-support
.. only:: subproject and html
Indices
=======
* :ref:`genindex`

232
Documentation/rust/quick-start.rst Normal file
View File

@ -0,0 +1,232 @@
.. SPDX-License-Identifier: GPL-2.0
Quick Start
===========
This document describes how to get started with kernel development in Rust.
Requirements: Building
----------------------
This section explains how to fetch the tools needed for building.
Some of these requirements might be available from Linux distributions
under names like ``rustc``, ``rust-src``, ``rust-bindgen``, etc. However,
at the time of writing, they are likely not to be recent enough unless
the distribution tracks the latest releases.
To easily check whether the requirements are met, the following target
can be used::
make LLVM=1 rustavailable
This triggers the same logic used by Kconfig to determine whether
``RUST_IS_AVAILABLE`` should be enabled; but it also explains why not
if that is the case.
rustc
*****
A particular version of the Rust compiler is required. Newer versions may or
may not work because, for the moment, the kernel depends on some unstable
Rust features.
If ``rustup`` is being used, enter the checked out source code directory
and run::
rustup override set $(scripts/min-tool-version.sh rustc)
Otherwise, fetch a standalone installer or install ``rustup`` from:
https://www.rust-lang.org
Rust standard library source
****************************
The Rust standard library source is required because the build system will
cross-compile ``core`` and ``alloc``.
If ``rustup`` is being used, run::
rustup component add rust-src
The components are installed per toolchain, thus upgrading the Rust compiler
version later on requires re-adding the component.
Otherwise, if a standalone installer is used, the Rust repository may be cloned
into the installation folder of the toolchain::
git clone --recurse-submodules \
--branch $(scripts/min-tool-version.sh rustc) \
https://github.com/rust-lang/rust \
$(rustc --print sysroot)/lib/rustlib/src/rust
In this case, upgrading the Rust compiler version later on requires manually
updating this clone.
libclang
********
``libclang`` (part of LLVM) is used by ``bindgen`` to understand the C code
in the kernel, which means LLVM needs to be installed; like when the kernel
is compiled with ``CC=clang`` or ``LLVM=1``.
Linux distributions are likely to have a suitable one available, so it is
best to check that first.
There are also some binaries for several systems and architectures uploaded at:
https://releases.llvm.org/download.html
Otherwise, building LLVM takes quite a while, but it is not a complex process:
https://llvm.org/docs/GettingStarted.html#getting-the-source-code-and-building-llvm
Please see Documentation/kbuild/llvm.rst for more information and further ways
to fetch pre-built releases and distribution packages.
bindgen
*******
The bindings to the C side of the kernel are generated at build time using
the ``bindgen`` tool. A particular version is required.
Install it via (note that this will download and build the tool from source)::
cargo install --locked --version $(scripts/min-tool-version.sh bindgen) bindgen
Requirements: Developing
------------------------
This section explains how to fetch the tools needed for developing. That is,
they are not needed when just building the kernel.
rustfmt
*******
The ``rustfmt`` tool is used to automatically format all the Rust kernel code,
including the generated C bindings (for details, please see
coding-guidelines.rst).
If ``rustup`` is being used, its ``default`` profile already installs the tool,
thus nothing needs to be done. If another profile is being used, the component
can be installed manually::
rustup component add rustfmt
The standalone installers also come with ``rustfmt``.
clippy
******
``clippy`` is a Rust linter. Running it provides extra warnings for Rust code.
It can be run by passing ``CLIPPY=1`` to ``make`` (for details, please see
general-information.rst).
If ``rustup`` is being used, its ``default`` profile already installs the tool,
thus nothing needs to be done. If another profile is being used, the component
can be installed manually::
rustup component add clippy
The standalone installers also come with ``clippy``.
cargo
*****
``cargo`` is the Rust native build system. It is currently required to run
the tests since it is used to build a custom standard library that contains
the facilities provided by the custom ``alloc`` in the kernel. The tests can
be run using the ``rusttest`` Make target.
If ``rustup`` is being used, all the profiles already install the tool,
thus nothing needs to be done.
The standalone installers also come with ``cargo``.
rustdoc
*******
``rustdoc`` is the documentation tool for Rust. It generates pretty HTML
documentation for Rust code (for details, please see
general-information.rst).
``rustdoc`` is also used to test the examples provided in documented Rust code
(called doctests or documentation tests). The ``rusttest`` Make target uses
this feature.
If ``rustup`` is being used, all the profiles already install the tool,
thus nothing needs to be done.
The standalone installers also come with ``rustdoc``.
rust-analyzer
*************
The `rust-analyzer <https://rust-analyzer.github.io/>`_ language server can
be used with many editors to enable syntax highlighting, completion, go to
definition, and other features.
``rust-analyzer`` needs a configuration file, ``rust-project.json``, which
can be generated by the ``rust-analyzer`` Make target.
Configuration
-------------
``Rust support`` (``CONFIG_RUST``) needs to be enabled in the ``General setup``
menu. The option is only shown if a suitable Rust toolchain is found (see
above), as long as the other requirements are met. In turn, this will make
visible the rest of options that depend on Rust.
Afterwards, go to::
Kernel hacking
-> Sample kernel code
-> Rust samples
And enable some sample modules either as built-in or as loadable.
Building
--------
Building a kernel with a complete LLVM toolchain is the best supported setup
at the moment. That is::
make LLVM=1
For architectures that do not support a full LLVM toolchain, use::
make CC=clang
Using GCC also works for some configurations, but it is very experimental at
the moment.
Hacking
-------
To dive deeper, take a look at the source code of the samples
at ``samples/rust/``, the Rust support code under ``rust/`` and
the ``Rust hacking`` menu under ``Kernel hacking``.
If GDB/Binutils is used and Rust symbols are not getting demangled, the reason
is the toolchain does not support Rust's new v0 mangling scheme yet.
There are a few ways out:
- Install a newer release (GDB >= 10.2, Binutils >= 2.36).
- Some versions of GDB (e.g. vanilla GDB 10.1) are able to use
the pre-demangled names embedded in the debug info (``CONFIG_DEBUG_INFO``).

18
MAINTAINERS
View File

@ -17755,6 +17755,24 @@ F: include/rv/
F: kernel/trace/rv/
F: tools/verification/
RUST
M: Miguel Ojeda <ojeda@kernel.org>
M: Alex Gaynor <alex.gaynor@gmail.com>
M: Wedson Almeida Filho <wedsonaf@gmail.com>
R: Boqun Feng <boqun.feng@gmail.com>
R: Gary Guo <gary@garyguo.net>
R: Björn Roy Baron <bjorn3_gh@protonmail.com>
L: rust-for-linux@vger.kernel.org
S: Supported
W: https://github.com/Rust-for-Linux/linux
B: https://github.com/Rust-for-Linux/linux/issues
T: git https://github.com/Rust-for-Linux/linux.git rust-next
F: Documentation/rust/
F: rust/
F: samples/rust/
F: scripts/*rust*
K: \b(?i:rust)\b
RXRPC SOCKETS (AF_RXRPC)
M: David Howells <dhowells@redhat.com>
M: Marc Dionne <marc.dionne@auristor.com>

172
Makefile
View File

@ -120,6 +120,15 @@ endif
export KBUILD_CHECKSRC
# Enable "clippy" (a linter) as part of the Rust compilation.
#
# Use 'make CLIPPY=1' to enable it.
ifeq ("$(origin CLIPPY)", "command line")
KBUILD_CLIPPY := $(CLIPPY)
endif
export KBUILD_CLIPPY
# Use make M=dir or set the environment variable KBUILD_EXTMOD to specify the
# directory of external module to build. Setting M= takes precedence.
ifeq ("$(origin M)", "command line")
@ -270,14 +279,14 @@ no-dot-config-targets := $(clean-targets) \
cscope gtags TAGS tags help% %docs check% coccicheck \
$(version_h) headers headers_% archheaders archscripts \
%asm-generic kernelversion %src-pkg dt_binding_check \
outputmakefile
outputmakefile rustavailable rustfmt rustfmtcheck
# Installation targets should not require compiler. Unfortunately, vdso_install
# is an exception where build artifacts may be updated. This must be fixed.
no-compiler-targets := $(no-dot-config-targets) install dtbs_install \
headers_install modules_install kernelrelease image_name
no-sync-config-targets := $(no-dot-config-targets) %install kernelrelease \
image_name
single-targets := %.a %.i %.ko %.lds %.ll %.lst %.mod %.o %.s %.symtypes %/
single-targets := %.a %.i %.rsi %.ko %.lds %.ll %.lst %.mod %.o %.s %.symtypes %/
config-build :=
mixed-build :=
@ -439,6 +448,7 @@ else
HOSTCC = gcc
HOSTCXX = g++
endif
HOSTRUSTC = rustc
HOSTPKG_CONFIG = pkg-config
KBUILD_USERHOSTCFLAGS := -Wall -Wmissing-prototypes -Wstrict-prototypes \
@ -447,8 +457,26 @@ KBUILD_USERHOSTCFLAGS := -Wall -Wmissing-prototypes -Wstrict-prototypes \
KBUILD_USERCFLAGS := $(KBUILD_USERHOSTCFLAGS) $(USERCFLAGS)
KBUILD_USERLDFLAGS := $(USERLDFLAGS)
# These flags apply to all Rust code in the tree, including the kernel and
# host programs.
export rust_common_flags := --edition=2021 \
-Zbinary_dep_depinfo=y \
-Dunsafe_op_in_unsafe_fn -Drust_2018_idioms \
-Dunreachable_pub -Dnon_ascii_idents \
-Wmissing_docs \
-Drustdoc::missing_crate_level_docs \
-Dclippy::correctness -Dclippy::style \
-Dclippy::suspicious -Dclippy::complexity \
-Dclippy::perf \
-Dclippy::let_unit_value -Dclippy::mut_mut \
-Dclippy::needless_bitwise_bool \
-Dclippy::needless_continue \
-Wclippy::dbg_macro
KBUILD_HOSTCFLAGS := $(KBUILD_USERHOSTCFLAGS) $(HOST_LFS_CFLAGS) $(HOSTCFLAGS)
KBUILD_HOSTCXXFLAGS := -Wall -O2 $(HOST_LFS_CFLAGS) $(HOSTCXXFLAGS)
KBUILD_HOSTRUSTFLAGS := $(rust_common_flags) -O -Cstrip=debuginfo \
-Zallow-features= $(HOSTRUSTFLAGS)
KBUILD_HOSTLDFLAGS := $(HOST_LFS_LDFLAGS) $(HOSTLDFLAGS)
KBUILD_HOSTLDLIBS := $(HOST_LFS_LIBS) $(HOSTLDLIBS)
@ -473,6 +501,12 @@ OBJDUMP = $(CROSS_COMPILE)objdump
READELF = $(CROSS_COMPILE)readelf
STRIP = $(CROSS_COMPILE)strip
endif
RUSTC = rustc
RUSTDOC = rustdoc
RUSTFMT = rustfmt
CLIPPY_DRIVER = clippy-driver
BINDGEN = bindgen
CARGO = cargo
PAHOLE = pahole
RESOLVE_BTFIDS = $(objtree)/tools/bpf/resolve_btfids/resolve_btfids
LEX = flex
@ -498,9 +532,11 @@ CHECKFLAGS := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \
-Wbitwise -Wno-return-void -Wno-unknown-attribute $(CF)
NOSTDINC_FLAGS :=
CFLAGS_MODULE =
RUSTFLAGS_MODULE =
AFLAGS_MODULE =
LDFLAGS_MODULE =
CFLAGS_KERNEL =
RUSTFLAGS_KERNEL =
AFLAGS_KERNEL =
LDFLAGS_vmlinux =
@ -529,15 +565,43 @@ KBUILD_CFLAGS := -Wall -Wundef -Werror=strict-prototypes -Wno-trigraphs \
-Werror=return-type -Wno-format-security \
-std=gnu11
KBUILD_CPPFLAGS := -D__KERNEL__
KBUILD_RUSTFLAGS := $(rust_common_flags) \
--target=$(objtree)/rust/target.json \
-Cpanic=abort -Cembed-bitcode=n -Clto=n \
-Cforce-unwind-tables=n -Ccodegen-units=1 \
-Csymbol-mangling-version=v0 \
-Crelocation-model=static \
-Zfunction-sections=n \
-Dclippy::float_arithmetic
KBUILD_AFLAGS_KERNEL :=
KBUILD_CFLAGS_KERNEL :=
KBUILD_RUSTFLAGS_KERNEL :=
KBUILD_AFLAGS_MODULE := -DMODULE
KBUILD_CFLAGS_MODULE := -DMODULE
KBUILD_RUSTFLAGS_MODULE := --cfg MODULE
KBUILD_LDFLAGS_MODULE :=
KBUILD_LDFLAGS :=
CLANG_FLAGS :=
ifeq ($(KBUILD_CLIPPY),1)
RUSTC_OR_CLIPPY_QUIET := CLIPPY
RUSTC_OR_CLIPPY = $(CLIPPY_DRIVER)
else
RUSTC_OR_CLIPPY_QUIET := RUSTC
RUSTC_OR_CLIPPY = $(RUSTC)
endif
ifdef RUST_LIB_SRC
export RUST_LIB_SRC
endif
# Allows the usage of unstable features in stable compilers.
export RUSTC_BOOTSTRAP := 1
export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE LD CC HOSTPKG_CONFIG
export RUSTC RUSTDOC RUSTFMT RUSTC_OR_CLIPPY_QUIET RUSTC_OR_CLIPPY BINDGEN CARGO
export HOSTRUSTC KBUILD_HOSTRUSTFLAGS
export CPP AR NM STRIP OBJCOPY OBJDUMP READELF PAHOLE RESOLVE_BTFIDS LEX YACC AWK INSTALLKERNEL
export PERL PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
export KGZIP KBZIP2 KLZOP LZMA LZ4 XZ ZSTD
@ -546,9 +610,10 @@ export KBUILD_USERCFLAGS KBUILD_USERLDFLAGS
export KBUILD_CPPFLAGS NOSTDINC_FLAGS LINUXINCLUDE OBJCOPYFLAGS KBUILD_LDFLAGS
export KBUILD_CFLAGS CFLAGS_KERNEL CFLAGS_MODULE
export KBUILD_RUSTFLAGS RUSTFLAGS_KERNEL RUSTFLAGS_MODULE
export KBUILD_AFLAGS AFLAGS_KERNEL AFLAGS_MODULE
export KBUILD_AFLAGS_MODULE KBUILD_CFLAGS_MODULE KBUILD_LDFLAGS_MODULE
export KBUILD_AFLAGS_KERNEL KBUILD_CFLAGS_KERNEL
export KBUILD_AFLAGS_MODULE KBUILD_CFLAGS_MODULE KBUILD_RUSTFLAGS_MODULE KBUILD_LDFLAGS_MODULE
export KBUILD_AFLAGS_KERNEL KBUILD_CFLAGS_KERNEL KBUILD_RUSTFLAGS_KERNEL
export PAHOLE_FLAGS
# Files to ignore in find ... statements
@ -729,7 +794,7 @@ $(KCONFIG_CONFIG):
#
# Do not use $(call cmd,...) here. That would suppress prompts from syncconfig,
# so you cannot notice that Kconfig is waiting for the user input.
%/config/auto.conf %/config/auto.conf.cmd %/generated/autoconf.h: $(KCONFIG_CONFIG)
%/config/auto.conf %/config/auto.conf.cmd %/generated/autoconf.h %/generated/rustc_cfg: $(KCONFIG_CONFIG)
$(Q)$(kecho) " SYNC $@"
$(Q)$(MAKE) -f $(srctree)/Makefile syncconfig
else # !may-sync-config
@ -758,10 +823,17 @@ KBUILD_CFLAGS += $(call cc-disable-warning, address-of-packed-member)
ifdef CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE
KBUILD_CFLAGS += -O2
KBUILD_RUSTFLAGS += -Copt-level=2
else ifdef CONFIG_CC_OPTIMIZE_FOR_SIZE
KBUILD_CFLAGS += -Os
KBUILD_RUSTFLAGS += -Copt-level=s
endif
# Always set `debug-assertions` and `overflow-checks` because their default
# depends on `opt-level` and `debug-assertions`, respectively.
KBUILD_RUSTFLAGS += -Cdebug-assertions=$(if $(CONFIG_RUST_DEBUG_ASSERTIONS),y,n)
KBUILD_RUSTFLAGS += -Coverflow-checks=$(if $(CONFIG_RUST_OVERFLOW_CHECKS),y,n)
# Tell gcc to never replace conditional load with a non-conditional one
ifdef CONFIG_CC_IS_GCC
# gcc-10 renamed --param=allow-store-data-races=0 to
@ -792,6 +864,9 @@ KBUILD_CFLAGS-$(CONFIG_WERROR) += -Werror
KBUILD_CFLAGS-$(CONFIG_CC_NO_ARRAY_BOUNDS) += -Wno-array-bounds
KBUILD_CFLAGS += $(KBUILD_CFLAGS-y) $(CONFIG_CC_IMPLICIT_FALLTHROUGH)
KBUILD_RUSTFLAGS-$(CONFIG_WERROR) += -Dwarnings
KBUILD_RUSTFLAGS += $(KBUILD_RUSTFLAGS-y)
ifdef CONFIG_CC_IS_CLANG
KBUILD_CPPFLAGS += -Qunused-arguments
# The kernel builds with '-std=gnu11' so use of GNU extensions is acceptable.
@ -812,12 +887,15 @@ KBUILD_CFLAGS += $(call cc-disable-warning, dangling-pointer)
ifdef CONFIG_FRAME_POINTER
KBUILD_CFLAGS += -fno-omit-frame-pointer -fno-optimize-sibling-calls
KBUILD_RUSTFLAGS += -Cforce-frame-pointers=y
else
# Some targets (ARM with Thumb2, for example), can't be built with frame
# pointers. For those, we don't have FUNCTION_TRACER automatically
# select FRAME_POINTER. However, FUNCTION_TRACER adds -pg, and this is
# incompatible with -fomit-frame-pointer with current GCC, so we don't use
# -fomit-frame-pointer with FUNCTION_TRACER.
# In the Rust target specification, "frame-pointer" is set explicitly
# to "may-omit".
ifndef CONFIG_FUNCTION_TRACER
KBUILD_CFLAGS += -fomit-frame-pointer
endif
@ -882,8 +960,10 @@ ifdef CONFIG_DEBUG_SECTION_MISMATCH
KBUILD_CFLAGS += -fno-inline-functions-called-once
endif
# `rustc`'s `-Zfunction-sections` applies to data too (as of 1.59.0).
ifdef CONFIG_LD_DEAD_CODE_DATA_ELIMINATION
KBUILD_CFLAGS_KERNEL += -ffunction-sections -fdata-sections
KBUILD_RUSTFLAGS_KERNEL += -Zfunction-sections=y
LDFLAGS_vmlinux += --gc-sections
endif
@ -1026,10 +1106,11 @@ include $(addprefix $(srctree)/, $(include-y))
# Do not add $(call cc-option,...) below this line. When you build the kernel
# from the clean source tree, the GCC plugins do not exist at this point.
# Add user supplied CPPFLAGS, AFLAGS and CFLAGS as the last assignments
# Add user supplied CPPFLAGS, AFLAGS, CFLAGS and RUSTFLAGS as the last assignments
KBUILD_CPPFLAGS += $(KCPPFLAGS)
KBUILD_AFLAGS += $(KAFLAGS)
KBUILD_CFLAGS += $(KCFLAGS)
KBUILD_RUSTFLAGS += $(KRUSTFLAGS)
KBUILD_LDFLAGS_MODULE += --build-id=sha1
LDFLAGS_vmlinux += --build-id=sha1
@ -1104,6 +1185,7 @@ ifeq ($(KBUILD_EXTMOD),)
core-y += kernel/ certs/ mm/ fs/ ipc/ security/ crypto/
core-$(CONFIG_BLOCK) += block/
core-$(CONFIG_IO_URING) += io_uring/
core-$(CONFIG_RUST) += rust/
vmlinux-dirs := $(patsubst %/,%,$(filter %/, \
$(core-y) $(core-m) $(drivers-y) $(drivers-m) \
@ -1206,6 +1288,10 @@ prepare0: archprepare
# All the preparing..
prepare: prepare0
ifdef CONFIG_RUST
$(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh -v
$(Q)$(MAKE) $(build)=rust
endif
PHONY += remove-stale-files
remove-stale-files:
@ -1499,7 +1585,7 @@ endif # CONFIG_MODULES
# Directories & files removed with 'make clean'
CLEAN_FILES += include/ksym vmlinux.symvers modules-only.symvers \
modules.builtin modules.builtin.modinfo modules.nsdeps \
compile_commands.json .thinlto-cache
compile_commands.json .thinlto-cache rust/test rust/doc
# Directories & files removed with 'make mrproper'
MRPROPER_FILES += include/config include/generated \
@ -1510,7 +1596,8 @@ MRPROPER_FILES += include/config include/generated \
certs/signing_key.pem \
certs/x509.genkey \
vmlinux-gdb.py \
*.spec
*.spec \
rust/target.json rust/libmacros.so
# clean - Delete most, but leave enough to build external modules
#
@ -1535,6 +1622,9 @@ $(mrproper-dirs):
mrproper: clean $(mrproper-dirs)
$(call cmd,rmfiles)
@find . $(RCS_FIND_IGNORE) \
\( -name '*.rmeta' \) \
-type f -print | xargs rm -f
# distclean
#
@ -1622,6 +1712,24 @@ help:
@echo ' kselftest-merge - Merge all the config dependencies of'
@echo ' kselftest to existing .config.'
@echo ''
@echo 'Rust targets:'
@echo ' rustavailable - Checks whether the Rust toolchain is'
@echo ' available and, if not, explains why.'
@echo ' rustfmt - Reformat all the Rust code in the kernel'
@echo ' rustfmtcheck - Checks if all the Rust code in the kernel'
@echo ' is formatted, printing a diff otherwise.'
@echo ' rustdoc - Generate Rust documentation'
@echo ' (requires kernel .config)'
@echo ' rusttest - Runs the Rust tests'
@echo ' (requires kernel .config; downloads external repos)'
@echo ' rust-analyzer - Generate rust-project.json rust-analyzer support file'
@echo ' (requires kernel .config)'
@echo ' dir/file.[os] - Build specified target only'
@echo ' dir/file.rsi - Build macro expanded source, similar to C preprocessing.'
@echo ' Run with RUSTFMT=n to skip reformatting if needed.'
@echo ' The output is not intended to be compilable.'
@echo ' dir/file.ll - Build the LLVM assembly file'
@echo ''
@$(if $(dtstree), \
echo 'Devicetree:'; \
echo '* dtbs - Build device tree blobs for enabled boards'; \
@ -1694,6 +1802,52 @@ PHONY += $(DOC_TARGETS)
$(DOC_TARGETS):
$(Q)$(MAKE) $(build)=Documentation $@
# Rust targets
# ---------------------------------------------------------------------------
# "Is Rust available?" target
PHONY += rustavailable
rustavailable:
$(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh -v && echo "Rust is available!"
# Documentation target
#
# Using the singular to avoid running afoul of `no-dot-config-targets`.
PHONY += rustdoc
rustdoc: prepare
$(Q)$(MAKE) $(build)=rust $@
# Testing target
PHONY += rusttest
rusttest: prepare
$(Q)$(MAKE) $(build)=rust $@
# Formatting targets
PHONY += rustfmt rustfmtcheck
# We skip `rust/alloc` since we want to minimize the diff w.r.t. upstream.
#
# We match using absolute paths since `find` does not resolve them
# when matching, which is a problem when e.g. `srctree` is `..`.
# We `grep` afterwards in order to remove the directory entry itself.
rustfmt:
$(Q)find $(abs_srctree) -type f -name '*.rs' \
-o -path $(abs_srctree)/rust/alloc -prune \
-o -path $(abs_objtree)/rust/test -prune \
| grep -Fv $(abs_srctree)/rust/alloc \
| grep -Fv $(abs_objtree)/rust/test \
| grep -Fv generated \
| xargs $(RUSTFMT) $(rustfmt_flags)
rustfmtcheck: rustfmt_flags = --check
rustfmtcheck: rustfmt
# IDE support targets
PHONY += rust-analyzer
rust-analyzer:
$(Q)$(MAKE) $(build)=rust $@
# Misc
# ---------------------------------------------------------------------------
@ -1861,7 +2015,7 @@ $(clean-dirs):
clean: $(clean-dirs)
$(call cmd,rmfiles)
@find $(or $(KBUILD_EXTMOD), .) $(RCS_FIND_IGNORE) \
\( -name '*.[aios]' -o -name '*.ko' -o -name '.*.cmd' \
\( -name '*.[aios]' -o -name '*.rsi' -o -name '*.ko' -o -name '.*.cmd' \
-o -name '*.ko.*' \
-o -name '*.dtb' -o -name '*.dtbo' -o -name '*.dtb.S' -o -name '*.dt.yaml' \
-o -name '*.dwo' -o -name '*.lst' \

6
arch/Kconfig
View File

@ -355,6 +355,12 @@ config HAVE_RSEQ
This symbol should be selected by an architecture if it
supports an implementation of restartable sequences.
config HAVE_RUST
bool
help
This symbol should be selected by an architecture if it
supports Rust.
config HAVE_FUNCTION_ARG_ACCESS_API
bool
help

1
arch/x86/Kconfig
View File

@ -257,6 +257,7 @@ config X86
select HAVE_STATIC_CALL_INLINE if HAVE_OBJTOOL
select HAVE_PREEMPT_DYNAMIC_CALL
select HAVE_RSEQ
select HAVE_RUST if X86_64
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_UACCESS_VALIDATION if HAVE_OBJTOOL
select HAVE_UNSTABLE_SCHED_CLOCK

10
arch/x86/Makefile
View File

@ -68,6 +68,7 @@ export BITS
# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53383
#
KBUILD_CFLAGS += -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx
KBUILD_RUSTFLAGS += -Ctarget-feature=-sse,-sse2,-sse3,-ssse3,-sse4.1,-sse4.2,-avx,-avx2
ifeq ($(CONFIG_X86_KERNEL_IBT),y)
#
@ -155,8 +156,17 @@ else
cflags-$(CONFIG_GENERIC_CPU) += -mtune=generic
KBUILD_CFLAGS += $(cflags-y)
rustflags-$(CONFIG_MK8) += -Ctarget-cpu=k8
rustflags-$(CONFIG_MPSC) += -Ctarget-cpu=nocona
rustflags-$(CONFIG_MCORE2) += -Ctarget-cpu=core2
rustflags-$(CONFIG_MATOM) += -Ctarget-cpu=atom
rustflags-$(CONFIG_GENERIC_CPU) += -Ztune-cpu=generic
KBUILD_RUSTFLAGS += $(rustflags-y)
KBUILD_CFLAGS += -mno-red-zone
KBUILD_CFLAGS += -mcmodel=kernel
KBUILD_RUSTFLAGS += -Cno-redzone=y
KBUILD_RUSTFLAGS += -Ccode-model=kernel
endif
#

6
include/linux/compiler_types.h
View File

@ -4,8 +4,12 @@
#ifndef __ASSEMBLY__
/*
* Skipped when running bindgen due to a libclang issue;
* see https://github.com/rust-lang/rust-bindgen/issues/2244.
*/
#if defined(CONFIG_DEBUG_INFO_BTF) && defined(CONFIG_PAHOLE_HAS_BTF_TAG) && \
__has_attribute(btf_type_tag)
__has_attribute(btf_type_tag) && !defined(__BINDGEN__)
# define BTF_TYPE_TAG(value) __attribute__((btf_type_tag(#value)))
#else
# define BTF_TYPE_TAG(value) /* nothing */

2
include/linux/kallsyms.h
View File

@ -15,7 +15,7 @@
#include <asm/sections.h>
#define KSYM_NAME_LEN 128
#define KSYM_NAME_LEN 512
#define KSYM_SYMBOL_LEN (sizeof("%s+%#lx/%#lx [%s %s]") + \
(KSYM_NAME_LEN - 1) + \
2*(BITS_PER_LONG*3/10) + (MODULE_NAME_LEN - 1) + \

46
init/Kconfig
View File

@ -60,6 +60,17 @@ config LLD_VERSION
default $(ld-version) if LD_IS_LLD
default 0
config RUST_IS_AVAILABLE
def_bool $(success,$(srctree)/scripts/rust_is_available.sh)
help
This shows whether a suitable Rust toolchain is available (found).
Please see Documentation/rust/quick-start.rst for instructions on how
to satify the build requirements of Rust support.
In particular, the Makefile target 'rustavailable' is useful to check
why the Rust toolchain is not being detected.
config CC_CAN_LINK
bool
default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(USERCFLAGS) $(USERLDFLAGS) $(m64-flag)) if 64BIT
@ -147,7 +158,8 @@ config WERROR
default COMPILE_TEST
help
A kernel build should not cause any compiler warnings, and this
enables the '-Werror' flag to enforce that rule by default.
enables the '-Werror' (for C) and '-Dwarnings' (for Rust) flags
to enforce that rule by default.
However, if you have a new (or very old) compiler with odd and
unusual warnings, or you have some architecture with problems,
@ -1899,6 +1911,38 @@ config PROFILING
Say Y here to enable the extended profiling support mechanisms used
by profilers.
config RUST
bool "Rust support"
depends on HAVE_RUST
depends on RUST_IS_AVAILABLE
depends on !MODVERSIONS
depends on !GCC_PLUGINS
depends on !RANDSTRUCT
depends on !DEBUG_INFO_BTF
select CONSTRUCTORS
help
Enables Rust support in the kernel.
This allows other Rust-related options, like drivers written in Rust,
to be selected.
It is also required to be able to load external kernel modules
written in Rust.
See Documentation/rust/ for more information.
If unsure, say N.
config RUSTC_VERSION_TEXT
string
depends on RUST
default $(shell,command -v $(RUSTC) >/dev/null 2>&1 && $(RUSTC) --version || echo n)
config BINDGEN_VERSION_TEXT
string
depends on RUST
default $(shell,command -v $(BINDGEN) >/dev/null 2>&1 && $(BINDGEN) --version || echo n)
#
# Place an empty function call at each tracepoint site. Can be
# dynamically changed for a probe function.

1
kernel/configs/rust.config Normal file
View File

@ -0,0 +1 @@
CONFIG_RUST=y

26
kernel/kallsyms.c
View File

@ -50,12 +50,20 @@ static unsigned int kallsyms_expand_symbol(unsigned int off,
data = &kallsyms_names[off];
len = *data;
data++;
off++;
/* If MSB is 1, it is a "big" symbol, so needs an additional byte. */
if ((len & 0x80) != 0) {
len = (len & 0x7F) | (*data << 7);
data++;
off++;
}
/*
* Update the offset to return the offset for the next symbol on
* the compressed stream.
*/
off += len + 1;
off += len;
/*
* For every byte on the compressed symbol data, copy the table
@ -108,7 +116,7 @@ static char kallsyms_get_symbol_type(unsigned int off)
static unsigned int get_symbol_offset(unsigned long pos)
{
const u8 *name;
int i;
int i, len;
/*
* Use the closest marker we have. We have markers every 256 positions,
@ -122,8 +130,18 @@ static unsigned int get_symbol_offset(unsigned long pos)
* so we just need to add the len to the current pointer for every
* symbol we wish to skip.
*/
for (i = 0; i < (pos & 0xFF); i++)
name = name + (*name) + 1;
for (i = 0; i < (pos & 0xFF); i++) {
len = *name;
/*
* If MSB is 1, it is a "big" symbol, so we need to look into
* the next byte (and skip it, too).
*/
if ((len & 0x80) != 0)
len = ((len & 0x7F) | (name[1] << 7)) + 1;
name = name + len + 1;
}
return name - kallsyms_names;
}

4
kernel/livepatch/core.c
View File

@ -213,7 +213,7 @@ static int klp_resolve_symbols(Elf_Shdr *sechdrs, const char *strtab,
* we use the smallest/strictest upper bound possible (56, based on
* the current definition of MODULE_NAME_LEN) to prevent overflows.
*/
BUILD_BUG_ON(MODULE_NAME_LEN < 56 || KSYM_NAME_LEN != 128);
BUILD_BUG_ON(MODULE_NAME_LEN < 56 || KSYM_NAME_LEN != 512);
relas = (Elf_Rela *) relasec->sh_addr;
/* For each rela in this klp relocation section */
@ -227,7 +227,7 @@ static int klp_resolve_symbols(Elf_Shdr *sechdrs, const char *strtab,
/* Format: .klp.sym.sym_objname.sym_name,sympos */
cnt = sscanf(strtab + sym->st_name,
".klp.sym.%55[^.].%127[^,],%lu",
".klp.sym.%55[^.].%511[^,],%lu",
sym_objname, sym_name, &sympos);
if (cnt != 3) {
pr_err("symbol %s has an incorrectly formatted name\n",

34
lib/Kconfig.debug
View File

@ -2710,6 +2710,40 @@ config HYPERV_TESTING
endmenu # "Kernel Testing and Coverage"
menu "Rust hacking"
config RUST_DEBUG_ASSERTIONS
bool "Debug assertions"
depends on RUST
help
Enables rustc's `-Cdebug-assertions` codegen option.
This flag lets you turn `cfg(debug_assertions)` conditional
compilation on or off. This can be used to enable extra debugging
code in development but not in production. For example, it controls
the behavior of the standard library's `debug_assert!` macro.
Note that this will apply to all Rust code, including `core`.
If unsure, say N.
config RUST_OVERFLOW_CHECKS
bool "Overflow checks"
default y
depends on RUST
help
Enables rustc's `-Coverflow-checks` codegen option.
This flag allows you to control the behavior of runtime integer
overflow. When overflow-checks are enabled, a Rust panic will occur
on overflow.
Note that this will apply to all Rust code, including `core`.
If unsure, say Y.
endmenu # "Rust"
source "Documentation/Kconfig"
endmenu # Kernel hacking

13
lib/vsprintf.c
View File

@ -2246,6 +2246,9 @@ int __init no_hash_pointers_enable(char *str)
}
early_param("no_hash_pointers", no_hash_pointers_enable);
/* Used for Rust formatting ('%pA'). */
char *rust_fmt_argument(char *buf, char *end, void *ptr);
/*
* Show a '%p' thing. A kernel extension is that the '%p' is followed
* by an extra set of alphanumeric characters that are extended format
@ -2372,6 +2375,10 @@ early_param("no_hash_pointers", no_hash_pointers_enable);
*
* Note: The default behaviour (unadorned %p) is to hash the address,
* rendering it useful as a unique identifier.
*
* There is also a '%pA' format specifier, but it is only intended to be used
* from Rust code to format core::fmt::Arguments. Do *not* use it from C.
* See rust/kernel/print.rs for details.
*/
static noinline_for_stack
char *pointer(const char *fmt, char *buf, char *end, void *ptr,
@ -2444,6 +2451,12 @@ char *pointer(const char *fmt, char *buf, char *end, void *ptr,
return device_node_string(buf, end, ptr, spec, fmt + 1);
case 'f':
return fwnode_string(buf, end, ptr, spec, fmt + 1);
case 'A':
if (!IS_ENABLED(CONFIG_RUST)) {
WARN_ONCE(1, "Please remove %%pA from non-Rust code\n");
return error_string(buf, end, "(%pA?)", spec);
}
return rust_fmt_argument(buf, end, ptr);
case 'x':
return pointer_string(buf, end, ptr, spec);
case 'e':

8
rust/.gitignore vendored Normal file
View File

@ -0,0 +1,8 @@
# SPDX-License-Identifier: GPL-2.0
target.json
bindings_generated.rs
bindings_helpers_generated.rs
exports_*_generated.h
doc/
test/

381
rust/Makefile Normal file
View File

@ -0,0 +1,381 @@
# SPDX-License-Identifier: GPL-2.0
always-$(CONFIG_RUST) += target.json
no-clean-files += target.json
obj-$(CONFIG_RUST) += core.o compiler_builtins.o
always-$(CONFIG_RUST) += exports_core_generated.h
# Missing prototypes are expected in the helpers since these are exported
# for Rust only, thus there is no header nor prototypes.
obj-$(CONFIG_RUST) += helpers.o
CFLAGS_REMOVE_helpers.o = -Wmissing-prototypes -Wmissing-declarations
always-$(CONFIG_RUST) += libmacros.so
no-clean-files += libmacros.so
always-$(CONFIG_RUST) += bindings/bindings_generated.rs bindings/bindings_helpers_generated.rs
obj-$(CONFIG_RUST) += alloc.o bindings.o kernel.o
always-$(CONFIG_RUST) += exports_alloc_generated.h exports_bindings_generated.h \
exports_kernel_generated.h
obj-$(CONFIG_RUST) += exports.o
# Avoids running `$(RUSTC)` for the sysroot when it may not be available.
ifdef CONFIG_RUST
# `$(rust_flags)` is passed in case the user added `--sysroot`.
rustc_sysroot := $(shell $(RUSTC) $(rust_flags) --print sysroot)
rustc_host_target := $(shell $(RUSTC) --version --verbose | grep -F 'host: ' | cut -d' ' -f2)
RUST_LIB_SRC ?= $(rustc_sysroot)/lib/rustlib/src/rust/library
ifeq ($(quiet),silent_)
cargo_quiet=-q
rust_test_quiet=-q
rustdoc_test_quiet=--test-args -q
else ifeq ($(quiet),quiet_)
rust_test_quiet=-q
rustdoc_test_quiet=--test-args -q
else
cargo_quiet=--verbose
endif
core-cfgs = \
--cfg no_fp_fmt_parse
alloc-cfgs = \
--cfg no_fmt \
--cfg no_global_oom_handling \
--cfg no_macros \
--cfg no_rc \
--cfg no_str \
--cfg no_string \
--cfg no_sync \
--cfg no_thin
quiet_cmd_rustdoc = RUSTDOC $(if $(rustdoc_host),H, ) $<
cmd_rustdoc = \
OBJTREE=$(abspath $(objtree)) \
$(RUSTDOC) $(if $(rustdoc_host),$(rust_common_flags),$(rust_flags)) \
$(rustc_target_flags) -L$(objtree)/$(obj) \
--output $(objtree)/$(obj)/doc \
--crate-name $(subst rustdoc-,,$@) \
@$(objtree)/include/generated/rustc_cfg $<
# The `html_logo_url` and `html_favicon_url` forms of the `doc` attribute
# can be used to specify a custom logo. However:
# - The given value is used as-is, thus it cannot be relative or a local file
# (unlike the non-custom case) since the generated docs have subfolders.
# - It requires adding it to every crate.
# - It requires changing `core` which comes from the sysroot.
#
# Using `-Zcrate-attr` would solve the last two points, but not the first.
# The https://github.com/rust-lang/rfcs/pull/3226 RFC suggests two new
# command-like flags to solve the issue. Meanwhile, we use the non-custom case
# and then retouch the generated files.
rustdoc: rustdoc-core rustdoc-macros rustdoc-compiler_builtins \
rustdoc-alloc rustdoc-kernel
$(Q)cp $(srctree)/Documentation/images/logo.svg $(objtree)/$(obj)/doc
$(Q)cp $(srctree)/Documentation/images/COPYING-logo $(objtree)/$(obj)/doc
$(Q)find $(objtree)/$(obj)/doc -name '*.html' -type f -print0 | xargs -0 sed -Ei \
-e 's:rust-logo\.svg:logo.svg:g' \
-e 's:rust-logo\.png:logo.svg:g' \
-e 's:favicon\.svg:logo.svg:g' \
-e 's:<link rel="alternate icon" type="image/png" href="[./]*favicon-(16x16|32x32)\.png">::g'
$(Q)echo '.logo-container > img { object-fit: contain; }' \
>> $(objtree)/$(obj)/doc/rustdoc.css
rustdoc-macros: private rustdoc_host = yes
rustdoc-macros: private rustc_target_flags = --crate-type proc-macro \
--extern proc_macro
rustdoc-macros: $(src)/macros/lib.rs FORCE
$(call if_changed,rustdoc)
rustdoc-core: private rustc_target_flags = $(core-cfgs)
rustdoc-core: $(RUST_LIB_SRC)/core/src/lib.rs FORCE
$(call if_changed,rustdoc)
rustdoc-compiler_builtins: $(src)/compiler_builtins.rs rustdoc-core FORCE
$(call if_changed,rustdoc)
# We need to allow `rustdoc::broken_intra_doc_links` because some
# `no_global_oom_handling` functions refer to non-`no_global_oom_handling`
# functions. Ideally `rustdoc` would have a way to distinguish broken links
# due to things that are "configured out" vs. entirely non-existing ones.
rustdoc-alloc: private rustc_target_flags = $(alloc-cfgs) \
-Arustdoc::broken_intra_doc_links
rustdoc-alloc: $(src)/alloc/lib.rs rustdoc-core rustdoc-compiler_builtins FORCE
$(call if_changed,rustdoc)
rustdoc-kernel: private rustc_target_flags = --extern alloc \
--extern macros=$(objtree)/$(obj)/libmacros.so \
--extern bindings
rustdoc-kernel: $(src)/kernel/lib.rs rustdoc-core rustdoc-macros \
rustdoc-compiler_builtins rustdoc-alloc $(obj)/libmacros.so \
$(obj)/bindings.o FORCE
$(call if_changed,rustdoc)
quiet_cmd_rustc_test_library = RUSTC TL $<
cmd_rustc_test_library = \
OBJTREE=$(abspath $(objtree)) \
$(RUSTC) $(rust_common_flags) \
@$(objtree)/include/generated/rustc_cfg $(rustc_target_flags) \
--crate-type $(if $(rustc_test_library_proc),proc-macro,rlib) \
--out-dir $(objtree)/$(obj)/test --cfg testlib \
--sysroot $(objtree)/$(obj)/test/sysroot \
-L$(objtree)/$(obj)/test \
--crate-name $(subst rusttest-,,$(subst rusttestlib-,,$@)) $<
rusttestlib-macros: private rustc_target_flags = --extern proc_macro
rusttestlib-macros: private rustc_test_library_proc = yes
rusttestlib-macros: $(src)/macros/lib.rs rusttest-prepare FORCE
$(call if_changed,rustc_test_library)
rusttestlib-bindings: $(src)/bindings/lib.rs rusttest-prepare FORCE
$(call if_changed,rustc_test_library)
quiet_cmd_rustdoc_test = RUSTDOC T $<
cmd_rustdoc_test = \
OBJTREE=$(abspath $(objtree)) \
$(RUSTDOC) --test $(rust_common_flags) \
@$(objtree)/include/generated/rustc_cfg \
$(rustc_target_flags) $(rustdoc_test_target_flags) \
--sysroot $(objtree)/$(obj)/test/sysroot $(rustdoc_test_quiet) \
-L$(objtree)/$(obj)/test --output $(objtree)/$(obj)/doc \
--crate-name $(subst rusttest-,,$@) $<
# We cannot use `-Zpanic-abort-tests` because some tests are dynamic,
# so for the moment we skip `-Cpanic=abort`.
quiet_cmd_rustc_test = RUSTC T $<
cmd_rustc_test = \
OBJTREE=$(abspath $(objtree)) \
$(RUSTC) --test $(rust_common_flags) \
@$(objtree)/include/generated/rustc_cfg \
$(rustc_target_flags) --out-dir $(objtree)/$(obj)/test \
--sysroot $(objtree)/$(obj)/test/sysroot \
-L$(objtree)/$(obj)/test \
--crate-name $(subst rusttest-,,$@) $<; \
$(objtree)/$(obj)/test/$(subst rusttest-,,$@) $(rust_test_quiet) \
$(rustc_test_run_flags)
rusttest: rusttest-macros rusttest-kernel
# This prepares a custom sysroot with our custom `alloc` instead of
# the standard one.
#
# This requires several hacks:
# - Unlike `core` and `alloc`, `std` depends on more than a dozen crates,
# including third-party crates that need to be downloaded, plus custom
# `build.rs` steps. Thus hardcoding things here is not maintainable.
# - `cargo` knows how to build the standard library, but it is an unstable
# feature so far (`-Zbuild-std`).
# - `cargo` only considers the use case of building the standard library
# to use it in a given package. Thus we need to create a dummy package
# and pick the generated libraries from there.
# - Since we only keep a subset of upstream `alloc` in-tree, we need
# to recreate it on the fly by putting our sources on top.
# - The usual ways of modifying the dependency graph in `cargo` do not seem
# to apply for the `-Zbuild-std` steps, thus we have to mislead it
# by modifying the sources in the sysroot.
# - To avoid messing with the user's Rust installation, we create a clone
# of the sysroot. However, `cargo` ignores `RUSTFLAGS` in the `-Zbuild-std`
# steps, thus we use a wrapper binary passed via `RUSTC` to pass the flag.
#
# In the future, we hope to avoid the whole ordeal by either:
# - Making the `test` crate not depend on `std` (either improving upstream
# or having our own custom crate).
# - Making the tests run in kernel space (requires the previous point).
# - Making `std` and friends be more like a "normal" crate, so that
# `-Zbuild-std` and related hacks are not needed.
quiet_cmd_rustsysroot = RUSTSYSROOT
cmd_rustsysroot = \
rm -rf $(objtree)/$(obj)/test; \
mkdir -p $(objtree)/$(obj)/test; \
cp -a $(rustc_sysroot) $(objtree)/$(obj)/test/sysroot; \
cp -r $(srctree)/$(src)/alloc/* \
$(objtree)/$(obj)/test/sysroot/lib/rustlib/src/rust/library/alloc/src; \
echo '\#!/bin/sh' > $(objtree)/$(obj)/test/rustc_sysroot; \
echo "$(RUSTC) --sysroot=$(abspath $(objtree)/$(obj)/test/sysroot) \"\$$@\"" \
>> $(objtree)/$(obj)/test/rustc_sysroot; \
chmod u+x $(objtree)/$(obj)/test/rustc_sysroot; \
$(CARGO) -q new $(objtree)/$(obj)/test/dummy; \
RUSTC=$(objtree)/$(obj)/test/rustc_sysroot $(CARGO) $(cargo_quiet) \
test -Zbuild-std --target $(rustc_host_target) \
--manifest-path $(objtree)/$(obj)/test/dummy/Cargo.toml; \
rm $(objtree)/$(obj)/test/sysroot/lib/rustlib/$(rustc_host_target)/lib/*; \
cp $(objtree)/$(obj)/test/dummy/target/$(rustc_host_target)/debug/deps/* \
$(objtree)/$(obj)/test/sysroot/lib/rustlib/$(rustc_host_target)/lib
rusttest-prepare: FORCE
$(call if_changed,rustsysroot)
rusttest-macros: private rustc_target_flags = --extern proc_macro
rusttest-macros: private rustdoc_test_target_flags = --crate-type proc-macro
rusttest-macros: $(src)/macros/lib.rs rusttest-prepare FORCE
$(call if_changed,rustc_test)
$(call if_changed,rustdoc_test)
rusttest-kernel: private rustc_target_flags = --extern alloc \
--extern macros --extern bindings
rusttest-kernel: $(src)/kernel/lib.rs rusttest-prepare \
rusttestlib-macros rusttestlib-bindings FORCE
$(call if_changed,rustc_test)
$(call if_changed,rustc_test_library)
filechk_rust_target = $(objtree)/scripts/generate_rust_target < $<
$(obj)/target.json: $(objtree)/include/config/auto.conf FORCE
$(call filechk,rust_target)
ifdef CONFIG_CC_IS_CLANG
bindgen_c_flags = $(c_flags)
else
# bindgen relies on libclang to parse C. Ideally, bindgen would support a GCC
# plugin backend and/or the Clang driver would be perfectly compatible with GCC.
#
# For the moment, here we are tweaking the flags on the fly. This is a hack,
# and some kernel configurations may not work (e.g. `GCC_PLUGIN_RANDSTRUCT`
# if we end up using one of those structs).
bindgen_skip_c_flags := -mno-fp-ret-in-387 -mpreferred-stack-boundary=% \
-mskip-rax-setup -mgeneral-regs-only -msign-return-address=% \
-mindirect-branch=thunk-extern -mindirect-branch-register \
-mfunction-return=thunk-extern -mrecord-mcount -mabi=lp64 \
-mindirect-branch-cs-prefix -mstack-protector-guard% -mtraceback=no \
-mno-pointers-to-nested-functions -mno-string \
-mno-strict-align -mstrict-align \
-fconserve-stack -falign-jumps=% -falign-loops=% \
-femit-struct-debug-baseonly -fno-ipa-cp-clone -fno-ipa-sra \
-fno-partial-inlining -fplugin-arg-arm_ssp_per_task_plugin-% \
-fno-reorder-blocks -fno-allow-store-data-races -fasan-shadow-offset=% \
-fzero-call-used-regs=% -fno-stack-clash-protection \
-fno-inline-functions-called-once \
--param=% --param asan-%
# Derived from `scripts/Makefile.clang`.
BINDGEN_TARGET_x86 := x86_64-linux-gnu
BINDGEN_TARGET := $(BINDGEN_TARGET_$(SRCARCH))
# All warnings are inhibited since GCC builds are very experimental,
# many GCC warnings are not supported by Clang, they may only appear in
# some configurations, with new GCC versions, etc.
bindgen_extra_c_flags = -w --target=$(BINDGEN_TARGET)
bindgen_c_flags = $(filter-out $(bindgen_skip_c_flags), $(c_flags)) \
$(bindgen_extra_c_flags)
endif
ifdef CONFIG_LTO
bindgen_c_flags_lto = $(filter-out $(CC_FLAGS_LTO), $(bindgen_c_flags))
else
bindgen_c_flags_lto = $(bindgen_c_flags)
endif
bindgen_c_flags_final = $(bindgen_c_flags_lto) -D__BINDGEN__
quiet_cmd_bindgen = BINDGEN $@
cmd_bindgen = \
$(BINDGEN) $< $(bindgen_target_flags) \
--use-core --with-derive-default --ctypes-prefix core::ffi --no-layout-tests \
--no-debug '.*' \
--size_t-is-usize -o $@ -- $(bindgen_c_flags_final) -DMODULE \
$(bindgen_target_cflags) $(bindgen_target_extra)
$(obj)/bindings/bindings_generated.rs: private bindgen_target_flags = \
$(shell grep -v '^\#\|^$$' $(srctree)/$(src)/bindgen_parameters)
$(obj)/bindings/bindings_generated.rs: $(src)/bindings/bindings_helper.h \
$(src)/bindgen_parameters FORCE
$(call if_changed_dep,bindgen)
# See `CFLAGS_REMOVE_helpers.o` above. In addition, Clang on C does not warn
# with `-Wmissing-declarations` (unlike GCC), so it is not strictly needed here
# given it is `libclang`; but for consistency, future Clang changes and/or
# a potential future GCC backend for `bindgen`, we disable it too.
$(obj)/bindings/bindings_helpers_generated.rs: private bindgen_target_flags = \
--blacklist-type '.*' --whitelist-var '' \
--whitelist-function 'rust_helper_.*'
$(obj)/bindings/bindings_helpers_generated.rs: private bindgen_target_cflags = \
-I$(objtree)/$(obj) -Wno-missing-prototypes -Wno-missing-declarations
$(obj)/bindings/bindings_helpers_generated.rs: private bindgen_target_extra = ; \
sed -Ei 's/pub fn rust_helper_([a-zA-Z0-9_]*)/#[link_name="rust_helper_\1"]\n pub fn \1/g' $@
$(obj)/bindings/bindings_helpers_generated.rs: $(src)/helpers.c FORCE
$(call if_changed_dep,bindgen)
quiet_cmd_exports = EXPORTS $@
cmd_exports = \
$(NM) -p --defined-only $< \
| grep -E ' (T|R|D) ' | cut -d ' ' -f 3 \
| xargs -Isymbol \
echo 'EXPORT_SYMBOL_RUST_GPL(symbol);' > $@
$(obj)/exports_core_generated.h: $(obj)/core.o FORCE
$(call if_changed,exports)
$(obj)/exports_alloc_generated.h: $(obj)/alloc.o FORCE
$(call if_changed,exports)
$(obj)/exports_bindings_generated.h: $(obj)/bindings.o FORCE
$(call if_changed,exports)
$(obj)/exports_kernel_generated.h: $(obj)/kernel.o FORCE
$(call if_changed,exports)
quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@
cmd_rustc_procmacro = \
$(RUSTC_OR_CLIPPY) $(rust_common_flags) \
--emit=dep-info,link --extern proc_macro \
--crate-type proc-macro --out-dir $(objtree)/$(obj) \
--crate-name $(patsubst lib%.so,%,$(notdir $@)) $<; \
mv $(objtree)/$(obj)/$(patsubst lib%.so,%,$(notdir $@)).d $(depfile); \
sed -i '/^\#/d' $(depfile)
# Procedural macros can only be used with the `rustc` that compiled it.
# Therefore, to get `libmacros.so` automatically recompiled when the compiler
# version changes, we add `core.o` as a dependency (even if it is not needed).
$(obj)/libmacros.so: $(src)/macros/lib.rs $(obj)/core.o FORCE
$(call if_changed_dep,rustc_procmacro)
quiet_cmd_rustc_library = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) L $@
cmd_rustc_library = \
OBJTREE=$(abspath $(objtree)) \
$(if $(skip_clippy),$(RUSTC),$(RUSTC_OR_CLIPPY)) \
$(filter-out $(skip_flags),$(rust_flags) $(rustc_target_flags)) \
--emit=dep-info,obj,metadata --crate-type rlib \
--out-dir $(objtree)/$(obj) -L$(objtree)/$(obj) \
--crate-name $(patsubst %.o,%,$(notdir $@)) $<; \
mv $(objtree)/$(obj)/$(patsubst %.o,%,$(notdir $@)).d $(depfile); \
sed -i '/^\#/d' $(depfile) \
$(if $(rustc_objcopy),;$(OBJCOPY) $(rustc_objcopy) $@)
rust-analyzer:
$(Q)$(srctree)/scripts/generate_rust_analyzer.py $(srctree) $(objtree) \
$(RUST_LIB_SRC) > $(objtree)/rust-project.json
$(obj)/core.o: private skip_clippy = 1
$(obj)/core.o: private skip_flags = -Dunreachable_pub
$(obj)/core.o: private rustc_target_flags = $(core-cfgs)
$(obj)/core.o: $(RUST_LIB_SRC)/core/src/lib.rs $(obj)/target.json FORCE
$(call if_changed_dep,rustc_library)
$(obj)/compiler_builtins.o: private rustc_objcopy = -w -W '__*'
$(obj)/compiler_builtins.o: $(src)/compiler_builtins.rs $(obj)/core.o FORCE
$(call if_changed_dep,rustc_library)
$(obj)/alloc.o: private skip_clippy = 1
$(obj)/alloc.o: private skip_flags = -Dunreachable_pub
$(obj)/alloc.o: private rustc_target_flags = $(alloc-cfgs)
$(obj)/alloc.o: $(src)/alloc/lib.rs $(obj)/compiler_builtins.o FORCE
$(call if_changed_dep,rustc_library)
$(obj)/bindings.o: $(src)/bindings/lib.rs \
$(obj)/compiler_builtins.o \
$(obj)/bindings/bindings_generated.rs \
$(obj)/bindings/bindings_helpers_generated.rs FORCE
$(call if_changed_dep,rustc_library)
$(obj)/kernel.o: private rustc_target_flags = --extern alloc \
--extern macros --extern bindings
$(obj)/kernel.o: $(src)/kernel/lib.rs $(obj)/alloc.o \
$(obj)/libmacros.so $(obj)/bindings.o FORCE
$(call if_changed_dep,rustc_library)
endif # CONFIG_RUST

33
rust/alloc/README.md Normal file
View File

@ -0,0 +1,33 @@
# `alloc`
These source files come from the Rust standard library, hosted in
the <https://github.com/rust-lang/rust> repository, licensed under
"Apache-2.0 OR MIT" and adapted for kernel use. For copyright details,
see <https://github.com/rust-lang/rust/blob/master/COPYRIGHT>.
Please note that these files should be kept as close as possible to
upstream. In general, only additions should be performed (e.g. new
methods). Eventually, changes should make it into upstream so that,
at some point, this fork can be dropped from the kernel tree.
## Rationale
On one hand, kernel folks wanted to keep `alloc` in-tree to have more
freedom in both workflow and actual features if actually needed
(e.g. receiver types if we ended up using them), which is reasonable.
On the other hand, Rust folks wanted to keep `alloc` as close as
upstream as possible and avoid as much divergence as possible, which
is also reasonable.
We agreed on a middle-ground: we would keep a subset of `alloc`
in-tree that would be as small and as close as possible to upstream.
Then, upstream can start adding the functions that we add to `alloc`
etc., until we reach a point where the kernel already knows exactly
what it needs in `alloc` and all the new methods are merged into
upstream, so that we can drop `alloc` from the kernel tree and go back
to using the upstream one.
By doing this, the kernel can go a bit faster now, and Rust can
slowly incorporate and discuss the changes as needed.

440
rust/alloc/alloc.rs Normal file
View File

@ -0,0 +1,440 @@
// SPDX-License-Identifier: Apache-2.0 OR MIT
//! Memory allocation APIs
#![stable(feature = "alloc_module", since = "1.28.0")]
#[cfg(not(test))]
use core::intrinsics;
use core::intrinsics::{min_align_of_val, size_of_val};
use core::ptr::Unique;
#[cfg(not(test))]
use core::ptr::{self, NonNull};
#[stable(feature = "alloc_module", since = "1.28.0")]
#[doc(inline)]
pub use core::alloc::*;
use core::marker::Destruct;
#[cfg(test)]
mod tests;
extern "Rust" {
// These are the magic symbols to call the global allocator. rustc generates
// them to call `__rg_alloc` etc. if there is a `#[global_allocator]` attribute
// (the code expanding that attribute macro generates those functions), or to call
// the default implementations in libstd (`__rdl_alloc` etc. in `library/std/src/alloc.rs`)
// otherwise.
// The rustc fork of LLVM also special-cases these function names to be able to optimize them
// like `malloc`, `realloc`, and `free`, respectively.
#[rustc_allocator]
#[rustc_allocator_nounwind]
fn __rust_alloc(size: usize, align: usize) -> *mut u8;
#[rustc_allocator_nounwind]
fn __rust_dealloc(ptr: *mut u8, size: usize, align: usize);
#[rustc_allocator_nounwind]
fn __rust_realloc(ptr: *mut u8, old_size: usize, align: usize, new_size: usize) -> *mut u8;
#[rustc_allocator_nounwind]
fn __rust_alloc_zeroed(size: usize, align: usize) -> *mut u8;
}
/// The global memory allocator.
///
/// This type implements the [`Allocator`] trait by forwarding calls
/// to the allocator registered with the `#[global_allocator]` attribute
/// if there is one, or the `std` crate’s default.
///
/// Note: while this type is unstable, the functionality it provides can be
/// accessed through the [free functions in `alloc`](self#functions).
#[unstable(feature = "allocator_api", issue = "32838")]
#[derive(Copy, Clone, Default, Debug)]
#[cfg(not(test))]
pub struct Global;
#[cfg(test)]
pub use std::alloc::Global;
/// Allocate memory with the global allocator.
///
/// This function forwards calls to the [`GlobalAlloc::alloc`] method
/// of the allocator registered with the `#[global_allocator]` attribute
/// if there is one, or the `std` crate’s default.
///
/// This function is expected to be deprecated in favor of the `alloc` method
/// of the [`Global`] type when it and the [`Allocator`] trait become stable.
///
/// # Safety
///
/// See [`GlobalAlloc::alloc`].
///
/// # Examples
///
/// ```
/// use std::alloc::{alloc, dealloc, Layout};
///
/// unsafe {
/// let layout = Layout::new::<u16>();
/// let ptr = alloc(layout);
///
/// *(ptr as *mut u16) = 42;
/// assert_eq!(*(ptr as *mut u16), 42);
///
/// dealloc(ptr, layout);
/// }
/// ```
#[stable(feature = "global_alloc", since = "1.28.0")]
#[must_use = "losing the pointer will leak memory"]
#[inline]
pub unsafe fn alloc(layout: Layout) -> *mut u8 {
unsafe { __rust_alloc(layout.size(), layout.align()) }
}
/// Deallocate memory with the global allocator.
///
/// This function forwards calls to the [`GlobalAlloc::dealloc`] method
/// of the allocator registered with the `#[global_allocator]` attribute
/// if there is one, or the `std` crate’s default.
///
/// This function is expected to be deprecated in favor of the `dealloc` method
/// of the [`Global`] type when it and the [`Allocator`] trait become stable.
///
/// # Safety
///
/// See [`GlobalAlloc::dealloc`].
#[stable(feature = "global_alloc", since = "1.28.0")]
#[inline]
pub unsafe fn dealloc(ptr: *mut u8, layout: Layout) {
unsafe { __rust_dealloc(ptr, layout.size(), layout.align()) }
}
/// Reallocate memory with the global allocator.
///
/// This function forwards calls to the [`GlobalAlloc::realloc`] method
/// of the allocator registered with the `#[global_allocator]` attribute
/// if there is one, or the `std` crate’s default.
///
/// This function is expected to be deprecated in favor of the `realloc` method
/// of the [`Global`] type when it and the [`Allocator`] trait become stable.
///
/// # Safety
///
/// See [`GlobalAlloc::realloc`].
#[stable(feature = "global_alloc", since = "1.28.0")]
#[must_use = "losing the pointer will leak memory"]
#[inline]
pub unsafe fn realloc(ptr: *mut u8, layout: Layout, new_size: usize) -> *mut u8 {
unsafe { __rust_realloc(ptr, layout.size(), layout.align(), new_size) }
}
/// Allocate zero-initialized memory with the global allocator.
///
/// This function forwards calls to the [`GlobalAlloc::alloc_zeroed`] method
/// of the allocator registered with the `#[global_allocator]` attribute
/// if there is one, or the `std` crate’s default.
///
/// This function is expected to be deprecated in favor of the `alloc_zeroed` method
/// of the [`Global`] type when it and the [`Allocator`] trait become stable.
///
/// # Safety
///
/// See [`GlobalAlloc::alloc_zeroed`].
///
/// # Examples
///
/// ```
/// use std::alloc::{alloc_zeroed, dealloc, Layout};
///
/// unsafe {
/// let layout = Layout::new::<u16>();
/// let ptr = alloc_zeroed(layout);
///
/// assert_eq!(*(ptr as *mut u16), 0);
///
/// dealloc(ptr, layout);
/// }
/// ```
#[stable(feature = "global_alloc", since = "1.28.0")]
#[must_use = "losing the pointer will leak memory"]
#[inline]
pub unsafe fn alloc_zeroed(layout: Layout) -> *mut u8 {
unsafe { __rust_alloc_zeroed(layout.size(), layout.align()) }
}
#[cfg(not(test))]
impl Global {
#[inline]
fn alloc_impl(&self, layout: Layout, zeroed: bool) -> Result<NonNull<[u8]>, AllocError> {
match layout.size() {
0 => Ok(NonNull::slice_from_raw_parts(layout.dangling(), 0)),
// SAFETY: `layout` is non-zero in size,
size => unsafe {
let raw_ptr = if zeroed { alloc_zeroed(layout) } else { alloc(layout) };
let ptr = NonNull::new(raw_ptr).ok_or(AllocError)?;
Ok(NonNull::slice_from_raw_parts(ptr, size))
},
}
}
// SAFETY: Same as `Allocator::grow`
#[inline]
unsafe fn grow_impl(
&self,
ptr: NonNull<u8>,
old_layout: Layout,
new_layout: Layout,
zeroed: bool,
) -> Result<NonNull<[u8]>, AllocError> {
debug_assert!(
new_layout.size() >= old_layout.size(),
"`new_layout.size()` must be greater than or equal to `old_layout.size()`"
);
match old_layout.size() {
0 => self.alloc_impl(new_layout, zeroed),
// SAFETY: `new_size` is non-zero as `old_size` is greater than or equal to `new_size`
// as required by safety conditions. Other conditions must be upheld by the caller
old_size if old_layout.align() == new_layout.align() => unsafe {
let new_size = new_layout.size();
// `realloc` probably checks for `new_size >= old_layout.size()` or something similar.
intrinsics::assume(new_size >= old_layout.size());
let raw_ptr = realloc(ptr.as_ptr(), old_layout, new_size);
let ptr = NonNull::new(raw_ptr).ok_or(AllocError)?;
if zeroed {
raw_ptr.add(old_size).write_bytes(0, new_size - old_size);
}
Ok(NonNull::slice_from_raw_parts(ptr, new_size))
},
// SAFETY: because `new_layout.size()` must be greater than or equal to `old_size`,
// both the old and new memory allocation are valid for reads and writes for `old_size`
// bytes. Also, because the old allocation wasn't yet deallocated, it cannot overlap
// `new_ptr`. Thus, the call to `copy_nonoverlapping` is safe. The safety contract
// for `dealloc` must be upheld by the caller.
old_size => unsafe {
let new_ptr = self.alloc_impl(new_layout, zeroed)?;
ptr::copy_nonoverlapping(ptr.as_ptr(), new_ptr.as_mut_ptr(), old_size);
self.deallocate(ptr, old_layout);
Ok(new_ptr)
},
}
}
}
#[unstable(feature = "allocator_api", issue = "32838")]
#[cfg(not(test))]
unsafe impl Allocator for Global {
#[inline]
fn allocate(&self, layout: Layout) -> Result<NonNull<[u8]>, AllocError> {
self.alloc_impl(layout, false)
}
#[inline]
fn allocate_zeroed(&self, layout: Layout) -> Result<NonNull<[u8]>, AllocError> {
self.alloc_impl(layout, true)
}
#[inline]
unsafe fn deallocate(&self, ptr: NonNull<u8>, layout: Layout) {
if layout.size() != 0 {
// SAFETY: `layout` is non-zero in size,
// other conditions must be upheld by the caller
unsafe { dealloc(ptr.as_ptr(), layout) }
}
}
#[inline]
unsafe fn grow(
&self,
ptr: NonNull<u8>,
old_layout: Layout,
new_layout: Layout,
) -> Result<NonNull<[u8]>, AllocError> {
// SAFETY: all conditions must be upheld by the caller
unsafe { self.grow_impl(ptr, old_layout, new_layout, false) }
}
#[inline]
unsafe fn grow_zeroed(
&self,
ptr: NonNull<u8>,
old_layout: Layout,
new_layout: Layout,
) -> Result<NonNull<[u8]>, AllocError> {
// SAFETY: all conditions must be upheld by the caller
unsafe { self.grow_impl(ptr, old_layout, new_layout, true) }
}
#[inline]
unsafe fn shrink(
&self,
ptr: NonNull<u8>,
old_layout: Layout,
new_layout: Layout,
) -> Result<NonNull<[u8]>, AllocError> {
debug_assert!(
new_layout.size() <= old_layout.size(),
"`new_layout.size()` must be smaller than or equal to `old_layout.size()`"
);
match new_layout.size() {
// SAFETY: conditions must be upheld by the caller
0 => unsafe {
self.deallocate(ptr, old_layout);
Ok(NonNull::slice_from_raw_parts(new_layout.dangling(), 0))
},
// SAFETY: `new_size` is non-zero. Other conditions must be upheld by the caller
new_size if old_layout.align() == new_layout.align() => unsafe {
// `realloc` probably checks for `new_size <= old_layout.size()` or something similar.
intrinsics::assume(new_size <= old_layout.size());
let raw_ptr = realloc(ptr.as_ptr(), old_layout, new_size);
let ptr = NonNull::new(raw_ptr).ok_or(AllocError)?;
Ok(NonNull::slice_from_raw_parts(ptr, new_size))
},
// SAFETY: because `new_size` must be smaller than or equal to `old_layout.size()`,
// both the old and new memory allocation are valid for reads and writes for `new_size`
// bytes. Also, because the old allocation wasn't yet deallocated, it cannot overlap
// `new_ptr`. Thus, the call to `copy_nonoverlapping` is safe. The safety contract
// for `dealloc` must be upheld by the caller.
new_size => unsafe {
let new_ptr = self.allocate(new_layout)?;
ptr::copy_nonoverlapping(ptr.as_ptr(), new_ptr.as_mut_ptr(), new_size);
self.deallocate(ptr, old_layout);
Ok(new_ptr)
},
}
}
}
/// The allocator for unique pointers.
#[cfg(all(not(no_global_oom_handling), not(test)))]
#[lang = "exchange_malloc"]
#[inline]
unsafe fn exchange_malloc(size: usize, align: usize) -> *mut u8 {
let layout = unsafe { Layout::from_size_align_unchecked(size, align) };
match Global.allocate(layout) {
Ok(ptr) => ptr.as_mut_ptr(),
Err(_) => handle_alloc_error(layout),
}
}
#[cfg_attr(not(test), lang = "box_free")]
#[inline]
#[rustc_const_unstable(feature = "const_box", issue = "92521")]
// This signature has to be the same as `Box`, otherwise an ICE will happen.
// When an additional parameter to `Box` is added (like `A: Allocator`), this has to be added here as
// well.
// For example if `Box` is changed to `struct Box<T: ?Sized, A: Allocator>(Unique<T>, A)`,
// this function has to be changed to `fn box_free<T: ?Sized, A: Allocator>(Unique<T>, A)` as well.
pub(crate) const unsafe fn box_free<T: ?Sized, A: ~const Allocator + ~const Destruct>(
ptr: Unique<T>,
alloc: A,
) {
unsafe {
let size = size_of_val(ptr.as_ref());
let align = min_align_of_val(ptr.as_ref());
let layout = Layout::from_size_align_unchecked(size, align);
alloc.deallocate(From::from(ptr.cast()), layout)
}
}
// # Allocation error handler
#[cfg(not(no_global_oom_handling))]
extern "Rust" {
// This is the magic symbol to call the global alloc error handler. rustc generates
// it to call `__rg_oom` if there is a `#[alloc_error_handler]`, or to call the
// default implementations below (`__rdl_oom`) otherwise.
fn __rust_alloc_error_handler(size: usize, align: usize) -> !;
}
/// Abort on memory allocation error or failure.
///
/// Callers of memory allocation APIs wishing to abort computation
/// in response to an allocation error are encouraged to call this function,
/// rather than directly invoking `panic!` or similar.
///
/// The default behavior of this function is to print a message to standard error
/// and abort the process.
/// It can be replaced with [`set_alloc_error_hook`] and [`take_alloc_error_hook`].
///
/// [`set_alloc_error_hook`]: ../../std/alloc/fn.set_alloc_error_hook.html
/// [`take_alloc_error_hook`]: ../../std/alloc/fn.take_alloc_error_hook.html
#[stable(feature = "global_alloc", since = "1.28.0")]
#[rustc_const_unstable(feature = "const_alloc_error", issue = "92523")]
#[cfg(all(not(no_global_oom_handling), not(test)))]
#[cold]
pub const fn handle_alloc_error(layout: Layout) -> ! {
const fn ct_error(_: Layout) -> ! {
panic!("allocation failed");
}
fn rt_error(layout: Layout) -> ! {
unsafe {
__rust_alloc_error_handler(layout.size(), layout.align());
}
}
unsafe { core::intrinsics::const_eval_select((layout,), ct_error, rt_error) }
}
// For alloc test `std::alloc::handle_alloc_error` can be used directly.
#[cfg(all(not(no_global_oom_handling), test))]
pub use std::alloc::handle_alloc_error;
#[cfg(all(not(no_global_oom_handling), not(test)))]
#[doc(hidden)]
#[allow(unused_attributes)]
#[unstable(feature = "alloc_internals", issue = "none")]
pub mod __alloc_error_handler {
use crate::alloc::Layout;
// called via generated `__rust_alloc_error_handler`
// if there is no `#[alloc_error_handler]`
#[rustc_std_internal_symbol]
pub unsafe extern "C-unwind" fn __rdl_oom(size: usize, _align: usize) -> ! {
panic!("memory allocation of {size} bytes failed")
}
// if there is an `#[alloc_error_handler]`
#[rustc_std_internal_symbol]
pub unsafe extern "C-unwind" fn __rg_oom(size: usize, align: usize) -> ! {
let layout = unsafe { Layout::from_size_align_unchecked(size, align) };
extern "Rust" {
#[lang = "oom"]
fn oom_impl(layout: Layout) -> !;
}
unsafe { oom_impl(layout) }
}
}
/// Specialize clones into pre-allocated, uninitialized memory.
/// Used by `Box::clone` and `Rc`/`Arc::make_mut`.
pub(crate) trait WriteCloneIntoRaw: Sized {
unsafe fn write_clone_into_raw(&self, target: *mut Self);
}
impl<T: Clone> WriteCloneIntoRaw for T {
#[inline]
default unsafe fn write_clone_into_raw(&self, target: *mut Self) {
// Having allocated *first* may allow the optimizer to create
// the cloned value in-place, skipping the local and move.
unsafe { target.write(self.clone()) };
}
}
impl<T: Copy> WriteCloneIntoRaw for T {
#[inline]
unsafe fn write_clone_into_raw(&self, target: *mut Self) {
// We can always copy in-place, without ever involving a local value.
unsafe { target.copy_from_nonoverlapping(self, 1) };
}
}

498
rust/alloc/borrow.rs Normal file
View File

@ -0,0 +1,498 @@
// SPDX-License-Identifier: Apache-2.0 OR MIT
//! A module for working with borrowed data.
#![stable(feature = "rust1", since = "1.0.0")]
use core::cmp::Ordering;
use core::hash::{Hash, Hasher};
use core::ops::Deref;
#[cfg(not(no_global_oom_handling))]
use core::ops::{Add, AddAssign};
#[stable(feature = "rust1", since = "1.0.0")]
pub use core::borrow::{Borrow, BorrowMut};
use core::fmt;
#[cfg(not(no_global_oom_handling))]
use crate::string::String;
use Cow::*;
#[stable(feature = "rust1", since = "1.0.0")]
impl<'a, B: ?Sized> Borrow<B> for Cow<'a, B>
where
B: ToOwned,
<B as ToOwned>::Owned: 'a,
{
fn borrow(&self) -> &B {
&**self
}
}
/// A generalization of `Clone` to borrowed data.
///
/// Some types make it possible to go from borrowed to owned, usually by
/// implementing the `Clone` trait. But `Clone` works only for going from `&T`
/// to `T`. The `ToOwned` trait generalizes `Clone` to construct owned data
/// from any borrow of a given type.
#[cfg_attr(not(test), rustc_diagnostic_item = "ToOwned")]
#[stable(feature = "rust1", since = "1.0.0")]
pub trait ToOwned {
/// The resulting type after obtaining ownership.
#[stable(feature = "rust1", since = "1.0.0")]
type Owned: Borrow<Self>;
/// Creates owned data from borrowed data, usually by cloning.
///
/// # Examples
///
/// Basic usage:
///
/// ```
/// let s: &str = "a";
/// let ss: String = s.to_owned();
///
/// let v: &[i32] = &[1, 2];
/// let vv: Vec<i32> = v.to_owned();
/// ```
#[stable(feature = "rust1", since = "1.0.0")]
#[must_use = "cloning is often expensive and is not expected to have side effects"]
fn to_owned(&self) -> Self::Owned;
/// Uses borrowed data to replace owned data, usually by cloning.
///
/// This is borrow-generalized version of `Clone::clone_from`.
///
/// # Examples
///
/// Basic usage:
///
/// ```
/// # #![feature(toowned_clone_into)]
/// let mut s: String = String::new();
/// "hello".clone_into(&mut s);
///
/// let mut v: Vec<i32> = Vec::new();
/// [1, 2][..].clone_into(&mut v);
/// ```
#[unstable(feature = "toowned_clone_into", reason = "recently added", issue = "41263")]
fn clone_into(&self, target: &mut Self::Owned) {
*target = self.to_owned();
}
}
#[stable(feature = "rust1", since = "1.0.0")]
impl<T> ToOwned for T
where
T: Clone,
{
type Owned = T;
fn to_owned(&self) -> T {
self.clone()
}
fn clone_into(&self, target: &mut T) {
target.clone_from(self);
}
}
/// A clone-on-write smart pointer.
///
/// The type `Cow` is a smart pointer providing clone-on-write functionality: it
/// can enclose and provide immutable access to borrowed data, and clone the
/// data lazily when mutation or ownership is required. The type is designed to
/// work with general borrowed data via the `Borrow` trait.
///
/// `Cow` implements `Deref`, which means that you can call
/// non-mutating methods directly on the data it encloses. If mutation
/// is desired, `to_mut` will obtain a mutable reference to an owned
/// value, cloning if necessary.
///
/// If you need reference-counting pointers, note that
/// [`Rc::make_mut`][crate::rc::Rc::make_mut] and
/// [`Arc::make_mut`][crate::sync::Arc::make_mut] can provide clone-on-write
/// functionality as well.
///
/// # Examples
///
/// ```
/// use std::borrow::Cow;
///
/// fn abs_all(input: &mut Cow<[i32]>) {
/// for i in 0..input.len() {
/// let v = input[i];
/// if v < 0 {
/// // Clones into a vector if not already owned.
/// input.to_mut()[i] = -v;
/// }
/// }
/// }
///
/// // No clone occurs because `input` doesn't need to be mutated.
/// let slice = [0, 1, 2];
/// let mut input = Cow::from(&slice[..]);
/// abs_all(&mut input);
///
/// // Clone occurs because `input` needs to be mutated.
/// let slice = [-1, 0, 1];
/// let mut input = Cow::from(&slice[..]);
/// abs_all(&mut input);
///
/// // No clone occurs because `input` is already owned.
/// let mut input = Cow::from(vec![-1, 0, 1]);
/// abs_all(&mut input);
/// ```
///
/// Another example showing how to keep `Cow` in a struct:
///
/// ```
/// use std::borrow::Cow;
///
/// struct Items<'a, X: 'a> where [X]: ToOwned<Owned = Vec<X>> {
/// values: Cow<'a, [X]>,
/// }
///
/// impl<'a, X: Clone + 'a> Items<'a, X> where [X]: ToOwned<Owned = Vec<X>> {
/// fn new(v: Cow<'a, [X]>) -> Self {
/// Items { values: v }
/// }
/// }
///
/// // Creates a container from borrowed values of a slice
/// let readonly = [1, 2];
/// let borrowed = Items::new((&readonly[..]).into());
/// match borrowed {
/// Items { values: Cow::Borrowed(b) } => println!("borrowed {b:?}"),
/// _ => panic!("expect borrowed value"),
/// }
///
/// let mut clone_on_write = borrowed;
/// // Mutates the data from slice into owned vec and pushes a new value on top
/// clone_on_write.values.to_mut().push(3);
/// println!("clone_on_write = {:?}", clone_on_write.values);
///
/// // The data was mutated. Let's check it out.
/// match clone_on_write {
/// Items { values: Cow::Owned(_) } => println!("clone_on_write contains owned data"),
/// _ => panic!("expect owned data"),
/// }
/// ```
#[stable(feature = "rust1", since = "1.0.0")]
#[cfg_attr(not(test), rustc_diagnostic_item = "Cow")]
pub enum Cow<'a, B: ?Sized + 'a>
where
B: ToOwned,
{
/// Borrowed data.
#[stable(feature = "rust1", since = "1.0.0")]
Borrowed(#[stable(feature = "rust1", since = "1.0.0")] &'a B),
/// Owned data.
#[stable(feature = "rust1", since = "1.0.0")]
Owned(#[stable(feature = "rust1", since = "1.0.0")] <B as ToOwned>::Owned),
}
#[stable(feature = "rust1", since = "1.0.0")]
impl<B: ?Sized + ToOwned> Clone for Cow<'_, B> {
fn clone(&self) -> Self {
match *self {
Borrowed(b) => Borrowed(b),
Owned(ref o) => {
let b: &B = o.borrow();
Owned(b.to_owned())
}
}
}
fn clone_from(&mut self, source: &Self) {
match (self, source) {
(&mut Owned(ref mut dest), &Owned(ref o)) => o.borrow().clone_into(dest),
(t, s) => *t = s.clone(),
}
}
}
impl<B: ?Sized + ToOwned> Cow<'_, B> {
/// Returns true if the data is borrowed, i.e. if `to_mut` would require additional work.
///
/// # Examples
///
/// ```
/// #![feature(cow_is_borrowed)]
/// use std::borrow::Cow;
///
/// let cow = Cow::Borrowed("moo");
/// assert!(cow.is_borrowed());
///
/// let bull: Cow<'_, str> = Cow::Owned("...moo?".to_string());
/// assert!(!bull.is_borrowed());
/// ```
#[unstable(feature = "cow_is_borrowed", issue = "65143")]
#[rustc_const_unstable(feature = "const_cow_is_borrowed", issue = "65143")]
pub const fn is_borrowed(&self) -> bool {
match *self {
Borrowed(_) => true,
Owned(_) => false,
}
}
/// Returns true if the data is owned, i.e. if `to_mut` would be a no-op.
///
/// # Examples
///
/// ```
/// #![feature(cow_is_borrowed)]
/// use std::borrow::Cow;
///
/// let cow: Cow<'_, str> = Cow::Owned("moo".to_string());
/// assert!(cow.is_owned());
///
/// let bull = Cow::Borrowed("...moo?");
/// assert!(!bull.is_owned());
/// ```
#[unstable(feature = "cow_is_borrowed", issue = "65143")]
#[rustc_const_unstable(feature = "const_cow_is_borrowed", issue = "65143")]
pub const fn is_owned(&self) -> bool {
!self.is_borrowed()
}
/// Acquires a mutable reference to the owned form of the data.
///
/// Clones the data if it is not already owned.
///
/// # Examples
///
/// ```
/// use std::borrow::Cow;
///
/// let mut cow = Cow::Borrowed("foo");
/// cow.to_mut().make_ascii_uppercase();
///
/// assert_eq!(
/// cow,
/// Cow::Owned(String::from("FOO")) as Cow<str>
/// );
/// ```
#[stable(feature = "rust1", since = "1.0.0")]
pub fn to_mut(&mut self) -> &mut <B as ToOwned>::Owned {
match *self {
Borrowed(borrowed) => {
*self = Owned(borrowed.to_owned());
match *self {
Borrowed(..) => unreachable!(),
Owned(ref mut owned) => owned,
}
}
Owned(ref mut owned) => owned,
}
}
/// Extracts the owned data.
///
/// Clones the data if it is not already owned.
///
/// # Examples
///
/// Calling `into_owned` on a `Cow::Borrowed` returns a clone of the borrowed data:
///
/// ```
/// use std::borrow::Cow;
///
/// let s = "Hello world!";
/// let cow = Cow::Borrowed(s);
///
/// assert_eq!(
/// cow.into_owned(),
/// String::from(s)
/// );
/// ```
///
/// Calling `into_owned` on a `Cow::Owned` returns the owned data. The data is moved out of the
/// `Cow` without being cloned.
///
/// ```
/// use std::borrow::Cow;
///
/// let s = "Hello world!";
/// let cow: Cow<str> = Cow::Owned(String::from(s));
///
/// assert_eq!(
/// cow.into_owned(),
/// String::from(s)
/// );
/// ```
#[stable(feature = "rust1", since = "1.0.0")]
pub fn into_owned(self) -> <B as ToOwned>::Owned {
match self {
Borrowed(borrowed) => borrowed.to_owned(),
Owned(owned) => owned,
}
}
}
#[stable(feature = "rust1", since = "1.0.0")]
#[rustc_const_unstable(feature = "const_deref", issue = "88955")]
impl<B: ?Sized + ToOwned> const Deref for Cow<'_, B>
where
B::Owned: ~const Borrow<B>,
{
type Target = B;
fn deref(&self) -> &B {
match *self {
Borrowed(borrowed) => borrowed,
Owned(ref owned) => owned.borrow(),
}
}
}
#[stable(feature = "rust1", since = "1.0.0")]
impl<B: ?Sized> Eq for Cow<'_, B> where B: Eq + ToOwned {}
#[stable(feature = "rust1", since = "1.0.0")]
impl<B: ?Sized> Ord for Cow<'_, B>
where
B: Ord + ToOwned,
{
#[inline]
fn cmp(&self, other: &Self) -> Ordering {
Ord::cmp(&**self, &**other)
}
}
#[stable(feature = "rust1", since = "1.0.0")]
impl<'a, 'b, B: ?Sized, C: ?Sized> PartialEq<Cow<'b, C>> for Cow<'a, B>
where
B: PartialEq<C> + ToOwned,
C: ToOwned,
{
#[inline]
fn eq(&self, other: &Cow<'b, C>) -> bool {
PartialEq::eq(&**self, &**other)
}
}
#[stable(feature = "rust1", since = "1.0.0")]
impl<'a, B: ?Sized> PartialOrd for Cow<'a, B>
where
B: PartialOrd + ToOwned,
{
#[inline]
fn partial_cmp(&self, other: &Cow<'a, B>) -> Option<Ordering> {
PartialOrd::partial_cmp(&**self, &**other)
}
}
#[stable(feature = "rust1", since = "1.0.0")]
impl<B: ?Sized> fmt::Debug for Cow<'_, B>
where
B: fmt::Debug + ToOwned<Owned: fmt::Debug>,
{
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match *self {
Borrowed(ref b) => fmt::Debug::fmt(b, f),
Owned(ref o) => fmt::Debug::fmt(o, f),
}
}
}
#[stable(feature = "rust1", since = "1.0.0")]
impl<B: ?Sized> fmt::Display for Cow<'_, B>
where
B: fmt::Display + ToOwned<Owned: fmt::Display>,
{
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match *self {
Borrowed(ref b) => fmt::Display::fmt(b, f),
Owned(ref o) => fmt::Display::fmt(o, f),
}
}
}
#[stable(feature = "default", since = "1.11.0")]
impl<B: ?Sized> Default for Cow<'_, B>
where
B: ToOwned<Owned: Default>,
{
/// Creates an owned Cow<'a, B> with the default value for the contained owned value.
fn default() -> Self {
Owned(<B as ToOwned>::Owned::default())
}
}
#[stable(feature = "rust1", since = "1.0.0")]
impl<B: ?Sized> Hash for Cow<'_, B>
where
B: Hash + ToOwned,
{
#[inline]
fn hash<H: Hasher>(&self, state: &mut H) {
Hash::hash(&**self, state)
}
}
#[stable(feature = "rust1", since = "1.0.0")]
impl<T: ?Sized + ToOwned> AsRef<T> for Cow<'_, T> {
fn as_ref(&self) -> &T {
self
}
}
#[cfg(not(no_global_oom_handling))]
#[stable(feature = "cow_add", since = "1.14.0")]
impl<'a> Add<&'a str> for Cow<'a, str> {
type Output = Cow<'a, str>;
#[inline]
fn add(mut self, rhs: &'a str) -> Self::Output {
self += rhs;
self
}
}
#[cfg(not(no_global_oom_handling))]
#[stable(feature = "cow_add", since = "1.14.0")]
impl<'a> Add<Cow<'a, str>> for Cow<'a, str> {
type Output = Cow<'a, str>;
#[inline]
fn add(mut self, rhs: Cow<'a, str>) -> Self::Output {
self += rhs;
self
}
}
#[cfg(not(no_global_oom_handling))]
#[stable(feature = "cow_add", since = "1.14.0")]
impl<'a> AddAssign<&'a str> for Cow<'a, str> {
fn add_assign(&mut self, rhs: &'a str) {
if self.is_empty() {
*self = Cow::Borrowed(rhs)
} else if !rhs.is_empty() {
if let Cow::Borrowed(lhs) = *self {
let mut s = String::with_capacity(lhs.len() + rhs.len());
s.push_str(lhs);
*self = Cow::Owned(s);
}
self.to_mut().push_str(rhs);
}
}
}
#[cfg(not(no_global_oom_handling))]
#[stable(feature = "cow_add", since = "1.14.0")]
impl<'a> AddAssign<Cow<'a, str>> for Cow<'a, str> {
fn add_assign(&mut self, rhs: Cow<'a, str>) {
if self.is_empty() {
*self = rhs
} else if !rhs.is_empty() {
if let Cow::Borrowed(lhs) = *self {
let mut s = String::with_capacity(lhs.len() + rhs.len());
s.push_str(lhs);
*self = Cow::Owned(s);
}
self.to_mut().push_str(&rhs);
}
}
}

2028
rust/alloc/boxed.rs Normal file
View File

File diff suppressed because it is too large Load Diff

156
rust/alloc/collections/mod.rs Normal file
View File

@ -0,0 +1,156 @@
// SPDX-License-Identifier: Apache-2.0 OR MIT
//! Collection types.
#![stable(feature = "rust1", since = "1.0.0")]
#[cfg(not(no_global_oom_handling))]
pub mod binary_heap;
#[cfg(not(no_global_oom_handling))]
mod btree;
#[cfg(not(no_global_oom_handling))]
pub mod linked_list;
#[cfg(not(no_global_oom_handling))]
pub mod vec_deque;
#[cfg(not(no_global_oom_handling))]
#[stable(feature = "rust1", since = "1.0.0")]
pub mod btree_map {
//! An ordered map based on a B-Tree.
#[stable(feature = "rust1", since = "1.0.0")]
pub use super::btree::map::*;
}
#[cfg(not(no_global_oom_handling))]
#[stable(feature = "rust1", since = "1.0.0")]
pub mod btree_set {
//! An ordered set based on a B-Tree.
#[stable(feature = "rust1", since = "1.0.0")]
pub use super::btree::set::*;
}
#[cfg(not(no_global_oom_handling))]
#[stable(feature = "rust1", since = "1.0.0")]
#[doc(no_inline)]
pub use binary_heap::BinaryHeap;
#[cfg(not(no_global_oom_handling))]
#[stable(feature = "rust1", since = "1.0.0")]
#[doc(no_inline)]
pub use btree_map::BTreeMap;
#[cfg(not(no_global_oom_handling))]
#[stable(feature = "rust1", since = "1.0.0")]
#[doc(no_inline)]
pub use btree_set::BTreeSet;
#[cfg(not(no_global_oom_handling))]
#[stable(feature = "rust1", since = "1.0.0")]
#[doc(no_inline)]
pub use linked_list::LinkedList;
#[cfg(not(no_global_oom_handling))]
#[stable(feature = "rust1", since = "1.0.0")]
#[doc(no_inline)]
pub use vec_deque::VecDeque;
use crate::alloc::{Layout, LayoutError};
use core::fmt::Display;
/// The error type for `try_reserve` methods.
#[derive(Clone, PartialEq, Eq, Debug)]
#[stable(feature = "try_reserve", since = "1.57.0")]
pub struct TryReserveError {
kind: TryReserveErrorKind,
}
impl TryReserveError {
/// Details about the allocation that caused the error
#[inline]
#[must_use]
#[unstable(
feature = "try_reserve_kind",
reason = "Uncertain how much info should be exposed",
issue = "48043"
)]
pub fn kind(&self) -> TryReserveErrorKind {
self.kind.clone()
}
}
/// Details of the allocation that caused a `TryReserveError`
#[derive(Clone, PartialEq, Eq, Debug)]
#[unstable(
feature = "try_reserve_kind",
reason = "Uncertain how much info should be exposed",
issue = "48043"
)]
pub enum TryReserveErrorKind {
/// Error due to the computed capacity exceeding the collection's maximum
/// (usually `isize::MAX` bytes).
CapacityOverflow,
/// The memory allocator returned an error
AllocError {
/// The layout of allocation request that failed
layout: Layout,
#[doc(hidden)]
#[unstable(
feature = "container_error_extra",
issue = "none",
reason = "\
Enable exposing the allocator’s custom error value \
if an associated type is added in the future: \
https://github.com/rust-lang/wg-allocators/issues/23"
)]
non_exhaustive: (),
},
}
#[unstable(
feature = "try_reserve_kind",
reason = "Uncertain how much info should be exposed",
issue = "48043"
)]
impl From<TryReserveErrorKind> for TryReserveError {
#[inline]
fn from(kind: TryReserveErrorKind) -> Self {
Self { kind }
}
}
#[unstable(feature = "try_reserve_kind", reason = "new API", issue = "48043")]
impl From<LayoutError> for TryReserveErrorKind {
/// Always evaluates to [`TryReserveErrorKind::CapacityOverflow`].
#[inline]
fn from(_: LayoutError) -> Self {
TryReserveErrorKind::CapacityOverflow
}
}
#[stable(feature = "try_reserve", since = "1.57.0")]
impl Display for TryReserveError {
fn fmt(
&self,
fmt: &mut core::fmt::Formatter<'_>,
) -> core::result::Result<(), core::fmt::Error> {
fmt.write_str("memory allocation failed")?;
let reason = match self.kind {
TryReserveErrorKind::CapacityOverflow => {
" because the computed capacity exceeded the collection's maximum"
}
TryReserveErrorKind::AllocError { .. } => {
" because the memory allocator returned a error"
}
};
fmt.write_str(reason)
}
}
/// An intermediate trait for specialization of `Extend`.
#[doc(hidden)]
trait SpecExtend<I: IntoIterator> {
/// Extends `self` with the contents of the given iterator.
fn spec_extend(&mut self, iter: I);
}

244
rust/alloc/lib.rs Normal file
View File

@ -0,0 +1,244 @@
// SPDX-License-Identifier: Apache-2.0 OR MIT
//! # The Rust core allocation and collections library
//!
//! This library provides smart pointers and collections for managing
//! heap-allocated values.
//!
//! This library, like libcore, normally doesn’t need to be used directly
//! since its contents are re-exported in the [`std` crate](../std/index.html).
//! Crates that use the `#![no_std]` attribute however will typically
//! not depend on `std`, so they’d use this crate instead.
//!
//! ## Boxed values
//!
//! The [`Box`] type is a smart pointer type. There can only be one owner of a
//! [`Box`], and the owner can decide to mutate the contents, which live on the
//! heap.
//!
//! This type can be sent among threads efficiently as the size of a `Box` value
//! is the same as that of a pointer. Tree-like data structures are often built
//! with boxes because each node often has only one owner, the parent.
//!
//! ## Reference counted pointers
//!
//! The [`Rc`] type is a non-threadsafe reference-counted pointer type intended
//! for sharing memory within a thread. An [`Rc`] pointer wraps a type, `T`, and
//! only allows access to `&T`, a shared reference.
//!
//! This type is useful when inherited mutability (such as using [`Box`]) is too
//! constraining for an application, and is often paired with the [`Cell`] or
//! [`RefCell`] types in order to allow mutation.
//!
//! ## Atomically reference counted pointers
//!
//! The [`Arc`] type is the threadsafe equivalent of the [`Rc`] type. It
//! provides all the same functionality of [`Rc`], except it requires that the
//! contained type `T` is shareable. Additionally, [`Arc<T>`][`Arc`] is itself
//! sendable while [`Rc<T>`][`Rc`] is not.
//!
//! This type allows for shared access to the contained data, and is often
//! paired with synchronization primitives such as mutexes to allow mutation of
//! shared resources.
//!
//! ## Collections
//!
//! Implementations of the most common general purpose data structures are
//! defined in this library. They are re-exported through the
//! [standard collections library](../std/collections/index.html).
//!
//! ## Heap interfaces
//!
//! The [`alloc`](alloc/index.html) module defines the low-level interface to the
//! default global allocator. It is not compatible with the libc allocator API.
//!
//! [`Arc`]: sync
//! [`Box`]: boxed
//! [`Cell`]: core::cell
//! [`Rc`]: rc
//! [`RefCell`]: core::cell
// To run liballoc tests without x.py without ending up with two copies of liballoc, Miri needs to be
// able to "empty" this crate. See <https://github.com/rust-lang/miri-test-libstd/issues/4>.
// rustc itself never sets the feature, so this line has no affect there.
#![cfg(any(not(feature = "miri-test-libstd"), test, doctest))]
#![allow(unused_attributes)]
#![stable(feature = "alloc", since = "1.36.0")]
#![doc(
html_playground_url = "https://play.rust-lang.org/",
issue_tracker_base_url = "https://github.com/rust-lang/rust/issues/",
test(no_crate_inject, attr(allow(unused_variables), deny(warnings)))
)]
#![doc(cfg_hide(
not(test),
not(any(test, bootstrap)),
any(not(feature = "miri-test-libstd"), test, doctest),
no_global_oom_handling,
not(no_global_oom_handling),
target_has_atomic = "ptr"
))]
#![no_std]
#![needs_allocator]
//
// Lints:
#![deny(unsafe_op_in_unsafe_fn)]
#![warn(deprecated_in_future)]
#![warn(missing_debug_implementations)]
#![warn(missing_docs)]
#![allow(explicit_outlives_requirements)]
//
// Library features:
#![cfg_attr(not(no_global_oom_handling), feature(alloc_c_string))]
#![feature(alloc_layout_extra)]
#![feature(allocator_api)]
#![feature(array_chunks)]
#![feature(array_methods)]
#![feature(array_windows)]
#![feature(assert_matches)]
#![feature(async_iterator)]
#![feature(coerce_unsized)]
#![cfg_attr(not(no_global_oom_handling), feature(const_alloc_error))]
#![feature(const_box)]
#![cfg_attr(not(no_global_oom_handling), feature(const_btree_new))]
#![feature(const_cow_is_borrowed)]
#![feature(const_convert)]
#![feature(const_size_of_val)]
#![feature(const_align_of_val)]
#![feature(const_ptr_read)]
#![feature(const_maybe_uninit_write)]
#![feature(const_maybe_uninit_as_mut_ptr)]
#![feature(const_refs_to_cell)]
#![feature(core_c_str)]
#![feature(core_intrinsics)]
#![feature(core_ffi_c)]
#![feature(const_eval_select)]
#![feature(const_pin)]
#![feature(cstr_from_bytes_until_nul)]
#![feature(dispatch_from_dyn)]
#![feature(exact_size_is_empty)]
#![feature(extend_one)]
#![feature(fmt_internals)]
#![feature(fn_traits)]
#![feature(hasher_prefixfree_extras)]
#![feature(inplace_iteration)]
#![feature(iter_advance_by)]
#![feature(layout_for_ptr)]
#![feature(maybe_uninit_slice)]
#![cfg_attr(test, feature(new_uninit))]
#![feature(nonnull_slice_from_raw_parts)]
#![feature(pattern)]
#![feature(ptr_internals)]
#![feature(ptr_metadata)]
#![feature(ptr_sub_ptr)]
#![feature(receiver_trait)]
#![feature(set_ptr_value)]
#![feature(slice_group_by)]
#![feature(slice_ptr_get)]
#![feature(slice_ptr_len)]
#![feature(slice_range)]
#![feature(str_internals)]
#![feature(strict_provenance)]
#![feature(trusted_len)]
#![feature(trusted_random_access)]
#![feature(try_trait_v2)]
#![feature(unchecked_math)]
#![feature(unicode_internals)]
#![feature(unsize)]
//
// Language features:
#![feature(allocator_internals)]
#![feature(allow_internal_unstable)]
#![feature(associated_type_bounds)]
#![feature(box_syntax)]
#![feature(cfg_sanitize)]
#![feature(const_deref)]
#![feature(const_mut_refs)]
#![feature(const_ptr_write)]
#![feature(const_precise_live_drops)]
#![feature(const_trait_impl)]
#![feature(const_try)]
#![feature(dropck_eyepatch)]
#![feature(exclusive_range_pattern)]
#![feature(fundamental)]
#![cfg_attr(not(test), feature(generator_trait))]
#![feature(hashmap_internals)]
#![feature(lang_items)]
#![feature(let_else)]
#![feature(min_specialization)]
#![feature(negative_impls)]
#![feature(never_type)]
#![feature(nll)] // Not necessary, but here to test the `nll` feature.
#![feature(rustc_allow_const_fn_unstable)]
#![feature(rustc_attrs)]
#![feature(slice_internals)]
#![feature(staged_api)]
#![cfg_attr(test, feature(test))]
#![feature(unboxed_closures)]
#![feature(unsized_fn_params)]
#![feature(c_unwind)]
//
// Rustdoc features:
#![feature(doc_cfg)]
#![feature(doc_cfg_hide)]
// Technically, this is a bug in rustdoc: rustdoc sees the documentation on `#[lang = slice_alloc]`
// blocks is for `&[T]`, which also has documentation using this feature in `core`, and gets mad
// that the feature-gate isn't enabled. Ideally, it wouldn't check for the feature gate for docs
// from other crates, but since this can only appear for lang items, it doesn't seem worth fixing.
#![feature(intra_doc_pointers)]
// Allow testing this library
#[cfg(test)]
#[macro_use]
extern crate std;
#[cfg(test)]
extern crate test;
// Module with internal macros used by other modules (needs to be included before other modules).
#[cfg(not(no_macros))]
#[macro_use]
mod macros;
mod raw_vec;
// Heaps provided for low-level allocation strategies
pub mod alloc;
// Primitive types using the heaps above
// Need to conditionally define the mod from `boxed.rs` to avoid
// duplicating the lang-items when building in test cfg; but also need
// to allow code to have `use boxed::Box;` declarations.
#[cfg(not(test))]
pub mod boxed;
#[cfg(test)]
mod boxed {
pub use std::boxed::Box;
}
pub mod borrow;
pub mod collections;
#[cfg(not(no_global_oom_handling))]
pub mod ffi;
#[cfg(not(no_fmt))]
pub mod fmt;
#[cfg(not(no_rc))]
pub mod rc;
pub mod slice;
#[cfg(not(no_str))]
pub mod str;
#[cfg(not(no_string))]
pub mod string;
#[cfg(not(no_sync))]
#[cfg(target_has_atomic = "ptr")]
pub mod sync;
#[cfg(all(not(no_global_oom_handling), target_has_atomic = "ptr"))]
pub mod task;
#[cfg(test)]
mod tests;
pub mod vec;
#[doc(hidden)]
#[unstable(feature = "liballoc_internals", issue = "none", reason = "implementation detail")]
pub mod __export {
pub use core::format_args;
}

527
rust/alloc/raw_vec.rs Normal file
View File

@ -0,0 +1,527 @@
// SPDX-License-Identifier: Apache-2.0 OR MIT
#![unstable(feature = "raw_vec_internals", reason = "unstable const warnings", issue = "none")]
use core::alloc::LayoutError;
use core::cmp;
use core::intrinsics;
use core::mem::{self, ManuallyDrop, MaybeUninit};
use core::ops::Drop;
use core::ptr::{self, NonNull, Unique};
use core::slice;
#[cfg(not(no_global_oom_handling))]
use crate::alloc::handle_alloc_error;
use crate::alloc::{Allocator, Global, Layout};
use crate::boxed::Box;
use crate::collections::TryReserveError;
use crate::collections::TryReserveErrorKind::*;
#[cfg(test)]
mod tests;
#[cfg(not(no_global_oom_handling))]
enum AllocInit {
/// The contents of the new memory are uninitialized.
Uninitialized,
/// The new memory is guaranteed to be zeroed.
Zeroed,
}
/// A low-level utility for more ergonomically allocating, reallocating, and deallocating
/// a buffer of memory on the heap without having to worry about all the corner cases
/// involved. This type is excellent for building your own data structures like Vec and VecDeque.
/// In particular:
///
/// * Produces `Unique::dangling()` on zero-sized types.
/// * Produces `Unique::dangling()` on zero-length allocations.
/// * Avoids freeing `Unique::dangling()`.
/// * Catches all overflows in capacity computations (promotes them to "capacity overflow" panics).
/// * Guards against 32-bit systems allocating more than isize::MAX bytes.
/// * Guards against overflowing your length.
/// * Calls `handle_alloc_error` for fallible allocations.
/// * Contains a `ptr::Unique` and thus endows the user with all related benefits.
/// * Uses the excess returned from the allocator to use the largest available capacity.
///
/// This type does not in anyway inspect the memory that it manages. When dropped it *will*
/// free its memory, but it *won't* try to drop its contents. It is up to the user of `RawVec`
/// to handle the actual things *stored* inside of a `RawVec`.
///
/// Note that the excess of a zero-sized types is always infinite, so `capacity()` always returns
/// `usize::MAX`. This means that you need to be careful when round-tripping this type with a
/// `Box<[T]>`, since `capacity()` won't yield the length.
#[allow(missing_debug_implementations)]
pub(crate) struct RawVec<T, A: Allocator = Global> {
ptr: Unique<T>,
cap: usize,
alloc: A,
}
impl<T> RawVec<T, Global> {
/// HACK(Centril): This exists because stable `const fn` can only call stable `const fn`, so
/// they cannot call `Self::new()`.
///
/// If you change `RawVec<T>::new` or dependencies, please take care to not introduce anything
/// that would truly const-call something unstable.
pub const NEW: Self = Self::new();
/// Creates the biggest possible `RawVec` (on the system heap)
/// without allocating. If `T` has positive size, then this makes a
/// `RawVec` with capacity `0`. If `T` is zero-sized, then it makes a
/// `RawVec` with capacity `usize::MAX`. Useful for implementing
/// delayed allocation.
#[must_use]
pub const fn new() -> Self {
Self::new_in(Global)
}
/// Creates a `RawVec` (on the system heap) with exactly the
/// capacity and alignment requirements for a `[T; capacity]`. This is
/// equivalent to calling `RawVec::new` when `capacity` is `0` or `T` is
/// zero-sized. Note that if `T` is zero-sized this means you will
/// *not* get a `RawVec` with the requested capacity.
///
/// # Panics
///
/// Panics if the requested capacity exceeds `isize::MAX` bytes.
///
/// # Aborts
///
/// Aborts on OOM.
#[cfg(not(any(no_global_oom_handling, test)))]
#[must_use]
#[inline]
pub fn with_capacity(capacity: usize) -> Self {
Self::with_capacity_in(capacity, Global)
}
/// Like `with_capacity`, but guarantees the buffer is zeroed.
#[cfg(not(any(no_global_oom_handling, test)))]
#[must_use]
#[inline]
pub fn with_capacity_zeroed(capacity: usize) -> Self {
Self::with_capacity_zeroed_in(capacity, Global)
}
}
impl<T, A: Allocator> RawVec<T, A> {
// Tiny Vecs are dumb. Skip to:
// - 8 if the element size is 1, because any heap allocators is likely
// to round up a request of less than 8 bytes to at least 8 bytes.
// - 4 if elements are moderate-sized (<= 1 KiB).
// - 1 otherwise, to avoid wasting too much space for very short Vecs.
pub(crate) const MIN_NON_ZERO_CAP: usize = if mem::size_of::<T>() == 1 {
8
} else if mem::size_of::<T>() <= 1024 {
4
} else {
1
};
/// Like `new`, but parameterized over the choice of allocator for
/// the returned `RawVec`.
pub const fn new_in(alloc: A) -> Self {
// `cap: 0` means "unallocated". zero-sized types are ignored.
Self { ptr: Unique::dangling(), cap: 0, alloc }
}
/// Like `with_capacity`, but parameterized over the choice of
/// allocator for the returned `RawVec`.
#[cfg(not(no_global_oom_handling))]
#[inline]
pub fn with_capacity_in(capacity: usize, alloc: A) -> Self {
Self::allocate_in(capacity, AllocInit::Uninitialized, alloc)
}
/// Like `with_capacity_zeroed`, but parameterized over the choice
/// of allocator for the returned `RawVec`.
#[cfg(not(no_global_oom_handling))]
#[inline]
pub fn with_capacity_zeroed_in(capacity: usize, alloc: A) -> Self {
Self::allocate_in(capacity, AllocInit::Zeroed, alloc)
}
/// Converts the entire buffer into `Box<[MaybeUninit<T>]>` with the specified `len`.
///
/// Note that this will correctly reconstitute any `cap` changes
/// that may have been performed. (See description of type for details.)
///
/// # Safety
///
/// * `len` must be greater than or equal to the most recently requested capacity, and
/// * `len` must be less than or equal to `self.capacity()`.
///
/// Note, that the requested capacity and `self.capacity()` could differ, as
/// an allocator could overallocate and return a greater memory block than requested.
pub unsafe fn into_box(self, len: usize) -> Box<[MaybeUninit<T>], A> {
// Sanity-check one half of the safety requirement (we cannot check the other half).
debug_assert!(
len <= self.capacity(),
"`len` must be smaller than or equal to `self.capacity()`"
);
let me = ManuallyDrop::new(self);
unsafe {
let slice = slice::from_raw_parts_mut(me.ptr() as *mut MaybeUninit<T>, len);
Box::from_raw_in(slice, ptr::read(&me.alloc))
}
}
#[cfg(not(no_global_oom_handling))]
fn allocate_in(capacity: usize, init: AllocInit, alloc: A) -> Self {
// Don't allocate here because `Drop` will not deallocate when `capacity` is 0.
if mem::size_of::<T>() == 0 || capacity == 0 {
Self::new_in(alloc)
} else {
// We avoid `unwrap_or_else` here because it bloats the amount of
// LLVM IR generated.
let layout = match Layout::array::<T>(capacity) {
Ok(layout) => layout,
Err(_) => capacity_overflow(),
};
match alloc_guard(layout.size()) {
Ok(_) => {}
Err(_) => capacity_overflow(),
}
let result = match init {
AllocInit::Uninitialized => alloc.allocate(layout),
AllocInit::Zeroed => alloc.allocate_zeroed(layout),
};
let ptr = match result {
Ok(ptr) => ptr,
Err(_) => handle_alloc_error(layout),
};
// Allocators currently return a `NonNull<[u8]>` whose length
// matches the size requested. If that ever changes, the capacity
// here should change to `ptr.len() / mem::size_of::<T>()`.
Self {
ptr: unsafe { Unique::new_unchecked(ptr.cast().as_ptr()) },
cap: capacity,
alloc,
}
}
}
/// Reconstitutes a `RawVec` from a pointer, capacity, and allocator.
///
/// # Safety
///
/// The `ptr` must be allocated (via the given allocator `alloc`), and with the given
/// `capacity`.
/// The `capacity` cannot exceed `isize::MAX` for sized types. (only a concern on 32-bit
/// systems). ZST vectors may have a capacity up to `usize::MAX`.
/// If the `ptr` and `capacity` come from a `RawVec` created via `alloc`, then this is
/// guaranteed.
#[inline]
pub unsafe fn from_raw_parts_in(ptr: *mut T, capacity: usize, alloc: A) -> Self {
Self { ptr: unsafe { Unique::new_unchecked(ptr) }, cap: capacity, alloc }
}
/// Gets a raw pointer to the start of the allocation. Note that this is
/// `Unique::dangling()` if `capacity == 0` or `T` is zero-sized. In the former case, you must
/// be careful.
#[inline]
pub fn ptr(&self) -> *mut T {
self.ptr.as_ptr()
}
/// Gets the capacity of the allocation.
///
/// This will always be `usize::MAX` if `T` is zero-sized.
#[inline(always)]
pub fn capacity(&self) -> usize {
if mem::size_of::<T>() == 0 { usize::MAX } else { self.cap }
}
/// Returns a shared reference to the allocator backing this `RawVec`.
pub fn allocator(&self) -> &A {
&self.alloc
}
fn current_memory(&self) -> Option<(NonNull<u8>, Layout)> {
if mem::size_of::<T>() == 0 || self.cap == 0 {
None
} else {
// We have an allocated chunk of memory, so we can bypass runtime
// checks to get our current layout.
unsafe {
let layout = Layout::array::<T>(self.cap).unwrap_unchecked();
Some((self.ptr.cast().into(), layout))
}
}
}
/// Ensures that the buffer contains at least enough space to hold `len +
/// additional` elements. If it doesn't already have enough capacity, will
/// reallocate enough space plus comfortable slack space to get amortized
/// *O*(1) behavior. Will limit this behavior if it would needlessly cause
/// itself to panic.
///
/// If `len` exceeds `self.capacity()`, this may fail to actually allocate
/// the requested space. This is not really unsafe, but the unsafe
/// code *you* write that relies on the behavior of this function may break.
///
/// This is ideal for implementing a bulk-push operation like `extend`.
///
/// # Panics
///
/// Panics if the new capacity exceeds `isize::MAX` bytes.
///
/// # Aborts
///
/// Aborts on OOM.
#[cfg(not(no_global_oom_handling))]
#[inline]
pub fn reserve(&mut self, len: usize, additional: usize) {
// Callers expect this function to be very cheap when there is already sufficient capacity.
// Therefore, we move all the resizing and error-handling logic from grow_amortized and
// handle_reserve behind a call, while making sure that this function is likely to be
// inlined as just a comparison and a call if the comparison fails.
#[cold]
fn do_reserve_and_handle<T, A: Allocator>(
slf: &mut RawVec<T, A>,
len: usize,
additional: usize,
) {
handle_reserve(slf.grow_amortized(len, additional));
}
if self.needs_to_grow(len, additional) {
do_reserve_and_handle(self, len, additional);
}
}
/// A specialized version of `reserve()` used only by the hot and
/// oft-instantiated `Vec::push()`, which does its own capacity check.
#[cfg(not(no_global_oom_handling))]
#[inline(never)]
pub fn reserve_for_push(&mut self, len: usize) {
handle_reserve(self.grow_amortized(len, 1));
}
/// The same as `reserve`, but returns on errors instead of panicking or aborting.
pub fn try_reserve(&mut self, len: usize, additional: usize) -> Result<(), TryReserveError> {
if self.needs_to_grow(len, additional) {
self.grow_amortized(len, additional)
} else {
Ok(())
}
}
/// The same as `reserve_for_push`, but returns on errors instead of panicking or aborting.
#[inline(never)]
pub fn try_reserve_for_push(&mut self, len: usize) -> Result<(), TryReserveError> {
self.grow_amortized(len, 1)
}
/// Ensures that the buffer contains at least enough space to hold `len +
/// additional` elements. If it doesn't already, will reallocate the
/// minimum possible amount of memory necessary. Generally this will be
/// exactly the amount of memory necessary, but in principle the allocator
/// is free to give back more than we asked for.
///
/// If `len` exceeds `self.capacity()`, this may fail to actually allocate
/// the requested space. This is not really unsafe, but the unsafe code
/// *you* write that relies on the behavior of this function may break.
///
/// # Panics
///
/// Panics if the new capacity exceeds `isize::MAX` bytes.
///
/// # Aborts
///
/// Aborts on OOM.
#[cfg(not(no_global_oom_handling))]
pub fn reserve_exact(&mut self, len: usize, additional: usize) {
handle_reserve(self.try_reserve_exact(len, additional));
}
/// The same as `reserve_exact`, but returns on errors instead of panicking or aborting.
pub fn try_reserve_exact(
&mut self,
len: usize,
additional: usize,
) -> Result<(), TryReserveError> {
if self.needs_to_grow(len, additional) { self.grow_exact(len, additional) } else { Ok(()) }
}
/// Shrinks the buffer down to the specified capacity. If the given amount
/// is 0, actually completely deallocates.
///
/// # Panics
///
/// Panics if the given amount is *larger* than the current capacity.
///
/// # Aborts
///
/// Aborts on OOM.
#[cfg(not(no_global_oom_handling))]
pub fn shrink_to_fit(&mut self, cap: usize) {
handle_reserve(self.shrink(cap));
}
}
impl<T, A: Allocator> RawVec<T, A> {
/// Returns if the buffer needs to grow to fulfill the needed extra capacity.
/// Mainly used to make inlining reserve-calls possible without inlining `grow`.
fn needs_to_grow(&self, len: usize, additional: usize) -> bool {
additional > self.capacity().wrapping_sub(len)
}
fn set_ptr_and_cap(&mut self, ptr: NonNull<[u8]>, cap: usize) {
// Allocators currently return a `NonNull<[u8]>` whose length matches
// the size requested. If that ever changes, the capacity here should
// change to `ptr.len() / mem::size_of::<T>()`.
self.ptr = unsafe { Unique::new_unchecked(ptr.cast().as_ptr()) };
self.cap = cap;
}
// This method is usually instantiated many times. So we want it to be as
// small as possible, to improve compile times. But we also want as much of
// its contents to be statically computable as possible, to make the
// generated code run faster. Therefore, this method is carefully written
// so that all of the code that depends on `T` is within it, while as much
// of the code that doesn't depend on `T` as possible is in functions that
// are non-generic over `T`.
fn grow_amortized(&mut self, len: usize, additional: usize) -> Result<(), TryReserveError> {
// This is ensured by the calling contexts.
debug_assert!(additional > 0);
if mem::size_of::<T>() == 0 {
// Since we return a capacity of `usize::MAX` when `elem_size` is
// 0, getting to here necessarily means the `RawVec` is overfull.
return Err(CapacityOverflow.into());
}
// Nothing we can really do about these checks, sadly.
let required_cap = len.checked_add(additional).ok_or(CapacityOverflow)?;
// This guarantees exponential growth. The doubling cannot overflow
// because `cap <= isize::MAX` and the type of `cap` is `usize`.
let cap = cmp::max(self.cap * 2, required_cap);
let cap = cmp::max(Self::MIN_NON_ZERO_CAP, cap);
let new_layout = Layout::array::<T>(cap);
// `finish_grow` is non-generic over `T`.
let ptr = finish_grow(new_layout, self.current_memory(), &mut self.alloc)?;
self.set_ptr_and_cap(ptr, cap);
Ok(())
}
// The constraints on this method are much the same as those on
// `grow_amortized`, but this method is usually instantiated less often so
// it's less critical.
fn grow_exact(&mut self, len: usize, additional: usize) -> Result<(), TryReserveError> {
if mem::size_of::<T>() == 0 {
// Since we return a capacity of `usize::MAX` when the type size is
// 0, getting to here necessarily means the `RawVec` is overfull.
return Err(CapacityOverflow.into());
}
let cap = len.checked_add(additional).ok_or(CapacityOverflow)?;
let new_layout = Layout::array::<T>(cap);
// `finish_grow` is non-generic over `T`.
let ptr = finish_grow(new_layout, self.current_memory(), &mut self.alloc)?;
self.set_ptr_and_cap(ptr, cap);
Ok(())
}
#[allow(dead_code)]
fn shrink(&mut self, cap: usize) -> Result<(), TryReserveError> {
assert!(cap <= self.capacity(), "Tried to shrink to a larger capacity");
let (ptr, layout) = if let Some(mem) = self.current_memory() { mem } else { return Ok(()) };
let ptr = unsafe {
// `Layout::array` cannot overflow here because it would have
// overflowed earlier when capacity was larger.
let new_layout = Layout::array::<T>(cap).unwrap_unchecked();
self.alloc
.shrink(ptr, layout, new_layout)
.map_err(|_| AllocError { layout: new_layout, non_exhaustive: () })?
};
self.set_ptr_and_cap(ptr, cap);
Ok(())
}
}
// This function is outside `RawVec` to minimize compile times. See the comment
// above `RawVec::grow_amortized` for details. (The `A` parameter isn't
// significant, because the number of different `A` types seen in practice is
// much smaller than the number of `T` types.)
#[inline(never)]
fn finish_grow<A>(
new_layout: Result<Layout, LayoutError>,
current_memory: Option<(NonNull<u8>, Layout)>,
alloc: &mut A,
) -> Result<NonNull<[u8]>, TryReserveError>
where
A: Allocator,
{
// Check for the error here to minimize the size of `RawVec::grow_*`.
let new_layout = new_layout.map_err(|_| CapacityOverflow)?;
alloc_guard(new_layout.size())?;
let memory = if let Some((ptr, old_layout)) = current_memory {
debug_assert_eq!(old_layout.align(), new_layout.align());
unsafe {
// The allocator checks for alignment equality
intrinsics::assume(old_layout.align() == new_layout.align());
alloc.grow(ptr, old_layout, new_layout)
}
} else {
alloc.allocate(new_layout)
};
memory.map_err(|_| AllocError { layout: new_layout, non_exhaustive: () }.into())
}
unsafe impl<#[may_dangle] T, A: Allocator> Drop for RawVec<T, A> {
/// Frees the memory owned by the `RawVec` *without* trying to drop its contents.
fn drop(&mut self) {
if let Some((ptr, layout)) = self.current_memory() {
unsafe { self.alloc.deallocate(ptr, layout) }
}
}
}
// Central function for reserve error handling.
#[cfg(not(no_global_oom_handling))]
#[inline]
fn handle_reserve(result: Result<(), TryReserveError>) {
match result.map_err(|e| e.kind()) {
Err(CapacityOverflow) => capacity_overflow(),
Err(AllocError { layout, .. }) => handle_alloc_error(layout),
Ok(()) => { /* yay */ }
}
}
// We need to guarantee the following:
// * We don't ever allocate `> isize::MAX` byte-size objects.
// * We don't overflow `usize::MAX` and actually allocate too little.
//
// On 64-bit we just need to check for overflow since trying to allocate
// `> isize::MAX` bytes will surely fail. On 32-bit and 16-bit we need to add
// an extra guard for this in case we're running on a platform which can use
// all 4GB in user-space, e.g., PAE or x32.
#[inline]
fn alloc_guard(alloc_size: usize) -> Result<(), TryReserveError> {
if usize::BITS < 64 && alloc_size > isize::MAX as usize {
Err(CapacityOverflow.into())
} else {
Ok(())
}
}
// One central function responsible for reporting capacity overflows. This'll
// ensure that the code generation related to these panics is minimal as there's
// only one location which panics rather than a bunch throughout the module.
#[cfg(not(no_global_oom_handling))]
fn capacity_overflow() -> ! {
panic!("capacity overflow");
}

1204
rust/alloc/slice.rs Normal file
View File

File diff suppressed because it is too large Load Diff

186
rust/alloc/vec/drain.rs Normal file
View File

@ -0,0 +1,186 @@
// SPDX-License-Identifier: Apache-2.0 OR MIT
use crate::alloc::{Allocator, Global};
use core::fmt;
use core::iter::{FusedIterator, TrustedLen};
use core::mem;
use core::ptr::{self, NonNull};
use core::slice::{self};
use super::Vec;
/// A draining iterator for `Vec<T>`.
///
/// This `struct` is created by [`Vec::drain`].
/// See its documentation for more.
///
/// # Example
///
/// ```
/// let mut v = vec![0, 1, 2];
/// let iter: std::vec::Drain<_> = v.drain(..);
/// ```
#[stable(feature = "drain", since = "1.6.0")]
pub struct Drain<
'a,
T: 'a,
#[unstable(feature = "allocator_api", issue = "32838")] A: Allocator + 'a = Global,
> {
/// Index of tail to preserve
pub(super) tail_start: usize,
/// Length of tail
pub(super) tail_len: usize,
/// Current remaining range to remove
pub(super) iter: slice::Iter<'a, T>,
pub(super) vec: NonNull<Vec<T, A>>,
}
#[stable(feature = "collection_debug", since = "1.17.0")]
impl<T: fmt::Debug, A: Allocator> fmt::Debug for Drain<'_, T, A> {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.debug_tuple("Drain").field(&self.iter.as_slice()).finish()
}
}
impl<'a, T, A: Allocator> Drain<'a, T, A> {
/// Returns the remaining items of this iterator as a slice.
///
/// # Examples
///
/// ```
/// let mut vec = vec!['a', 'b', 'c'];
/// let mut drain = vec.drain(..);
/// assert_eq!(drain.as_slice(), &['a', 'b', 'c']);
/// let _ = drain.next().unwrap();
/// assert_eq!(drain.as_slice(), &['b', 'c']);
/// ```
#[must_use]
#[stable(feature = "vec_drain_as_slice", since = "1.46.0")]
pub fn as_slice(&self) -> &[T] {
self.iter.as_slice()
}
/// Returns a reference to the underlying allocator.
#[unstable(feature = "allocator_api", issue = "32838")]
#[must_use]
#[inline]
pub fn allocator(&self) -> &A {
unsafe { self.vec.as_ref().allocator() }
}
}
#[stable(feature = "vec_drain_as_slice", since = "1.46.0")]
impl<'a, T, A: Allocator> AsRef<[T]> for Drain<'a, T, A> {
fn as_ref(&self) -> &[T] {
self.as_slice()
}
}
#[stable(feature = "drain", since = "1.6.0")]
unsafe impl<T: Sync, A: Sync + Allocator> Sync for Drain<'_, T, A> {}
#[stable(feature = "drain", since = "1.6.0")]
unsafe impl<T: Send, A: Send + Allocator> Send for Drain<'_, T, A> {}
#[stable(feature = "drain", since = "1.6.0")]
impl<T, A: Allocator> Iterator for Drain<'_, T, A> {
type Item = T;
#[inline]
fn next(&mut self) -> Option<T> {
self.iter.next().map(|elt| unsafe { ptr::read(elt as *const _) })
}
fn size_hint(&self) -> (usize, Option<usize>) {
self.iter.size_hint()
}
}
#[stable(feature = "drain", since = "1.6.0")]
impl<T, A: Allocator> DoubleEndedIterator for Drain<'_, T, A> {
#[inline]
fn next_back(&mut self) -> Option<T> {
self.iter.next_back().map(|elt| unsafe { ptr::read(elt as *const _) })
}
}
#[stable(feature = "drain", since = "1.6.0")]
impl<T, A: Allocator> Drop for Drain<'_, T, A> {
fn drop(&mut self) {
/// Moves back the un-`Drain`ed elements to restore the original `Vec`.
struct DropGuard<'r, 'a, T, A: Allocator>(&'r mut Drain<'a, T, A>);
impl<'r, 'a, T, A: Allocator> Drop for DropGuard<'r, 'a, T, A> {
fn drop(&mut self) {
if self.0.tail_len > 0 {
unsafe {
let source_vec = self.0.vec.as_mut();
// memmove back untouched tail, update to new length
let start = source_vec.len();
let tail = self.0.tail_start;
if tail != start {
let src = source_vec.as_ptr().add(tail);
let dst = source_vec.as_mut_ptr().add(start);
ptr::copy(src, dst, self.0.tail_len);
}
source_vec.set_len(start + self.0.tail_len);
}
}
}
}
let iter = mem::replace(&mut self.iter, (&mut []).iter());
let drop_len = iter.len();
let mut vec = self.vec;
if mem::size_of::<T>() == 0 {
// ZSTs have no identity, so we don't need to move them around, we only need to drop the correct amount.
// this can be achieved by manipulating the Vec length instead of moving values out from `iter`.
unsafe {
let vec = vec.as_mut();
let old_len = vec.len();
vec.set_len(old_len + drop_len + self.tail_len);
vec.truncate(old_len + self.tail_len);
}
return;
}
// ensure elements are moved back into their appropriate places, even when drop_in_place panics
let _guard = DropGuard(self);
if drop_len == 0 {
return;
}
// as_slice() must only be called when iter.len() is > 0 because
// vec::Splice modifies vec::Drain fields and may grow the vec which would invalidate
// the iterator's internal pointers. Creating a reference to deallocated memory
// is invalid even when it is zero-length
let drop_ptr = iter.as_slice().as_ptr();
unsafe {
// drop_ptr comes from a slice::Iter which only gives us a &[T] but for drop_in_place
// a pointer with mutable provenance is necessary. Therefore we must reconstruct
// it from the original vec but also avoid creating a &mut to the front since that could
// invalidate raw pointers to it which some unsafe code might rely on.
let vec_ptr = vec.as_mut().as_mut_ptr();
let drop_offset = drop_ptr.sub_ptr(vec_ptr);
let to_drop = ptr::slice_from_raw_parts_mut(vec_ptr.add(drop_offset), drop_len);
ptr::drop_in_place(to_drop);
}
}
}
#[stable(feature = "drain", since = "1.6.0")]
impl<T, A: Allocator> ExactSizeIterator for Drain<'_, T, A> {
fn is_empty(&self) -> bool {
self.iter.is_empty()
}
}
#[unstable(feature = "trusted_len", issue = "37572")]
unsafe impl<T, A: Allocator> TrustedLen for Drain<'_, T, A> {}
#[stable(feature = "fused", since = "1.26.0")]
impl<T, A: Allocator> FusedIterator for Drain<'_, T, A> {}

145
rust/alloc/vec/drain_filter.rs Normal file
View File

@ -0,0 +1,145 @@
// SPDX-License-Identifier: Apache-2.0 OR MIT
use crate::alloc::{Allocator, Global};
use core::ptr::{self};
use core::slice::{self};
use super::Vec;
/// An iterator which uses a closure to determine if an element should be removed.
///
/// This struct is created by [`Vec::drain_filter`].
/// See its documentation for more.
///
/// # Example
///
/// ```
/// #![feature(drain_filter)]
///
/// let mut v = vec![0, 1, 2];
/// let iter: std::vec::DrainFilter<_, _> = v.drain_filter(|x| *x % 2 == 0);
/// ```
#[unstable(feature = "drain_filter", reason = "recently added", issue = "43244")]
#[derive(Debug)]
pub struct DrainFilter<
'a,
T,
F,
#[unstable(feature = "allocator_api", issue = "32838")] A: Allocator = Global,
> where
F: FnMut(&mut T) -> bool,
{
pub(super) vec: &'a mut Vec<T, A>,
/// The index of the item that will be inspected by the next call to `next`.
pub(super) idx: usize,
/// The number of items that have been drained (removed) thus far.
pub(super) del: usize,
/// The original length of `vec` prior to draining.
pub(super) old_len: usize,
/// The filter test predicate.
pub(super) pred: F,
/// A flag that indicates a panic has occurred in the filter test predicate.
/// This is used as a hint in the drop implementation to prevent consumption
/// of the remainder of the `DrainFilter`. Any unprocessed items will be
/// backshifted in the `vec`, but no further items will be dropped or
/// tested by the filter predicate.
pub(super) panic_flag: bool,
}
impl<T, F, A: Allocator> DrainFilter<'_, T, F, A>
where
F: FnMut(&mut T) -> bool,
{
/// Returns a reference to the underlying allocator.
#[unstable(feature = "allocator_api", issue = "32838")]
#[inline]
pub fn allocator(&self) -> &A {
self.vec.allocator()
}
}
#[unstable(feature = "drain_filter", reason = "recently added", issue = "43244")]
impl<T, F, A: Allocator> Iterator for DrainFilter<'_, T, F, A>
where
F: FnMut(&mut T) -> bool,
{
type Item = T;
fn next(&mut self) -> Option<T> {
unsafe {
while self.idx < self.old_len {
let i = self.idx;
let v = slice::from_raw_parts_mut(self.vec.as_mut_ptr(), self.old_len);
self.panic_flag = true;
let drained = (self.pred)(&mut v[i]);
self.panic_flag = false;
// Update the index *after* the predicate is called. If the index
// is updated prior and the predicate panics, the element at this
// index would be leaked.
self.idx += 1;
if drained {
self.del += 1;
return Some(ptr::read(&v[i]));
} else if self.del > 0 {
let del = self.del;
let src: *const T = &v[i];
let dst: *mut T = &mut v[i - del];
ptr::copy_nonoverlapping(src, dst, 1);
}
}
None
}
}
fn size_hint(&self) -> (usize, Option<usize>) {
(0, Some(self.old_len - self.idx))
}
}
#[unstable(feature = "drain_filter", reason = "recently added", issue = "43244")]
impl<T, F, A: Allocator> Drop for DrainFilter<'_, T, F, A>
where
F: FnMut(&mut T) -> bool,
{
fn drop(&mut self) {
struct BackshiftOnDrop<'a, 'b, T, F, A: Allocator>
where
F: FnMut(&mut T) -> bool,
{
drain: &'b mut DrainFilter<'a, T, F, A>,
}
impl<'a, 'b, T, F, A: Allocator> Drop for BackshiftOnDrop<'a, 'b, T, F, A>
where
F: FnMut(&mut T) -> bool,
{
fn drop(&mut self) {
unsafe {
if self.drain.idx < self.drain.old_len && self.drain.del > 0 {
// This is a pretty messed up state, and there isn't really an
// obviously right thing to do. We don't want to keep trying
// to execute `pred`, so we just backshift all the unprocessed
// elements and tell the vec that they still exist. The backshift
// is required to prevent a double-drop of the last successfully
// drained item prior to a panic in the predicate.
let ptr = self.drain.vec.as_mut_ptr();
let src = ptr.add(self.drain.idx);
let dst = src.sub(self.drain.del);
let tail_len = self.drain.old_len - self.drain.idx;
src.copy_to(dst, tail_len);
}
self.drain.vec.set_len(self.drain.old_len - self.drain.del);
}
}
}
let backshift = BackshiftOnDrop { drain: self };
// Attempt to consume any remaining elements if the filter predicate
// has not yet panicked. We'll backshift any remaining elements
// whether we've already panicked or if the consumption here panics.
if !backshift.drain.panic_flag {
backshift.drain.for_each(drop);
}
}
}

366
rust/alloc/vec/into_iter.rs Normal file
View File

@ -0,0 +1,366 @@
// SPDX-License-Identifier: Apache-2.0 OR MIT
#[cfg(not(no_global_oom_handling))]
use super::AsVecIntoIter;
use crate::alloc::{Allocator, Global};
use crate::raw_vec::RawVec;
use core::fmt;
use core::intrinsics::arith_offset;
use core::iter::{
FusedIterator, InPlaceIterable, SourceIter, TrustedLen, TrustedRandomAccessNoCoerce,
};
use core::marker::PhantomData;
use core::mem::{self, ManuallyDrop};
#[cfg(not(no_global_oom_handling))]
use core::ops::Deref;
use core::ptr::{self, NonNull};
use core::slice::{self};
/// An iterator that moves out of a vector.
///
/// This `struct` is created by the `into_iter` method on [`Vec`](super::Vec)
/// (provided by the [`IntoIterator`] trait).
///
/// # Example
///
/// ```
/// let v = vec![0, 1, 2];
/// let iter: std::vec::IntoIter<_> = v.into_iter();
/// ```
#[stable(feature = "rust1", since = "1.0.0")]
#[rustc_insignificant_dtor]
pub struct IntoIter<
T,
#[unstable(feature = "allocator_api", issue = "32838")] A: Allocator = Global,
> {
pub(super) buf: NonNull<T>,
pub(super) phantom: PhantomData<T>,
pub(super) cap: usize,
// the drop impl reconstructs a RawVec from buf, cap and alloc
// to avoid dropping the allocator twice we need to wrap it into ManuallyDrop
pub(super) alloc: ManuallyDrop<A>,
pub(super) ptr: *const T,
pub(super) end: *const T,
}
#[stable(feature = "vec_intoiter_debug", since = "1.13.0")]
impl<T: fmt::Debug, A: Allocator> fmt::Debug for IntoIter<T, A> {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.debug_tuple("IntoIter").field(&self.as_slice()).finish()
}
}
impl<T, A: Allocator> IntoIter<T, A> {
/// Returns the remaining items of this iterator as a slice.
///
/// # Examples
///
/// ```
/// let vec = vec!['a', 'b', 'c'];
/// let mut into_iter = vec.into_iter();
/// assert_eq!(into_iter.as_slice(), &['a', 'b', 'c']);
/// let _ = into_iter.next().unwrap();
/// assert_eq!(into_iter.as_slice(), &['b', 'c']);
/// ```
#[stable(feature = "vec_into_iter_as_slice", since = "1.15.0")]
pub fn as_slice(&self) -> &[T] {
unsafe { slice::from_raw_parts(self.ptr, self.len()) }
}
/// Returns the remaining items of this iterator as a mutable slice.
///
/// # Examples
///
/// ```
/// let vec = vec!['a', 'b', 'c'];
/// let mut into_iter = vec.into_iter();
/// assert_eq!(into_iter.as_slice(), &['a', 'b', 'c']);
/// into_iter.as_mut_slice()[2] = 'z';
/// assert_eq!(into_iter.next().unwrap(), 'a');
/// assert_eq!(into_iter.next().unwrap(), 'b');
/// assert_eq!(into_iter.next().unwrap(), 'z');
/// ```
#[stable(feature = "vec_into_iter_as_slice", since = "1.15.0")]
pub fn as_mut_slice(&mut self) -> &mut [T] {
unsafe { &mut *self.as_raw_mut_slice() }
}
/// Returns a reference to the underlying allocator.
#[unstable(feature = "allocator_api", issue = "32838")]
#[inline]
pub fn allocator(&self) -> &A {
&self.alloc
}
fn as_raw_mut_slice(&mut self) -> *mut [T] {
ptr::slice_from_raw_parts_mut(self.ptr as *mut T, self.len())
}
/// Drops remaining elements and relinquishes the backing allocation.
///
/// This is roughly equivalent to the following, but more efficient
///
/// ```
/// # let mut into_iter = Vec::<u8>::with_capacity(10).into_iter();
/// (&mut into_iter).for_each(core::mem::drop);
/// unsafe { core::ptr::write(&mut into_iter, Vec::new().into_iter()); }
/// ```
///
/// This method is used by in-place iteration, refer to the vec::in_place_collect
/// documentation for an overview.
#[cfg(not(no_global_oom_handling))]
pub(super) fn forget_allocation_drop_remaining(&mut self) {
let remaining = self.as_raw_mut_slice();
// overwrite the individual fields instead of creating a new
// struct and then overwriting &mut self.
// this creates less assembly
self.cap = 0;
self.buf = unsafe { NonNull::new_unchecked(RawVec::NEW.ptr()) };
self.ptr = self.buf.as_ptr();
self.end = self.buf.as_ptr();
unsafe {
ptr::drop_in_place(remaining);
}
}
/// Forgets to Drop the remaining elements while still allowing the backing allocation to be freed.
#[allow(dead_code)]
pub(crate) fn forget_remaining_elements(&mut self) {
self.ptr = self.end;
}
}
#[stable(feature = "vec_intoiter_as_ref", since = "1.46.0")]
impl<T, A: Allocator> AsRef<[T]> for IntoIter<T, A> {
fn as_ref(&self) -> &[T] {
self.as_slice()
}
}
#[stable(feature = "rust1", since = "1.0.0")]
unsafe impl<T: Send, A: Allocator + Send> Send for IntoIter<T, A> {}
#[stable(feature = "rust1", since = "1.0.0")]
unsafe impl<T: Sync, A: Allocator + Sync> Sync for IntoIter<T, A> {}
#[stable(feature = "rust1", since = "1.0.0")]
impl<T, A: Allocator> Iterator for IntoIter<T, A> {
type Item = T;
#[inline]
fn next(&mut self) -> Option<T> {
if self.ptr as *const _ == self.end {
None
} else if mem::size_of::<T>() == 0 {
// purposefully don't use 'ptr.offset' because for
// vectors with 0-size elements this would return the
// same pointer.
self.ptr = unsafe { arith_offset(self.ptr as *const i8, 1) as *mut T };
// Make up a value of this ZST.
Some(unsafe { mem::zeroed() })
} else {
let old = self.ptr;
self.ptr = unsafe { self.ptr.offset(1) };
Some(unsafe { ptr::read(old) })
}
}
#[inline]
fn size_hint(&self) -> (usize, Option<usize>) {
let exact = if mem::size_of::<T>() == 0 {
self.end.addr().wrapping_sub(self.ptr.addr())
} else {
unsafe { self.end.sub_ptr(self.ptr) }
};
(exact, Some(exact))
}
#[inline]
fn advance_by(&mut self, n: usize) -> Result<(), usize> {
let step_size = self.len().min(n);
let to_drop = ptr::slice_from_raw_parts_mut(self.ptr as *mut T, step_size);
if mem::size_of::<T>() == 0 {
// SAFETY: due to unchecked casts of unsigned amounts to signed offsets the wraparound
// effectively results in unsigned pointers representing positions 0..usize::MAX,
// which is valid for ZSTs.
self.ptr = unsafe { arith_offset(self.ptr as *const i8, step_size as isize) as *mut T }
} else {
// SAFETY: the min() above ensures that step_size is in bounds
self.ptr = unsafe { self.ptr.add(step_size) };
}
// SAFETY: the min() above ensures that step_size is in bounds
unsafe {
ptr::drop_in_place(to_drop);
}
if step_size < n {
return Err(step_size);
}
Ok(())
}
#[inline]
fn count(self) -> usize {
self.len()
}
unsafe fn __iterator_get_unchecked(&mut self, i: usize) -> Self::Item
where
Self: TrustedRandomAccessNoCoerce,
{
// SAFETY: the caller must guarantee that `i` is in bounds of the
// `Vec<T>`, so `i` cannot overflow an `isize`, and the `self.ptr.add(i)`
// is guaranteed to pointer to an element of the `Vec<T>` and
// thus guaranteed to be valid to dereference.
//
// Also note the implementation of `Self: TrustedRandomAccess` requires
// that `T: Copy` so reading elements from the buffer doesn't invalidate
// them for `Drop`.
unsafe {
if mem::size_of::<T>() == 0 { mem::zeroed() } else { ptr::read(self.ptr.add(i)) }
}
}
}
#[stable(feature = "rust1", since = "1.0.0")]
impl<T, A: Allocator> DoubleEndedIterator for IntoIter<T, A> {
#[inline]
fn next_back(&mut self) -> Option<T> {
if self.end == self.ptr {
None
} else if mem::size_of::<T>() == 0 {
// See above for why 'ptr.offset' isn't used
self.end = unsafe { arith_offset(self.end as *const i8, -1) as *mut T };
// Make up a value of this ZST.
Some(unsafe { mem::zeroed() })
} else {
self.end = unsafe { self.end.offset(-1) };
Some(unsafe { ptr::read(self.end) })
}
}
#[inline]
fn advance_back_by(&mut self, n: usize) -> Result<(), usize> {
let step_size = self.len().min(n);
if mem::size_of::<T>() == 0 {
// SAFETY: same as for advance_by()
self.end = unsafe {
arith_offset(self.end as *const i8, step_size.wrapping_neg() as isize) as *mut T
}
} else {
// SAFETY: same as for advance_by()
self.end = unsafe { self.end.offset(step_size.wrapping_neg() as isize) };
}
let to_drop = ptr::slice_from_raw_parts_mut(self.end as *mut T, step_size);
// SAFETY: same as for advance_by()
unsafe {
ptr::drop_in_place(to_drop);
}
if step_size < n {
return Err(step_size);
}
Ok(())
}
}
#[stable(feature = "rust1", since = "1.0.0")]
impl<T, A: Allocator> ExactSizeIterator for IntoIter<T, A> {
fn is_empty(&self) -> bool {
self.ptr == self.end
}
}
#[stable(feature = "fused", since = "1.26.0")]
impl<T, A: Allocator> FusedIterator for IntoIter<T, A> {}
#[unstable(feature = "trusted_len", issue = "37572")]
unsafe impl<T, A: Allocator> TrustedLen for IntoIter<T, A> {}
#[doc(hidden)]
#[unstable(issue = "none", feature = "std_internals")]
#[rustc_unsafe_specialization_marker]
pub trait NonDrop {}
// T: Copy as approximation for !Drop since get_unchecked does not advance self.ptr
// and thus we can't implement drop-handling
#[unstable(issue = "none", feature = "std_internals")]
impl<T: Copy> NonDrop for T {}
#[doc(hidden)]
#[unstable(issue = "none", feature = "std_internals")]
// TrustedRandomAccess (without NoCoerce) must not be implemented because
// subtypes/supertypes of `T` might not be `NonDrop`
unsafe impl<T, A: Allocator> TrustedRandomAccessNoCoerce for IntoIter<T, A>
where
T: NonDrop,
{
const MAY_HAVE_SIDE_EFFECT: bool = false;
}
#[cfg(not(no_global_oom_handling))]
#[stable(feature = "vec_into_iter_clone", since = "1.8.0")]
impl<T: Clone, A: Allocator + Clone> Clone for IntoIter<T, A> {
#[cfg(not(test))]
fn clone(&self) -> Self {
self.as_slice().to_vec_in(self.alloc.deref().clone()).into_iter()
}
#[cfg(test)]
fn clone(&self) -> Self {
crate::slice::to_vec(self.as_slice(), self.alloc.deref().clone()).into_iter()
}
}
#[stable(feature = "rust1", since = "1.0.0")]
unsafe impl<#[may_dangle] T, A: Allocator> Drop for IntoIter<T, A> {
fn drop(&mut self) {
struct DropGuard<'a, T, A: Allocator>(&'a mut IntoIter<T, A>);
impl<T, A: Allocator> Drop for DropGuard<'_, T, A> {
fn drop(&mut self) {
unsafe {
// `IntoIter::alloc` is not used anymore after this and will be dropped by RawVec
let alloc = ManuallyDrop::take(&mut self.0.alloc);
// RawVec handles deallocation
let _ = RawVec::from_raw_parts_in(self.0.buf.as_ptr(), self.0.cap, alloc);
}
}
}
let guard = DropGuard(self);
// destroy the remaining elements
unsafe {
ptr::drop_in_place(guard.0.as_raw_mut_slice());
}
// now `guard` will be dropped and do the rest
}
}
// In addition to the SAFETY invariants of the following three unsafe traits
// also refer to the vec::in_place_collect module documentation to get an overview
#[unstable(issue = "none", feature = "inplace_iteration")]
#[doc(hidden)]
unsafe impl<T, A: Allocator> InPlaceIterable for IntoIter<T, A> {}
#[unstable(issue = "none", feature = "inplace_iteration")]
#[doc(hidden)]
unsafe impl<T, A: Allocator> SourceIter for IntoIter<T, A> {
type Source = Self;
#[inline]
unsafe fn as_inner(&mut self) -> &mut Self::Source {
self
}
}
#[cfg(not(no_global_oom_handling))]
unsafe impl<T> AsVecIntoIter for IntoIter<T> {
type Item = T;
fn as_into_iter(&mut self) -> &mut IntoIter<Self::Item> {
self
}
}

120
rust/alloc/vec/is_zero.rs Normal file
View File

@ -0,0 +1,120 @@
// SPDX-License-Identifier: Apache-2.0 OR MIT
use crate::boxed::Box;
#[rustc_specialization_trait]
pub(super) unsafe trait IsZero {
/// Whether this value's representation is all zeros
fn is_zero(&self) -> bool;
}
macro_rules! impl_is_zero {
($t:ty, $is_zero:expr) => {
unsafe impl IsZero for $t {
#[inline]
fn is_zero(&self) -> bool {
$is_zero(*self)
}
}
};
}
impl_is_zero!(i16, |x| x == 0);
impl_is_zero!(i32, |x| x == 0);
impl_is_zero!(i64, |x| x == 0);
impl_is_zero!(i128, |x| x == 0);
impl_is_zero!(isize, |x| x == 0);
impl_is_zero!(u16, |x| x == 0);
impl_is_zero!(u32, |x| x == 0);
impl_is_zero!(u64, |x| x == 0);
impl_is_zero!(u128, |x| x == 0);
impl_is_zero!(usize, |x| x == 0);
impl_is_zero!(bool, |x| x == false);
impl_is_zero!(char, |x| x == '\0');
impl_is_zero!(f32, |x: f32| x.to_bits() == 0);
impl_is_zero!(f64, |x: f64| x.to_bits() == 0);
unsafe impl<T> IsZero for *const T {
#[inline]
fn is_zero(&self) -> bool {
(*self).is_null()
}
}
unsafe impl<T> IsZero for *mut T {
#[inline]
fn is_zero(&self) -> bool {
(*self).is_null()
}
}
unsafe impl<T: IsZero, const N: usize> IsZero for [T; N] {
#[inline]
fn is_zero(&self) -> bool {
// Because this is generated as a runtime check, it's not obvious that
// it's worth doing if the array is really long. The threshold here
// is largely arbitrary, but was picked because as of 2022-05-01 LLVM
// can const-fold the check in `vec![[0; 32]; n]` but not in
// `vec![[0; 64]; n]`: https://godbolt.org/z/WTzjzfs5b
// Feel free to tweak if you have better evidence.
N <= 32 && self.iter().all(IsZero::is_zero)
}
}
// `Option<&T>` and `Option<Box<T>>` are guaranteed to represent `None` as null.
// For fat pointers, the bytes that would be the pointer metadata in the `Some`
// variant are padding in the `None` variant, so ignoring them and
// zero-initializing instead is ok.
// `Option<&mut T>` never implements `Clone`, so there's no need for an impl of
// `SpecFromElem`.
unsafe impl<T: ?Sized> IsZero for Option<&T> {
#[inline]
fn is_zero(&self) -> bool {
self.is_none()
}
}
unsafe impl<T: ?Sized> IsZero for Option<Box<T>> {
#[inline]
fn is_zero(&self) -> bool {
self.is_none()
}
}
// `Option<num::NonZeroU32>` and similar have a representation guarantee that
// they're the same size as the corresponding `u32` type, as well as a guarantee
// that transmuting between `NonZeroU32` and `Option<num::NonZeroU32>` works.
// While the documentation officially makes it UB to transmute from `None`,
// we're the standard library so we can make extra inferences, and we know that
// the only niche available to represent `None` is the one that's all zeros.
macro_rules! impl_is_zero_option_of_nonzero {
($($t:ident,)+) => {$(
unsafe impl IsZero for Option<core::num::$t> {
#[inline]
fn is_zero(&self) -> bool {
self.is_none()
}
}
)+};
}
impl_is_zero_option_of_nonzero!(
NonZeroU8,
NonZeroU16,
NonZeroU32,
NonZeroU64,
NonZeroU128,
NonZeroI8,
NonZeroI16,
NonZeroI32,
NonZeroI64,
NonZeroI128,
NonZeroUsize,
NonZeroIsize,
);

3140
rust/alloc/vec/mod.rs Normal file
View File

File diff suppressed because it is too large Load Diff

49
rust/alloc/vec/partial_eq.rs Normal file
View File

@ -0,0 +1,49 @@
// SPDX-License-Identifier: Apache-2.0 OR MIT
use crate::alloc::Allocator;
#[cfg(not(no_global_oom_handling))]
use crate::borrow::Cow;
use super::Vec;
macro_rules! __impl_slice_eq1 {
([$($vars:tt)*] $lhs:ty, $rhs:ty $(where $ty:ty: $bound:ident)?, #[$stability:meta]) => {
#[$stability]
impl<T, U, $($vars)*> PartialEq<$rhs> for $lhs
where
T: PartialEq<U>,
$($ty: $bound)?
{
#[inline]
fn eq(&self, other: &$rhs) -> bool { self[..] == other[..] }
#[inline]
fn ne(&self, other: &$rhs) -> bool { self[..] != other[..] }
}
}
}
__impl_slice_eq1! { [A1: Allocator, A2: Allocator] Vec<T, A1>, Vec<U, A2>, #[stable(feature = "rust1", since = "1.0.0")] }
__impl_slice_eq1! { [A: Allocator] Vec<T, A>, &[U], #[stable(feature = "rust1", since = "1.0.0")] }
__impl_slice_eq1! { [A: Allocator] Vec<T, A>, &mut [U], #[stable(feature = "rust1", since = "1.0.0")] }
__impl_slice_eq1! { [A: Allocator] &[T], Vec<U, A>, #[stable(feature = "partialeq_vec_for_ref_slice", since = "1.46.0")] }
__impl_slice_eq1! { [A: Allocator] &mut [T], Vec<U, A>, #[stable(feature = "partialeq_vec_for_ref_slice", since = "1.46.0")] }
__impl_slice_eq1! { [A: Allocator] Vec<T, A>, [U], #[stable(feature = "partialeq_vec_for_slice", since = "1.48.0")] }
__impl_slice_eq1! { [A: Allocator] [T], Vec<U, A>, #[stable(feature = "partialeq_vec_for_slice", since = "1.48.0")] }
#[cfg(not(no_global_oom_handling))]
__impl_slice_eq1! { [A: Allocator] Cow<'_, [T]>, Vec<U, A> where T: Clone, #[stable(feature = "rust1", since = "1.0.0")] }
#[cfg(not(no_global_oom_handling))]
__impl_slice_eq1! { [] Cow<'_, [T]>, &[U] where T: Clone, #[stable(feature = "rust1", since = "1.0.0")] }
#[cfg(not(no_global_oom_handling))]
__impl_slice_eq1! { [] Cow<'_, [T]>, &mut [U] where T: Clone, #[stable(feature = "rust1", since = "1.0.0")] }
__impl_slice_eq1! { [A: Allocator, const N: usize] Vec<T, A>, [U; N], #[stable(feature = "rust1", since = "1.0.0")] }
__impl_slice_eq1! { [A: Allocator, const N: usize] Vec<T, A>, &[U; N], #[stable(feature = "rust1", since = "1.0.0")] }
// NOTE: some less important impls are omitted to reduce code bloat
// FIXME(Centril): Reconsider this?
//__impl_slice_eq1! { [const N: usize] Vec<A>, &mut [B; N], }
//__impl_slice_eq1! { [const N: usize] [A; N], Vec<B>, }
//__impl_slice_eq1! { [const N: usize] &[A; N], Vec<B>, }
//__impl_slice_eq1! { [const N: usize] &mut [A; N], Vec<B>, }
//__impl_slice_eq1! { [const N: usize] Cow<'a, [A]>, [B; N], }
//__impl_slice_eq1! { [const N: usize] Cow<'a, [A]>, &[B; N], }
//__impl_slice_eq1! { [const N: usize] Cow<'a, [A]>, &mut [B; N], }

21
rust/bindgen_parameters Normal file
View File

@ -0,0 +1,21 @@
# SPDX-License-Identifier: GPL-2.0
--opaque-type xregs_state
--opaque-type desc_struct
--opaque-type arch_lbr_state
--opaque-type local_apic
# Packed type cannot transitively contain a `#[repr(align)]` type.
--opaque-type x86_msi_data
--opaque-type x86_msi_addr_lo
# `try` is a reserved keyword since Rust 2018; solved in `bindgen` v0.59.2,
# commit 2aed6b021680 ("context: Escape the try keyword properly").
--opaque-type kunit_try_catch
# If SMP is disabled, `arch_spinlock_t` is defined as a ZST which triggers a Rust
# warning. We don't need to peek into it anyway.
--opaque-type spinlock
# `seccomp`'s comment gets understood as a doctest
--no-doc-comments

13
rust/bindings/bindings_helper.h Normal file
View File

@ -0,0 +1,13 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Header that contains the code (mostly headers) for which Rust bindings
* will be automatically generated by `bindgen`.
*
* Sorted alphabetically.
*/
#include <linux/slab.h>
/* `bindgen` gets confused at certain things. */
const gfp_t BINDINGS_GFP_KERNEL = GFP_KERNEL;
const gfp_t BINDINGS___GFP_ZERO = __GFP_ZERO;

53
rust/bindings/lib.rs Normal file
View File

@ -0,0 +1,53 @@
// SPDX-License-Identifier: GPL-2.0
//! Bindings.
//!
//! Imports the generated bindings by `bindgen`.
//!
//! This crate may not be directly used. If you need a kernel C API that is
//! not ported or wrapped in the `kernel` crate, then do so first instead of
//! using this crate.
#![no_std]
#![feature(core_ffi_c)]
// See <https://github.com/rust-lang/rust-bindgen/issues/1651>.
#![cfg_attr(test, allow(deref_nullptr))]
#![cfg_attr(test, allow(unaligned_references))]
#![cfg_attr(test, allow(unsafe_op_in_unsafe_fn))]
#![allow(
clippy::all,
missing_docs,
non_camel_case_types,
non_upper_case_globals,
non_snake_case,
improper_ctypes,
unreachable_pub,
unsafe_op_in_unsafe_fn
)]
mod bindings_raw {
// Use glob import here to expose all helpers.
// Symbols defined within the module will take precedence to the glob import.
pub use super::bindings_helper::*;
include!(concat!(
env!("OBJTREE"),
"/rust/bindings/bindings_generated.rs"
));
}
// When both a directly exposed symbol and a helper exists for the same function,
// the directly exposed symbol is preferred and the helper becomes dead code, so
// ignore the warning here.
#[allow(dead_code)]
mod bindings_helper {
// Import the generated bindings for types.
include!(concat!(
env!("OBJTREE"),
"/rust/bindings/bindings_helpers_generated.rs"
));
}
pub use bindings_raw::*;
pub const GFP_KERNEL: gfp_t = BINDINGS_GFP_KERNEL;
pub const __GFP_ZERO: gfp_t = BINDINGS___GFP_ZERO;

63
rust/compiler_builtins.rs Normal file
View File

@ -0,0 +1,63 @@
// SPDX-License-Identifier: GPL-2.0
//! Our own `compiler_builtins`.
//!
//! Rust provides [`compiler_builtins`] as a port of LLVM's [`compiler-rt`].
//! Since we do not need the vast majority of them, we avoid the dependency
//! by providing this file.
//!
//! At the moment, some builtins are required that should not be. For instance,
//! [`core`] has 128-bit integers functionality which we should not be compiling
//! in. We will work with upstream [`core`] to provide feature flags to disable
//! the parts we do not need. For the moment, we define them to [`panic!`] at
//! runtime for simplicity to catch mistakes, instead of performing surgery
//! on `core.o`.
//!
//! In any case, all these symbols are weakened to ensure we do not override
//! those that may be provided by the rest of the kernel.
//!
//! [`compiler_builtins`]: https://github.com/rust-lang/compiler-builtins
//! [`compiler-rt`]: https://compiler-rt.llvm.org/
#![feature(compiler_builtins)]
#![compiler_builtins]
#![no_builtins]
#![no_std]
macro_rules! define_panicking_intrinsics(
($reason: tt, { $($ident: ident, )* }) => {
$(
#[doc(hidden)]
#[no_mangle]
pub extern "C" fn $ident() {
panic!($reason);
}
)*
}
);
define_panicking_intrinsics!("`f32` should not be used", {
__eqsf2,
__gesf2,
__lesf2,
__nesf2,
__unordsf2,
});
define_panicking_intrinsics!("`f64` should not be used", {
__unorddf2,
});
define_panicking_intrinsics!("`i128` should not be used", {
__ashrti3,
__muloti4,
__multi3,
});
define_panicking_intrinsics!("`u128` should not be used", {
__ashlti3,
__lshrti3,
__udivmodti4,
__udivti3,
__umodti3,
});

21
rust/exports.c Normal file
View File

@ -0,0 +1,21 @@
// SPDX-License-Identifier: GPL-2.0
/*
* A hack to export Rust symbols for loadable modules without having to redo
* the entire `include/linux/export.h` logic in Rust.
*
* This requires the Rust's new/future `v0` mangling scheme because the default
* one ("legacy") uses invalid characters for C identifiers (thus we cannot use
* the `EXPORT_SYMBOL_*` macros).
*
* All symbols are exported as GPL-only to guarantee no GPL-only feature is
* accidentally exposed.
*/
#include <linux/module.h>
#define EXPORT_SYMBOL_RUST_GPL(sym) extern int sym; EXPORT_SYMBOL_GPL(sym)
#include "exports_core_generated.h"
#include "exports_alloc_generated.h"
#include "exports_bindings_generated.h"
#include "exports_kernel_generated.h"

51
rust/helpers.c Normal file
View File

@ -0,0 +1,51 @@
// SPDX-License-Identifier: GPL-2.0
/*
* Non-trivial C macros cannot be used in Rust. Similarly, inlined C functions
* cannot be called either. This file explicitly creates functions ("helpers")
* that wrap those so that they can be called from Rust.
*
* Even though Rust kernel modules should never use directly the bindings, some
* of these helpers need to be exported because Rust generics and inlined
* functions may not get their code generated in the crate where they are
* defined. Other helpers, called from non-inline functions, may not be
* exported, in principle. However, in general, the Rust compiler does not
* guarantee codegen will be performed for a non-inline function either.
* Therefore, this file exports all the helpers. In the future, this may be
* revisited to reduce the number of exports after the compiler is informed
* about the places codegen is required.
*
* All symbols are exported as GPL-only to guarantee no GPL-only feature is
* accidentally exposed.
*/
#include <linux/bug.h>
#include <linux/build_bug.h>
__noreturn void rust_helper_BUG(void)
{
BUG();
}
EXPORT_SYMBOL_GPL(rust_helper_BUG);
/*
* We use `bindgen`'s `--size_t-is-usize` option to bind the C `size_t` type
* as the Rust `usize` type, so we can use it in contexts where Rust
* expects a `usize` like slice (array) indices. `usize` is defined to be
* the same as C's `uintptr_t` type (can hold any pointer) but not
* necessarily the same as `size_t` (can hold the size of any single
* object). Most modern platforms use the same concrete integer type for
* both of them, but in case we find ourselves on a platform where
* that's not true, fail early instead of risking ABI or
* integer-overflow issues.
*
* If your platform fails this assertion, it means that you are in
* danger of integer-overflow bugs (even if you attempt to remove
* `--size_t-is-usize`). It may be easiest to change the kernel ABI on
* your platform such that `size_t` matches `uintptr_t` (i.e., to increase
* `size_t`, because `uintptr_t` has to be at least as big as `size_t`).
*/
static_assert(
sizeof(size_t) == sizeof(uintptr_t) &&
__alignof__(size_t) == __alignof__(uintptr_t),
"Rust code expects C `size_t` to match Rust `usize`"
);

64
rust/kernel/allocator.rs Normal file
View File

@ -0,0 +1,64 @@
// SPDX-License-Identifier: GPL-2.0
//! Allocator support.
use core::alloc::{GlobalAlloc, Layout};
use core::ptr;
use crate::bindings;
struct KernelAllocator;
unsafe impl GlobalAlloc for KernelAllocator {
unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
// `krealloc()` is used instead of `kmalloc()` because the latter is
// an inline function and cannot be bound to as a result.
unsafe { bindings::krealloc(ptr::null(), layout.size(), bindings::GFP_KERNEL) as *mut u8 }
}
unsafe fn dealloc(&self, ptr: *mut u8, _layout: Layout) {
unsafe {
bindings::kfree(ptr as *const core::ffi::c_void);
}
}
}
#[global_allocator]
static ALLOCATOR: KernelAllocator = KernelAllocator;
// `rustc` only generates these for some crate types. Even then, we would need
// to extract the object file that has them from the archive. For the moment,
// let's generate them ourselves instead.
//
// Note that `#[no_mangle]` implies exported too, nowadays.
#[no_mangle]
fn __rust_alloc(size: usize, _align: usize) -> *mut u8 {
unsafe { bindings::krealloc(core::ptr::null(), size, bindings::GFP_KERNEL) as *mut u8 }
}
#[no_mangle]
fn __rust_dealloc(ptr: *mut u8, _size: usize, _align: usize) {
unsafe { bindings::kfree(ptr as *const core::ffi::c_void) };
}
#[no_mangle]
fn __rust_realloc(ptr: *mut u8, _old_size: usize, _align: usize, new_size: usize) -> *mut u8 {
unsafe {
bindings::krealloc(
ptr as *const core::ffi::c_void,
new_size,
bindings::GFP_KERNEL,
) as *mut u8
}
}
#[no_mangle]
fn __rust_alloc_zeroed(size: usize, _align: usize) -> *mut u8 {
unsafe {
bindings::krealloc(
core::ptr::null(),
size,
bindings::GFP_KERNEL | bindings::__GFP_ZERO,
) as *mut u8
}
}

59
rust/kernel/error.rs Normal file
View File

@ -0,0 +1,59 @@
// SPDX-License-Identifier: GPL-2.0
//! Kernel errors.
//!
//! C header: [`include/uapi/asm-generic/errno-base.h`](../../../include/uapi/asm-generic/errno-base.h)
use alloc::collections::TryReserveError;
/// Contains the C-compatible error codes.
pub mod code {
/// Out of memory.
pub const ENOMEM: super::Error = super::Error(-(crate::bindings::ENOMEM as i32));
}
/// Generic integer kernel error.
///
/// The kernel defines a set of integer generic error codes based on C and
/// POSIX ones. These codes may have a more specific meaning in some contexts.
///
/// # Invariants
///
/// The value is a valid `errno` (i.e. `>= -MAX_ERRNO && < 0`).
#[derive(Clone, Copy, PartialEq, Eq)]
pub struct Error(core::ffi::c_int);
impl Error {
/// Returns the kernel error code.
pub fn to_kernel_errno(self) -> core::ffi::c_int {
self.0
}
}
impl From<TryReserveError> for Error {
fn from(_: TryReserveError) -> Error {
code::ENOMEM
}
}
/// A [`Result`] with an [`Error`] error type.
///
/// To be used as the return type for functions that may fail.
///
/// # Error codes in C and Rust
///
/// In C, it is common that functions indicate success or failure through
/// their return value; modifying or returning extra data through non-`const`
/// pointer parameters. In particular, in the kernel, functions that may fail
/// typically return an `int` that represents a generic error code. We model
/// those as [`Error`].
///
/// In Rust, it is idiomatic to model functions that may fail as returning
/// a [`Result`]. Since in the kernel many functions return an error code,
/// [`Result`] is a type alias for a [`core::result::Result`] that uses
/// [`Error`] as its error type.
///
/// Note that even if a function does not return anything when it succeeds,
/// it should still be modeled as returning a `Result` rather than
/// just an [`Error`].
pub type Result<T = ()> = core::result::Result<T, Error>;

78
rust/kernel/lib.rs Normal file
View File

@ -0,0 +1,78 @@
// SPDX-License-Identifier: GPL-2.0
//! The `kernel` crate.
//!
//! This crate contains the kernel APIs that have been ported or wrapped for
//! usage by Rust code in the kernel and is shared by all of them.
//!
//! In other words, all the rest of the Rust code in the kernel (e.g. kernel
//! modules written in Rust) depends on [`core`], [`alloc`] and this crate.
//!
//! If you need a kernel C API that is not ported or wrapped yet here, then
//! do so first instead of bypassing this crate.
#![no_std]
#![feature(core_ffi_c)]
// Ensure conditional compilation based on the kernel configuration works;
// otherwise we may silently break things like initcall handling.
#[cfg(not(CONFIG_RUST))]
compile_error!("Missing kernel configuration for conditional compilation");
#[cfg(not(test))]
#[cfg(not(testlib))]
mod allocator;
pub mod error;
pub mod prelude;
pub mod print;
pub mod str;
#[doc(hidden)]
pub use bindings;
pub use macros;
/// Prefix to appear before log messages printed from within the `kernel` crate.
const __LOG_PREFIX: &[u8] = b"rust_kernel\0";
/// The top level entrypoint to implementing a kernel module.
///
/// For any teardown or cleanup operations, your type may implement [`Drop`].
pub trait Module: Sized + Sync {
/// Called at module initialization time.
///
/// Use this method to perform whatever setup or registration your module
/// should do.
///
/// Equivalent to the `module_init` macro in the C API.
fn init(module: &'static ThisModule) -> error::Result<Self>;
}
/// Equivalent to `THIS_MODULE` in the C API.
///
/// C header: `include/linux/export.h`
pub struct ThisModule(*mut bindings::module);
// SAFETY: `THIS_MODULE` may be used from all threads within a module.
unsafe impl Sync for ThisModule {}
impl ThisModule {
/// Creates a [`ThisModule`] given the `THIS_MODULE` pointer.
///
/// # Safety
///
/// The pointer must be equal to the right `THIS_MODULE`.
pub const unsafe fn from_ptr(ptr: *mut bindings::module) -> ThisModule {
ThisModule(ptr)
}
}
#[cfg(not(any(testlib, test)))]
#[panic_handler]
fn panic(info: &core::panic::PanicInfo<'_>) -> ! {
pr_emerg!("{}\n", info);
// SAFETY: FFI call.
unsafe { bindings::BUG() };
// Bindgen currently does not recognize `__noreturn` so `BUG` returns `()`
// instead of `!`. See <https://github.com/rust-lang/rust-bindgen/issues/2094>.
loop {}
}

20
rust/kernel/prelude.rs Normal file
View File

@ -0,0 +1,20 @@
// SPDX-License-Identifier: GPL-2.0
//! The `kernel` prelude.
//!
//! These are the most common items used by Rust code in the kernel,
//! intended to be imported by all Rust code, for convenience.
//!
//! # Examples
//!
//! ```
//! use kernel::prelude::*;
//! ```
pub use super::{
error::{Error, Result},
pr_emerg, pr_info, ThisModule,
};
pub use alloc::{boxed::Box, vec::Vec};
pub use core::pin::Pin;
pub use macros::module;

198
rust/kernel/print.rs Normal file
View File

@ -0,0 +1,198 @@
// SPDX-License-Identifier: GPL-2.0
//! Printing facilities.
//!
//! C header: [`include/linux/printk.h`](../../../../include/linux/printk.h)
//!
//! Reference: <https://www.kernel.org/doc/html/latest/core-api/printk-basics.html>
use core::{
ffi::{c_char, c_void},
fmt,
};
use crate::str::RawFormatter;
#[cfg(CONFIG_PRINTK)]
use crate::bindings;
// Called from `vsprintf` with format specifier `%pA`.
#[no_mangle]
unsafe fn rust_fmt_argument(buf: *mut c_char, end: *mut c_char, ptr: *const c_void) -> *mut c_char {
use fmt::Write;
// SAFETY: The C contract guarantees that `buf` is valid if it's less than `end`.
let mut w = unsafe { RawFormatter::from_ptrs(buf.cast(), end.cast()) };
let _ = w.write_fmt(unsafe { *(ptr as *const fmt::Arguments<'_>) });
w.pos().cast()
}
/// Format strings.
///
/// Public but hidden since it should only be used from public macros.
#[doc(hidden)]
pub mod format_strings {
use crate::bindings;
/// The length we copy from the `KERN_*` kernel prefixes.
const LENGTH_PREFIX: usize = 2;
/// The length of the fixed format strings.
pub const LENGTH: usize = 10;
/// Generates a fixed format string for the kernel's [`_printk`].
///
/// The format string is always the same for a given level, i.e. for a
/// given `prefix`, which are the kernel's `KERN_*` constants.
///
/// [`_printk`]: ../../../../include/linux/printk.h
const fn generate(is_cont: bool, prefix: &[u8; 3]) -> [u8; LENGTH] {
// Ensure the `KERN_*` macros are what we expect.
assert!(prefix[0] == b'\x01');
if is_cont {
assert!(prefix[1] == b'c');
} else {
assert!(prefix[1] >= b'0' && prefix[1] <= b'7');
}
assert!(prefix[2] == b'\x00');
let suffix: &[u8; LENGTH - LENGTH_PREFIX] = if is_cont {
b"%pA\0\0\0\0\0"
} else {
b"%s: %pA\0"
};
[
prefix[0], prefix[1], suffix[0], suffix[1], suffix[2], suffix[3], suffix[4], suffix[5],
suffix[6], suffix[7],
]
}
// Generate the format strings at compile-time.
//
// This avoids the compiler generating the contents on the fly in the stack.
//
// Furthermore, `static` instead of `const` is used to share the strings
// for all the kernel.
pub static EMERG: [u8; LENGTH] = generate(false, bindings::KERN_EMERG);
pub static INFO: [u8; LENGTH] = generate(false, bindings::KERN_INFO);
}
/// Prints a message via the kernel's [`_printk`].
///
/// Public but hidden since it should only be used from public macros.
///
/// # Safety
///
/// The format string must be one of the ones in [`format_strings`], and
/// the module name must be null-terminated.
///
/// [`_printk`]: ../../../../include/linux/_printk.h
#[doc(hidden)]
#[cfg_attr(not(CONFIG_PRINTK), allow(unused_variables))]
pub unsafe fn call_printk(
format_string: &[u8; format_strings::LENGTH],
module_name: &[u8],
args: fmt::Arguments<'_>,
) {
// `_printk` does not seem to fail in any path.
#[cfg(CONFIG_PRINTK)]
unsafe {
bindings::_printk(
format_string.as_ptr() as _,
module_name.as_ptr(),
&args as *const _ as *const c_void,
);
}
}
/// Performs formatting and forwards the string to [`call_printk`].
///
/// Public but hidden since it should only be used from public macros.
#[doc(hidden)]
#[cfg(not(testlib))]
#[macro_export]
#[allow(clippy::crate_in_macro_def)]
macro_rules! print_macro (
// The non-continuation cases (most of them, e.g. `INFO`).
($format_string:path, $($arg:tt)+) => (
// SAFETY: This hidden macro should only be called by the documented
// printing macros which ensure the format string is one of the fixed
// ones. All `__LOG_PREFIX`s are null-terminated as they are generated
// by the `module!` proc macro or fixed values defined in a kernel
// crate.
unsafe {
$crate::print::call_printk(
&$format_string,
crate::__LOG_PREFIX,
format_args!($($arg)+),
);
}
);
);
/// Stub for doctests
#[cfg(testlib)]
#[macro_export]
macro_rules! print_macro (
($format_string:path, $e:expr, $($arg:tt)+) => (
()
);
);
// We could use a macro to generate these macros. However, doing so ends
// up being a bit ugly: it requires the dollar token trick to escape `$` as
// well as playing with the `doc` attribute. Furthermore, they cannot be easily
// imported in the prelude due to [1]. So, for the moment, we just write them
// manually, like in the C side; while keeping most of the logic in another
// macro, i.e. [`print_macro`].
//
// [1]: https://github.com/rust-lang/rust/issues/52234
/// Prints an emergency-level message (level 0).
///
/// Use this level if the system is unusable.
///
/// Equivalent to the kernel's [`pr_emerg`] macro.
///
/// Mimics the interface of [`std::print!`]. See [`core::fmt`] and
/// `alloc::format!` for information about the formatting syntax.
///
/// [`pr_emerg`]: https://www.kernel.org/doc/html/latest/core-api/printk-basics.html#c.pr_emerg
/// [`std::print!`]: https://doc.rust-lang.org/std/macro.print.html
///
/// # Examples
///
/// ```
/// pr_emerg!("hello {}\n", "there");
/// ```
#[macro_export]
macro_rules! pr_emerg (
($($arg:tt)*) => (
$crate::print_macro!($crate::print::format_strings::EMERG, $($arg)*)
)
);
/// Prints an info-level message (level 6).
///
/// Use this level for informational messages.
///
/// Equivalent to the kernel's [`pr_info`] macro.
///
/// Mimics the interface of [`std::print!`]. See [`core::fmt`] and
/// `alloc::format!` for information about the formatting syntax.
///
/// [`pr_info`]: https://www.kernel.org/doc/html/latest/core-api/printk-basics.html#c.pr_info
/// [`std::print!`]: https://doc.rust-lang.org/std/macro.print.html
///
/// # Examples
///
/// ```
/// pr_info!("hello {}\n", "there");
/// ```
#[macro_export]
#[doc(alias = "print")]
macro_rules! pr_info (
($($arg:tt)*) => (
$crate::print_macro!($crate::print::format_strings::INFO, $($arg)*)
)
);

72
rust/kernel/str.rs Normal file
View File

@ -0,0 +1,72 @@
// SPDX-License-Identifier: GPL-2.0
//! String representations.
use core::fmt;
/// Allows formatting of [`fmt::Arguments`] into a raw buffer.
///
/// It does not fail if callers write past the end of the buffer so that they can calculate the
/// size required to fit everything.
///
/// # Invariants
///
/// The memory region between `pos` (inclusive) and `end` (exclusive) is valid for writes if `pos`
/// is less than `end`.
pub(crate) struct RawFormatter {
// Use `usize` to use `saturating_*` functions.
#[allow(dead_code)]
beg: usize,
pos: usize,
end: usize,
}
impl RawFormatter {
/// Creates a new instance of [`RawFormatter`] with the given buffer pointers.
///
/// # Safety
///
/// If `pos` is less than `end`, then the region between `pos` (inclusive) and `end`
/// (exclusive) must be valid for writes for the lifetime of the returned [`RawFormatter`].
pub(crate) unsafe fn from_ptrs(pos: *mut u8, end: *mut u8) -> Self {
// INVARIANT: The safety requierments guarantee the type invariants.
Self {
beg: pos as _,
pos: pos as _,
end: end as _,
}
}
/// Returns the current insert position.
///
/// N.B. It may point to invalid memory.
pub(crate) fn pos(&self) -> *mut u8 {
self.pos as _
}
}
impl fmt::Write for RawFormatter {
fn write_str(&mut self, s: &str) -> fmt::Result {
// `pos` value after writing `len` bytes. This does not have to be bounded by `end`, but we
// don't want it to wrap around to 0.
let pos_new = self.pos.saturating_add(s.len());
// Amount that we can copy. `saturating_sub` ensures we get 0 if `pos` goes past `end`.
let len_to_copy = core::cmp::min(pos_new, self.end).saturating_sub(self.pos);
if len_to_copy > 0 {
// SAFETY: If `len_to_copy` is non-zero, then we know `pos` has not gone past `end`
// yet, so it is valid for write per the type invariants.
unsafe {
core::ptr::copy_nonoverlapping(
s.as_bytes().as_ptr(),
self.pos as *mut u8,
len_to_copy,
)
};
}
self.pos = pos_new;
Ok(())
}
}

51
rust/macros/helpers.rs Normal file
View File

@ -0,0 +1,51 @@
// SPDX-License-Identifier: GPL-2.0
use proc_macro::{token_stream, TokenTree};
pub(crate) fn try_ident(it: &mut token_stream::IntoIter) -> Option<String> {
if let Some(TokenTree::Ident(ident)) = it.next() {
Some(ident.to_string())
} else {
None
}
}
pub(crate) fn try_literal(it: &mut token_stream::IntoIter) -> Option<String> {
if let Some(TokenTree::Literal(literal)) = it.next() {
Some(literal.to_string())
} else {
None
}
}
pub(crate) fn try_byte_string(it: &mut token_stream::IntoIter) -> Option<String> {
try_literal(it).and_then(|byte_string| {
if byte_string.starts_with("b\"") && byte_string.ends_with('\"') {
Some(byte_string[2..byte_string.len() - 1].to_string())
} else {
None
}
})
}
pub(crate) fn expect_ident(it: &mut token_stream::IntoIter) -> String {
try_ident(it).expect("Expected Ident")
}
pub(crate) fn expect_punct(it: &mut token_stream::IntoIter) -> char {
if let TokenTree::Punct(punct) = it.next().expect("Reached end of token stream for Punct") {
punct.as_char()
} else {
panic!("Expected Punct");
}
}
pub(crate) fn expect_byte_string(it: &mut token_stream::IntoIter) -> String {
try_byte_string(it).expect("Expected byte string")
}
pub(crate) fn expect_end(it: &mut token_stream::IntoIter) {
if it.next().is_some() {
panic!("Expected end");
}
}

72
rust/macros/lib.rs Normal file
View File

@ -0,0 +1,72 @@
// SPDX-License-Identifier: GPL-2.0
//! Crate for all kernel procedural macros.
mod helpers;
mod module;
use proc_macro::TokenStream;
/// Declares a kernel module.
///
/// The `type` argument should be a type which implements the [`Module`]
/// trait. Also accepts various forms of kernel metadata.
///
/// C header: [`include/linux/moduleparam.h`](../../../include/linux/moduleparam.h)
///
/// [`Module`]: ../kernel/trait.Module.html
///
/// # Examples
///
/// ```ignore
/// use kernel::prelude::*;
///
/// module!{
/// type: MyModule,
/// name: b"my_kernel_module",
/// author: b"Rust for Linux Contributors",
/// description: b"My very own kernel module!",
/// license: b"GPL",
/// params: {
/// my_i32: i32 {
/// default: 42,
/// permissions: 0o000,
/// description: b"Example of i32",
/// },
/// writeable_i32: i32 {
/// default: 42,
/// permissions: 0o644,
/// description: b"Example of i32",
/// },
/// },
/// }
///
/// struct MyModule;
///
/// impl kernel::Module for MyModule {
/// fn init() -> Result<Self> {
/// // If the parameter is writeable, then the kparam lock must be
/// // taken to read the parameter:
/// {
/// let lock = THIS_MODULE.kernel_param_lock();
/// pr_info!("i32 param is: {}\n", writeable_i32.read(&lock));
/// }
/// // If the parameter is read only, it can be read without locking
/// // the kernel parameters:
/// pr_info!("i32 param is: {}\n", my_i32.read());
/// Ok(Self)
/// }
/// }
/// ```
///
/// # Supported argument types
/// - `type`: type which implements the [`Module`] trait (required).
/// - `name`: byte array of the name of the kernel module (required).
/// - `author`: byte array of the author of the kernel module.
/// - `description`: byte array of the description of the kernel module.
/// - `license`: byte array of the license of the kernel module (required).
/// - `alias`: byte array of alias name of the kernel module.
#[proc_macro]
pub fn module(ts: TokenStream) -> TokenStream {
module::module(ts)
}

282
rust/macros/module.rs Normal file
View File

@ -0,0 +1,282 @@
// SPDX-License-Identifier: GPL-2.0
use crate::helpers::*;
use proc_macro::{token_stream, Literal, TokenStream, TokenTree};
use std::fmt::Write;
struct ModInfoBuilder<'a> {
module: &'a str,
counter: usize,
buffer: String,
}
impl<'a> ModInfoBuilder<'a> {
fn new(module: &'a str) -> Self {
ModInfoBuilder {
module,
counter: 0,
buffer: String::new(),
}
}
fn emit_base(&mut self, field: &str, content: &str, builtin: bool) {
let string = if builtin {
// Built-in modules prefix their modinfo strings by `module.`.
format!(
"{module}.{field}={content}\0",
module = self.module,
field = field,
content = content
)
} else {
// Loadable modules' modinfo strings go as-is.
format!("{field}={content}\0", field = field, content = content)
};
write!(
&mut self.buffer,
"
{cfg}
#[doc(hidden)]
#[link_section = \".modinfo\"]
#[used]
pub static __{module}_{counter}: [u8; {length}] = *{string};
",
cfg = if builtin {
"#[cfg(not(MODULE))]"
} else {
"#[cfg(MODULE)]"
},
module = self.module.to_uppercase(),
counter = self.counter,
length = string.len(),
string = Literal::byte_string(string.as_bytes()),
)
.unwrap();
self.counter += 1;
}
fn emit_only_builtin(&mut self, field: &str, content: &str) {
self.emit_base(field, content, true)
}
fn emit_only_loadable(&mut self, field: &str, content: &str) {
self.emit_base(field, content, false)
}
fn emit(&mut self, field: &str, content: &str) {
self.emit_only_builtin(field, content);
self.emit_only_loadable(field, content);
}
}
#[derive(Debug, Default)]
struct ModuleInfo {
type_: String,
license: String,
name: String,
author: Option<String>,
description: Option<String>,
alias: Option<String>,
}
impl ModuleInfo {
fn parse(it: &mut token_stream::IntoIter) -> Self {
let mut info = ModuleInfo::default();
const EXPECTED_KEYS: &[&str] =
&["type", "name", "author", "description", "license", "alias"];
const REQUIRED_KEYS: &[&str] = &["type", "name", "license"];
let mut seen_keys = Vec::new();
loop {
let key = match it.next() {
Some(TokenTree::Ident(ident)) => ident.to_string(),
Some(_) => panic!("Expected Ident or end"),
None => break,
};
if seen_keys.contains(&key) {
panic!(
"Duplicated key \"{}\". Keys can only be specified once.",
key
);
}
assert_eq!(expect_punct(it), ':');
match key.as_str() {
"type" => info.type_ = expect_ident(it),
"name" => info.name = expect_byte_string(it),
"author" => info.author = Some(expect_byte_string(it)),
"description" => info.description = Some(expect_byte_string(it)),
"license" => info.license = expect_byte_string(it),
"alias" => info.alias = Some(expect_byte_string(it)),
_ => panic!(
"Unknown key \"{}\". Valid keys are: {:?}.",
key, EXPECTED_KEYS
),
}
assert_eq!(expect_punct(it), ',');
seen_keys.push(key);
}
expect_end(it);
for key in REQUIRED_KEYS {
if !seen_keys.iter().any(|e| e == key) {
panic!("Missing required key \"{}\".", key);
}
}
let mut ordered_keys: Vec<&str> = Vec::new();
for key in EXPECTED_KEYS {
if seen_keys.iter().any(|e| e == key) {
ordered_keys.push(key);
}
}
if seen_keys != ordered_keys {
panic!(
"Keys are not ordered as expected. Order them like: {:?}.",
ordered_keys
);
}
info
}
}
pub(crate) fn module(ts: TokenStream) -> TokenStream {
let mut it = ts.into_iter();
let info = ModuleInfo::parse(&mut it);
let mut modinfo = ModInfoBuilder::new(info.name.as_ref());
if let Some(author) = info.author {
modinfo.emit("author", &author);
}
if let Some(description) = info.description {
modinfo.emit("description", &description);
}
modinfo.emit("license", &info.license);
if let Some(alias) = info.alias {
modinfo.emit("alias", &alias);
}
// Built-in modules also export the `file` modinfo string.
let file =
std::env::var("RUST_MODFILE").expect("Unable to fetch RUST_MODFILE environmental variable");
modinfo.emit_only_builtin("file", &file);
format!(
"
/// The module name.
///
/// Used by the printing macros, e.g. [`info!`].
const __LOG_PREFIX: &[u8] = b\"{name}\\0\";
/// The \"Rust loadable module\" mark, for `scripts/is_rust_module.sh`.
//
// This may be best done another way later on, e.g. as a new modinfo
// key or a new section. For the moment, keep it simple.
#[cfg(MODULE)]
#[doc(hidden)]
#[used]
static __IS_RUST_MODULE: () = ();
static mut __MOD: Option<{type_}> = None;
// SAFETY: `__this_module` is constructed by the kernel at load time and will not be
// freed until the module is unloaded.
#[cfg(MODULE)]
static THIS_MODULE: kernel::ThisModule = unsafe {{
kernel::ThisModule::from_ptr(&kernel::bindings::__this_module as *const _ as *mut _)
}};
#[cfg(not(MODULE))]
static THIS_MODULE: kernel::ThisModule = unsafe {{
kernel::ThisModule::from_ptr(core::ptr::null_mut())
}};
// Loadable modules need to export the `{{init,cleanup}}_module` identifiers.
#[cfg(MODULE)]
#[doc(hidden)]
#[no_mangle]
pub extern \"C\" fn init_module() -> core::ffi::c_int {{
__init()
}}
#[cfg(MODULE)]
#[doc(hidden)]
#[no_mangle]
pub extern \"C\" fn cleanup_module() {{
__exit()
}}
// Built-in modules are initialized through an initcall pointer
// and the identifiers need to be unique.
#[cfg(not(MODULE))]
#[cfg(not(CONFIG_HAVE_ARCH_PREL32_RELOCATIONS))]
#[doc(hidden)]
#[link_section = \"{initcall_section}\"]
#[used]
pub static __{name}_initcall: extern \"C\" fn() -> core::ffi::c_int = __{name}_init;
#[cfg(not(MODULE))]
#[cfg(CONFIG_HAVE_ARCH_PREL32_RELOCATIONS)]
core::arch::global_asm!(
r#\".section \"{initcall_section}\", \"a\"
__{name}_initcall:
.long __{name}_init - .
.previous
\"#
);
#[cfg(not(MODULE))]
#[doc(hidden)]
#[no_mangle]
pub extern \"C\" fn __{name}_init() -> core::ffi::c_int {{
__init()
}}
#[cfg(not(MODULE))]
#[doc(hidden)]
#[no_mangle]
pub extern \"C\" fn __{name}_exit() {{
__exit()
}}
fn __init() -> core::ffi::c_int {{
match <{type_} as kernel::Module>::init(&THIS_MODULE) {{
Ok(m) => {{
unsafe {{
__MOD = Some(m);
}}
return 0;
}}
Err(e) => {{
return e.to_kernel_errno();
}}
}}
}}
fn __exit() {{
unsafe {{
// Invokes `drop()` on `__MOD`, which should be used for cleanup.
__MOD = None;
}}
}}
{modinfo}
",
type_ = info.type_,
name = info.name,
modinfo = modinfo.buffer,
initcall_section = ".initcall6.init"
)
.parse()
.expect("Error parsing formatted string into token stream.")
}

2
samples/Kconfig
View File

@ -263,6 +263,8 @@ config SAMPLE_CORESIGHT_SYSCFG
This demonstrates how a user may create their own CoreSight
configurations and easily load them into the system at runtime.
source "samples/rust/Kconfig"
endif # SAMPLES
config HAVE_SAMPLE_FTRACE_DIRECT

1
samples/Makefile
View File

@ -35,3 +35,4 @@ subdir-$(CONFIG_SAMPLE_WATCH_QUEUE) += watch_queue
obj-$(CONFIG_DEBUG_KMEMLEAK_TEST) += kmemleak/
obj-$(CONFIG_SAMPLE_CORESIGHT_SYSCFG) += coresight/
obj-$(CONFIG_SAMPLE_FPROBE) += fprobe/
obj-$(CONFIG_SAMPLES_RUST) += rust/

30
samples/rust/Kconfig Normal file
View File

@ -0,0 +1,30 @@
# SPDX-License-Identifier: GPL-2.0
menuconfig SAMPLES_RUST
bool "Rust samples"
depends on RUST
help
You can build sample Rust kernel code here.
If unsure, say N.
if SAMPLES_RUST
config SAMPLE_RUST_MINIMAL
tristate "Minimal"
help
This option builds the Rust minimal module sample.
To compile this as a module, choose M here:
the module will be called rust_minimal.
If unsure, say N.
config SAMPLE_RUST_HOSTPROGS
bool "Host programs"
help
This option builds the Rust host program samples.
If unsure, say N.
endif # SAMPLES_RUST

5
samples/rust/Makefile Normal file
View File

@ -0,0 +1,5 @@
# SPDX-License-Identifier: GPL-2.0
obj-$(CONFIG_SAMPLE_RUST_MINIMAL) += rust_minimal.o
subdir-$(CONFIG_SAMPLE_RUST_HOSTPROGS) += hostprogs

3
samples/rust/hostprogs/.gitignore vendored Normal file
View File

@ -0,0 +1,3 @@
# SPDX-License-Identifier: GPL-2.0
single

5
samples/rust/hostprogs/Makefile Normal file
View File

@ -0,0 +1,5 @@
# SPDX-License-Identifier: GPL-2.0
hostprogs-always-y := single
single-rust := y

7
samples/rust/hostprogs/a.rs Normal file
View File

@ -0,0 +1,7 @@
// SPDX-License-Identifier: GPL-2.0
//! Rust single host program sample: module `a`.
pub(crate) fn f(x: i32) {
println!("The number is {}.", x);
}

5
samples/rust/hostprogs/b.rs Normal file
View File

@ -0,0 +1,5 @@
// SPDX-License-Identifier: GPL-2.0
//! Rust single host program sample: module `b`.
pub(crate) const CONSTANT: i32 = 42;

12
samples/rust/hostprogs/single.rs Normal file
View File

@ -0,0 +1,12 @@
// SPDX-License-Identifier: GPL-2.0
//! Rust single host program sample.
mod a;
mod b;
fn main() {
println!("Hello world!");
a::f(b::CONSTANT);
}

38
samples/rust/rust_minimal.rs Normal file
View File

@ -0,0 +1,38 @@
// SPDX-License-Identifier: GPL-2.0
//! Rust minimal sample.
use kernel::prelude::*;
module! {
type: RustMinimal,
name: b"rust_minimal",
author: b"Rust for Linux Contributors",
description: b"Rust minimal sample",
license: b"GPL",
}
struct RustMinimal {
numbers: Vec<i32>,
}
impl kernel::Module for RustMinimal {
fn init(_module: &'static ThisModule) -> Result<Self> {
pr_info!("Rust minimal sample (init)\n");
pr_info!("Am I built-in? {}\n", !cfg!(MODULE));
let mut numbers = Vec::new();
numbers.try_push(72)?;
numbers.try_push(108)?;
numbers.try_push(200)?;
Ok(RustMinimal { numbers })
}
}
impl Drop for RustMinimal {
fn drop(&mut self) {
pr_info!("My numbers are {:?}\n", self.numbers);
pr_info!("Rust minimal sample (exit)\n");
}
}

1
scripts/.gitignore vendored
View File

@ -1,6 +1,7 @@
# SPDX-License-Identifier: GPL-2.0-only
/asn1_compiler
/bin2c
/generate_rust_target
/insert-sys-cert
/kallsyms
/module.lds

6
scripts/Kconfig.include
View File

@ -36,12 +36,12 @@ ld-option = $(success,$(LD) -v $(1))
as-instr = $(success,printf "%b\n" "$(1)" | $(CC) $(CLANG_FLAGS) -c -x assembler -o /dev/null -)
# check if $(CC) and $(LD) exist
$(error-if,$(failure,command -v $(CC)),compiler '$(CC)' not found)
$(error-if,$(failure,command -v $(CC)),C compiler '$(CC)' not found)
$(error-if,$(failure,command -v $(LD)),linker '$(LD)' not found)
# Get the compiler name, version, and error out if it is not supported.
# Get the C compiler name, version, and error out if it is not supported.
cc-info := $(shell,$(srctree)/scripts/cc-version.sh $(CC))
$(error-if,$(success,test -z "$(cc-info)"),Sorry$(comma) this compiler is not supported.)
$(error-if,$(success,test -z "$(cc-info)"),Sorry$(comma) this C compiler is not supported.)
cc-name := $(shell,set -- $(cc-info) && echo $1)
cc-version := $(shell,set -- $(cc-info) && echo $2)

3
scripts/Makefile
View File

@ -10,6 +10,9 @@ hostprogs-always-$(CONFIG_BUILDTIME_TABLE_SORT) += sorttable
hostprogs-always-$(CONFIG_ASN1) += asn1_compiler
hostprogs-always-$(CONFIG_MODULE_SIG_FORMAT) += sign-file
hostprogs-always-$(CONFIG_SYSTEM_EXTRA_CERTIFICATE) += insert-sys-cert
hostprogs-always-$(CONFIG_RUST) += generate_rust_target
generate_rust_target-rust := y
HOSTCFLAGS_sorttable.o = -I$(srctree)/tools/include
HOSTLDLIBS_sorttable = -lpthread

60
scripts/Makefile.build
View File

@ -26,6 +26,7 @@ EXTRA_CPPFLAGS :=
EXTRA_LDFLAGS :=
asflags-y :=
ccflags-y :=
rustflags-y :=
cppflags-y :=
ldflags-y :=
@ -271,6 +272,65 @@ quiet_cmd_cc_lst_c = MKLST $@
$(obj)/%.lst: $(src)/%.c FORCE
$(call if_changed_dep,cc_lst_c)
# Compile Rust sources (.rs)
# ---------------------------------------------------------------------------
rust_allowed_features := core_ffi_c
rust_common_cmd = \
RUST_MODFILE=$(modfile) $(RUSTC_OR_CLIPPY) $(rust_flags) \
-Zallow-features=$(rust_allowed_features) \
-Zcrate-attr=no_std \
-Zcrate-attr='feature($(rust_allowed_features))' \
--extern alloc --extern kernel \
--crate-type rlib --out-dir $(obj) -L $(objtree)/rust/ \
--crate-name $(basename $(notdir $@))
rust_handle_depfile = \
mv $(obj)/$(basename $(notdir $@)).d $(depfile); \
sed -i '/^\#/d' $(depfile)
# `--emit=obj`, `--emit=asm` and `--emit=llvm-ir` imply a single codegen unit
# will be used. We explicitly request `-Ccodegen-units=1` in any case, and
# the compiler shows a warning if it is not 1. However, if we ever stop
# requesting it explicitly and we start using some other `--emit` that does not
# imply it (and for which codegen is performed), then we would be out of sync,
# i.e. the outputs we would get for the different single targets (e.g. `.ll`)
# would not match each other.
quiet_cmd_rustc_o_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@
cmd_rustc_o_rs = \
$(rust_common_cmd) --emit=dep-info,obj $<; \
$(rust_handle_depfile)
$(obj)/%.o: $(src)/%.rs FORCE
$(call if_changed_dep,rustc_o_rs)
quiet_cmd_rustc_rsi_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@
cmd_rustc_rsi_rs = \
$(rust_common_cmd) --emit=dep-info -Zunpretty=expanded $< >$@; \
command -v $(RUSTFMT) >/dev/null && $(RUSTFMT) $@; \
$(rust_handle_depfile)
$(obj)/%.rsi: $(src)/%.rs FORCE
$(call if_changed_dep,rustc_rsi_rs)
quiet_cmd_rustc_s_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@
cmd_rustc_s_rs = \
$(rust_common_cmd) --emit=dep-info,asm $<; \
$(rust_handle_depfile)
$(obj)/%.s: $(src)/%.rs FORCE
$(call if_changed_dep,rustc_s_rs)
quiet_cmd_rustc_ll_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@
cmd_rustc_ll_rs = \
$(rust_common_cmd) --emit=dep-info,llvm-ir $<; \
$(rust_handle_depfile)
$(obj)/%.ll: $(src)/%.rs FORCE
$(call if_changed_dep,rustc_ll_rs)
# Compile assembler sources (.S)
# ---------------------------------------------------------------------------

8
scripts/Makefile.debug
View File

@ -1,4 +1,6 @@
DEBUG_CFLAGS :=
DEBUG_RUSTFLAGS :=
debug-flags-y := -g
ifdef CONFIG_DEBUG_INFO_SPLIT
@ -17,9 +19,12 @@ KBUILD_AFLAGS += $(debug-flags-y)
ifdef CONFIG_DEBUG_INFO_REDUCED
DEBUG_CFLAGS += -fno-var-tracking
DEBUG_RUSTFLAGS += -Cdebuginfo=1
ifdef CONFIG_CC_IS_GCC
DEBUG_CFLAGS += -femit-struct-debug-baseonly
endif
else
DEBUG_RUSTFLAGS += -Cdebuginfo=2
endif
ifdef CONFIG_DEBUG_INFO_COMPRESSED
@ -30,3 +35,6 @@ endif
KBUILD_CFLAGS += $(DEBUG_CFLAGS)
export DEBUG_CFLAGS
KBUILD_RUSTFLAGS += $(DEBUG_RUSTFLAGS)
export DEBUG_RUSTFLAGS

34
scripts/Makefile.host
View File

@ -22,6 +22,8 @@ $(obj)/%.tab.c $(obj)/%.tab.h: $(src)/%.y FORCE
# to preprocess a data file.
#
# Both C and C++ are supported, but preferred language is C for such utilities.
# Rust is also supported, but it may only be used in scenarios where a Rust
# toolchain is required to be available (e.g. when `CONFIG_RUST` is enabled).
#
# Sample syntax (see Documentation/kbuild/makefiles.rst for reference)
# hostprogs := bin2hex
@ -37,15 +39,20 @@ $(obj)/%.tab.c $(obj)/%.tab.h: $(src)/%.y FORCE
# qconf-objs := menu.o
# Will compile qconf as a C++ program, and menu as a C program.
# They are linked as C++ code to the executable qconf
#
# hostprogs := target
# target-rust := y
# Will compile `target` as a Rust program, using `target.rs` as the crate root.
# The crate may consist of several source files.
# C code
# Executables compiled from a single .c file
host-csingle := $(foreach m,$(hostprogs), \
$(if $($(m)-objs)$($(m)-cxxobjs),,$(m)))
$(if $($(m)-objs)$($(m)-cxxobjs)$($(m)-rust),,$(m)))
# C executables linked based on several .o files
host-cmulti := $(foreach m,$(hostprogs),\
$(if $($(m)-cxxobjs),,$(if $($(m)-objs),$(m))))
$(if $($(m)-cxxobjs)$($(m)-rust),,$(if $($(m)-objs),$(m))))
# Object (.o) files compiled from .c files
host-cobjs := $(sort $(foreach m,$(hostprogs),$($(m)-objs)))
@ -58,11 +65,17 @@ host-cxxmulti := $(foreach m,$(hostprogs),$(if $($(m)-cxxobjs),$(m)))
# C++ Object (.o) files compiled from .cc files
host-cxxobjs := $(sort $(foreach m,$(host-cxxmulti),$($(m)-cxxobjs)))
# Rust code
# Executables compiled from a single Rust crate (which may consist of
# one or more .rs files)
host-rust := $(foreach m,$(hostprogs),$(if $($(m)-rust),$(m)))
host-csingle := $(addprefix $(obj)/,$(host-csingle))
host-cmulti := $(addprefix $(obj)/,$(host-cmulti))
host-cobjs := $(addprefix $(obj)/,$(host-cobjs))
host-cxxmulti := $(addprefix $(obj)/,$(host-cxxmulti))
host-cxxobjs := $(addprefix $(obj)/,$(host-cxxobjs))
host-rust := $(addprefix $(obj)/,$(host-rust))
#####
# Handle options to gcc. Support building with separate output directory
@ -71,6 +84,8 @@ _hostc_flags = $(KBUILD_HOSTCFLAGS) $(HOST_EXTRACFLAGS) \
$(HOSTCFLAGS_$(target-stem).o)
_hostcxx_flags = $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
$(HOSTCXXFLAGS_$(target-stem).o)
_hostrust_flags = $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \
$(HOSTRUSTFLAGS_$(target-stem))
# $(objtree)/$(obj) for including generated headers from checkin source files
ifeq ($(KBUILD_EXTMOD),)
@ -82,6 +97,7 @@ endif
hostc_flags = -Wp,-MMD,$(depfile) $(_hostc_flags)
hostcxx_flags = -Wp,-MMD,$(depfile) $(_hostcxx_flags)
hostrust_flags = $(_hostrust_flags)
#####
# Compile programs on the host
@ -128,5 +144,17 @@ quiet_cmd_host-cxxobjs = HOSTCXX $@
$(host-cxxobjs): $(obj)/%.o: $(src)/%.cc FORCE
$(call if_changed_dep,host-cxxobjs)
# Create executable from a single Rust crate (which may consist of
# one or more `.rs` files)
# host-rust -> Executable
quiet_cmd_host-rust = HOSTRUSTC $@
cmd_host-rust = \
$(HOSTRUSTC) $(hostrust_flags) --emit=dep-info,link \
--out-dir=$(obj)/ $<; \
mv $(obj)/$(target-stem).d $(depfile); \
sed -i '/^\#/d' $(depfile)
$(host-rust): $(obj)/%: $(src)/%.rs FORCE
$(call if_changed_dep,host-rust)
targets += $(host-csingle) $(host-cmulti) $(host-cobjs) \
$(host-cxxmulti) $(host-cxxobjs)
$(host-cxxmulti) $(host-cxxobjs) $(host-rust)

12
scripts/Makefile.lib
View File

@ -8,6 +8,7 @@ ldflags-y += $(EXTRA_LDFLAGS)
# flags that take effect in current and sub directories
KBUILD_AFLAGS += $(subdir-asflags-y)
KBUILD_CFLAGS += $(subdir-ccflags-y)
KBUILD_RUSTFLAGS += $(subdir-rustflags-y)
# Figure out what we need to build from the various variables
# ===========================================================================
@ -128,6 +129,10 @@ _c_flags = $(filter-out $(CFLAGS_REMOVE_$(target-stem).o), \
$(filter-out $(ccflags-remove-y), \
$(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS) $(ccflags-y)) \
$(CFLAGS_$(target-stem).o))
_rust_flags = $(filter-out $(RUSTFLAGS_REMOVE_$(target-stem).o), \
$(filter-out $(rustflags-remove-y), \
$(KBUILD_RUSTFLAGS) $(rustflags-y)) \
$(RUSTFLAGS_$(target-stem).o))
_a_flags = $(filter-out $(AFLAGS_REMOVE_$(target-stem).o), \
$(filter-out $(asflags-remove-y), \
$(KBUILD_CPPFLAGS) $(KBUILD_AFLAGS) $(asflags-y)) \
@ -202,6 +207,11 @@ modkern_cflags = \
$(KBUILD_CFLAGS_MODULE) $(CFLAGS_MODULE), \
$(KBUILD_CFLAGS_KERNEL) $(CFLAGS_KERNEL) $(modfile_flags))
modkern_rustflags = \
$(if $(part-of-module), \
$(KBUILD_RUSTFLAGS_MODULE) $(RUSTFLAGS_MODULE), \
$(KBUILD_RUSTFLAGS_KERNEL) $(RUSTFLAGS_KERNEL))
modkern_aflags = $(if $(part-of-module), \
$(KBUILD_AFLAGS_MODULE) $(AFLAGS_MODULE), \
$(KBUILD_AFLAGS_KERNEL) $(AFLAGS_KERNEL))
@ -211,6 +221,8 @@ c_flags = -Wp,-MMD,$(depfile) $(NOSTDINC_FLAGS) $(LINUXINCLUDE) \
$(_c_flags) $(modkern_cflags) \
$(basename_flags) $(modname_flags)
rust_flags = $(_rust_flags) $(modkern_rustflags) @$(objtree)/include/generated/rustc_cfg
a_flags = -Wp,-MMD,$(depfile) $(NOSTDINC_FLAGS) $(LINUXINCLUDE) \
$(_a_flags) $(modkern_aflags)

8
scripts/Makefile.modfinal
View File

@ -39,11 +39,13 @@ quiet_cmd_ld_ko_o = LD [M] $@
quiet_cmd_btf_ko = BTF [M] $@
cmd_btf_ko = \
if [ -f vmlinux ]; then \
if [ ! -f vmlinux ]; then \
printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \
elif [ -n "$(CONFIG_RUST)" ] && $(srctree)/scripts/is_rust_module.sh $@; then \
printf "Skipping BTF generation for %s because it's a Rust module\n" $@ 1>&2; \
else \
LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) --btf_base vmlinux $@; \
$(RESOLVE_BTFIDS) -b vmlinux $@; \
else \
printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \
fi;
# Same as newer-prereqs, but allows to exclude specified extra dependencies

12
scripts/cc-version.sh
View File

@ -1,13 +1,13 @@
#!/bin/sh
# SPDX-License-Identifier: GPL-2.0
#
# Print the compiler name and its version in a 5 or 6-digit form.
# Print the C compiler name and its version in a 5 or 6-digit form.
# Also, perform the minimum version check.
set -e
# Print the compiler name and some version components.
get_compiler_info()
# Print the C compiler name and some version components.
get_c_compiler_info()
{
cat <<- EOF | "$@" -E -P -x c - 2>/dev/null
#if defined(__clang__)
@ -32,7 +32,7 @@ get_canonical_version()
# $@ instead of $1 because multiple words might be given, e.g. CC="ccache gcc".
orig_args="$@"
set -- $(get_compiler_info "$@")
set -- $(get_c_compiler_info "$@")
name=$1
@ -52,7 +52,7 @@ ICC)
min_version=$($min_tool_version icc)
;;
*)
echo "$orig_args: unknown compiler" >&2
echo "$orig_args: unknown C compiler" >&2
exit 1
;;
esac
@ -62,7 +62,7 @@ min_cversion=$(get_canonical_version $min_version)
if [ "$cversion" -lt "$min_cversion" ]; then
echo >&2 "***"
echo >&2 "*** Compiler is too old."
echo >&2 "*** C compiler is too old."
echo >&2 "*** Your $name version: $version"
echo >&2 "*** Minimum $name version: $min_version"
echo >&2 "***"

12
scripts/checkpatch.pl
View File

@ -3616,7 +3616,7 @@ sub process {
my $comment = "";
if ($realfile =~ /\.(h|s|S)$/) {
$comment = '/*';
} elsif ($realfile =~ /\.(c|dts|dtsi)$/) {
} elsif ($realfile =~ /\.(c|rs|dts|dtsi)$/) {
$comment = '//';
} elsif (($checklicenseline == 2) || $realfile =~ /\.(sh|pl|py|awk|tc|yaml)$/) {
$comment = '#';
@ -3664,7 +3664,7 @@ sub process {
}
# check we are in a valid source file if not then ignore this hunk
next if ($realfile !~ /\.(h|c|s|S|sh|dtsi|dts)$/);
next if ($realfile !~ /\.(h|c|rs|s|S|sh|dtsi|dts)$/);
# check for using SPDX-License-Identifier on the wrong line number
if ($realline != $checklicenseline &&
@ -6783,15 +6783,19 @@ sub process {
}
if ($bad_specifier ne "") {
my $stat_real = get_stat_real($linenr, $lc);
my $msg_level = \&WARN;
my $ext_type = "Invalid";
my $use = "";
if ($bad_specifier =~ /p[Ff]/) {
$use = " - use %pS instead";
$use =~ s/pS/ps/ if ($bad_specifier =~ /pf/);
} elsif ($bad_specifier =~ /pA/) {
$use = " - '%pA' is only intended to be used from Rust code";
$msg_level = \&ERROR;
}
WARN("VSPRINTF_POINTER_EXTENSION",
"$ext_type vsprintf pointer extension '$bad_specifier'$use\n" . "$here\n$stat_real\n");
&{$msg_level}("VSPRINTF_POINTER_EXTENSION",
"$ext_type vsprintf pointer extension '$bad_specifier'$use\n" . "$here\n$stat_real\n");
}
}
}

14
scripts/decode_stacktrace.sh
View File

@ -8,6 +8,14 @@ usage() {
echo " $0 -r <release> | <vmlinux> [<base path>|auto] [<modules path>]"
}
# Try to find a Rust demangler
if type llvm-cxxfilt >/dev/null 2>&1 ; then
cppfilt=llvm-cxxfilt
elif type c++filt >/dev/null 2>&1 ; then
cppfilt=c++filt
cppfilt_opts=-i
fi
if [[ $1 == "-r" ]] ; then
vmlinux=""
basepath="auto"
@ -180,6 +188,12 @@ parse_symbol() {
# In the case of inlines, move everything to same line
code=${code//$'\n'/' '}
# Demangle if the name looks like a Rust symbol and if
# we got a Rust demangler
if [[ $name =~ ^_R && $cppfilt != "" ]] ; then
name=$("$cppfilt" "$cppfilt_opts" "$name")
fi
# Replace old address with pretty line numbers
symbol="$segment$name ($code)"
}

135
scripts/generate_rust_analyzer.py Executable file
View File

@ -0,0 +1,135 @@
#!/usr/bin/env python3
# SPDX-License-Identifier: GPL-2.0
"""generate_rust_analyzer - Generates the `rust-project.json` file for `rust-analyzer`.
"""
import argparse
import json
import logging
import pathlib
import sys
def generate_crates(srctree, objtree, sysroot_src):
# Generate the configuration list.
cfg = []
with open(objtree / "include" / "generated" / "rustc_cfg") as fd:
for line in fd:
line = line.replace("--cfg=", "")
line = line.replace("\n", "")
cfg.append(line)
# Now fill the crates list -- dependencies need to come first.
#
# Avoid O(n^2) iterations by keeping a map of indexes.
crates = []
crates_indexes = {}
def append_crate(display_name, root_module, deps, cfg=[], is_workspace_member=True, is_proc_macro=False):
crates_indexes[display_name] = len(crates)
crates.append({
"display_name": display_name,
"root_module": str(root_module),
"is_workspace_member": is_workspace_member,
"is_proc_macro": is_proc_macro,
"deps": [{"crate": crates_indexes[dep], "name": dep} for dep in deps],
"cfg": cfg,
"edition": "2021",
"env": {
"RUST_MODFILE": "This is only for rust-analyzer"
}
})
# First, the ones in `rust/` since they are a bit special.
append_crate(
"core",
sysroot_src / "core" / "src" / "lib.rs",
[],
is_workspace_member=False,
)
append_crate(
"compiler_builtins",
srctree / "rust" / "compiler_builtins.rs",
[],
)
append_crate(
"alloc",
srctree / "rust" / "alloc" / "lib.rs",
["core", "compiler_builtins"],
)
append_crate(
"macros",
srctree / "rust" / "macros" / "lib.rs",
[],
is_proc_macro=True,
)
crates[-1]["proc_macro_dylib_path"] = "rust/libmacros.so"
append_crate(
"bindings",
srctree / "rust"/ "bindings" / "lib.rs",
["core"],
cfg=cfg,
)
crates[-1]["env"]["OBJTREE"] = str(objtree.resolve(True))
append_crate(
"kernel",
srctree / "rust" / "kernel" / "lib.rs",
["core", "alloc", "macros", "bindings"],
cfg=cfg,
)
crates[-1]["source"] = {
"include_dirs": [
str(srctree / "rust" / "kernel"),
str(objtree / "rust")
],
"exclude_dirs": [],
}
# Then, the rest outside of `rust/`.
#
# We explicitly mention the top-level folders we want to cover.
for folder in ("samples", "drivers"):
for path in (srctree / folder).rglob("*.rs"):
logging.info("Checking %s", path)
name = path.name.replace(".rs", "")
# Skip those that are not crate roots.
if f"{name}.o" not in open(path.parent / "Makefile").read():
continue
logging.info("Adding %s", name)
append_crate(
name,
path,
["core", "alloc", "kernel"],
cfg=cfg,
)
return crates
def main():
parser = argparse.ArgumentParser()
parser.add_argument('--verbose', '-v', action='store_true')
parser.add_argument("srctree", type=pathlib.Path)
parser.add_argument("objtree", type=pathlib.Path)
parser.add_argument("sysroot_src", type=pathlib.Path)
args = parser.parse_args()
logging.basicConfig(
format="[%(asctime)s] [%(levelname)s] %(message)s",
level=logging.INFO if args.verbose else logging.WARNING
)
rust_project = {
"crates": generate_crates(args.srctree, args.objtree, args.sysroot_src),
"sysroot_src": str(args.sysroot_src),
}
json.dump(rust_project, sys.stdout, sort_keys=True, indent=4)
if __name__ == "__main__":
main()

182
scripts/generate_rust_target.rs Normal file
View File

@ -0,0 +1,182 @@
// SPDX-License-Identifier: GPL-2.0
//! The custom target specification file generator for `rustc`.
//!
//! To configure a target from scratch, a JSON-encoded file has to be passed
//! to `rustc` (introduced in [RFC 131]). These options and the file itself are
//! unstable. Eventually, `rustc` should provide a way to do this in a stable
//! manner. For instance, via command-line arguments. Therefore, this file
//! should avoid using keys which can be set via `-C` or `-Z` options.
//!
//! [RFC 131]: https://rust-lang.github.io/rfcs/0131-target-specification.html
use std::{
collections::HashMap,
fmt::{Display, Formatter, Result},
io::BufRead,
};
enum Value {
Boolean(bool),
Number(i32),
String(String),
Object(Object),
}
type Object = Vec<(String, Value)>;
/// Minimal "almost JSON" generator (e.g. no `null`s, no arrays, no escaping),
/// enough for this purpose.
impl Display for Value {
fn fmt(&self, formatter: &mut Formatter<'_>) -> Result {
match self {
Value::Boolean(boolean) => write!(formatter, "{}", boolean),
Value::Number(number) => write!(formatter, "{}", number),
Value::String(string) => write!(formatter, "\"{}\"", string),
Value::Object(object) => {
formatter.write_str("{")?;
if let [ref rest @ .., ref last] = object[..] {
for (key, value) in rest {
write!(formatter, "\"{}\": {},", key, value)?;
}
write!(formatter, "\"{}\": {}", last.0, last.1)?;
}
formatter.write_str("}")
}
}
}
}
struct TargetSpec(Object);
impl TargetSpec {
fn new() -> TargetSpec {
TargetSpec(Vec::new())
}
}
trait Push<T> {
fn push(&mut self, key: &str, value: T);
}
impl Push<bool> for TargetSpec {
fn push(&mut self, key: &str, value: bool) {
self.0.push((key.to_string(), Value::Boolean(value)));
}
}
impl Push<i32> for TargetSpec {
fn push(&mut self, key: &str, value: i32) {
self.0.push((key.to_string(), Value::Number(value)));
}
}
impl Push<String> for TargetSpec {
fn push(&mut self, key: &str, value: String) {
self.0.push((key.to_string(), Value::String(value)));
}
}
impl Push<&str> for TargetSpec {
fn push(&mut self, key: &str, value: &str) {
self.push(key, value.to_string());
}
}
impl Push<Object> for TargetSpec {
fn push(&mut self, key: &str, value: Object) {
self.0.push((key.to_string(), Value::Object(value)));
}
}
impl Display for TargetSpec {
fn fmt(&self, formatter: &mut Formatter<'_>) -> Result {
// We add some newlines for clarity.
formatter.write_str("{\n")?;
if let [ref rest @ .., ref last] = self.0[..] {
for (key, value) in rest {
write!(formatter, " \"{}\": {},\n", key, value)?;
}
write!(formatter, " \"{}\": {}\n", last.0, last.1)?;
}
formatter.write_str("}")
}
}
struct KernelConfig(HashMap<String, String>);
impl KernelConfig {
/// Parses `include/config/auto.conf` from `stdin`.
fn from_stdin() -> KernelConfig {
let mut result = HashMap::new();
let stdin = std::io::stdin();
let mut handle = stdin.lock();
let mut line = String::new();
loop {
line.clear();
if handle.read_line(&mut line).unwrap() == 0 {
break;
}
if line.starts_with('#') {
continue;
}
let (key, value) = line.split_once('=').expect("Missing `=` in line.");
result.insert(key.to_string(), value.trim_end_matches('\n').to_string());
}
KernelConfig(result)
}
/// Does the option exist in the configuration (any value)?
///
/// The argument must be passed without the `CONFIG_` prefix.
/// This avoids repetition and it also avoids `fixdep` making us
/// depend on it.
fn has(&self, option: &str) -> bool {
let option = "CONFIG_".to_owned() + option;
self.0.contains_key(&option)
}
}
fn main() {
let cfg = KernelConfig::from_stdin();
let mut ts = TargetSpec::new();
// `llvm-target`s are taken from `scripts/Makefile.clang`.
if cfg.has("X86_64") {
ts.push("arch", "x86_64");
ts.push(
"data-layout",
"e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128",
);
let mut features = "-3dnow,-3dnowa,-mmx,+soft-float".to_string();
if cfg.has("RETPOLINE") {
features += ",+retpoline-external-thunk";
}
ts.push("features", features);
ts.push("llvm-target", "x86_64-linux-gnu");
ts.push("target-pointer-width", "64");
} else {
panic!("Unsupported architecture");
}
ts.push("emit-debug-gdb-scripts", false);
ts.push("frame-pointer", "may-omit");
ts.push(
"stack-probes",
vec![("kind".to_string(), Value::String("none".to_string()))],
);
// Everything else is LE, whether `CPU_LITTLE_ENDIAN` is declared or not
// (e.g. x86). It is also `rustc`'s default.
if cfg.has("CPU_BIG_ENDIAN") {
ts.push("target-endian", "big");
}
println!("{}", ts);
}

16
scripts/is_rust_module.sh Executable file
View File

@ -0,0 +1,16 @@
#!/bin/sh
# SPDX-License-Identifier: GPL-2.0
#
# is_rust_module.sh module.ko
#
# Returns `0` if `module.ko` is a Rust module, `1` otherwise.
set -e
# Using the `16_` prefix ensures other symbols with the same substring
# are not picked up (even if it would be unlikely). The last part is
# used just in case LLVM decides to use the `.` suffix.
#
# In the future, checking for the `.comment` section may be another
# option, see https://github.com/rust-lang/rust/pull/97550.
${NM} "$*" | grep -qE '^[0-9a-fA-F]+ r _R[^[:space:]]+16___IS_RUST_MODULE[^[:space:]]*$'

53
scripts/kallsyms.c
View File

@ -27,7 +27,23 @@
#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof(arr[0]))
#define KSYM_NAME_LEN 128
#define _stringify_1(x) #x
#define _stringify(x) _stringify_1(x)
#define KSYM_NAME_LEN 512
/*
* A substantially bigger size than the current maximum.
*
* It cannot be defined as an expression because it gets stringified
* for the fscanf() format string. Therefore, a _Static_assert() is
* used instead to maintain the relationship with KSYM_NAME_LEN.
*/
#define KSYM_NAME_LEN_BUFFER 2048
_Static_assert(
KSYM_NAME_LEN_BUFFER == KSYM_NAME_LEN * 4,
"Please keep KSYM_NAME_LEN_BUFFER in sync with KSYM_NAME_LEN"
);
struct sym_entry {
unsigned long long addr;
@ -198,15 +214,15 @@ static void check_symbol_range(const char *sym, unsigned long long addr,
static struct sym_entry *read_symbol(FILE *in)
{
char name[500], type;
char name[KSYM_NAME_LEN_BUFFER+1], type;
unsigned long long addr;
unsigned int len;
struct sym_entry *sym;
int rc;
rc = fscanf(in, "%llx %c %499s\n", &addr, &type, name);
rc = fscanf(in, "%llx %c %" _stringify(KSYM_NAME_LEN_BUFFER) "s\n", &addr, &type, name);
if (rc != 3) {
if (rc != EOF && fgets(name, 500, in) == NULL)
if (rc != EOF && fgets(name, ARRAY_SIZE(name), in) == NULL)
fprintf(stderr, "Read error or end of file.\n");
return NULL;
}
@ -471,12 +487,35 @@ static void write_src(void)
if ((i & 0xFF) == 0)
markers[i >> 8] = off;
printf("\t.byte 0x%02x", table[i]->len);
/* There cannot be any symbol of length zero. */
if (table[i]->len == 0) {
fprintf(stderr, "kallsyms failure: "
"unexpected zero symbol length\n");
exit(EXIT_FAILURE);
}
/* Only lengths that fit in up-to-two-byte ULEB128 are supported. */
if (table[i]->len > 0x3FFF) {
fprintf(stderr, "kallsyms failure: "
"unexpected huge symbol length\n");
exit(EXIT_FAILURE);
}
/* Encode length with ULEB128. */
if (table[i]->len <= 0x7F) {
/* Most symbols use a single byte for the length. */
printf("\t.byte 0x%02x", table[i]->len);
off += table[i]->len + 1;
} else {
/* "Big" symbols use two bytes. */
printf("\t.byte 0x%02x, 0x%02x",
(table[i]->len & 0x7F) | 0x80,
(table[i]->len >> 7) & 0x7F);
off += table[i]->len + 2;
}
for (k = 0; k < table[i]->len; k++)
printf(", 0x%02x", table[i]->sym[k]);
printf("\n");
off += table[i]->len + 1;
}
printf("\n");

75
scripts/kconfig/confdata.c
View File

@ -216,6 +216,13 @@ static const char *conf_get_autoheader_name(void)
return name ? name : "include/generated/autoconf.h";
}
static const char *conf_get_rustccfg_name(void)
{
char *name = getenv("KCONFIG_RUSTCCFG");
return name ? name : "include/generated/rustc_cfg";
}
static int conf_set_sym_val(struct symbol *sym, int def, int def_flags, char *p)
{
char *p2;
@ -605,6 +612,9 @@ static const struct comment_style comment_style_c = {
static void conf_write_heading(FILE *fp, const struct comment_style *cs)
{
if (!cs)
return;
fprintf(fp, "%s\n", cs->prefix);
fprintf(fp, "%s Automatically generated file; DO NOT EDIT.\n",
@ -745,6 +755,65 @@ static void print_symbol_for_c(FILE *fp, struct symbol *sym)
free(escaped);
}
static void print_symbol_for_rustccfg(FILE *fp, struct symbol *sym)
{
const char *val;
const char *val_prefix = "";
char *val_prefixed = NULL;
size_t val_prefixed_len;
char *escaped = NULL;
if (sym->type == S_UNKNOWN)
return;
val = sym_get_string_value(sym);
switch (sym->type) {
case S_BOOLEAN:
case S_TRISTATE:
/*
* We do not care about disabled ones, i.e. no need for
* what otherwise are "comments" in other printers.
*/
if (*val == 'n')
return;
/*
* To have similar functionality to the C macro `IS_ENABLED()`
* we provide an empty `--cfg CONFIG_X` here in both `y`
* and `m` cases.
*
* Then, the common `fprintf()` below will also give us
* a `--cfg CONFIG_X="y"` or `--cfg CONFIG_X="m"`, which can
* be used as the equivalent of `IS_BUILTIN()`/`IS_MODULE()`.
*/
fprintf(fp, "--cfg=%s%s\n", CONFIG_, sym->name);
break;
case S_HEX:
if (val[0] != '0' || (val[1] != 'x' && val[1] != 'X'))
val_prefix = "0x";
break;
default:
break;
}
if (strlen(val_prefix) > 0) {
val_prefixed_len = strlen(val) + strlen(val_prefix) + 1;
val_prefixed = xmalloc(val_prefixed_len);
snprintf(val_prefixed, val_prefixed_len, "%s%s", val_prefix, val);
val = val_prefixed;
}
/* All values get escaped: the `--cfg` option only takes strings */
escaped = escape_string_value(val);
val = escaped;
fprintf(fp, "--cfg=%s%s=%s\n", CONFIG_, sym->name, val);
free(escaped);
free(val_prefixed);
}
/*
* Write out a minimal config.
* All values that has default values are skipped as this is redundant.
@ -1132,6 +1201,12 @@ int conf_write_autoconf(int overwrite)
if (ret)
return ret;
ret = __conf_write_autoconf(conf_get_rustccfg_name(),
print_symbol_for_rustccfg,
NULL);
if (ret)
return ret;
/*
* Create include/config/auto.conf. This must be the last step because
* Kbuild has a dependency on auto.conf and this marks the successful

6
scripts/min-tool-version.sh
View File

@ -30,6 +30,12 @@ llvm)
echo 11.0.0
fi
;;
rustc)
echo 1.62.0
;;
bindgen)
echo 0.56.0
;;
*)
echo "$1: unknown tool" >&2
exit 1

160
scripts/rust_is_available.sh Executable file
View File

@ -0,0 +1,160 @@
#!/bin/sh
# SPDX-License-Identifier: GPL-2.0
#
# Tests whether a suitable Rust toolchain is available.
#
# Pass `-v` for human output and more checks (as warnings).
set -e
min_tool_version=$(dirname $0)/min-tool-version.sh
# Convert the version string x.y.z to a canonical up-to-7-digits form.
#
# Note that this function uses one more digit (compared to other
# instances in other version scripts) to give a bit more space to
# `rustc` since it will reach 1.100.0 in late 2026.
get_canonical_version()
{
IFS=.
set -- $1
echo $((100000 * $1 + 100 * $2 + $3))
}
# Check that the Rust compiler exists.
if ! command -v "$RUSTC" >/dev/null; then
if [ "$1" = -v ]; then
echo >&2 "***"
echo >&2 "*** Rust compiler '$RUSTC' could not be found."
echo >&2 "***"
fi
exit 1
fi
# Check that the Rust bindings generator exists.
if ! command -v "$BINDGEN" >/dev/null; then
if [ "$1" = -v ]; then
echo >&2 "***"
echo >&2 "*** Rust bindings generator '$BINDGEN' could not be found."
echo >&2 "***"
fi
exit 1
fi
# Check that the Rust compiler version is suitable.
#
# Non-stable and distributions' versions may have a version suffix, e.g. `-dev`.
rust_compiler_version=$( \
LC_ALL=C "$RUSTC" --version 2>/dev/null \
| head -n 1 \
| grep -oE '[0-9]+\.[0-9]+\.[0-9]+' \
)
rust_compiler_min_version=$($min_tool_version rustc)
rust_compiler_cversion=$(get_canonical_version $rust_compiler_version)
rust_compiler_min_cversion=$(get_canonical_version $rust_compiler_min_version)
if [ "$rust_compiler_cversion" -lt "$rust_compiler_min_cversion" ]; then
if [ "$1" = -v ]; then
echo >&2 "***"
echo >&2 "*** Rust compiler '$RUSTC' is too old."
echo >&2 "*** Your version: $rust_compiler_version"
echo >&2 "*** Minimum version: $rust_compiler_min_version"
echo >&2 "***"
fi
exit 1
fi
if [ "$1" = -v ] && [ "$rust_compiler_cversion" -gt "$rust_compiler_min_cversion" ]; then
echo >&2 "***"
echo >&2 "*** Rust compiler '$RUSTC' is too new. This may or may not work."
echo >&2 "*** Your version: $rust_compiler_version"
echo >&2 "*** Expected version: $rust_compiler_min_version"
echo >&2 "***"
fi
# Check that the Rust bindings generator is suitable.
#
# Non-stable and distributions' versions may have a version suffix, e.g. `-dev`.
rust_bindings_generator_version=$( \
LC_ALL=C "$BINDGEN" --version 2>/dev/null \
| head -n 1 \
| grep -oE '[0-9]+\.[0-9]+\.[0-9]+' \
)
rust_bindings_generator_min_version=$($min_tool_version bindgen)
rust_bindings_generator_cversion=$(get_canonical_version $rust_bindings_generator_version)
rust_bindings_generator_min_cversion=$(get_canonical_version $rust_bindings_generator_min_version)
if [ "$rust_bindings_generator_cversion" -lt "$rust_bindings_generator_min_cversion" ]; then
if [ "$1" = -v ]; then
echo >&2 "***"
echo >&2 "*** Rust bindings generator '$BINDGEN' is too old."
echo >&2 "*** Your version: $rust_bindings_generator_version"
echo >&2 "*** Minimum version: $rust_bindings_generator_min_version"
echo >&2 "***"
fi
exit 1
fi
if [ "$1" = -v ] && [ "$rust_bindings_generator_cversion" -gt "$rust_bindings_generator_min_cversion" ]; then
echo >&2 "***"
echo >&2 "*** Rust bindings generator '$BINDGEN' is too new. This may or may not work."
echo >&2 "*** Your version: $rust_bindings_generator_version"
echo >&2 "*** Expected version: $rust_bindings_generator_min_version"
echo >&2 "***"
fi
# Check that the `libclang` used by the Rust bindings generator is suitable.
bindgen_libclang_version=$( \
LC_ALL=C "$BINDGEN" $(dirname $0)/rust_is_available_bindgen_libclang.h 2>&1 >/dev/null \
| grep -F 'clang version ' \
| grep -oE '[0-9]+\.[0-9]+\.[0-9]+' \
| head -n 1 \
)
bindgen_libclang_min_version=$($min_tool_version llvm)
bindgen_libclang_cversion=$(get_canonical_version $bindgen_libclang_version)
bindgen_libclang_min_cversion=$(get_canonical_version $bindgen_libclang_min_version)
if [ "$bindgen_libclang_cversion" -lt "$bindgen_libclang_min_cversion" ]; then
if [ "$1" = -v ]; then
echo >&2 "***"
echo >&2 "*** libclang (used by the Rust bindings generator '$BINDGEN') is too old."
echo >&2 "*** Your version: $bindgen_libclang_version"
echo >&2 "*** Minimum version: $bindgen_libclang_min_version"
echo >&2 "***"
fi
exit 1
fi
# If the C compiler is Clang, then we can also check whether its version
# matches the `libclang` version used by the Rust bindings generator.
#
# In the future, we might be able to perform a full version check, see
# https://github.com/rust-lang/rust-bindgen/issues/2138.
if [ "$1" = -v ]; then
cc_name=$($(dirname $0)/cc-version.sh "$CC" | cut -f1 -d' ')
if [ "$cc_name" = Clang ]; then
clang_version=$( \
LC_ALL=C "$CC" --version 2>/dev/null \
| sed -nE '1s:.*version ([0-9]+\.[0-9]+\.[0-9]+).*:\1:p'
)
if [ "$clang_version" != "$bindgen_libclang_version" ]; then
echo >&2 "***"
echo >&2 "*** libclang (used by the Rust bindings generator '$BINDGEN')"
echo >&2 "*** version does not match Clang's. This may be a problem."
echo >&2 "*** libclang version: $bindgen_libclang_version"
echo >&2 "*** Clang version: $clang_version"
echo >&2 "***"
fi
fi
fi
# Check that the source code for the `core` standard library exists.
#
# `$KRUSTFLAGS` is passed in case the user added `--sysroot`.
rustc_sysroot=$("$RUSTC" $KRUSTFLAGS --print sysroot)
rustc_src=${RUST_LIB_SRC:-"$rustc_sysroot/lib/rustlib/src/rust/library"}
rustc_src_core="$rustc_src/core/src/lib.rs"
if [ ! -e "$rustc_src_core" ]; then
if [ "$1" = -v ]; then
echo >&2 "***"
echo >&2 "*** Source code for the 'core' standard library could not be found"
echo >&2 "*** at '$rustc_src_core'."
echo >&2 "***"
fi
exit 1
fi

2
scripts/rust_is_available_bindgen_libclang.h Normal file
View File

@ -0,0 +1,2 @@
/* SPDX-License-Identifier: GPL-2.0 */
#pragma message("clang version " __clang_version__)

2
tools/include/linux/kallsyms.h
View File

@ -6,7 +6,7 @@
#include <stdio.h>
#include <unistd.h>
#define KSYM_NAME_LEN 128
#define KSYM_NAME_LEN 512
struct module;

2
tools/lib/perf/include/perf/event.h
View File

@ -97,7 +97,7 @@ struct perf_record_throttle {
};
#ifndef KSYM_NAME_LEN
#define KSYM_NAME_LEN 256
#define KSYM_NAME_LEN 512
#endif
struct perf_record_ksymbol {

2
tools/lib/symbol/kallsyms.h
View File

@ -7,7 +7,7 @@
#include <linux/types.h>
#ifndef KSYM_NAME_LEN
#define KSYM_NAME_LEN 256
#define KSYM_NAME_LEN 512
#endif
static inline u8 kallsyms2elf_binding(char type)