Since the felix DSA driver will need to poll the CPU port module for
extracted frames as well, let's create some common functions that read
an Extraction Frame Header, and then an skb, from a CPU extraction
group.
We abuse the struct ocelot_ops :: port_to_netdev function a little bit,
in order to retrieve the DSA port net_device or the ocelot switchdev
net_device based on the source port information from the Extraction
Frame Header, but it's all in the benefit of code simplification -
netdev_alloc_skb needs it. Originally, the port_to_netdev method was
intended for parsing act->dev from tc flower offload code.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The ocelot tagger is a hot mess currently, it relies on memory
initialized by the attached driver for basic frame transmission.
This is against all that DSA tagging protocols stand for, which is that
the transmission and reception of a DSA-tagged frame, the data path,
should be independent from the switch control path, because the tag
protocol is in principle hot-pluggable and reusable across switches
(even if in practice it wasn't until very recently). But if another
driver like dsa_loop wants to make use of tag_ocelot, it couldn't.
This was done to have common code between Felix and Ocelot, which have
one bit difference in the frame header format. Quoting from commit
67c2404922 ("net: dsa: felix: create a template for the DSA tags on
xmit"):
Other alternatives have been analyzed, such as:
- Create a separate tag_seville.c: too much code duplication for just 1
bit field difference.
- Create a separate DSA_TAG_PROTO_SEVILLE under tag_ocelot.c, just like
tag_brcm.c, which would have a separate .xmit function. Again, too
much code duplication for just 1 bit field difference.
- Allocate the template from the init function of the tag_ocelot.c
module, instead of from the driver: couldn't figure out a method of
accessing the correct port template corresponding to the correct
tagger in the .xmit function.
The really interesting part is that Seville should have had its own
tagging protocol defined - it is not compatible on the wire with Ocelot,
even for that single bit. In principle, a packet generated by
DSA_TAG_PROTO_OCELOT when booted on NXP LS1028A would look in a certain
way, but when booted on NXP T1040 it would look differently. The reverse
is also true: a packet generated by a Seville switch would be
interpreted incorrectly by Wireshark if it was told it was generated by
an Ocelot switch.
Actually things are a bit more nuanced. If we concentrate only on the
DSA tag, what I said above is true, but Ocelot/Seville also support an
optional DSA tag prefix, which can be short or long, and it is possible
to distinguish the two taggers based on an integer constant put in that
prefix. Nonetheless, creating a separate tagger is still justified,
since the tag prefix is optional, and without it, there is again no way
to distinguish.
Claiming backwards binary compatibility is a bit more tough, since I've
already changed the format of tag_ocelot once, in commit 5124197ce5
("net: dsa: tag_ocelot: use a short prefix on both ingress and egress").
Therefore I am not very concerned with treating this as a bugfix and
backporting it to stable kernels (which would be another mess due to the
fact that there would be lots of conflicts with the other DSA_TAG_PROTO*
definitions). It's just simpler to say that the string values of the
taggers have ABI value starting with kernel 5.12, which will be when the
changing of tag protocol via /sys/class/net/<dsa-master>/dsa/tagging
goes live.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
There is one place where we cannot avoid accessing driver data, and that
is 2-step PTP TX timestamping, since the switch wants us to provide a
timestamp request ID through the injection header, which naturally must
come from a sequence number kept by the driver (it is generated by the
.port_txtstamp method prior to the tagger's xmit).
However, since other drivers like dsa_loop do not claim PTP support
anyway, the DSA_SKB_CB(skb)->clone will always be NULL anyway, so if we
move all PTP-related dereferences of struct ocelot and struct ocelot_port
into a separate function, we can effectively ensure that this is dead
code when the ocelot tagger is attached to non-ocelot switches, and the
stateful portion of the tagger is more self-contained.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The Injection Frame Header and Extraction Frame Header that the switch
prepends to frames over the NPI port is also prepended to frames
delivered over the CPU port module's queues.
Let's unify the handling of the frame headers by making the ocelot
driver call some helpers exported by the DSA tagger. Among other things,
this allows us to get rid of the strange cpu_to_be32 when transmitting
the Injection Frame Header on ocelot, since the packing API uses
network byte order natively (when "quirks" is 0).
The comments above ocelot_gen_ifh talk about setting pop_cnt to 3, and
the cpu extraction queue mask to something, but the code doesn't do it,
so we don't do it either.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Taggers should be written to do something valid irrespective of the
switch driver that they are attached to. This is even more true now,
because since the introduction of the .change_tag_protocol method, a
certain tagger is not necessarily strictly associated with a driver any
longer, and I would like to be able to test all taggers with dsa_loop in
the future.
In the case of ocelot, it needs to move the classified VLAN from the DSA
tag into the skb if the port is VLAN-aware. We can allow it to do that
by looking at the dp->vlan_filtering property, no need to invoke
structures which are specific to ocelot.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The felix DSA driver will inject some frames through register MMIO, same
as ocelot switchdev currently does. So we need to be able to reuse the
common code.
Also create some shim definitions, since the DSA tagger can be compiled
without support for the switch driver.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This looks a bit nicer than the open-coded "(x + 3) % 4" idiom.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The ocelot_rx_frame_word() function can return a negative error code,
however this isn't being checked for consistently. Errors being ignored
have not been seen in practice though.
Also, some constructs can be simplified by using "goto" instead of
repeated "break" statements.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
It appears that the intention of this snippet of code is to not exit
ocelot_xtr_irq_handler() while in the middle of extracting a frame.
The problem in extracting it word by word is that future extraction
attempts are really easy to get desynchronized, since the IRQ handler
assumes that the first 16 bytes are the IFH, which give further
information about the frame, such as frame length.
But during normal operation, "err" will not be 0, but 4, set from here:
for (i = 0; i < OCELOT_TAG_LEN / 4; i++) {
err = ocelot_rx_frame_word(ocelot, grp, true, &ifh[i]);
if (err != 4)
break;
}
if (err != 4)
break;
In that case, draining the extraction queue is a no-op. So explicitly
make this code execute only on negative err.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Since the xtr (extraction) IRQ of the ocelot switch is not shared, then
if it fired, it means that some data must be present in the queues of
the CPU port module. So simplify the code.
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Michael Chan says:
====================
bnxt_en: Error recovery optimizations.
This series implements some optimizations to error recovery. One
patch adds an echo/reply mechanism with firmware to enhance error
detection. The other patches speed up the recovery process by
polling config space earlier and to selectively initialize
context memory during re-initialization.
====================
Signed-off-by: David S. Miller <davem@davemloft.net>
We currently only log the error recovery settings if it is enabled.
In some cases, firmware disables error recovery after it was
initially enabled. Without logging anything, the user will not be
aware of this change in setting.
Log it when error recovery is disabled. Also, change the reset count
value from hexadecimal to decimal.
Reviewed-by: Edwin Peer <edwin.peer@broadcom.com>
Reviewed-by: Pavan Chebbi <pavan.chebbi@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This is a new async message that the firmware can send to check if it
can communicate with the driver. This is an added error detection
scheme that firmware can use if it suspects errors in the PCIe
interface. When the driver receives this async message, it will reply
back echoing some data in the async message. If the firmware is not
getting the reply with the proper data after some retries, error
recovery will kick in.
Reviewed-by: Andy Gospodarek <gospo@broadcom.com>
Reviewed-by: Edwin Peer <edwin.peer@broadcom.com>
Reviewed-by: Vasundhara Volam <vasundhara-v.volam@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
If firmware provides the offset to the "context kind" field of the
relevant context memory blocks, we'll initialize just that field for
each block instead of initializing all of context memory.
Populate the bnxt_mem_init structure with the proper offset returned
by firmware. If it is older firmware and the information is not
available, we set the offset to an invalid value and fall back to
the old behavior of initializing every byte. Otherwise, we initialize
only the "context kind" byte at the offset.
Reviewed-by: Edwin Peer <edwin.peer@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Currently, the driver calls memset() to set all relevant context memory
used by the chip to the initial value. This can take many milliseconds
with the potentially large number of context pages allocated for the
chip.
To make this faster, we only need to initialize the "context kind" field
of each block of context memory. This patch sets up the infrastructure
to do that with the bnxt_mem_init structure. In the next patch, we'll
add the logic to obtain the offset of the "context kind" from the
firmware. This patch is not changing the current behavior of calling
memset() to initialize all relevant context memory.
Reviewed-by: Pavan Chebbi <pavan.chebbi@broadcom.com>
Reviewed-by: Edwin Peer <edwin.peer@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
During some fatal firmware error conditions, the PCI config space
register 0x2e which normally contains the subsystem ID will become
0xffff. This register will revert back to the normal value after
the chip has completed core reset. If we detect this condition,
we can poll this config register immediately for the value to revert.
Because we use config read cycles to poll this register, there is no
possibility of Master Abort if we happen to read it during core reset.
This speeds up recovery significantly as we don't have to wait for the
conservative min_time before polling MMIO to see if the firmware has
come out of reset. As soon as this register changes value we can
proceed to re-initialize the device.
Reviewed-by: Edwin Peer <edwin.peer@broadcom.com>
Reviewed-by: Vasundhara Volam <vasundhara-v.volam@broadcom.com>
Reviewed-by: Andy Gospodarek <gospo@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Newer devices may have local context memory instead of relying on the
host for backing store. In these cases, HWRM_FUNC_BACKING_STORE_QCAPS
will return a zero entry size to indicate contexts for which the host
should not allocate backing store.
Selectively allocate context memory based on device capabilities and
only enable backing store for the appropriate contexts.
Signed-off-by: Edwin Peer <edwin.peer@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The main changes are the echo request/response from firmware for error
detection and the NO_FCS feature to transmit frames without FCS.
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The PCA95xx GPIO expander requires GPIO controller properties to operate
properly.
Signed-off-by: Krzysztof Kozlowski <krzk@kernel.org>
Acked-by: Joel Stanley <joel@jms.id.au>
Link: https://lore.kernel.org/r/20210212162640.66677-1-krzk@kernel.org'
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Pull LED fix from Pavel Machek:
"One-liner fixing a build problem"
* 'for-rc8-5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/pavel/linux-leds:
leds: rt8515: add V4L2_FLASH_LED_CLASS dependency
- Fix CONFIG_TRIM_UNUSED_KSYMS build for ppc64
- Use pkg-config for scripts/sign-file.c CFLAGS
-----BEGIN PGP SIGNATURE-----
iQJJBAABCgAzFiEEbmPs18K1szRHjPqEPYsBB53g2wYFAmApVnIVHG1hc2FoaXJv
eUBrZXJuZWwub3JnAAoJED2LAQed4NsG/cQP/iEd73lUabEPZzyBzOSFVS1nsMIf
xU+rtASlqqd2n+yDx9tv5cIgGdDXBPtf360tQ8W6RPXgjpaMLS+FJQfsvSpmZNwx
UbSwNvPzLw6I5Bs63V4bqkFnnL+14grPonbqUdRHoa/0h1C1vCpGlP5MZMCfoL9k
wjla90h/TYysLyS1C5w71P1tr9yAF0UvG0u1d0h49AM+BaRkyxXTvokhKbfD47iz
SP7b9Y7VarfVq1YOPVh37v2nueWaG41YtPjBLVM42oa2gXdrsasb3LVIgahsgc3N
QsiKMmz+3pE8GPG0smKbuFLXEi2JoXVnVWysSlJjJNIjRr2R6TB5pG6H2ZmmAqkV
jWjRPvPky9LbZC/psuwEvPjnDi+Q0byYKy/L1xeVMhnPcITK19jF6KNUAZ6d8ABI
YX5ZkLh2EHOmVvLxD4vsvOURSQERI7UtnMZ7dwhp3VsXvN2mSsZOlnTQKWJ1L+WD
JB8dP6dJZtiUI2q46hyA8NCsHaoJ7o+NnaanMZjdyzEvCDwUfAeY3rk/3p4u42jg
izEUYv0KR05qEbXOzJ4mr6Omb9onePmDHj3J9l92yeTexWJfSJeQFk3I+aIFnQos
L2VHLTro9KOtnT4qjMnZRnqxUzPQ/zI0+JvPXPb1kYu91bfn3kQl2U5L3THvpDXc
EohIJNYbUe07vS2f
=Pp2I
-----END PGP SIGNATURE-----
Merge tag 'kbuild-fixes-v5.11-3' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild
Pull Kbuild fixes from Masahiro Yamada:
- Fix CONFIG_TRIM_UNUSED_KSYMS build for ppc64
- Use pkg-config for scripts/sign-file.c CFLAGS
* tag 'kbuild-fixes-v5.11-3' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild:
scripts: set proper OpenSSL include dir also for sign-file
sparc: remove wrong comment from arch/sparc/include/asm/Kbuild
kbuild: fix CONFIG_TRIM_UNUSED_KSYMS build for ppc64
- A fix for the fix to disable CET instrumentation generation for kernel code.
We forgot 32-bit, which we seem to do very often nowadays.
- A Xen PV fix to irqdomain init ordering.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEzv7L6UO9uDPlPSfHEsHwGGHeVUoFAmAo+tUACgkQEsHwGGHe
VUqR0hAAqdYjPIk3ecxNKqjdhd9ggV66hmkwnq7iWkWGi8p33q7y9hAA1dG2PBc1
dMuGs9jOqGs5U5FrYjFD8nROVQnvNGt5ISOK02ABMqtiGSUVfInaib1nlCcyLTlt
y9a1PMISyuexrhgGwRe1LjTSV6BQmH+jlBSpioJ4lCYugfhttOPpghRKY3M2HKQ3
LhTh0j3+ls2onvgr7DP3wLfz+77ccxK7gRRc0BczqnRjVdpIdXpmxOOOaXA0Yhyd
FKnXDZ4ApIFwGB6gUvv7SlKKMULL7ev3LJQnRK4A7AUOJF37e6yMCDI7neGs6Qde
r3j7xwqNzge0B2vGaT6IOsKcytduxcB6/gpZVpExHhVBWgCl1AA7Ech+5kGdtWky
xMEYyGZZgpVvFND8iTYuoXvrkUOX3pAdXgsNLvcN5OxQrNGGfzpSzikwtZAsYqM4
Ckdv/XkMG9ALM46cNAR4kdecjOypP2zSt7k/olWbma4/MMLBjD+vAm1Vzjws7oMw
GSgENGMUa0zjLmi4gN5IZnMCbfBHkzvMlyFBop7cSt983nGLap8z/vx9J6R0jXHy
Nhlc129Pd5lRIptxFg0d6Fsn4IimOASFyfgxp1kGSR7nd9jnogXsDfMqC4TLCnwW
p1d+1JkUWIlrC2hqB+whQ23s81H9Myt4lp87GOT7s+FWmtAc7p0=
=9CtU
-----END PGP SIGNATURE-----
Merge tag 'x86_urgent_for_v5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull x86 fixes from Borislav Petkov:
"I kinda knew while typing 'I hope this is the last batch of x86/urgent
updates' last week, Murphy was reading too and uttered 'Hold my
beer!'.
So here's more fixes... Thanks Murphy.
Anyway, three more x86/urgent fixes for 5.11 final. We should be
finally ready (famous last words). :-)
- An SGX use after free fix
- A fix for the fix to disable CET instrumentation generation for
kernel code. We forgot 32-bit, which we seem to do very often
nowadays
- A Xen PV fix to irqdomain init ordering"
* tag 'x86_urgent_for_v5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
x86/pci: Create PCI/MSI irqdomain after x86_init.pci.arch_init()
x86/build: Disable CET instrumentation in the kernel for 32-bit too
x86/sgx: Maintain encl->refcount for each encl->mm_list entry
The leds-rt8515 driver can optionall use the v4l2 flash led class,
but it causes a link error when that class is in a loadable module
and the rt8515 driver itself is built-in:
ld.lld: error: undefined symbol: v4l2_flash_init
>>> referenced by leds-rt8515.c
>>> leds/flash/leds-rt8515.o:(rt8515_probe) in archive
drivers/built-in.a
Adding 'depends on V4L2_FLASH_LED_CLASS' in Kconfig would avoid that,
but it would make it impossible to use the driver without the
v4l2 support.
Add the same dependency that the other users of this class have
instead, which just prevents the broken configuration.
Fixes: e1c6edcbea ("leds: rt8515: Add Richtek RT8515 LED driver")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Pavel Machek <pavel@ucw.cz>
These are NOT exported to userspace.
The headers listed in arch/sparc/include/uapi/asm/Kbuild are exported.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Merely enabling CONFIG_COMPILE_TEST should not enable additional code.
To fix this, restrict the automatic enabling of IMX_INTMUX to ARCH_MXC,
and ask the user in case of compile-testing.
Fixes: 66968d7dfc ("irqchip: Add COMPILE_TEST support for IMX_INTMUX")
Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20210208145605.422943-1-geert+renesas@glider.be
Alexander Lobakin says:
====================
skbuff: introduce skbuff_heads bulking and reusing
Currently, all sorts of skb allocation always do allocate
skbuff_heads one by one via kmem_cache_alloc().
On the other hand, we have percpu napi_alloc_cache to store
skbuff_heads queued up for freeing and flush them by bulks.
We can use this cache not only for bulk-wiping, but also to obtain
heads for new skbs and avoid unconditional allocations, as well as
for bulk-allocating (like XDP's cpumap code and veth driver already
do).
As this might affect latencies, cache pressure and lots of hardware
and driver-dependent stuff, this new feature is mostly optional and
can be issued via:
- a new napi_build_skb() function (as a replacement for build_skb());
- existing {,__}napi_alloc_skb() and napi_get_frags() functions;
- __alloc_skb() with passing SKB_ALLOC_NAPI in flags.
iperf3 showed 35-70 Mbps bumps for both TCP and UDP while performing
VLAN NAT on 1.2 GHz MIPS board. The boost is likely to be bigger
on more powerful hosts and NICs with tens of Mpps.
Note on skbuff_heads from distant slabs or pfmemalloc'ed slabs:
- kmalloc()/kmem_cache_alloc() itself allows by default allocating
memory from the remote nodes to defragment their slabs. This is
controlled by sysctl, but according to this, skbuff_head from a
remote node is an OK case;
- The easiest way to check if the slab of skbuff_head is remote or
pfmemalloc'ed is:
if (!dev_page_is_reusable(virt_to_head_page(skb)))
/* drop it */;
...*but*, regarding that most slabs are built of compound pages,
virt_to_head_page() will hit unlikely-branch every single call.
This check costed at least 20 Mbps in test scenarios and seems
like it'd be better to _not_ do this.
Since v5 [4]:
- revert flags-to-bool conversion and simplify flags testing in
__alloc_skb() (Alexander Duyck).
Since v4 [3]:
- rebase on top of net-next and address kernel build robot issue;
- reorder checks a bit in __alloc_skb() to make new condition even
more harmless.
Since v3 [2]:
- make the feature mostly optional, so driver developers could
decide whether to use it or not (Paolo Abeni).
This reuses the old flag for __alloc_skb() and introduces
a new napi_build_skb();
- reduce bulk-allocation size from 32 to 16 elements (also Paolo).
This equals to the value of XDP's devmap and veth batch processing
(which were tested a lot) and should be sane enough;
- don't waste cycles on explicit in_serving_softirq() check.
Since v2 [1]:
- also cover {,__}alloc_skb() and {,__}build_skb() cases (became handy
after the changes that pass tiny skbs requests to kmalloc layer);
- cover the cache with KASAN instrumentation (suggested by Eric
Dumazet, help of Dmitry Vyukov);
- completely drop redundant __kfree_skb_flush() (also Eric);
- lots of code cleanups;
- expand the commit message with NUMA and pfmemalloc points (Jakub).
Since v1 [0]:
- use one unified cache instead of two separate to greatly simplify
the logics and reduce hotpath overhead (Edward Cree);
- new: recycle also GRO_MERGED_FREE skbs instead of immediate
freeing;
- correct performance numbers after optimizations and performing
lots of tests for different use cases.
[0] https://lore.kernel.org/netdev/20210111182655.12159-1-alobakin@pm.me
[1] https://lore.kernel.org/netdev/20210113133523.39205-1-alobakin@pm.me
[2] https://lore.kernel.org/netdev/20210209204533.327360-1-alobakin@pm.me
[3] https://lore.kernel.org/netdev/20210210162732.80467-1-alobakin@pm.me
[4] https://lore.kernel.org/netdev/20210211185220.9753-1-alobakin@pm.me
====================
Reviewed-by: Alexander Duyck <alexanderduyck@fb.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
napi_frags_finish() and napi_skb_finish() can only be called inside
NAPI Rx context, so we can feed NAPI cache with skbuff_heads that
got NAPI_MERGED_FREE verdict instead of immediate freeing.
Replace __kfree_skb() with __kfree_skb_defer() in napi_skb_finish()
and move napi_skb_free_stolen_head() to skbuff.c, so it can drop skbs
to NAPI cache.
As many drivers call napi_alloc_skb()/napi_get_frags() on their
receive path, this becomes especially useful.
Signed-off-by: Alexander Lobakin <alobakin@pm.me>
Signed-off-by: David S. Miller <davem@davemloft.net>
{,__}napi_alloc_skb() is mostly used either for optional non-linear
receive methods (usually controlled via Ethtool private flags and off
by default) and/or for Rx copybreaks.
Use __napi_build_skb() here for obtaining skbuff_heads from NAPI cache
instead of inplace allocations. This includes both kmalloc and page
frag paths.
Signed-off-by: Alexander Lobakin <alobakin@pm.me>
Signed-off-by: David S. Miller <davem@davemloft.net>
Reuse the old and forgotten SKB_ALLOC_NAPI to add an option to get
an skbuff_head from the NAPI cache instead of inplace allocation
inside __alloc_skb().
This implies that the function is called from softirq or BH-off
context, not for allocating a clone or from a distant node.
Cc: Alexander Duyck <alexander.duyck@gmail.com> # Simplified flags check
Signed-off-by: Alexander Lobakin <alobakin@pm.me>
Signed-off-by: David S. Miller <davem@davemloft.net>
Instead of just bulk-flushing skbuff_heads queued up through
napi_consume_skb() or __kfree_skb_defer(), try to reuse them
on allocation path.
If the cache is empty on allocation, bulk-allocate the first
16 elements, which is more efficient than per-skb allocation.
If the cache is full on freeing, bulk-wipe the second half of
the cache (32 elements).
This also includes custom KASAN poisoning/unpoisoning to be
double sure there are no use-after-free cases.
To not change current behaviour, introduce a new function,
napi_build_skb(), to optionally use a new approach later
in drivers.
Note on selected bulk size, 16:
- this equals to XDP_BULK_QUEUE_SIZE, DEV_MAP_BULK_SIZE
and especially VETH_XDP_BATCH, which is also used to
bulk-allocate skbuff_heads and was tested on powerful
setups;
- this also showed the best performance in the actual
test series (from the array of {8, 16, 32}).
Suggested-by: Edward Cree <ecree.xilinx@gmail.com> # Divide on two halves
Suggested-by: Eric Dumazet <edumazet@google.com> # KASAN poisoning
Cc: Dmitry Vyukov <dvyukov@google.com> # Help with KASAN
Cc: Paolo Abeni <pabeni@redhat.com> # Reduced batch size
Signed-off-by: Alexander Lobakin <alobakin@pm.me>
Signed-off-by: David S. Miller <davem@davemloft.net>
NAPI cache structures will be used for allocating skbuff_heads,
so move their declarations a bit upper.
Signed-off-by: Alexander Lobakin <alobakin@pm.me>
Signed-off-by: David S. Miller <davem@davemloft.net>
This function isn't much needed as NAPI skb queue gets bulk-freed
anyway when there's no more room, and even may reduce the efficiency
of bulk operations.
It will be even less needed after reusing skb cache on allocation path,
so remove it and this way lighten network softirqs a bit.
Suggested-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Alexander Lobakin <alobakin@pm.me>
Signed-off-by: David S. Miller <davem@davemloft.net>
Just call __build_skb_around() instead of open-coding it.
Signed-off-by: Alexander Lobakin <alobakin@pm.me>
Signed-off-by: David S. Miller <davem@davemloft.net>
Use unlikely() annotations for skbuff_head and data similarly to the
two other allocation functions and remove totally redundant goto.
Signed-off-by: Alexander Lobakin <alobakin@pm.me>
Signed-off-by: David S. Miller <davem@davemloft.net>
__build_skb_around() can never fail and always returns passed skb.
Make it return void to simplify and optimize the code.
Signed-off-by: Alexander Lobakin <alobakin@pm.me>
Signed-off-by: David S. Miller <davem@davemloft.net>
Eversince the introduction of __kmalloc_reserve(), "ip" argument
hasn't been used. _RET_IP_ is embedded inside
kmalloc_node_track_caller().
Remove the redundant macro and rename the function after it.
Signed-off-by: Alexander Lobakin <alobakin@pm.me>
Signed-off-by: David S. Miller <davem@davemloft.net>
In preparation before reusing several functions in all three skb
allocation variants, move __alloc_skb() next to the
__netdev_alloc_skb() and __napi_alloc_skb().
No functional changes.
Signed-off-by: Alexander Lobakin <alobakin@pm.me>
Signed-off-by: David S. Miller <davem@davemloft.net>
out the correct rate to use. This fixes displays running 4k@60Hz and
some other resolutions that haven't been exercised and fully understood
until now.
-----BEGIN PGP SIGNATURE-----
iQJFBAABCAAvFiEE9L57QeeUxqYDyoaDrQKIl8bklSUFAmAnMhERHHNib3lkQGtl
cm5lbC5vcmcACgkQrQKIl8bklSVlthAAp0bpcXZUmB/MJ8sp0hjUDkSjtHTYU4bo
nVSvap+HAqQ2EsHxeWQDojwbLQ03+qYwwnw0WEzImlEzrBZ7G5U8z1BovDGf48f1
AObOsHKkzPZHOiEKV+4Fi6UjNY4HtL0YxTIbQPK0oaXXLtd2LKyxGDwj688Y7OIu
WOmf62V9zvR/42pnl7oRK9MZ2lW/0RRPJuvll1XOh+CNFuxL1zQR5EPp0veG0QEz
MKcaMrBXCYfuoSsIXLhL3Dj8Ot3TmsJ6YFZGLX0vzdATRH5dsZlrdSL6p72GLvuz
TsEaLjU2yooR1QHvGnrVo1/nI4i7T8rrv14L5ebxcEfgEexo9qgVroyB+5caQvWk
49tnb9yE6pSDeYIucI55WZiweRrM6PMDuJiQjaYwjBTtbgEmhGBNVK/WePpi9l/v
kU3iPdCZix1k9KwhBc5lEeR1FyxjsycsQDxfniLppZtU1DqN3IHsaCCVfLb3/qMz
ADG2o0a0mmAB9z0TFlUyU+wp3E1PmMxge0/iJvgspqSzcem0u/9CKg/ssHul1lUs
tRXv/xrD3qg2GdN+se3aj+cdQrIHjUlrcykdYlJYv7RKQXDUzLxH8RIdHrzneJMe
V+DQ3YinKS5OpmnFucv9Fw0lBBzVm2O9qbdaaE92iuWs7bUx6x5c+8B4y2CbalZk
/T5PHxYZs3Y=
=jPmu
-----END PGP SIGNATURE-----
Merge tag 'clk-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/clk/linux
Pull clk fix from Stephen Boyd:
"One small fix for the Allwinner clk driver so that display clks figure
out the correct rate to use.
This fixes displays running 4k@60Hz and some other resolutions that
haven't been exercised and fully understood until now"
* tag 'clk-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/clk/linux:
clk: sunxi-ng: mp: fix parent rate change flag check
One fix for scsi_debug that fixes a memory leak on module removal.
Signed-off-by: James E.J. Bottomley <jejb@linux.ibm.com>
-----BEGIN PGP SIGNATURE-----
iJwEABMIAEQWIQTnYEDbdso9F2cI+arnQslM7pishQUCYCg/NiYcamFtZXMuYm90
dG9tbGV5QGhhbnNlbnBhcnRuZXJzaGlwLmNvbQAKCRDnQslM7pishaFcAP0UQ9j6
1t9uUiQmYO7elxHuDvf3qLX2i5ZIupa6CXaPsQD/e04hIV4G3OStBJtYz3dY0UjH
RG7aDuJGIYXBiK0pVrk=
=5rfC
-----END PGP SIGNATURE-----
Merge tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi
Pull SCSI fix from James Bottomley:
"One fix for scsi_debug that fixes a memory leak on module removal"
* tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi:
scsi: scsi_debug: Fix a memory leak
This patch fixes the following checkpatch.pl check:
CHECK: Alignment should match open parenthesis
Signed-off-by: Mukul Mehar <mukulmehar02@gmail.com>
Link: https://lore.kernel.org/r/20210213120556.73579-1-mukulmehar02@gmail.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Fix sparse byte-order warnings in the i2400m_bm_cmd_prepare()
function:
wimax/i2400m/fw.c:194:36: warning: restricted __le32 degrades to integer
wimax/i2400m/fw.c:195:34: warning: invalid assignment: +=
wimax/i2400m/fw.c:195:34: left side has type unsigned int
wimax/i2400m/fw.c:195:34: right side has type restricted __le32
wimax/i2400m/fw.c:196:32: warning: restricted __le32 degrades to integer
wimax/i2400m/fw.c:196:47: warning: restricted __le32 degrades to integer
wimax/i2400m/fw.c:196:66: warning: restricted __le32 degrades to integer
Signed-off-by: Anirudh Rayabharam <mail@anirudhrb.com>
Link: https://lore.kernel.org/r/20210212153843.8554-1-mail@anirudhrb.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
