2
0
mirror of https://github.com/edk2-porting/linux-next.git synced 2024-12-25 21:54:06 +08:00
linux-next/include/net/netfilter
Florian Westphal 87e389b4c2 netfilter: conntrack: small conntrack lookup optimization
____nf_conntrack_find() performs checks on the conntrack objects in
this order:

1. if (nf_ct_is_expired(ct))

This fetches ct->timeout, in third cache line.

The hnnode that is used to store the list pointers resides in the first
(origin) or second (reply tuple) cache lines.

This test rarely passes, but its necessary to reap obsolete entries.

2. if (nf_ct_is_dying(ct))

This fetches ct->status, also in third cache line.

The test is useless, and can be removed:
  Consider:
     cpu0                                           cpu1
    ct = ____nf_conntrack_find()
    atomic_inc_not_zero(ct) -> ok
    nf_ct_key_equal -> ok
    is_dying -> DYING bit not set, ok
                                                    set_bit(ct, DYING);
						    ... unhash ... etc.
    return ct
    -> returning a ct with dying bit set, despite
    having a test for it.

This (unlikely) case is fine - refcount prevents ct from getting free'd.

3. if (nf_ct_key_equal(h, tuple, zone, net))

nf_ct_key_equal checks in following order:

1. Tuple equal (first or second cacheline)
2. Zone equal (third cacheline)
3. confirmed bit set (->status, third cacheline)
4. net namespace match (third cacheline).

Swapping "timeout" and "cpu" places timeout in the first cacheline.
This has two advantages:

1. For a conntrack that won't even match the original tuple,
   we will now only fetch the first and maybe the second cacheline
   instead of always accessing the 3rd one as well.

2.  in case of TCP ct->timeout changes frequently because we
    reduce/increase it when there are packets outstanding in the network.

The first cacheline contains both the reference count and the ct spinlock,
i.e. moving timeout there avoids writes to 3rd cacheline.

The restart sequence in __nf_conntrack_find() is removed, if we found a
candidate, but then fail to increment the refcount or discover the tuple
has changed (object recycling), just pretend we did not find an entry.

A second lookup won't find anything until another CPU adds a new conntrack
with identical tuple into the hash table, which is very unlikely.

We have the confirmation-time checks (when we hold hash lock) that deal
with identical entries and even perform clash resolution in some cases.

Signed-off-by: Florian Westphal <fw@strlen.de>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2019-06-17 16:35:30 +02:00
..
ipv4 netfilter: nf_nat: merge ip/ip6 masquerade headers 2019-04-11 20:59:21 +02:00
ipv6 netfilter: nf_nat: merge ip/ip6 masquerade headers 2019-04-11 20:59:21 +02:00
br_netfilter.h netfilter: physdev: relax br_netfilter dependency 2019-01-18 15:02:33 +01:00
nf_conntrack_acct.h netfilter: conntrack: remove empty pernet fini stubs 2018-12-21 00:51:54 +01:00
nf_conntrack_bridge.h netfilter: bridge: add connection tracking system 2019-05-30 14:18:18 -07:00
nf_conntrack_core.h netfilter: bridge: add connection tracking system 2019-05-30 14:18:18 -07:00
nf_conntrack_count.h netfilter: nf_conncount: speculative garbage collection on empty lists 2018-12-29 02:45:22 +01:00
nf_conntrack_ecache.h netfilter: conntrack: remove empty pernet fini stubs 2018-12-21 00:51:54 +01:00
nf_conntrack_expect.h netfilter: replace NF_NAT_NEEDED with IS_ENABLED(CONFIG_NF_NAT) 2019-04-08 23:02:52 +02:00
nf_conntrack_extend.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
nf_conntrack_helper.h netfilter: add API to manage NAT helpers. 2019-04-30 14:19:55 +02:00
nf_conntrack_l4proto.h netfilter: conntrack: don't set related state for different outer address 2019-04-13 14:52:57 +02:00
nf_conntrack_labels.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
nf_conntrack_seqadj.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
nf_conntrack_synproxy.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
nf_conntrack_timeout.h netfilter: Export nf_ct_{set,destroy}_timeout() 2019-03-28 16:53:29 -07:00
nf_conntrack_timestamp.h netfilter: conntrack: remove empty pernet fini stubs 2018-12-21 00:51:54 +01:00
nf_conntrack_tuple.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
nf_conntrack_zones.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
nf_conntrack.h netfilter: conntrack: small conntrack lookup optimization 2019-06-17 16:35:30 +02:00
nf_dup_netdev.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
nf_flow_table.h netfilter: nf_flow_table: remove unnecessary variable in flow_offload_tuple 2019-05-31 18:02:42 +02:00
nf_log.h netfilter: check if the socket netns is correct. 2018-06-28 22:21:32 +09:00
nf_nat_helper.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
nf_nat_masquerade.h netfilter: nf_nat_masquerade: unify ipv4/6 notifier registration 2019-04-11 20:59:34 +02:00
nf_nat_redirect.h netfilter: add NAT support for shifted portmap ranges 2018-04-24 10:29:12 +02:00
nf_nat.h netfilter: nat: add inet family nat support 2019-04-08 23:01:39 +02:00
nf_queue.h bridge: netfilter: unroll NF_HOOK helper in bridge input path 2019-04-12 01:47:39 +02:00
nf_reject.h netfilter: reject: skip csum verification for protocols that don't support it 2019-02-13 10:03:53 +01:00
nf_socket.h netfilter: Decrease code duplication regarding transparent socket option 2018-06-03 00:02:01 +02:00
nf_tables_core.h netfilter: nf_tables: add direct calls for all builtin expressions 2019-01-18 15:02:33 +01:00
nf_tables_ipv4.h netfilter: nf_tables_inet: don't use multihook infrastructure anymore 2018-01-08 18:01:20 +01:00
nf_tables_ipv6.h netfilter: nf_tables_inet: don't use multihook infrastructure anymore 2018-01-08 18:01:20 +01:00
nf_tables.h netfilter: nf_tables: drop include of module.h from nf_tables.h 2019-04-30 13:35:32 +02:00
nf_tproxy.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/pablo/nf-next 2018-07-20 22:28:28 -07:00
nft_fib.h netfilter: nft_fib: Fix existence check support 2019-05-21 16:10:38 +02:00
nft_reject.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
xt_rateest.h netfilter: make xt_rateest hash table per net 2018-03-05 23:15:44 +01:00