linux/include/net
Shawn Bohrer 421b3885bf udp: ipv4: Add udp early demux
The removal of the routing cache introduced a performance regression for
some UDP workloads since a dst lookup must be done for each packet.
This change caches the dst per socket in a similar manner to what we do
for TCP by implementing early_demux.

For UDP multicast we can only cache the dst if there is only one
receiving socket on the host.  Since caching only works when there is
one receiving socket we do the multicast socket lookup using RCU.

For UDP unicast we only demux sockets with an exact match in order to
not break forwarding setups.  Additionally since the hash chains may be
long we only check the first socket to see if it is a match and not
waste extra time searching the whole chain when we might not find an
exact match.

Benchmark results from a netperf UDP_RR test:
Before 87961.22 transactions/s
After  89789.68 transactions/s

Benchmark results from a fio 1 byte UDP multicast pingpong test
(Multicast one way unicast response):
Before 12.97us RTT
After  12.63us RTT

Signed-off-by: Shawn Bohrer <sbohrer@rgmadvisors.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2013-10-08 16:27:33 -04:00
..
9p for-linus-3.12-merge minor 9p fixes and tweaks for 3.12 merge window 2013-09-11 12:34:13 -07:00
bluetooth Merge git://git.kernel.org/pub/scm/linux/kernel/git/linville/wireless-next 2013-10-07 15:40:44 -04:00
caif caif_hsi.h: Remove extern from function prototypes 2013-09-23 16:29:41 -04:00
irda irda: Remove extern from function prototypes 2013-09-23 16:29:42 -04:00
iucv af_iucv: fix recvmsg by replacing skb_pull() function 2013-04-08 17:16:57 -04:00
netfilter Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2013-10-01 17:06:14 -04:00
netns net ipv4: Convert ipv4.ip_local_port_range to be per netns v3 2013-09-30 21:59:38 -07:00
nfc NFC: netlink: Add result of firmware operation to completion event 2013-08-14 01:12:58 +02:00
phonet
sctp sctp: Remove extern from function prototypes 2013-09-23 16:29:42 -04:00
tc_act
act_api.h net: Remove extern from include/net/ scheduling prototypes 2013-07-31 17:24:22 -07:00
addrconf.h IPv6 NAT: Do not drop DNATed 6to4/6rd packets 2013-09-28 15:56:15 -04:00
af_ieee802154.h
af_rxrpc.h af_rxrpc.h: Remove extern from function prototypes 2013-07-31 17:50:01 -07:00
af_unix.h af_unix: improve STREAM behavior with fragmented memory 2013-08-10 01:16:44 -07:00
af_vsock.h VSOCK: Move af_vsock.h and vsock_addr.h to include/net 2013-07-27 22:14:06 -07:00
ah.h
arp.h arp/neighbour.h: Remove extern from function prototypes 2013-07-31 17:50:02 -07:00
atmclip.h
ax25.h ax25.h: Remove extern from function prototypes 2013-07-31 17:50:02 -07:00
ax88796.h
busy_poll.h net: add cpu_relax to busy poll loop 2013-08-28 17:45:48 -04:00
cfg80211-wext.h
cfg80211.h Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/linville/wireless-next into for-davem 2013-08-29 14:08:24 -04:00
checksum.h checksum: Remove extern from function prototypes 2013-07-31 17:50:02 -07:00
cipso_ipv4.h
cls_cgroup.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next 2013-09-05 14:54:29 -07:00
codel.h codel: refine one condition to avoid a nul rec_inv_sqrt 2012-08-10 16:52:54 -07:00
compat.h compat.h: Remove extern from function prototypes 2013-09-20 14:49:32 -04:00
datalink.h
dcbevent.h dcbevent.h: Remove extern from function prototypes 2013-09-20 14:49:32 -04:00
dcbnl.h
dn_dev.h decnet (dn*.h): Remove extern from function prototypes 2013-09-20 14:49:32 -04:00
dn_fib.h decnet (dn*.h): Remove extern from function prototypes 2013-09-20 14:49:32 -04:00
dn_neigh.h decnet (dn*.h): Remove extern from function prototypes 2013-09-20 14:49:32 -04:00
dn_nsp.h decnet (dn*.h): Remove extern from function prototypes 2013-09-20 14:49:32 -04:00
dn_route.h decnet (dn*.h): Remove extern from function prototypes 2013-09-20 14:49:32 -04:00
dn.h decnet (dn*.h): Remove extern from function prototypes 2013-09-20 14:49:32 -04:00
dsa.h
dsfield.h ipv6: Optimize ipv6_change_dsfield(). 2013-01-09 23:59:53 -08:00
dst_ops.h net: Fix warnings in dst_ops.h 2012-07-19 10:43:03 -07:00
dst.h dst.h: Remove extern from function prototypes 2013-09-20 14:49:32 -04:00
esp.h esp.h: Remove extern from function prototypes 2013-09-20 14:49:32 -04:00
ethoc.h
fib_rules.h fib_rules.h: Remove extern from function prototypes 2013-09-20 14:49:33 -04:00
firewire.h firewire net, ipv4 arp: Extend hardware address and remove driver-level packet inspection. 2013-03-26 12:32:13 -04:00
flow_keys.h flow_dissector: factor out the ports extraction in skb_flow_get_ports 2013-10-03 15:36:37 -04:00
flow.h flow.h/flow_keys.h: Remove extern from function prototypes 2013-09-20 14:49:33 -04:00
garp.h garp.h: Remove extern from function prototypes 2013-09-20 14:49:33 -04:00
gen_stats.h gen_stats.h: Remove extern from function prototypes 2013-09-20 14:49:33 -04:00
genetlink.h genetlink.h: Remove extern from function prototypes 2013-09-20 14:49:33 -04:00
gre.h net: gre: move GSO functions to gre_offload 2013-07-03 14:37:39 -07:00
gro_cells.h gro: Fix kcalloc argument order 2013-01-27 22:46:33 -05:00
icmp.h icmp.h: Remove extern from function prototypes 2013-09-20 14:49:33 -04:00
ieee80211_radiotap.h mac80211: add radiotap flag and handling for 5/10 MHz 2013-07-16 09:58:05 +03:00
ieee802154_netdev.h ieee802154/nl-mac.c: make some MLME operations optional 2013-04-08 12:00:16 -04:00
ieee802154.h
if_inet6.h net: ipv6: mld: fix v1/v2 switchback timeout to rfc3810, 9.12. 2013-09-04 14:53:20 -04:00
inet6_connection_sock.h inet*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
inet6_hashtables.h inet*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
inet_common.h inet*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
inet_connection_sock.h inet*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
inet_ecn.h net: Correct comparisons and calculations using skb->tail and skb-transport_header 2013-05-28 23:49:07 -07:00
inet_frag.h net: frag, fix race conditions in LRU list maintenance 2013-05-06 11:06:51 -04:00
inet_hashtables.h inet: consolidate INET_TW_MATCH 2013-10-03 15:33:35 -04:00
inet_sock.h inet: consolidate INET_TW_MATCH 2013-10-03 15:33:35 -04:00
inet_timewait_sock.h tcp: shrink tcp6_timewait_sock by one cache line 2013-10-03 17:43:39 -04:00
inetpeer.h inet*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
ip6_checksum.h ipv6: move csum_ipv6_magic() and udp6_csum_init() into static library 2013-01-08 17:56:10 -08:00
ip6_fib.h ipv6: avoid high order memory allocations for /proc/net/ipv6_route 2013-09-27 17:32:16 -04:00
ip6_route.h ip*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
ip6_tunnel.h tunnels: harmonize cleanup done on skb on xmit path 2013-09-04 00:27:25 -04:00
ip_fib.h ip*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
ip_tunnels.h tunnels: harmonize cleanup done on skb on xmit path 2013-09-04 00:27:25 -04:00
ip_vs.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2013-10-01 17:06:14 -04:00
ip.h net ipv4: Convert ipv4.ip_local_port_range to be per netns v3 2013-09-30 21:59:38 -07:00
ipcomp.h
ipconfig.h
ipv6.h ip*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
ipx.h ipx.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
iw_handler.h iw_handler.h: Remove extern from function prototypes 2013-09-21 14:01:39 -04:00
lapb.h lapb.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
lib80211.h hostap: Don't use create_proc_read_entry() 2013-04-29 15:41:56 -04:00
llc_c_ac.h llc*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
llc_c_ev.h llc*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
llc_c_st.h
llc_conn.h llc*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
llc_if.h llc*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
llc_pdu.h llc*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
llc_s_ac.h llc*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
llc_s_ev.h llc*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
llc_s_st.h
llc_sap.h llc*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
llc.h llc*.h: Remove extern from function prototypes 2013-09-21 14:01:38 -04:00
mac80211.h Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/linville/wireless-next into for-davem 2013-08-29 14:08:24 -04:00
mac802154.h
mip6.h
mld.h net: ipv6: mld: get rid of MLDV2_MRC and simplify calculation 2013-09-04 14:53:20 -04:00
mrp.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2013-10-01 17:06:14 -04:00
ndisc.h ndisc.h: Remove extern from function prototypes 2013-09-21 14:01:39 -04:00
neighbour.h arp/neighbour.h: Remove extern from function prototypes 2013-07-31 17:50:02 -07:00
net_namespace.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2013-10-01 17:06:14 -04:00
net_ratelimit.h
netdma.h
netevent.h netevent/netlink.h: Remove extern from function prototypes 2013-09-21 14:01:39 -04:00
netlabel.h userns: Convert the audit loginuid to be a kuid 2012-09-17 18:08:54 -07:00
netlink.h netevent/netlink.h: Remove extern from function prototypes 2013-09-21 14:01:39 -04:00
netprio_cgroup.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next 2013-09-05 14:54:29 -07:00
netrom.h netrom.h: Remove extern from function prototypes 2013-09-21 14:01:39 -04:00
nexthop.h
nl802154.h
p8022.h p8022.h: Remove extern from function prototypes 2013-09-21 14:01:39 -04:00
ping.h ping.h: Remove extern from function prototypes 2013-09-23 01:51:07 -04:00
pkt_cls.h net: Remove extern from include/net/ scheduling prototypes 2013-07-31 17:24:22 -07:00
pkt_sched.h qdisc: allow setting default queuing discipline 2013-08-31 00:32:32 -04:00
protocol.h protocol.h: Remove extern from function prototypes 2013-09-23 01:51:08 -04:00
psnap.h psnap.h: Remove extern from function prototypes 2013-09-23 01:51:08 -04:00
raw.h raw/rawv6.h: Remove extern from function prototypes 2013-09-23 01:51:08 -04:00
rawv6.h raw/rawv6.h: Remove extern from function prototypes 2013-09-23 01:51:08 -04:00
red.h
regulatory.h regulatory: use RCU to protect last_request 2013-01-03 13:01:30 +01:00
request_sock.h request_sock.h: Remove extern from function prototypes 2013-09-23 01:51:08 -04:00
rose.h rose.h: Remove extern from function prototypes 2013-09-23 01:51:08 -04:00
route.h ipv4: processing ancillary IP_TOS or IP_TTL 2013-09-28 15:21:52 -07:00
rtnetlink.h rtnetlink.h: Remove extern from function prototypes 2013-09-23 01:51:09 -04:00
sch_generic.h net_sched: add u64 rate to psched_ratecfg_precompute() 2013-09-20 14:41:02 -04:00
scm.h scm.h: Remove extern from function prototypes 2013-09-23 01:51:09 -04:00
secure_seq.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2013-10-01 17:06:14 -04:00
slhc_vj.h
snmp.h net: avoid reloads in SNMP_UPD_PO_STATS 2012-08-06 13:40:47 -07:00
sock.h udp: ipv4: Add udp early demux 2013-10-08 16:27:33 -04:00
stp.h stp.h: Remove extern from function prototypes 2013-09-23 01:51:09 -04:00
tcp_memcontrol.h
tcp_states.h
tcp.h tcp: syncookies: reduce cookie lifetime to 128 seconds 2013-09-24 10:39:58 -04:00
timewait_sock.h
transp_v6.h transp_v6.h: style neatening 2013-06-04 16:43:42 -07:00
udp.h udp: ipv4: Add udp early demux 2013-10-08 16:27:33 -04:00
udplite.h udplite.h: Remove extern from function prototypes 2013-09-23 16:29:40 -04:00
vsock_addr.h VSOCK: Move af_vsock.h and vsock_addr.h to include/net 2013-07-27 22:14:06 -07:00
vxlan.h vxlan: Notify drivers for listening UDP port changes 2013-09-05 12:44:30 -04:00
wext.h wext.h: Remove extern from function prototypes 2013-09-23 16:29:40 -04:00
wimax.h wimax.h: Remove extern from function prototypes 2013-09-23 16:29:41 -04:00
wpan-phy.h
x25.h x25.h: Remove extern from function prototypes 2013-09-23 16:29:41 -04:00
x25device.h
xfrm.h Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/klassert/ipsec-next 2013-09-30 15:24:57 -04:00