Eric Dumazet 29f3490ba9 net: use indirect call wrappers for skb_copy_datagram_iter() ... TCP recvmsg() calls skb_copy_datagram_iter(), which calls an indirect function (cb pointing to simple_copy_to_iter()) for every MSS (fragment) present in the skb. CONFIG_RETPOLINE=y forces a very expensive operation that we can avoid thanks to indirect call wrappers. This patch gives a 13% increase of performance on a single flow, if the bottleneck is the thread reading the TCP socket. Fixes: 950fcaecd5 ("datagram: consolidate datagram copy to iter helpers") Signed-off-by: Eric Dumazet <edumazet@google.com> Acked-by: Paolo Abeni <pabeni@redhat.com> Acked-by: Willem de Bruijn <willemb@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>		2020-03-25 11:30:40 -07:00
..
bpf_sk_storage.c	Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next	2020-02-29 15:53:35 -08:00
datagram.c	net: use indirect call wrappers for skb_copy_datagram_iter()	2020-03-25 11:30:40 -07:00
datagram.h	net/core: Allow the compiler to verify declaration and definition consistency	2019-03-27 13:49:44 -07:00
dev_addr_lists.c	net: remove unnecessary variables and callback	2019-10-24 14:53:49 -07:00
dev_ioctl.c	net: Introduce peer to peer one step PTP time stamping.	2019-12-25 19:51:34 -08:00
dev.c	netfilter: revert introduction of egress hook	2020-03-18 16:35:48 -07:00
devlink.c	devlink: Only pass packet trap group identifier in trap structure	2020-03-23 21:40:40 -07:00
drop_monitor.c	net: core: Replace zero-length array with flexible-array member	2020-02-28 12:08:37 -08:00
dst_cache.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152	2019-05-30 11:26:32 -07:00
dst.c	net: print proper warning on dst underflow	2019-09-26 09:05:56 +02:00
failover.c	failover: allow name change on IFF_UP slave interfaces	2019-04-10 22:12:26 -07:00
fib_notifier.c	net: fib_notifier: propagate extack down to the notifier block callback	2019-10-04 11:10:56 -07:00
fib_rules.c	net: fib_rules: Correctly set table field when table number exceeds 8 bits	2020-02-16 18:38:24 -08:00
filter.c	bpf: Add bpf_trampoline_ name prefix for DECLARE_BPF_DISPATCHER	2020-03-13 12:49:51 -07:00
flow_dissector.c	bpf: Use bpf_prog_run_pin_on_cpu() at simple call sites.	2020-02-24 16:20:09 -08:00
flow_offload.c	flow_offload: Add flow_match_ct to get rule ct match	2020-03-12 15:00:39 -07:00
gen_estimator.c	net_sched: gen_estimator: extend packet counter to 64bit	2019-11-06 21:51:36 -08:00
gen_stats.c	net_sched: add TCA_STATS_PKT64 attribute	2019-11-05 18:20:55 -08:00
gro_cells.c	gro_cells: make sure device is up in gro_cells_receive()	2019-03-10 11:07:14 -07:00
hwbm.c	net: hwbm: Make the hwbm_pool lock a mutex	2019-06-09 19:40:10 -07:00
link_watch.c	net: link_watch: prevent starvation when processing linkwatch wq	2019-07-01 19:02:47 -07:00
lwt_bpf.c	net: ipv6_stub: use ip6_dst_lookup_flow instead of ip6_dst_lookup	2019-12-04 12:27:13 -08:00
lwtunnel.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152	2019-05-30 11:26:32 -07:00
Makefile	ethtool: move to its own directory	2019-12-12 17:07:05 -08:00
neighbour.c	net: neigh: remove unused NEIGH_SYSCTL_MS_JIFFIES_ENTRY	2020-02-20 10:02:23 -08:00
net_namespace.c	netns: Constify exported functions	2020-01-17 13:25:24 +01:00
net-procfs.c	net: procfs: use index hashlist instead of name hashlist	2019-10-01 14:47:19 -07:00
net-sysfs.c	net-sysfs: add queue_change_owner()	2020-02-26 20:07:26 -08:00
net-sysfs.h	net-sysfs: add netdev_change_owner()	2020-02-26 20:07:25 -08:00
net-traces.c	page_pool: add tracepoints for page_pool with details need by XDP	2019-06-19 11:23:13 -04:00
netclassid_cgroup.c	cgroup, netclassid: periodically release file_lock on classid updating	2020-03-09 18:13:39 -07:00
netevent.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152	2019-05-30 11:26:32 -07:00
netpoll.c	net: fix skb use after free in netpoll	2019-08-27 20:52:02 -07:00
netprio_cgroup.c	netprio: use css ID instead of cgroup ID	2019-11-12 08:18:03 -08:00
page_pool.c	net: page_pool: API cleanup and comments	2020-02-20 10:09:25 -08:00
pktgen.c	pktgen: Allow on loopback device	2020-03-10 15:44:59 -07:00
ptp_classifier.c	treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 295	2019-06-05 17:36:38 +02:00
request_sock.c	tcp: add rcu protection around tp->fastopen_rsk	2019-10-13 10:13:08 -07:00
rtnetlink.c	Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net	2020-02-21 13:39:34 -08:00
scm.c	y2038: socket: remove timespec reference in timestamping	2019-11-15 14:38:29 +01:00
secure_seq.c	treewide: Add SPDX license identifier for missed files	2019-05-21 10:50:45 +02:00
skbuff.c	net: Make skb_segment not to compute checksum if network controller supports checksumming	2020-03-23 12:59:53 -07:00
skmsg.c	bpf: Use bpf_prog_run_pin_on_cpu() at simple call sites.	2020-02-24 16:20:09 -08:00
sock_diag.c	sock: make cookie generation global instead of per netns	2019-08-09 13:14:46 -07:00
sock_map.c	bpf: sockmap: Add UDP support	2020-03-09 22:34:58 +01:00
sock_reuseport.c	net: Generate reuseport group ID on group creation	2020-02-21 22:29:45 +01:00
sock.c	Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net	2020-03-12 22:34:48 -07:00
stream.c	tcp: make sure EPOLLOUT wont be missed	2019-08-19 13:07:43 -07:00
sysctl_net_core.c	net, sysctl: Fix compiler warning when only cBPF is present	2019-12-19 17:17:51 +01:00
timestamping.c	net: Introduce a new MII time stamping interface.	2019-12-25 19:51:33 -08:00
tso.c	net: Use skb accessors in network core	2019-07-22 20:47:56 -07:00
utils.c	net: Fix skb->csum update in inet_proto_csum_replace16().	2020-01-24 20:54:30 +01:00
xdp.c	net: page_pool: API cleanup and comments	2020-02-20 10:09:25 -08:00