linux/net/sunrpc
Chuck Lever d8f532d20e xprtrdma: Invoke rpcrdma_reply_handler directly from RECV completion
I noticed that the soft IRQ thread looked pretty busy under heavy
I/O workloads. perf suggested one area that was expensive was the
queue_work() call in rpcrdma_wc_receive. That gave me some ideas.

Instead of scheduling a separate worker to process RPC Replies,
promote the Receive completion handler to IB_POLL_WORKQUEUE, and
invoke rpcrdma_reply_handler directly.

Note that the poll workqueue is single-threaded. In order to keep
memory invalidation from serializing all RPC Replies, handle any
necessary invalidation tasks in a separate multi-threaded workqueue.

This provides a two-tier scheme, similar to OS I/O interrupt
handlers: A fast interrupt handler that schedules the slow handler
and re-enables the interrupt, and a slower handler that is invoked
for any needed heavy lifting.

Benefits include:
- One less context switch for RPCs that don't register memory
- Receive completion handling is moved out of soft IRQ context to
  make room for other users of soft IRQ
- The same CPU core now DMA syncs and XDR decodes the Receive buffer

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
2017-11-17 13:47:54 -05:00
..
auth_gss NFS client updates for Linux 4.13 2017-07-13 14:35:37 -07:00
xprtrdma xprtrdma: Invoke rpcrdma_reply_handler directly from RECV completion 2017-11-17 13:47:54 -05:00
addr.c replace strict_strto calls 2014-07-12 18:45:49 -04:00
auth_generic.c NFS client updates for Linux 4.9 2016-10-13 21:28:20 -07:00
auth_null.c sunrpc: remove dead codes of cr_magic in rpc_cred 2017-02-08 17:02:46 -05:00
auth_unix.c sunrpc: rename NFS_NGROUPS to UNX_NGROUPS for auth unix 2017-02-08 17:02:45 -05:00
auth.c sched/headers: Prepare to remove <linux/cred.h> inclusion from <linux/sched.h> 2017-03-02 08:42:31 +01:00
backchannel_rqst.c SUNRPC: Don't hold the transport lock when receiving backchannel data 2017-08-16 15:10:16 -04:00
cache.c NFS client updates for Linux 4.11 2017-03-01 16:10:30 -08:00
clnt.c SUNRPC: remove some dead code. 2017-09-06 12:31:15 -04:00
debugfs.c sunrpc: record rpc client pointer in seq->private directly 2017-02-08 17:02:47 -05:00
Kconfig svcrdma: Introduce local rdma_rw API helpers 2017-04-25 17:25:55 -04:00
Makefile SUNRPC: Add a structure to track multiple transports 2016-02-05 18:48:54 -05:00
netns.h netns: make struct pernet_operations::id unsigned int 2016-11-18 10:59:15 -05:00
rpc_pipe.c fs: Replace CURRENT_TIME with current_time() for inode timestamps 2016-09-27 21:06:21 -04:00
rpcb_clnt.c sunrpc: mark all struct rpc_procinfo instances as const 2017-07-13 15:57:57 -04:00
sched.c sunrpc: don't check for failure from mempool_alloc() 2017-04-20 13:44:57 -04:00
socklib.c sunrpc: do not pull udp headers on receive 2016-04-11 15:31:33 -04:00
stats.c sunrpc: move pc_count out of struct svc_procinfo 2017-07-13 15:58:02 -04:00
sunrpc_syms.c SUNRPC: cleanup ida information when removing sunrpc module 2017-01-24 15:29:24 -05:00
sunrpc.h SUNRPC: track whether a request is coming from a loop-back interface. 2014-05-22 15:59:18 -04:00
svc_xprt.c sunrpc: Allocate up to RPCSVC_MAXPAGES per svc_rqst 2017-07-12 15:54:55 -04:00
svc.c sunrpc: Const-ify struct sv_serv_ops 2017-08-24 22:13:50 -04:00
svcauth_unix.c sunrpc: rename NFS_NGROUPS to UNX_NGROUPS for auth unix 2017-02-08 17:02:45 -05:00
svcauth.c locking/atomic, kref: Implement kref_put_lock() 2017-01-18 10:03:29 +01:00
svcsock.c NFS client updates for Linux 4.14 2017-09-11 22:01:44 -07:00
sysctl.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
timer.c net: cleanup unsigned to unsigned int 2012-04-15 12:44:40 -04:00
xdr.c sunrpc: Fix xdr_init_decode_pages() documenting comment 2017-04-25 16:12:34 -04:00
xprt.c xprtrdma: Use xprt_pin_rqst in rpcrdma_reply_handler 2017-09-05 18:27:07 -04:00
xprtmultipath.c SUNRPC search xprt switch for sockaddr 2016-09-19 13:08:36 -04:00
xprtsock.c sunrpc: remove redundant initialization of sock 2017-10-01 18:51:30 -04:00