linux/include/rdma
Israel Rukshin e9a53e73a2 RDMA/rw: Use IB_WR_REG_MR_INTEGRITY for PI handover
Replace the old signature handover API with the new one. The new API
simplifes PI handover code complexity for ULPs and improve performance.
For RW API it will reduce the maximum number of work requests per task
and the need of dealing with multiple MRs (and their registrations and
invalidations) per task. All the mappings and registration of the data
and the protection buffers is done by the LLD using a single WR and a
special MR type (IB_MR_TYPE_INTEGRITY) for the PI handover operation.

The setup of the tested benchmark (using iSER ULP):
 - 2 servers with 24 cores (1 initiator and 1 target)
 - ConnectX-4/ConnectX-5 adapters
 - 24 target sessions with 1 LUN each
 - ramdisk backstore
 - PI active

Performance results running fio (24 jobs, 128 iodepth) using
write_generate=1 and read_verify=1 (w/w.o patch):

bs      IOPS(read)        IOPS(write)
----    ----------        ----------
512   1243.3K/1182.3K    1725.1K/1680.2K
4k    571233/528835      743293/748259
32k   72388/71086        71789/93573

Using write_generate=0 and read_verify=0 (w/w.o patch):
bs      IOPS(read)        IOPS(write)
----    ----------        ----------
512   1572.1K/1427.2K    1823.5K/1724.3K
4k    921992/916194      753772/768267
32k   75052/73960        73180/95484

There is a performance degradation when writing big block sizes.
Degradation is caused by the complexity of combining multiple
indirections and perform RDMA READ operation from it. This will be
fixed in the following patches by reducing the indirections if
possible.

Signed-off-by: Israel Rukshin <israelr@mellanox.com>
Reviewed-by: Max Gurtovoy <maxg@mellanox.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2019-06-24 11:49:28 -03:00
..
ib_addr.h RDMA/core: Annotate timeout as unsigned long 2018-10-16 13:34:01 -04:00
ib_cache.h RDMA/cma: Use rdma_read_gid_attr_ndev_rcu to access netdev 2019-05-03 11:10:03 -03:00
ib_cm.h RDMA/core: Annotate timeout as unsigned long 2018-10-16 13:34:01 -04:00
ib_fmr_pool.h IB/core: Make function ib_fmr_pool_unmap return void 2018-11-21 16:13:02 -07:00
ib_hdrs.h IB/hfi1: Build TID RDMA WRITE request 2019-02-05 18:07:43 -05:00
ib_mad.h RDMA: Use __packed annotation instead of __attribute__ ((packed)) 2019-03-25 21:14:12 -03:00
ib_marshall.h IB/core: Convert ah_attr from OPA to IB when copying to user 2017-08-08 14:47:18 -04:00
ib_pack.h IB/core: Fix calculation of maximum RoCE MTU 2017-10-18 12:11:36 -04:00
ib_pma.h IB/core: Display extended counter set if available 2015-12-23 15:58:30 -05:00
ib_sa.h RDMA/core: Annotate timeout as unsigned long 2018-10-16 13:34:01 -04:00
ib_smi.h RDMA: Use __packed annotation instead of __attribute__ ((packed)) 2019-03-25 21:14:12 -03:00
ib_umem_odp.h RDMA/umem: Move page_shift from ib_umem to ib_odp_umem 2019-05-21 15:23:24 -03:00
ib_umem.h RDMA/umem: Move page_shift from ib_umem to ib_odp_umem 2019-05-21 15:23:24 -03:00
ib_verbs.h RDMA/core: Validate integrity handover device cap 2019-06-24 11:49:27 -03:00
ib.h RDMA/core: Return bool instead of int 2018-07-30 20:49:04 -06:00
iw_cm.h RDMA: Get rid of iw_cm_verbs 2019-05-03 10:56:56 -03:00
iw_portmap.h RDMA/iwpm: move kdoc comments to functions 2019-02-05 15:40:41 -07:00
mr_pool.h RDMA/core: Add an integrity MR pool support 2019-06-24 11:49:27 -03:00
opa_addr.h include/rdma/opa_addr.h: Fix an endianness issue 2018-07-03 14:11:34 -06:00
opa_port_info.h RDMA: Use __packed annotation instead of __attribute__ ((packed)) 2019-03-25 21:14:12 -03:00
opa_smi.h RDMA: Use __packed annotation instead of __attribute__ ((packed)) 2019-03-25 21:14:12 -03:00
opa_vnic.h IB/hfi1: Add support to receive 16B bypass packets 2017-08-22 14:22:37 -04:00
rdma_cm_ib.h RDMA/{cma, ucma}: Simplify and rename rdma_set_ib_paths 2018-01-10 22:00:33 -07:00
rdma_cm.h IB/cma: Define option to set ack timeout and pack tos_set 2019-02-08 16:14:21 -07:00
rdma_netlink.h RDMA: Add NLDEV_GET_CHARDEV to allow char dev discovery and autoload 2019-06-18 22:41:05 -04:00
rdma_vt.h RDMA: Move driver_id into struct ib_device_ops 2019-06-10 16:56:02 -03:00
rdmavt_cq.h IB/{hfi1, rdmavt, qib}: Implement CQ completion vector support 2018-05-09 15:53:30 -04:00
rdmavt_mr.h IB/rdmavt: Handle dereg of inuse MRs properly 2017-08-28 19:12:31 -04:00
rdmavt_qp.h IB/{rdmavt, qib, hfi1}: Convert to new completion API 2019-06-20 22:35:09 -04:00
restrack.h RDMA/restrack: Hide restrack DB from IB/core 2019-02-18 21:04:36 -07:00
rw.h RDMA/rw: Use IB_WR_REG_MR_INTEGRITY for PI handover 2019-06-24 11:49:28 -03:00
signature.h RDMA/core: Add signature attrs element for ib_mr structure 2019-06-24 11:49:26 -03:00
tid_rdma_defs.h IB/hfi1: Build TID RDMA WRITE request 2019-02-05 18:07:43 -05:00
uverbs_ioctl.h IB/verbs: Add helper function rdma_udata_to_drv_context 2019-02-15 11:17:25 -07:00
uverbs_named_ioctl.h RDMA/uverbs: Fix typo in string concatenation macro 2018-12-06 21:11:06 -07:00
uverbs_std_types.h IB: When attrs.udata/ufile is available use that instead of uobject 2019-04-08 13:05:25 -03:00
uverbs_types.h IB: Pass uverbs_attr_bundle down uobject destroy path 2019-04-01 14:55:36 -03:00