linux/include/rdma
Mike Marciniszyn 4fbc3a52cd RDMA/core: Fix umem iterator when PAGE_SIZE is greater then HCA pgsz
64k pages introduce the situation in this diagram when the HCA 4k page
size is being used:

 +-------------------------------------------+ <--- 64k aligned VA
 |                                           |
 |              HCA 4k page                  |
 |                                           |
 +-------------------------------------------+
 |                   o                       |
 |                                           |
 |                   o                       |
 |                                           |
 |                   o                       |
 +-------------------------------------------+
 |                                           |
 |              HCA 4k page                  |
 |                                           |
 +-------------------------------------------+ <--- Live HCA page
 |OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO| <--- offset
 |                                           | <--- VA
 |                MR data                    |
 +-------------------------------------------+
 |                                           |
 |              HCA 4k page                  |
 |                                           |
 +-------------------------------------------+
 |                   o                       |
 |                                           |
 |                   o                       |
 |                                           |
 |                   o                       |
 +-------------------------------------------+
 |                                           |
 |              HCA 4k page                  |
 |                                           |
 +-------------------------------------------+

The VA addresses are coming from rdma-core in this diagram can be
arbitrary, but for 64k pages, the VA may be offset by some number of HCA
4k pages and followed by some number of HCA 4k pages.

The current iterator doesn't account for either the preceding 4k pages or
the following 4k pages.

Fix the issue by extending the ib_block_iter to contain the number of DMA
pages like comment [1] says and by using __sg_advance to start the
iterator at the first live HCA page.

The changes are contained in a parallel set of iterator start and next
functions that are umem aware and specific to umem since there is one user
of the rdma_for_each_block() without umem.

These two fixes prevents the extra pages before and after the user MR
data.

Fix the preceding pages by using the __sq_advance field to start at the
first 4k page containing MR data.

Fix the following pages by saving the number of pgsz blocks in the
iterator state and downcounting on each next.

This fix allows for the elimination of the small page crutch noted in the
Fixes.

Fixes: 10c75ccb54 ("RDMA/umem: Prevent small pages from being returned by ib_umem_find_best_pgsz()")
Link: https://lore.kernel.org/r/20231129202143.1434-2-shiraz.saleem@intel.com
Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Signed-off-by: Shiraz Saleem <shiraz.saleem@intel.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2023-12-04 20:02:41 -04:00
..
ib_addr.h RDMA/cma: Always set static rate to 0 for RoCE 2023-06-11 11:26:02 +03:00
ib_cache.h RDMA: Support more than 255 rdma ports 2021-03-26 09:31:21 -03:00
ib_cm.h RDMA/cm: Use DLID from inbound/outbound PathRecords as the datapath DLID 2022-09-22 12:35:31 +03:00
ib_hdrs.h RDMA/mlx5: Add steering support in optional flow counters 2021-10-12 12:48:06 -03:00
ib_mad.h IB/mlx5: Expose XDR speed through MAD 2023-09-26 12:38:43 +03:00
ib_marshall.h RDMA/include: Replace license text with SPDX tags 2020-07-29 14:48:36 -03:00
ib_pack.h RDMA: Extend RDMA kernel verbs ABI to support flush 2022-12-09 19:36:01 -04:00
ib_pma.h RDMA/include: Replace license text with SPDX tags 2020-07-29 14:48:36 -03:00
ib_sa.h RDMA/cma: Refactor the inbound/outbound path records process flow 2023-01-10 10:49:50 +02:00
ib_smi.h RDMA/mad: Delete duplicated init_query_mad functions 2022-01-05 15:18:36 -04:00
ib_sysfs.h RDMA: Change ops->init_port to ops->port_groups 2021-06-16 20:58:31 -03:00
ib_umem_odp.h IB/core: Enable ODP sync without faulting 2020-10-01 16:44:05 -03:00
ib_umem.h RDMA/core: Fix umem iterator when PAGE_SIZE is greater then HCA pgsz 2023-12-04 20:02:41 -04:00
ib_verbs.h RDMA/core: Fix umem iterator when PAGE_SIZE is greater then HCA pgsz 2023-12-04 20:02:41 -04:00
ib.h uaccess: remove CONFIG_SET_FS 2022-02-25 09:36:06 +01:00
iba.h RDMA/cm: Add SET/GET implementations to hide IBA wire format 2020-01-25 15:05:59 -04:00
ibta_vol1_c12.h RDMA/cm: Add Enhanced Connection Establishment (ECE) bits 2020-05-27 16:05:05 -03:00
iw_cm.h RDMA Remove unused function declarations 2023-08-13 10:32:35 +03:00
iw_portmap.h RDMA/include: Replace license text with SPDX tags 2020-07-29 14:48:36 -03:00
lag.h RDMA/core: Add LAG functionality 2020-05-02 20:19:54 -03:00
mr_pool.h Linux 5.2-rc6 2019-06-28 21:18:23 -03:00
opa_addr.h RDMA/include: Replace license text with SPDX tags 2020-07-29 14:48:36 -03:00
opa_port_info.h RDMA/include: Replace license text with SPDX tags 2020-07-29 14:48:36 -03:00
opa_smi.h RDMA/include: Replace license text with SPDX tags 2020-07-29 14:48:36 -03:00
opa_vnic.h RDMA/opa_vnic: fix spelling typo in comment 2022-10-19 10:02:37 +03:00
rdma_cm_ib.h RDMA/include: Replace license text with SPDX tags 2020-07-29 14:48:36 -03:00
rdma_cm.h RDMA/cma: Refactor the inbound/outbound path records process flow 2023-01-10 10:49:50 +02:00
rdma_counter.h RDMA/counter: Add optional counter support 2021-10-12 12:48:05 -03:00
rdma_netlink.h RDMA/netlink: Add __maybe_unused to static inline in C file 2021-11-16 13:13:08 -04:00
rdma_vt.h IB: move from strlcpy with unused retval to strscpy 2022-08-21 14:18:02 +03:00
rdmavt_cq.h RDMA/include: Replace license text with SPDX tags 2020-07-29 14:48:36 -03:00
rdmavt_mr.h RDMA/include: Replace license text with SPDX tags 2020-07-29 14:48:36 -03:00
rdmavt_qp.h RDMA/rdmavt: Decouple QP and SGE lists allocations 2021-08-03 13:44:27 -03:00
restrack.h RDMA/restrack: Correct spelling 2023-02-07 11:25:10 +02:00
rw.h RDMA: Support more than 255 rdma ports 2021-03-26 09:31:21 -03:00
signature.h RDMA: Make most headers compile stand alone 2019-07-25 13:58:47 -03:00
tid_rdma_defs.h IB/hfi1: Build TID RDMA WRITE request 2019-02-05 18:07:43 -05:00
uverbs_ioctl.h rdma: fix INFINIBAND_USER_ACCESS dependency 2023-07-03 16:55:04 -07:00
uverbs_named_ioctl.h RDMA/uverbs: Make UVERBS_OBJECT_METHODS to consider line number 2021-04-13 19:36:35 -03:00
uverbs_std_types.h RDMA/include: Replace license text with SPDX tags 2020-07-29 14:48:36 -03:00
uverbs_types.h RDMA/uverbs: Allow drivers to create a new HW object during rereg_mr 2020-12-07 14:06:23 -04:00