2007-02-13 08:16:18 +08:00
|
|
|
/*
|
|
|
|
* Copyright (c) 2006 Chelsio, Inc. All rights reserved.
|
|
|
|
*
|
|
|
|
* This software is available to you under a choice of one of two
|
|
|
|
* licenses. You may choose to be licensed under the terms of the GNU
|
|
|
|
* General Public License (GPL) Version 2, available from the file
|
|
|
|
* COPYING in the main directory of this source tree, or the
|
|
|
|
* OpenIB.org BSD license below:
|
|
|
|
*
|
|
|
|
* Redistribution and use in source and binary forms, with or
|
|
|
|
* without modification, are permitted provided that the following
|
|
|
|
* conditions are met:
|
|
|
|
*
|
|
|
|
* - Redistributions of source code must retain the above
|
|
|
|
* copyright notice, this list of conditions and the following
|
|
|
|
* disclaimer.
|
|
|
|
*
|
|
|
|
* - Redistributions in binary form must reproduce the above
|
|
|
|
* copyright notice, this list of conditions and the following
|
|
|
|
* disclaimer in the documentation and/or other materials
|
|
|
|
* provided with the distribution.
|
|
|
|
*
|
|
|
|
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
|
|
|
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
|
|
|
* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
|
|
|
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
|
|
|
|
* BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
|
|
|
|
* ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
|
|
|
|
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
|
|
* SOFTWARE.
|
|
|
|
*/
|
|
|
|
#include <linux/module.h>
|
|
|
|
#include <linux/moduleparam.h>
|
|
|
|
#include <linux/device.h>
|
|
|
|
#include <linux/netdevice.h>
|
|
|
|
#include <linux/etherdevice.h>
|
|
|
|
#include <linux/delay.h>
|
|
|
|
#include <linux/errno.h>
|
|
|
|
#include <linux/list.h>
|
2009-10-07 21:09:06 +08:00
|
|
|
#include <linux/sched.h>
|
2007-02-13 08:16:18 +08:00
|
|
|
#include <linux/spinlock.h>
|
|
|
|
#include <linux/ethtool.h>
|
2007-11-27 01:28:44 +08:00
|
|
|
#include <linux/rtnetlink.h>
|
2009-05-28 05:42:36 +08:00
|
|
|
#include <linux/inetdevice.h>
|
include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files. percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.
percpu.h -> slab.h dependency is about to be removed. Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability. As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.
http://userweb.kernel.org/~tj/misc/slabh-sweep.py
The script does the followings.
* Scan files for gfp and slab usages and update includes such that
only the necessary includes are there. ie. if only gfp is used,
gfp.h, if slab is used, slab.h.
* When the script inserts a new include, it looks at the include
blocks and try to put the new include such that its order conforms
to its surrounding. It's put in the include block which contains
core kernel includes, in the same order that the rest are ordered -
alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
doesn't seem to be any matching order.
* If the script can't find a place to put a new include (mostly
because the file doesn't have fitting include block), it prints out
an error message indicating which .h file needs to be added to the
file.
The conversion was done in the following steps.
1. The initial automatic conversion of all .c files updated slightly
over 4000 files, deleting around 700 includes and adding ~480 gfp.h
and ~3000 slab.h inclusions. The script emitted errors for ~400
files.
2. Each error was manually checked. Some didn't need the inclusion,
some needed manual addition while adding it to implementation .h or
embedding .c file was more appropriate for others. This step added
inclusions to around 150 files.
3. The script was run again and the output was compared to the edits
from #2 to make sure no file was left behind.
4. Several build tests were done and a couple of problems were fixed.
e.g. lib/decompress_*.c used malloc/free() wrappers around slab
APIs requiring slab.h to be added manually.
5. The script was run on all .h files but without automatically
editing them as sprinkling gfp.h and slab.h inclusions around .h
files could easily lead to inclusion dependency hell. Most gfp.h
inclusion directives were ignored as stuff from gfp.h was usually
wildly available and often used in preprocessor macros. Each
slab.h inclusion directive was examined and added manually as
necessary.
6. percpu.h was updated not to include slab.h.
7. Build test were done on the following configurations and failures
were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
distributed build env didn't work with gcov compiles) and a few
more options had to be turned off depending on archs to make things
build (like ipr on powerpc/64 which failed due to missing writeq).
* x86 and x86_64 UP and SMP allmodconfig and a custom test config.
* powerpc and powerpc64 SMP allmodconfig
* sparc and sparc64 SMP allmodconfig
* ia64 SMP allmodconfig
* s390 SMP allmodconfig
* alpha SMP allmodconfig
* um on x86_64 SMP allmodconfig
8. percpu.h modifications were reverted so that it could be applied as
a separate patch and serve as bisection point.
Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
2010-03-24 16:04:11 +08:00
|
|
|
#include <linux/slab.h>
|
2007-02-13 08:16:18 +08:00
|
|
|
|
|
|
|
#include <asm/io.h>
|
|
|
|
#include <asm/irq.h>
|
|
|
|
#include <asm/byteorder.h>
|
|
|
|
|
|
|
|
#include <rdma/iw_cm.h>
|
|
|
|
#include <rdma/ib_verbs.h>
|
|
|
|
#include <rdma/ib_smi.h>
|
2007-03-05 08:15:11 +08:00
|
|
|
#include <rdma/ib_umem.h>
|
2007-02-13 08:16:18 +08:00
|
|
|
#include <rdma/ib_user_verbs.h>
|
|
|
|
|
|
|
|
#include "cxio_hal.h"
|
|
|
|
#include "iwch.h"
|
|
|
|
#include "iwch_provider.h"
|
|
|
|
#include "iwch_cm.h"
|
|
|
|
#include "iwch_user.h"
|
2008-07-15 14:48:48 +08:00
|
|
|
#include "common.h"
|
2007-02-13 08:16:18 +08:00
|
|
|
|
|
|
|
static struct ib_ah *iwch_ah_create(struct ib_pd *pd,
|
|
|
|
struct ib_ah_attr *ah_attr)
|
|
|
|
{
|
|
|
|
return ERR_PTR(-ENOSYS);
|
|
|
|
}
|
|
|
|
|
|
|
|
static int iwch_ah_destroy(struct ib_ah *ah)
|
|
|
|
{
|
|
|
|
return -ENOSYS;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int iwch_multicast_attach(struct ib_qp *ibqp, union ib_gid *gid, u16 lid)
|
|
|
|
{
|
|
|
|
return -ENOSYS;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int iwch_multicast_detach(struct ib_qp *ibqp, union ib_gid *gid, u16 lid)
|
|
|
|
{
|
|
|
|
return -ENOSYS;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int iwch_process_mad(struct ib_device *ibdev,
|
|
|
|
int mad_flags,
|
|
|
|
u8 port_num,
|
2015-06-01 05:15:30 +08:00
|
|
|
const struct ib_wc *in_wc,
|
|
|
|
const struct ib_grh *in_grh,
|
2015-06-07 02:38:31 +08:00
|
|
|
const struct ib_mad_hdr *in_mad,
|
|
|
|
size_t in_mad_size,
|
|
|
|
struct ib_mad_hdr *out_mad,
|
|
|
|
size_t *out_mad_size,
|
|
|
|
u16 *out_mad_pkey_index)
|
2007-02-13 08:16:18 +08:00
|
|
|
{
|
|
|
|
return -ENOSYS;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int iwch_dealloc_ucontext(struct ib_ucontext *context)
|
|
|
|
{
|
|
|
|
struct iwch_dev *rhp = to_iwch_dev(context->device);
|
|
|
|
struct iwch_ucontext *ucontext = to_iwch_ucontext(context);
|
|
|
|
struct iwch_mm_entry *mm, *tmp;
|
|
|
|
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s context %p\n", __func__, context);
|
2007-02-13 08:16:18 +08:00
|
|
|
list_for_each_entry_safe(mm, tmp, &ucontext->mmaps, entry)
|
|
|
|
kfree(mm);
|
|
|
|
cxio_release_ucontext(&rhp->rdev, &ucontext->uctx);
|
|
|
|
kfree(ucontext);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
static struct ib_ucontext *iwch_alloc_ucontext(struct ib_device *ibdev,
|
|
|
|
struct ib_udata *udata)
|
|
|
|
{
|
|
|
|
struct iwch_ucontext *context;
|
|
|
|
struct iwch_dev *rhp = to_iwch_dev(ibdev);
|
|
|
|
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s ibdev %p\n", __func__, ibdev);
|
2007-02-13 08:16:18 +08:00
|
|
|
context = kzalloc(sizeof(*context), GFP_KERNEL);
|
|
|
|
if (!context)
|
|
|
|
return ERR_PTR(-ENOMEM);
|
|
|
|
cxio_init_ucontext(&rhp->rdev, &context->uctx);
|
|
|
|
INIT_LIST_HEAD(&context->mmaps);
|
|
|
|
spin_lock_init(&context->mmap_lock);
|
|
|
|
return &context->ibucontext;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int iwch_destroy_cq(struct ib_cq *ib_cq)
|
|
|
|
{
|
|
|
|
struct iwch_cq *chp;
|
|
|
|
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s ib_cq %p\n", __func__, ib_cq);
|
2007-02-13 08:16:18 +08:00
|
|
|
chp = to_iwch_cq(ib_cq);
|
|
|
|
|
|
|
|
remove_handle(chp->rhp, &chp->rhp->cqidr, chp->cq.cqid);
|
|
|
|
atomic_dec(&chp->refcnt);
|
|
|
|
wait_event(chp->wait, !atomic_read(&chp->refcnt));
|
|
|
|
|
|
|
|
cxio_destroy_cq(&chp->rhp->rdev, &chp->cq);
|
|
|
|
kfree(chp);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2015-06-11 21:35:20 +08:00
|
|
|
static struct ib_cq *iwch_create_cq(struct ib_device *ibdev,
|
|
|
|
const struct ib_cq_init_attr *attr,
|
|
|
|
struct ib_ucontext *ib_context,
|
|
|
|
struct ib_udata *udata)
|
2007-02-13 08:16:18 +08:00
|
|
|
{
|
2015-06-11 21:35:20 +08:00
|
|
|
int entries = attr->cqe;
|
2007-02-13 08:16:18 +08:00
|
|
|
struct iwch_dev *rhp;
|
|
|
|
struct iwch_cq *chp;
|
|
|
|
struct iwch_create_cq_resp uresp;
|
|
|
|
struct iwch_create_cq_req ureq;
|
|
|
|
struct iwch_ucontext *ucontext = NULL;
|
2010-10-21 20:37:06 +08:00
|
|
|
static int warned;
|
|
|
|
size_t resplen;
|
2007-02-13 08:16:18 +08:00
|
|
|
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s ib_dev %p entries %d\n", __func__, ibdev, entries);
|
2015-06-11 21:35:20 +08:00
|
|
|
if (attr->flags)
|
|
|
|
return ERR_PTR(-EINVAL);
|
|
|
|
|
2007-02-13 08:16:18 +08:00
|
|
|
rhp = to_iwch_dev(ibdev);
|
|
|
|
chp = kzalloc(sizeof(*chp), GFP_KERNEL);
|
|
|
|
if (!chp)
|
|
|
|
return ERR_PTR(-ENOMEM);
|
|
|
|
|
|
|
|
if (ib_context) {
|
|
|
|
ucontext = to_iwch_ucontext(ib_context);
|
|
|
|
if (!t3a_device(rhp)) {
|
|
|
|
if (ib_copy_from_udata(&ureq, udata, sizeof (ureq))) {
|
|
|
|
kfree(chp);
|
|
|
|
return ERR_PTR(-EFAULT);
|
|
|
|
}
|
|
|
|
chp->user_rptr_addr = (u32 __user *)(unsigned long)ureq.user_rptr_addr;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
if (t3a_device(rhp)) {
|
|
|
|
|
|
|
|
/*
|
|
|
|
* T3A: Add some fluff to handle extra CQEs inserted
|
|
|
|
* for various errors.
|
|
|
|
* Additional CQE possibilities:
|
|
|
|
* TERMINATE,
|
|
|
|
* incoming RDMA WRITE Failures
|
|
|
|
* incoming RDMA READ REQUEST FAILUREs
|
|
|
|
* NOTE: We cannot ensure the CQ won't overflow.
|
|
|
|
*/
|
|
|
|
entries += 16;
|
|
|
|
}
|
|
|
|
entries = roundup_pow_of_two(entries);
|
|
|
|
chp->cq.size_log2 = ilog2(entries);
|
|
|
|
|
2010-01-28 04:22:34 +08:00
|
|
|
if (cxio_create_cq(&rhp->rdev, &chp->cq, !ucontext)) {
|
2007-02-13 08:16:18 +08:00
|
|
|
kfree(chp);
|
|
|
|
return ERR_PTR(-ENOMEM);
|
|
|
|
}
|
|
|
|
chp->rhp = rhp;
|
2008-03-10 04:54:12 +08:00
|
|
|
chp->ibcq.cqe = 1 << chp->cq.size_log2;
|
2007-02-13 08:16:18 +08:00
|
|
|
spin_lock_init(&chp->lock);
|
2011-10-24 23:50:22 +08:00
|
|
|
spin_lock_init(&chp->comp_handler_lock);
|
2007-02-13 08:16:18 +08:00
|
|
|
atomic_set(&chp->refcnt, 1);
|
|
|
|
init_waitqueue_head(&chp->wait);
|
2009-09-10 02:25:55 +08:00
|
|
|
if (insert_handle(rhp, &rhp->cqidr, chp, chp->cq.cqid)) {
|
|
|
|
cxio_destroy_cq(&chp->rhp->rdev, &chp->cq);
|
|
|
|
kfree(chp);
|
|
|
|
return ERR_PTR(-ENOMEM);
|
|
|
|
}
|
2007-02-13 08:16:18 +08:00
|
|
|
|
|
|
|
if (ucontext) {
|
|
|
|
struct iwch_mm_entry *mm;
|
|
|
|
|
|
|
|
mm = kmalloc(sizeof *mm, GFP_KERNEL);
|
|
|
|
if (!mm) {
|
|
|
|
iwch_destroy_cq(&chp->ibcq);
|
|
|
|
return ERR_PTR(-ENOMEM);
|
|
|
|
}
|
|
|
|
uresp.cqid = chp->cq.cqid;
|
|
|
|
uresp.size_log2 = chp->cq.size_log2;
|
|
|
|
spin_lock(&ucontext->mmap_lock);
|
|
|
|
uresp.key = ucontext->key;
|
|
|
|
ucontext->key += PAGE_SIZE;
|
|
|
|
spin_unlock(&ucontext->mmap_lock);
|
2010-10-21 20:37:06 +08:00
|
|
|
mm->key = uresp.key;
|
|
|
|
mm->addr = virt_to_phys(chp->cq.queue);
|
|
|
|
if (udata->outlen < sizeof uresp) {
|
|
|
|
if (!warned++)
|
|
|
|
printk(KERN_WARNING MOD "Warning - "
|
|
|
|
"downlevel libcxgb3 (non-fatal).\n");
|
|
|
|
mm->len = PAGE_ALIGN((1UL << uresp.size_log2) *
|
|
|
|
sizeof(struct t3_cqe));
|
|
|
|
resplen = sizeof(struct iwch_create_cq_resp_v0);
|
|
|
|
} else {
|
|
|
|
mm->len = PAGE_ALIGN(((1UL << uresp.size_log2) + 1) *
|
|
|
|
sizeof(struct t3_cqe));
|
|
|
|
uresp.memsize = mm->len;
|
2013-07-30 03:19:14 +08:00
|
|
|
uresp.reserved = 0;
|
2010-10-21 20:37:06 +08:00
|
|
|
resplen = sizeof uresp;
|
|
|
|
}
|
|
|
|
if (ib_copy_to_udata(udata, &uresp, resplen)) {
|
2007-02-13 08:16:18 +08:00
|
|
|
kfree(mm);
|
|
|
|
iwch_destroy_cq(&chp->ibcq);
|
|
|
|
return ERR_PTR(-EFAULT);
|
|
|
|
}
|
|
|
|
insert_mmap(ucontext, mm);
|
|
|
|
}
|
|
|
|
PDBG("created cqid 0x%0x chp %p size 0x%0x, dma_addr 0x%0llx\n",
|
|
|
|
chp->cq.cqid, chp, (1 << chp->cq.size_log2),
|
|
|
|
(unsigned long long) chp->cq.dma_addr);
|
|
|
|
return &chp->ibcq;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int iwch_resize_cq(struct ib_cq *cq, int cqe, struct ib_udata *udata)
|
|
|
|
{
|
|
|
|
#ifdef notyet
|
|
|
|
struct iwch_cq *chp = to_iwch_cq(cq);
|
|
|
|
struct t3_cq oldcq, newcq;
|
|
|
|
int ret;
|
|
|
|
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s ib_cq %p cqe %d\n", __func__, cq, cqe);
|
2007-02-13 08:16:18 +08:00
|
|
|
|
|
|
|
/* We don't downsize... */
|
|
|
|
if (cqe <= cq->cqe)
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
/* create new t3_cq with new size */
|
|
|
|
cqe = roundup_pow_of_two(cqe+1);
|
|
|
|
newcq.size_log2 = ilog2(cqe);
|
|
|
|
|
|
|
|
/* Dont allow resize to less than the current wce count */
|
|
|
|
if (cqe < Q_COUNT(chp->cq.rptr, chp->cq.wptr)) {
|
|
|
|
return -ENOMEM;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Quiesce all QPs using this CQ */
|
|
|
|
ret = iwch_quiesce_qps(chp);
|
|
|
|
if (ret) {
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
|
|
|
ret = cxio_create_cq(&chp->rhp->rdev, &newcq);
|
|
|
|
if (ret) {
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* copy CQEs */
|
|
|
|
memcpy(newcq.queue, chp->cq.queue, (1 << chp->cq.size_log2) *
|
|
|
|
sizeof(struct t3_cqe));
|
|
|
|
|
|
|
|
/* old iwch_qp gets new t3_cq but keeps old cqid */
|
|
|
|
oldcq = chp->cq;
|
|
|
|
chp->cq = newcq;
|
|
|
|
chp->cq.cqid = oldcq.cqid;
|
|
|
|
|
|
|
|
/* resize new t3_cq to update the HW context */
|
|
|
|
ret = cxio_resize_cq(&chp->rhp->rdev, &chp->cq);
|
|
|
|
if (ret) {
|
|
|
|
chp->cq = oldcq;
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
chp->ibcq.cqe = (1<<chp->cq.size_log2) - 1;
|
|
|
|
|
|
|
|
/* destroy old t3_cq */
|
|
|
|
oldcq.cqid = newcq.cqid;
|
|
|
|
ret = cxio_destroy_cq(&chp->rhp->rdev, &oldcq);
|
|
|
|
if (ret) {
|
|
|
|
printk(KERN_ERR MOD "%s - cxio_destroy_cq failed %d\n",
|
2008-04-17 12:01:10 +08:00
|
|
|
__func__, ret);
|
2007-02-13 08:16:18 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
/* add user hooks here */
|
|
|
|
|
|
|
|
/* resume qps */
|
|
|
|
ret = iwch_resume_qps(chp);
|
|
|
|
return ret;
|
|
|
|
#else
|
|
|
|
return -ENOSYS;
|
|
|
|
#endif
|
|
|
|
}
|
|
|
|
|
IB: Return "maybe missed event" hint from ib_req_notify_cq()
The semantics defined by the InfiniBand specification say that
completion events are only generated when a completions is added to a
completion queue (CQ) after completion notification is requested. In
other words, this means that the following race is possible:
while (CQ is not empty)
ib_poll_cq(CQ);
// new completion is added after while loop is exited
ib_req_notify_cq(CQ);
// no event is generated for the existing completion
To close this race, the IB spec recommends doing another poll of the
CQ after requesting notification.
However, it is not always possible to arrange code this way (for
example, we have found that NAPI for IPoIB cannot poll after
requesting notification). Also, some hardware (eg Mellanox HCAs)
actually will generate an event for completions added before the call
to ib_req_notify_cq() -- which is allowed by the spec, since there's
no way for any upper-layer consumer to know exactly when a completion
was really added -- so the extra poll of the CQ is just a waste.
Motivated by this, we add a new flag "IB_CQ_REPORT_MISSED_EVENTS" for
ib_req_notify_cq() so that it can return a hint about whether the a
completion may have been added before the request for notification.
The return value of ib_req_notify_cq() is extended so:
< 0 means an error occurred while requesting notification
== 0 means notification was requested successfully, and if
IB_CQ_REPORT_MISSED_EVENTS was passed in, then no
events were missed and it is safe to wait for another
event.
> 0 is only returned if IB_CQ_REPORT_MISSED_EVENTS was
passed in. It means that the consumer must poll the
CQ again to make sure it is empty to avoid the race
described above.
We add a flag to enable this behavior rather than turning it on
unconditionally, because checking for missed events may incur
significant overhead for some low-level drivers, and consumers that
don't care about the results of this test shouldn't be forced to pay
for the test.
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2007-05-07 12:02:48 +08:00
|
|
|
static int iwch_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags)
|
2007-02-13 08:16:18 +08:00
|
|
|
{
|
|
|
|
struct iwch_dev *rhp;
|
|
|
|
struct iwch_cq *chp;
|
|
|
|
enum t3_cq_opcode cq_op;
|
|
|
|
int err;
|
|
|
|
unsigned long flag;
|
|
|
|
u32 rptr;
|
|
|
|
|
|
|
|
chp = to_iwch_cq(ibcq);
|
|
|
|
rhp = chp->rhp;
|
IB: Return "maybe missed event" hint from ib_req_notify_cq()
The semantics defined by the InfiniBand specification say that
completion events are only generated when a completions is added to a
completion queue (CQ) after completion notification is requested. In
other words, this means that the following race is possible:
while (CQ is not empty)
ib_poll_cq(CQ);
// new completion is added after while loop is exited
ib_req_notify_cq(CQ);
// no event is generated for the existing completion
To close this race, the IB spec recommends doing another poll of the
CQ after requesting notification.
However, it is not always possible to arrange code this way (for
example, we have found that NAPI for IPoIB cannot poll after
requesting notification). Also, some hardware (eg Mellanox HCAs)
actually will generate an event for completions added before the call
to ib_req_notify_cq() -- which is allowed by the spec, since there's
no way for any upper-layer consumer to know exactly when a completion
was really added -- so the extra poll of the CQ is just a waste.
Motivated by this, we add a new flag "IB_CQ_REPORT_MISSED_EVENTS" for
ib_req_notify_cq() so that it can return a hint about whether the a
completion may have been added before the request for notification.
The return value of ib_req_notify_cq() is extended so:
< 0 means an error occurred while requesting notification
== 0 means notification was requested successfully, and if
IB_CQ_REPORT_MISSED_EVENTS was passed in, then no
events were missed and it is safe to wait for another
event.
> 0 is only returned if IB_CQ_REPORT_MISSED_EVENTS was
passed in. It means that the consumer must poll the
CQ again to make sure it is empty to avoid the race
described above.
We add a flag to enable this behavior rather than turning it on
unconditionally, because checking for missed events may incur
significant overhead for some low-level drivers, and consumers that
don't care about the results of this test shouldn't be forced to pay
for the test.
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2007-05-07 12:02:48 +08:00
|
|
|
if ((flags & IB_CQ_SOLICITED_MASK) == IB_CQ_SOLICITED)
|
2007-02-13 08:16:18 +08:00
|
|
|
cq_op = CQ_ARM_SE;
|
|
|
|
else
|
|
|
|
cq_op = CQ_ARM_AN;
|
|
|
|
if (chp->user_rptr_addr) {
|
|
|
|
if (get_user(rptr, chp->user_rptr_addr))
|
|
|
|
return -EFAULT;
|
|
|
|
spin_lock_irqsave(&chp->lock, flag);
|
|
|
|
chp->cq.rptr = rptr;
|
|
|
|
} else
|
|
|
|
spin_lock_irqsave(&chp->lock, flag);
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s rptr 0x%x\n", __func__, chp->cq.rptr);
|
2007-02-13 08:16:18 +08:00
|
|
|
err = cxio_hal_cq_op(&rhp->rdev, &chp->cq, cq_op, 0);
|
|
|
|
spin_unlock_irqrestore(&chp->lock, flag);
|
IB: Return "maybe missed event" hint from ib_req_notify_cq()
The semantics defined by the InfiniBand specification say that
completion events are only generated when a completions is added to a
completion queue (CQ) after completion notification is requested. In
other words, this means that the following race is possible:
while (CQ is not empty)
ib_poll_cq(CQ);
// new completion is added after while loop is exited
ib_req_notify_cq(CQ);
// no event is generated for the existing completion
To close this race, the IB spec recommends doing another poll of the
CQ after requesting notification.
However, it is not always possible to arrange code this way (for
example, we have found that NAPI for IPoIB cannot poll after
requesting notification). Also, some hardware (eg Mellanox HCAs)
actually will generate an event for completions added before the call
to ib_req_notify_cq() -- which is allowed by the spec, since there's
no way for any upper-layer consumer to know exactly when a completion
was really added -- so the extra poll of the CQ is just a waste.
Motivated by this, we add a new flag "IB_CQ_REPORT_MISSED_EVENTS" for
ib_req_notify_cq() so that it can return a hint about whether the a
completion may have been added before the request for notification.
The return value of ib_req_notify_cq() is extended so:
< 0 means an error occurred while requesting notification
== 0 means notification was requested successfully, and if
IB_CQ_REPORT_MISSED_EVENTS was passed in, then no
events were missed and it is safe to wait for another
event.
> 0 is only returned if IB_CQ_REPORT_MISSED_EVENTS was
passed in. It means that the consumer must poll the
CQ again to make sure it is empty to avoid the race
described above.
We add a flag to enable this behavior rather than turning it on
unconditionally, because checking for missed events may incur
significant overhead for some low-level drivers, and consumers that
don't care about the results of this test shouldn't be forced to pay
for the test.
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2007-05-07 12:02:48 +08:00
|
|
|
if (err < 0)
|
2007-02-13 08:16:18 +08:00
|
|
|
printk(KERN_ERR MOD "Error %d rearming CQID 0x%x\n", err,
|
|
|
|
chp->cq.cqid);
|
IB: Return "maybe missed event" hint from ib_req_notify_cq()
The semantics defined by the InfiniBand specification say that
completion events are only generated when a completions is added to a
completion queue (CQ) after completion notification is requested. In
other words, this means that the following race is possible:
while (CQ is not empty)
ib_poll_cq(CQ);
// new completion is added after while loop is exited
ib_req_notify_cq(CQ);
// no event is generated for the existing completion
To close this race, the IB spec recommends doing another poll of the
CQ after requesting notification.
However, it is not always possible to arrange code this way (for
example, we have found that NAPI for IPoIB cannot poll after
requesting notification). Also, some hardware (eg Mellanox HCAs)
actually will generate an event for completions added before the call
to ib_req_notify_cq() -- which is allowed by the spec, since there's
no way for any upper-layer consumer to know exactly when a completion
was really added -- so the extra poll of the CQ is just a waste.
Motivated by this, we add a new flag "IB_CQ_REPORT_MISSED_EVENTS" for
ib_req_notify_cq() so that it can return a hint about whether the a
completion may have been added before the request for notification.
The return value of ib_req_notify_cq() is extended so:
< 0 means an error occurred while requesting notification
== 0 means notification was requested successfully, and if
IB_CQ_REPORT_MISSED_EVENTS was passed in, then no
events were missed and it is safe to wait for another
event.
> 0 is only returned if IB_CQ_REPORT_MISSED_EVENTS was
passed in. It means that the consumer must poll the
CQ again to make sure it is empty to avoid the race
described above.
We add a flag to enable this behavior rather than turning it on
unconditionally, because checking for missed events may incur
significant overhead for some low-level drivers, and consumers that
don't care about the results of this test shouldn't be forced to pay
for the test.
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2007-05-07 12:02:48 +08:00
|
|
|
if (err > 0 && !(flags & IB_CQ_REPORT_MISSED_EVENTS))
|
|
|
|
err = 0;
|
2007-02-13 08:16:18 +08:00
|
|
|
return err;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int iwch_mmap(struct ib_ucontext *context, struct vm_area_struct *vma)
|
|
|
|
{
|
|
|
|
int len = vma->vm_end - vma->vm_start;
|
|
|
|
u32 key = vma->vm_pgoff << PAGE_SHIFT;
|
|
|
|
struct cxio_rdev *rdev_p;
|
|
|
|
int ret = 0;
|
|
|
|
struct iwch_mm_entry *mm;
|
|
|
|
struct iwch_ucontext *ucontext;
|
2007-03-03 06:06:36 +08:00
|
|
|
u64 addr;
|
2007-02-13 08:16:18 +08:00
|
|
|
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s pgoff 0x%lx key 0x%x len %d\n", __func__, vma->vm_pgoff,
|
2007-02-13 08:16:18 +08:00
|
|
|
key, len);
|
|
|
|
|
|
|
|
if (vma->vm_start & (PAGE_SIZE-1)) {
|
|
|
|
return -EINVAL;
|
|
|
|
}
|
|
|
|
|
|
|
|
rdev_p = &(to_iwch_dev(context->device)->rdev);
|
|
|
|
ucontext = to_iwch_ucontext(context);
|
|
|
|
|
|
|
|
mm = remove_mmap(ucontext, key, len);
|
|
|
|
if (!mm)
|
|
|
|
return -EINVAL;
|
2007-03-03 06:06:36 +08:00
|
|
|
addr = mm->addr;
|
2007-02-13 08:16:18 +08:00
|
|
|
kfree(mm);
|
|
|
|
|
2007-03-03 06:06:36 +08:00
|
|
|
if ((addr >= rdev_p->rnic_info.udbell_physbase) &&
|
|
|
|
(addr < (rdev_p->rnic_info.udbell_physbase +
|
2007-02-13 08:16:18 +08:00
|
|
|
rdev_p->rnic_info.udbell_len))) {
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Map T3 DB register.
|
|
|
|
*/
|
|
|
|
if (vma->vm_flags & VM_READ) {
|
|
|
|
return -EPERM;
|
|
|
|
}
|
|
|
|
|
|
|
|
vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
|
|
|
|
vma->vm_flags |= VM_DONTCOPY | VM_DONTEXPAND;
|
|
|
|
vma->vm_flags &= ~VM_MAYREAD;
|
|
|
|
ret = io_remap_pfn_range(vma, vma->vm_start,
|
2007-03-03 06:06:36 +08:00
|
|
|
addr >> PAGE_SHIFT,
|
2007-02-13 08:16:18 +08:00
|
|
|
len, vma->vm_page_prot);
|
|
|
|
} else {
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Map WQ or CQ contig dma memory...
|
|
|
|
*/
|
|
|
|
ret = remap_pfn_range(vma, vma->vm_start,
|
2007-03-03 06:06:36 +08:00
|
|
|
addr >> PAGE_SHIFT,
|
2007-02-13 08:16:18 +08:00
|
|
|
len, vma->vm_page_prot);
|
|
|
|
}
|
|
|
|
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int iwch_deallocate_pd(struct ib_pd *pd)
|
|
|
|
{
|
|
|
|
struct iwch_dev *rhp;
|
|
|
|
struct iwch_pd *php;
|
|
|
|
|
|
|
|
php = to_iwch_pd(pd);
|
|
|
|
rhp = php->rhp;
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s ibpd %p pdid 0x%x\n", __func__, pd, php->pdid);
|
2007-02-13 08:16:18 +08:00
|
|
|
cxio_hal_put_pdid(rhp->rdev.rscp, php->pdid);
|
|
|
|
kfree(php);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
static struct ib_pd *iwch_allocate_pd(struct ib_device *ibdev,
|
|
|
|
struct ib_ucontext *context,
|
|
|
|
struct ib_udata *udata)
|
|
|
|
{
|
|
|
|
struct iwch_pd *php;
|
|
|
|
u32 pdid;
|
|
|
|
struct iwch_dev *rhp;
|
|
|
|
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s ibdev %p\n", __func__, ibdev);
|
2007-02-13 08:16:18 +08:00
|
|
|
rhp = (struct iwch_dev *) ibdev;
|
|
|
|
pdid = cxio_hal_get_pdid(rhp->rdev.rscp);
|
|
|
|
if (!pdid)
|
|
|
|
return ERR_PTR(-EINVAL);
|
|
|
|
php = kzalloc(sizeof(*php), GFP_KERNEL);
|
|
|
|
if (!php) {
|
|
|
|
cxio_hal_put_pdid(rhp->rdev.rscp, pdid);
|
|
|
|
return ERR_PTR(-ENOMEM);
|
|
|
|
}
|
|
|
|
php->pdid = pdid;
|
|
|
|
php->rhp = rhp;
|
|
|
|
if (context) {
|
|
|
|
if (ib_copy_to_udata(udata, &php->pdid, sizeof (__u32))) {
|
|
|
|
iwch_deallocate_pd(&php->ibpd);
|
|
|
|
return ERR_PTR(-EFAULT);
|
|
|
|
}
|
|
|
|
}
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s pdid 0x%0x ptr 0x%p\n", __func__, pdid, php);
|
2007-02-13 08:16:18 +08:00
|
|
|
return &php->ibpd;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int iwch_dereg_mr(struct ib_mr *ib_mr)
|
|
|
|
{
|
|
|
|
struct iwch_dev *rhp;
|
|
|
|
struct iwch_mr *mhp;
|
|
|
|
u32 mmid;
|
|
|
|
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s ib_mr %p\n", __func__, ib_mr);
|
2007-02-13 08:16:18 +08:00
|
|
|
|
|
|
|
mhp = to_iwch_mr(ib_mr);
|
2015-10-14 00:11:29 +08:00
|
|
|
kfree(mhp->pages);
|
2007-02-13 08:16:18 +08:00
|
|
|
rhp = mhp->rhp;
|
|
|
|
mmid = mhp->attr.stag >> 8;
|
|
|
|
cxio_dereg_mem(&rhp->rdev, mhp->attr.stag, mhp->attr.pbl_size,
|
|
|
|
mhp->attr.pbl_addr);
|
2008-05-07 06:56:22 +08:00
|
|
|
iwch_free_pbl(mhp);
|
2007-02-13 08:16:18 +08:00
|
|
|
remove_handle(rhp, &rhp->mmidr, mmid);
|
|
|
|
if (mhp->kva)
|
|
|
|
kfree((void *) (unsigned long) mhp->kva);
|
2007-03-05 08:15:11 +08:00
|
|
|
if (mhp->umem)
|
|
|
|
ib_umem_release(mhp->umem);
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s mmid 0x%x ptr %p\n", __func__, mmid, mhp);
|
2007-02-13 08:16:18 +08:00
|
|
|
kfree(mhp);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2015-12-24 02:12:49 +08:00
|
|
|
static struct ib_mr *iwch_get_dma_mr(struct ib_pd *pd, int acc)
|
2007-02-13 08:16:18 +08:00
|
|
|
{
|
2015-12-24 02:12:49 +08:00
|
|
|
const u64 total_size = 0xffffffff;
|
|
|
|
const u64 mask = (total_size + PAGE_SIZE - 1) & PAGE_MASK;
|
|
|
|
struct iwch_pd *php = to_iwch_pd(pd);
|
|
|
|
struct iwch_dev *rhp = php->rhp;
|
2007-02-13 08:16:18 +08:00
|
|
|
struct iwch_mr *mhp;
|
2015-12-24 02:12:49 +08:00
|
|
|
__be64 *page_list;
|
|
|
|
int shift = 26, npages, ret, i;
|
2007-02-13 08:16:18 +08:00
|
|
|
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s ib_pd %p\n", __func__, pd);
|
2015-12-24 02:12:49 +08:00
|
|
|
|
|
|
|
/*
|
|
|
|
* T3 only supports 32 bits of size.
|
|
|
|
*/
|
|
|
|
if (sizeof(phys_addr_t) > 4) {
|
|
|
|
pr_warn_once(MOD "Cannot support dma_mrs on this platform.\n");
|
|
|
|
return ERR_PTR(-ENOTSUPP);
|
|
|
|
}
|
2007-02-13 08:16:18 +08:00
|
|
|
|
|
|
|
mhp = kzalloc(sizeof(*mhp), GFP_KERNEL);
|
|
|
|
if (!mhp)
|
|
|
|
return ERR_PTR(-ENOMEM);
|
|
|
|
|
2008-05-07 06:56:22 +08:00
|
|
|
mhp->rhp = rhp;
|
|
|
|
|
2015-12-24 02:12:49 +08:00
|
|
|
npages = (total_size + (1ULL << shift) - 1) >> shift;
|
|
|
|
if (!npages) {
|
2007-02-13 08:16:18 +08:00
|
|
|
ret = -EINVAL;
|
|
|
|
goto err;
|
|
|
|
}
|
|
|
|
|
2015-12-24 02:12:49 +08:00
|
|
|
page_list = kmalloc_array(npages, sizeof(u64), GFP_KERNEL);
|
|
|
|
if (!page_list) {
|
|
|
|
ret = -ENOMEM;
|
2007-02-13 08:16:18 +08:00
|
|
|
goto err;
|
|
|
|
}
|
|
|
|
|
2015-12-24 02:12:49 +08:00
|
|
|
for (i = 0; i < npages; i++)
|
|
|
|
page_list[i] = cpu_to_be64((u64)i << shift);
|
|
|
|
|
|
|
|
PDBG("%s mask 0x%llx shift %d len %lld pbl_size %d\n",
|
|
|
|
__func__, mask, shift, total_size, npages);
|
2007-02-13 08:16:18 +08:00
|
|
|
|
2008-05-07 06:56:22 +08:00
|
|
|
ret = iwch_alloc_pbl(mhp, npages);
|
|
|
|
if (ret) {
|
|
|
|
kfree(page_list);
|
|
|
|
goto err_pbl;
|
|
|
|
}
|
|
|
|
|
|
|
|
ret = iwch_write_pbl(mhp, page_list, npages, 0);
|
|
|
|
kfree(page_list);
|
|
|
|
if (ret)
|
|
|
|
goto err_pbl;
|
|
|
|
|
2007-02-13 08:16:18 +08:00
|
|
|
mhp->attr.pdid = php->pdid;
|
|
|
|
mhp->attr.zbva = 0;
|
|
|
|
|
2007-03-07 04:44:07 +08:00
|
|
|
mhp->attr.perms = iwch_ib_to_tpt_access(acc);
|
2015-12-24 02:12:49 +08:00
|
|
|
mhp->attr.va_fbo = 0;
|
2007-02-13 08:16:18 +08:00
|
|
|
mhp->attr.page_size = shift - 12;
|
|
|
|
|
|
|
|
mhp->attr.len = (u32) total_size;
|
|
|
|
mhp->attr.pbl_size = npages;
|
2008-05-07 06:56:22 +08:00
|
|
|
ret = iwch_register_mem(rhp, php, mhp, shift);
|
|
|
|
if (ret)
|
|
|
|
goto err_pbl;
|
|
|
|
|
2007-02-13 08:16:18 +08:00
|
|
|
return &mhp->ibmr;
|
2008-05-07 06:56:22 +08:00
|
|
|
|
|
|
|
err_pbl:
|
|
|
|
iwch_free_pbl(mhp);
|
|
|
|
|
2007-02-13 08:16:18 +08:00
|
|
|
err:
|
|
|
|
kfree(mhp);
|
|
|
|
return ERR_PTR(ret);
|
|
|
|
}
|
|
|
|
|
2007-03-05 08:15:11 +08:00
|
|
|
static struct ib_mr *iwch_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
|
|
|
|
u64 virt, int acc, struct ib_udata *udata)
|
2007-02-13 08:16:18 +08:00
|
|
|
{
|
|
|
|
__be64 *pages;
|
|
|
|
int shift, n, len;
|
2014-01-28 19:40:15 +08:00
|
|
|
int i, k, entry;
|
2007-02-13 08:16:18 +08:00
|
|
|
int err = 0;
|
|
|
|
struct iwch_dev *rhp;
|
|
|
|
struct iwch_pd *php;
|
|
|
|
struct iwch_mr *mhp;
|
|
|
|
struct iwch_reg_user_mr_resp uresp;
|
2014-01-28 19:40:15 +08:00
|
|
|
struct scatterlist *sg;
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s ib_pd %p\n", __func__, pd);
|
2007-02-13 08:16:18 +08:00
|
|
|
|
|
|
|
php = to_iwch_pd(pd);
|
|
|
|
rhp = php->rhp;
|
|
|
|
mhp = kzalloc(sizeof(*mhp), GFP_KERNEL);
|
|
|
|
if (!mhp)
|
|
|
|
return ERR_PTR(-ENOMEM);
|
|
|
|
|
2008-05-07 06:56:22 +08:00
|
|
|
mhp->rhp = rhp;
|
|
|
|
|
2008-04-29 16:00:34 +08:00
|
|
|
mhp->umem = ib_umem_get(pd->uobject->context, start, length, acc, 0);
|
2007-03-05 08:15:11 +08:00
|
|
|
if (IS_ERR(mhp->umem)) {
|
|
|
|
err = PTR_ERR(mhp->umem);
|
|
|
|
kfree(mhp);
|
|
|
|
return ERR_PTR(err);
|
|
|
|
}
|
|
|
|
|
|
|
|
shift = ffs(mhp->umem->page_size) - 1;
|
|
|
|
|
2014-01-28 19:40:15 +08:00
|
|
|
n = mhp->umem->nmap;
|
2007-02-13 08:16:18 +08:00
|
|
|
|
2008-05-07 06:56:22 +08:00
|
|
|
err = iwch_alloc_pbl(mhp, n);
|
|
|
|
if (err)
|
|
|
|
goto err;
|
|
|
|
|
|
|
|
pages = (__be64 *) __get_free_page(GFP_KERNEL);
|
2007-02-13 08:16:18 +08:00
|
|
|
if (!pages) {
|
|
|
|
err = -ENOMEM;
|
2008-05-07 06:56:22 +08:00
|
|
|
goto err_pbl;
|
2007-02-13 08:16:18 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
i = n = 0;
|
|
|
|
|
2014-01-28 19:40:15 +08:00
|
|
|
for_each_sg(mhp->umem->sg_head.sgl, sg, mhp->umem->nmap, entry) {
|
|
|
|
len = sg_dma_len(sg) >> shift;
|
2007-02-13 08:16:18 +08:00
|
|
|
for (k = 0; k < len; ++k) {
|
2014-01-28 19:40:15 +08:00
|
|
|
pages[i++] = cpu_to_be64(sg_dma_address(sg) +
|
2007-03-05 08:15:11 +08:00
|
|
|
mhp->umem->page_size * k);
|
2008-05-07 06:56:22 +08:00
|
|
|
if (i == PAGE_SIZE / sizeof *pages) {
|
|
|
|
err = iwch_write_pbl(mhp, pages, i, n);
|
|
|
|
if (err)
|
|
|
|
goto pbl_done;
|
|
|
|
n += i;
|
|
|
|
i = 0;
|
|
|
|
}
|
2007-02-13 08:16:18 +08:00
|
|
|
}
|
2014-01-28 19:40:15 +08:00
|
|
|
}
|
2007-02-13 08:16:18 +08:00
|
|
|
|
2008-05-07 06:56:22 +08:00
|
|
|
if (i)
|
|
|
|
err = iwch_write_pbl(mhp, pages, i, n);
|
|
|
|
|
|
|
|
pbl_done:
|
|
|
|
free_page((unsigned long) pages);
|
|
|
|
if (err)
|
|
|
|
goto err_pbl;
|
|
|
|
|
2007-02-13 08:16:18 +08:00
|
|
|
mhp->attr.pdid = php->pdid;
|
|
|
|
mhp->attr.zbva = 0;
|
2007-03-07 04:44:07 +08:00
|
|
|
mhp->attr.perms = iwch_ib_to_tpt_access(acc);
|
2007-03-05 08:15:11 +08:00
|
|
|
mhp->attr.va_fbo = virt;
|
2007-02-13 08:16:18 +08:00
|
|
|
mhp->attr.page_size = shift - 12;
|
2007-03-05 08:15:11 +08:00
|
|
|
mhp->attr.len = (u32) length;
|
2008-05-07 06:56:22 +08:00
|
|
|
|
|
|
|
err = iwch_register_mem(rhp, php, mhp, shift);
|
2007-02-13 08:16:18 +08:00
|
|
|
if (err)
|
2008-05-07 06:56:22 +08:00
|
|
|
goto err_pbl;
|
2007-02-13 08:16:18 +08:00
|
|
|
|
2008-01-25 06:30:16 +08:00
|
|
|
if (udata && !t3a_device(rhp)) {
|
2007-02-13 08:16:18 +08:00
|
|
|
uresp.pbl_addr = (mhp->attr.pbl_addr -
|
2008-05-07 06:56:22 +08:00
|
|
|
rhp->rdev.rnic_info.pbl_base) >> 3;
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s user resp pbl_addr 0x%x\n", __func__,
|
2007-02-13 08:16:18 +08:00
|
|
|
uresp.pbl_addr);
|
|
|
|
|
|
|
|
if (ib_copy_to_udata(udata, &uresp, sizeof (uresp))) {
|
|
|
|
iwch_dereg_mr(&mhp->ibmr);
|
|
|
|
err = -EFAULT;
|
|
|
|
goto err;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
return &mhp->ibmr;
|
|
|
|
|
2008-05-07 06:56:22 +08:00
|
|
|
err_pbl:
|
|
|
|
iwch_free_pbl(mhp);
|
|
|
|
|
2007-02-13 08:16:18 +08:00
|
|
|
err:
|
2007-03-05 08:15:11 +08:00
|
|
|
ib_umem_release(mhp->umem);
|
2007-02-13 08:16:18 +08:00
|
|
|
kfree(mhp);
|
|
|
|
return ERR_PTR(err);
|
|
|
|
}
|
|
|
|
|
2016-03-01 00:05:29 +08:00
|
|
|
static struct ib_mw *iwch_alloc_mw(struct ib_pd *pd, enum ib_mw_type type,
|
|
|
|
struct ib_udata *udata)
|
2007-02-13 08:16:18 +08:00
|
|
|
{
|
|
|
|
struct iwch_dev *rhp;
|
|
|
|
struct iwch_pd *php;
|
|
|
|
struct iwch_mw *mhp;
|
|
|
|
u32 mmid;
|
|
|
|
u32 stag = 0;
|
|
|
|
int ret;
|
|
|
|
|
2013-02-07 00:19:12 +08:00
|
|
|
if (type != IB_MW_TYPE_1)
|
|
|
|
return ERR_PTR(-EINVAL);
|
|
|
|
|
2007-02-13 08:16:18 +08:00
|
|
|
php = to_iwch_pd(pd);
|
|
|
|
rhp = php->rhp;
|
|
|
|
mhp = kzalloc(sizeof(*mhp), GFP_KERNEL);
|
|
|
|
if (!mhp)
|
|
|
|
return ERR_PTR(-ENOMEM);
|
|
|
|
ret = cxio_allocate_window(&rhp->rdev, &stag, php->pdid);
|
|
|
|
if (ret) {
|
|
|
|
kfree(mhp);
|
|
|
|
return ERR_PTR(ret);
|
|
|
|
}
|
|
|
|
mhp->rhp = rhp;
|
|
|
|
mhp->attr.pdid = php->pdid;
|
|
|
|
mhp->attr.type = TPT_MW;
|
|
|
|
mhp->attr.stag = stag;
|
|
|
|
mmid = (stag) >> 8;
|
2008-07-15 14:48:49 +08:00
|
|
|
mhp->ibmw.rkey = stag;
|
2009-09-10 02:25:55 +08:00
|
|
|
if (insert_handle(rhp, &rhp->mmidr, mhp, mmid)) {
|
|
|
|
cxio_deallocate_window(&rhp->rdev, mhp->attr.stag);
|
|
|
|
kfree(mhp);
|
|
|
|
return ERR_PTR(-ENOMEM);
|
|
|
|
}
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s mmid 0x%x mhp %p stag 0x%x\n", __func__, mmid, mhp, stag);
|
2007-02-13 08:16:18 +08:00
|
|
|
return &(mhp->ibmw);
|
|
|
|
}
|
|
|
|
|
|
|
|
static int iwch_dealloc_mw(struct ib_mw *mw)
|
|
|
|
{
|
|
|
|
struct iwch_dev *rhp;
|
|
|
|
struct iwch_mw *mhp;
|
|
|
|
u32 mmid;
|
|
|
|
|
|
|
|
mhp = to_iwch_mw(mw);
|
|
|
|
rhp = mhp->rhp;
|
|
|
|
mmid = (mw->rkey) >> 8;
|
|
|
|
cxio_deallocate_window(&rhp->rdev, mhp->attr.stag);
|
|
|
|
remove_handle(rhp, &rhp->mmidr, mmid);
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s ib_mw %p mmid 0x%x ptr %p\n", __func__, mw, mmid, mhp);
|
2013-01-15 03:34:09 +08:00
|
|
|
kfree(mhp);
|
2007-02-13 08:16:18 +08:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2015-07-30 15:32:45 +08:00
|
|
|
static struct ib_mr *iwch_alloc_mr(struct ib_pd *pd,
|
|
|
|
enum ib_mr_type mr_type,
|
|
|
|
u32 max_num_sg)
|
2008-07-15 14:48:45 +08:00
|
|
|
{
|
|
|
|
struct iwch_dev *rhp;
|
|
|
|
struct iwch_pd *php;
|
|
|
|
struct iwch_mr *mhp;
|
|
|
|
u32 mmid;
|
|
|
|
u32 stag = 0;
|
2009-09-10 02:25:55 +08:00
|
|
|
int ret = 0;
|
2008-07-15 14:48:45 +08:00
|
|
|
|
2015-07-30 15:32:45 +08:00
|
|
|
if (mr_type != IB_MR_TYPE_MEM_REG ||
|
|
|
|
max_num_sg > T3_MAX_FASTREG_DEPTH)
|
|
|
|
return ERR_PTR(-EINVAL);
|
|
|
|
|
2008-07-15 14:48:45 +08:00
|
|
|
php = to_iwch_pd(pd);
|
|
|
|
rhp = php->rhp;
|
|
|
|
mhp = kzalloc(sizeof(*mhp), GFP_KERNEL);
|
|
|
|
if (!mhp)
|
2009-09-10 02:25:55 +08:00
|
|
|
goto err;
|
2008-07-15 14:48:45 +08:00
|
|
|
|
2015-10-14 00:11:29 +08:00
|
|
|
mhp->pages = kcalloc(max_num_sg, sizeof(u64), GFP_KERNEL);
|
|
|
|
if (!mhp->pages) {
|
|
|
|
ret = -ENOMEM;
|
|
|
|
goto pl_err;
|
|
|
|
}
|
|
|
|
|
2008-07-15 14:48:45 +08:00
|
|
|
mhp->rhp = rhp;
|
2015-07-30 15:32:45 +08:00
|
|
|
ret = iwch_alloc_pbl(mhp, max_num_sg);
|
2009-09-10 02:25:55 +08:00
|
|
|
if (ret)
|
|
|
|
goto err1;
|
2015-07-30 15:32:45 +08:00
|
|
|
mhp->attr.pbl_size = max_num_sg;
|
2008-07-15 14:48:45 +08:00
|
|
|
ret = cxio_allocate_stag(&rhp->rdev, &stag, php->pdid,
|
|
|
|
mhp->attr.pbl_size, mhp->attr.pbl_addr);
|
2009-09-10 02:25:55 +08:00
|
|
|
if (ret)
|
|
|
|
goto err2;
|
2008-07-15 14:48:45 +08:00
|
|
|
mhp->attr.pdid = php->pdid;
|
|
|
|
mhp->attr.type = TPT_NON_SHARED_MR;
|
|
|
|
mhp->attr.stag = stag;
|
|
|
|
mhp->attr.state = 1;
|
|
|
|
mmid = (stag) >> 8;
|
|
|
|
mhp->ibmr.rkey = mhp->ibmr.lkey = stag;
|
2009-09-10 02:25:55 +08:00
|
|
|
if (insert_handle(rhp, &rhp->mmidr, mhp, mmid))
|
|
|
|
goto err3;
|
|
|
|
|
2008-07-15 14:48:45 +08:00
|
|
|
PDBG("%s mmid 0x%x mhp %p stag 0x%x\n", __func__, mmid, mhp, stag);
|
|
|
|
return &(mhp->ibmr);
|
2009-09-10 02:25:55 +08:00
|
|
|
err3:
|
|
|
|
cxio_dereg_mem(&rhp->rdev, stag, mhp->attr.pbl_size,
|
|
|
|
mhp->attr.pbl_addr);
|
|
|
|
err2:
|
|
|
|
iwch_free_pbl(mhp);
|
|
|
|
err1:
|
2015-10-14 00:11:29 +08:00
|
|
|
kfree(mhp->pages);
|
|
|
|
pl_err:
|
2009-09-10 02:25:55 +08:00
|
|
|
kfree(mhp);
|
|
|
|
err:
|
|
|
|
return ERR_PTR(ret);
|
2008-07-15 14:48:45 +08:00
|
|
|
}
|
|
|
|
|
2015-10-14 00:11:29 +08:00
|
|
|
static int iwch_set_page(struct ib_mr *ibmr, u64 addr)
|
|
|
|
{
|
|
|
|
struct iwch_mr *mhp = to_iwch_mr(ibmr);
|
|
|
|
|
|
|
|
if (unlikely(mhp->npages == mhp->attr.pbl_size))
|
|
|
|
return -ENOMEM;
|
|
|
|
|
|
|
|
mhp->pages[mhp->npages++] = addr;
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2016-05-04 00:01:04 +08:00
|
|
|
static int iwch_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg,
|
2016-05-13 01:49:15 +08:00
|
|
|
int sg_nents, unsigned int *sg_offset)
|
2015-10-14 00:11:29 +08:00
|
|
|
{
|
|
|
|
struct iwch_mr *mhp = to_iwch_mr(ibmr);
|
|
|
|
|
|
|
|
mhp->npages = 0;
|
|
|
|
|
2016-05-04 00:01:04 +08:00
|
|
|
return ib_sg_to_pages(ibmr, sg, sg_nents, sg_offset, iwch_set_page);
|
2015-10-14 00:11:29 +08:00
|
|
|
}
|
|
|
|
|
2007-02-13 08:16:18 +08:00
|
|
|
static int iwch_destroy_qp(struct ib_qp *ib_qp)
|
|
|
|
{
|
|
|
|
struct iwch_dev *rhp;
|
|
|
|
struct iwch_qp *qhp;
|
|
|
|
struct iwch_qp_attributes attrs;
|
|
|
|
struct iwch_ucontext *ucontext;
|
|
|
|
|
|
|
|
qhp = to_iwch_qp(ib_qp);
|
|
|
|
rhp = qhp->rhp;
|
|
|
|
|
2007-03-07 04:43:58 +08:00
|
|
|
attrs.next_state = IWCH_QP_STATE_ERROR;
|
|
|
|
iwch_modify_qp(rhp, qhp, IWCH_QP_ATTR_NEXT_STATE, &attrs, 0);
|
2007-02-13 08:16:18 +08:00
|
|
|
wait_event(qhp->wait, !qhp->ep);
|
|
|
|
|
|
|
|
remove_handle(rhp, &rhp->qpidr, qhp->wq.qpid);
|
|
|
|
|
|
|
|
atomic_dec(&qhp->refcnt);
|
|
|
|
wait_event(qhp->wait, !atomic_read(&qhp->refcnt));
|
|
|
|
|
|
|
|
ucontext = ib_qp->uobject ? to_iwch_ucontext(ib_qp->uobject->context)
|
|
|
|
: NULL;
|
|
|
|
cxio_destroy_qp(&rhp->rdev, &qhp->wq,
|
|
|
|
ucontext ? &ucontext->uctx : &rhp->rdev.uctx);
|
|
|
|
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s ib_qp %p qpid 0x%0x qhp %p\n", __func__,
|
2007-02-13 08:16:18 +08:00
|
|
|
ib_qp, qhp->wq.qpid, qhp);
|
|
|
|
kfree(qhp);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
static struct ib_qp *iwch_create_qp(struct ib_pd *pd,
|
|
|
|
struct ib_qp_init_attr *attrs,
|
|
|
|
struct ib_udata *udata)
|
|
|
|
{
|
|
|
|
struct iwch_dev *rhp;
|
|
|
|
struct iwch_qp *qhp;
|
|
|
|
struct iwch_pd *php;
|
|
|
|
struct iwch_cq *schp;
|
|
|
|
struct iwch_cq *rchp;
|
|
|
|
struct iwch_create_qp_resp uresp;
|
|
|
|
int wqsize, sqsize, rqsize;
|
|
|
|
struct iwch_ucontext *ucontext;
|
|
|
|
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s ib_pd %p\n", __func__, pd);
|
2007-02-13 08:16:18 +08:00
|
|
|
if (attrs->qp_type != IB_QPT_RC)
|
|
|
|
return ERR_PTR(-EINVAL);
|
|
|
|
php = to_iwch_pd(pd);
|
|
|
|
rhp = php->rhp;
|
|
|
|
schp = get_chp(rhp, ((struct iwch_cq *) attrs->send_cq)->cq.cqid);
|
|
|
|
rchp = get_chp(rhp, ((struct iwch_cq *) attrs->recv_cq)->cq.cqid);
|
|
|
|
if (!schp || !rchp)
|
|
|
|
return ERR_PTR(-EINVAL);
|
|
|
|
|
|
|
|
/* The RQT size must be # of entries + 1 rounded up to a power of two */
|
|
|
|
rqsize = roundup_pow_of_two(attrs->cap.max_recv_wr);
|
|
|
|
if (rqsize == attrs->cap.max_recv_wr)
|
|
|
|
rqsize = roundup_pow_of_two(attrs->cap.max_recv_wr+1);
|
|
|
|
|
|
|
|
/* T3 doesn't support RQT depth < 16 */
|
|
|
|
if (rqsize < 16)
|
|
|
|
rqsize = 16;
|
|
|
|
|
|
|
|
if (rqsize > T3_MAX_RQ_SIZE)
|
|
|
|
return ERR_PTR(-EINVAL);
|
|
|
|
|
2007-04-27 04:21:09 +08:00
|
|
|
if (attrs->cap.max_inline_data > T3_MAX_INLINE)
|
|
|
|
return ERR_PTR(-EINVAL);
|
|
|
|
|
2007-02-13 08:16:18 +08:00
|
|
|
/*
|
|
|
|
* NOTE: The SQ and total WQ sizes don't need to be
|
|
|
|
* a power of two. However, all the code assumes
|
|
|
|
* they are. EG: Q_FREECNT() and friends.
|
|
|
|
*/
|
|
|
|
sqsize = roundup_pow_of_two(attrs->cap.max_send_wr);
|
|
|
|
wqsize = roundup_pow_of_two(rqsize + sqsize);
|
2008-07-15 14:48:45 +08:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Kernel users need more wq space for fastreg WRs which can take
|
|
|
|
* 2 WR fragments.
|
|
|
|
*/
|
|
|
|
ucontext = pd->uobject ? to_iwch_ucontext(pd->uobject->context) : NULL;
|
|
|
|
if (!ucontext && wqsize < (rqsize + (2 * sqsize)))
|
|
|
|
wqsize = roundup_pow_of_two(rqsize +
|
|
|
|
roundup_pow_of_two(attrs->cap.max_send_wr * 2));
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s wqsize %d sqsize %d rqsize %d\n", __func__,
|
2007-02-13 08:16:18 +08:00
|
|
|
wqsize, sqsize, rqsize);
|
|
|
|
qhp = kzalloc(sizeof(*qhp), GFP_KERNEL);
|
|
|
|
if (!qhp)
|
|
|
|
return ERR_PTR(-ENOMEM);
|
|
|
|
qhp->wq.size_log2 = ilog2(wqsize);
|
|
|
|
qhp->wq.rq_size_log2 = ilog2(rqsize);
|
|
|
|
qhp->wq.sq_size_log2 = ilog2(sqsize);
|
|
|
|
if (cxio_create_qp(&rhp->rdev, !udata, &qhp->wq,
|
|
|
|
ucontext ? &ucontext->uctx : &rhp->rdev.uctx)) {
|
|
|
|
kfree(qhp);
|
|
|
|
return ERR_PTR(-ENOMEM);
|
|
|
|
}
|
2008-03-01 05:53:18 +08:00
|
|
|
|
2007-02-13 08:16:18 +08:00
|
|
|
attrs->cap.max_recv_wr = rqsize - 1;
|
|
|
|
attrs->cap.max_send_wr = sqsize;
|
2008-03-01 05:53:18 +08:00
|
|
|
attrs->cap.max_inline_data = T3_MAX_INLINE;
|
|
|
|
|
2007-02-13 08:16:18 +08:00
|
|
|
qhp->rhp = rhp;
|
|
|
|
qhp->attr.pd = php->pdid;
|
|
|
|
qhp->attr.scq = ((struct iwch_cq *) attrs->send_cq)->cq.cqid;
|
|
|
|
qhp->attr.rcq = ((struct iwch_cq *) attrs->recv_cq)->cq.cqid;
|
|
|
|
qhp->attr.sq_num_entries = attrs->cap.max_send_wr;
|
|
|
|
qhp->attr.rq_num_entries = attrs->cap.max_recv_wr;
|
|
|
|
qhp->attr.sq_max_sges = attrs->cap.max_send_sge;
|
|
|
|
qhp->attr.sq_max_sges_rdma_write = attrs->cap.max_send_sge;
|
|
|
|
qhp->attr.rq_max_sges = attrs->cap.max_recv_sge;
|
|
|
|
qhp->attr.state = IWCH_QP_STATE_IDLE;
|
|
|
|
qhp->attr.next_state = IWCH_QP_STATE_IDLE;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* XXX - These don't get passed in from the openib user
|
|
|
|
* at create time. The CM sets them via a QP modify.
|
|
|
|
* Need to fix... I think the CM should
|
|
|
|
*/
|
|
|
|
qhp->attr.enable_rdma_read = 1;
|
|
|
|
qhp->attr.enable_rdma_write = 1;
|
|
|
|
qhp->attr.enable_bind = 1;
|
|
|
|
qhp->attr.max_ord = 1;
|
|
|
|
qhp->attr.max_ird = 1;
|
|
|
|
|
|
|
|
spin_lock_init(&qhp->lock);
|
|
|
|
init_waitqueue_head(&qhp->wait);
|
|
|
|
atomic_set(&qhp->refcnt, 1);
|
2009-09-10 02:25:55 +08:00
|
|
|
|
|
|
|
if (insert_handle(rhp, &rhp->qpidr, qhp, qhp->wq.qpid)) {
|
|
|
|
cxio_destroy_qp(&rhp->rdev, &qhp->wq,
|
|
|
|
ucontext ? &ucontext->uctx : &rhp->rdev.uctx);
|
|
|
|
kfree(qhp);
|
|
|
|
return ERR_PTR(-ENOMEM);
|
|
|
|
}
|
2007-02-13 08:16:18 +08:00
|
|
|
|
|
|
|
if (udata) {
|
|
|
|
|
|
|
|
struct iwch_mm_entry *mm1, *mm2;
|
|
|
|
|
|
|
|
mm1 = kmalloc(sizeof *mm1, GFP_KERNEL);
|
|
|
|
if (!mm1) {
|
|
|
|
iwch_destroy_qp(&qhp->ibqp);
|
|
|
|
return ERR_PTR(-ENOMEM);
|
|
|
|
}
|
|
|
|
|
|
|
|
mm2 = kmalloc(sizeof *mm2, GFP_KERNEL);
|
|
|
|
if (!mm2) {
|
|
|
|
kfree(mm1);
|
|
|
|
iwch_destroy_qp(&qhp->ibqp);
|
|
|
|
return ERR_PTR(-ENOMEM);
|
|
|
|
}
|
|
|
|
|
|
|
|
uresp.qpid = qhp->wq.qpid;
|
|
|
|
uresp.size_log2 = qhp->wq.size_log2;
|
|
|
|
uresp.sq_size_log2 = qhp->wq.sq_size_log2;
|
|
|
|
uresp.rq_size_log2 = qhp->wq.rq_size_log2;
|
|
|
|
spin_lock(&ucontext->mmap_lock);
|
|
|
|
uresp.key = ucontext->key;
|
|
|
|
ucontext->key += PAGE_SIZE;
|
|
|
|
uresp.db_key = ucontext->key;
|
|
|
|
ucontext->key += PAGE_SIZE;
|
|
|
|
spin_unlock(&ucontext->mmap_lock);
|
|
|
|
if (ib_copy_to_udata(udata, &uresp, sizeof (uresp))) {
|
|
|
|
kfree(mm1);
|
|
|
|
kfree(mm2);
|
|
|
|
iwch_destroy_qp(&qhp->ibqp);
|
|
|
|
return ERR_PTR(-EFAULT);
|
|
|
|
}
|
|
|
|
mm1->key = uresp.key;
|
|
|
|
mm1->addr = virt_to_phys(qhp->wq.queue);
|
|
|
|
mm1->len = PAGE_ALIGN(wqsize * sizeof (union t3_wr));
|
|
|
|
insert_mmap(ucontext, mm1);
|
|
|
|
mm2->key = uresp.db_key;
|
|
|
|
mm2->addr = qhp->wq.udb & PAGE_MASK;
|
|
|
|
mm2->len = PAGE_SIZE;
|
|
|
|
insert_mmap(ucontext, mm2);
|
|
|
|
}
|
|
|
|
qhp->ibqp.qp_num = qhp->wq.qpid;
|
|
|
|
init_timer(&(qhp->timer));
|
|
|
|
PDBG("%s sq_num_entries %d, rq_num_entries %d "
|
2008-07-15 14:48:53 +08:00
|
|
|
"qpid 0x%0x qhp %p dma_addr 0x%llx size %d rq_addr 0x%x\n",
|
2008-04-17 12:01:10 +08:00
|
|
|
__func__, qhp->attr.sq_num_entries, qhp->attr.rq_num_entries,
|
2007-02-13 08:16:18 +08:00
|
|
|
qhp->wq.qpid, qhp, (unsigned long long) qhp->wq.dma_addr,
|
2008-07-15 14:48:53 +08:00
|
|
|
1 << qhp->wq.size_log2, qhp->wq.rq_addr);
|
2007-02-13 08:16:18 +08:00
|
|
|
return &qhp->ibqp;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int iwch_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
|
|
|
|
int attr_mask, struct ib_udata *udata)
|
|
|
|
{
|
|
|
|
struct iwch_dev *rhp;
|
|
|
|
struct iwch_qp *qhp;
|
|
|
|
enum iwch_qp_attr_mask mask = 0;
|
|
|
|
struct iwch_qp_attributes attrs;
|
|
|
|
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s ib_qp %p\n", __func__, ibqp);
|
2007-02-13 08:16:18 +08:00
|
|
|
|
|
|
|
/* iwarp does not support the RTR state */
|
|
|
|
if ((attr_mask & IB_QP_STATE) && (attr->qp_state == IB_QPS_RTR))
|
|
|
|
attr_mask &= ~IB_QP_STATE;
|
|
|
|
|
|
|
|
/* Make sure we still have something left to do */
|
|
|
|
if (!attr_mask)
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
memset(&attrs, 0, sizeof attrs);
|
|
|
|
qhp = to_iwch_qp(ibqp);
|
|
|
|
rhp = qhp->rhp;
|
|
|
|
|
|
|
|
attrs.next_state = iwch_convert_state(attr->qp_state);
|
|
|
|
attrs.enable_rdma_read = (attr->qp_access_flags &
|
|
|
|
IB_ACCESS_REMOTE_READ) ? 1 : 0;
|
|
|
|
attrs.enable_rdma_write = (attr->qp_access_flags &
|
|
|
|
IB_ACCESS_REMOTE_WRITE) ? 1 : 0;
|
|
|
|
attrs.enable_bind = (attr->qp_access_flags & IB_ACCESS_MW_BIND) ? 1 : 0;
|
|
|
|
|
|
|
|
|
|
|
|
mask |= (attr_mask & IB_QP_STATE) ? IWCH_QP_ATTR_NEXT_STATE : 0;
|
|
|
|
mask |= (attr_mask & IB_QP_ACCESS_FLAGS) ?
|
|
|
|
(IWCH_QP_ATTR_ENABLE_RDMA_READ |
|
|
|
|
IWCH_QP_ATTR_ENABLE_RDMA_WRITE |
|
|
|
|
IWCH_QP_ATTR_ENABLE_RDMA_BIND) : 0;
|
|
|
|
|
|
|
|
return iwch_modify_qp(rhp, qhp, mask, &attrs, 0);
|
|
|
|
}
|
|
|
|
|
|
|
|
void iwch_qp_add_ref(struct ib_qp *qp)
|
|
|
|
{
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s ib_qp %p\n", __func__, qp);
|
2007-02-13 08:16:18 +08:00
|
|
|
atomic_inc(&(to_iwch_qp(qp)->refcnt));
|
|
|
|
}
|
|
|
|
|
|
|
|
void iwch_qp_rem_ref(struct ib_qp *qp)
|
|
|
|
{
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s ib_qp %p\n", __func__, qp);
|
2007-02-13 08:16:18 +08:00
|
|
|
if (atomic_dec_and_test(&(to_iwch_qp(qp)->refcnt)))
|
|
|
|
wake_up(&(to_iwch_qp(qp)->wait));
|
|
|
|
}
|
|
|
|
|
2007-02-21 18:52:49 +08:00
|
|
|
static struct ib_qp *iwch_get_qp(struct ib_device *dev, int qpn)
|
2007-02-13 08:16:18 +08:00
|
|
|
{
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s ib_dev %p qpn 0x%x\n", __func__, dev, qpn);
|
2007-02-13 08:16:18 +08:00
|
|
|
return (struct ib_qp *)get_qhp(to_iwch_dev(dev), qpn);
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
static int iwch_query_pkey(struct ib_device *ibdev,
|
|
|
|
u8 port, u16 index, u16 * pkey)
|
|
|
|
{
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s ibdev %p\n", __func__, ibdev);
|
2007-02-13 08:16:18 +08:00
|
|
|
*pkey = 0;
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int iwch_query_gid(struct ib_device *ibdev, u8 port,
|
|
|
|
int index, union ib_gid *gid)
|
|
|
|
{
|
|
|
|
struct iwch_dev *dev;
|
|
|
|
|
|
|
|
PDBG("%s ibdev %p, port %d, index %d, gid %p\n",
|
2008-04-17 12:01:10 +08:00
|
|
|
__func__, ibdev, port, index, gid);
|
2007-02-13 08:16:18 +08:00
|
|
|
dev = to_iwch_dev(ibdev);
|
|
|
|
BUG_ON(port == 0 || port > 2);
|
|
|
|
memset(&(gid->raw[0]), 0, sizeof(gid->raw));
|
|
|
|
memcpy(&(gid->raw[0]), dev->rdev.port_info.lldevs[port-1]->dev_addr, 6);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2008-07-15 14:48:47 +08:00
|
|
|
static u64 fw_vers_string_to_u64(struct iwch_dev *iwch_dev)
|
|
|
|
{
|
|
|
|
struct ethtool_drvinfo info;
|
|
|
|
struct net_device *lldev = iwch_dev->rdev.t3cdev_p->lldev;
|
|
|
|
char *cp, *next;
|
|
|
|
unsigned fw_maj, fw_min, fw_mic;
|
|
|
|
|
|
|
|
lldev->ethtool_ops->get_drvinfo(lldev, &info);
|
|
|
|
|
|
|
|
next = info.fw_version + 1;
|
|
|
|
cp = strsep(&next, ".");
|
|
|
|
sscanf(cp, "%i", &fw_maj);
|
|
|
|
cp = strsep(&next, ".");
|
|
|
|
sscanf(cp, "%i", &fw_min);
|
|
|
|
cp = strsep(&next, ".");
|
|
|
|
sscanf(cp, "%i", &fw_mic);
|
|
|
|
|
|
|
|
return (((u64)fw_maj & 0xffff) << 32) | ((fw_min & 0xffff) << 16) |
|
|
|
|
(fw_mic & 0xffff);
|
|
|
|
}
|
|
|
|
|
2015-06-11 21:35:25 +08:00
|
|
|
static int iwch_query_device(struct ib_device *ibdev, struct ib_device_attr *props,
|
|
|
|
struct ib_udata *uhw)
|
2007-02-13 08:16:18 +08:00
|
|
|
{
|
|
|
|
|
|
|
|
struct iwch_dev *dev;
|
2015-06-11 21:35:25 +08:00
|
|
|
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s ibdev %p\n", __func__, ibdev);
|
2007-02-13 08:16:18 +08:00
|
|
|
|
2015-06-11 21:35:25 +08:00
|
|
|
if (uhw->inlen || uhw->outlen)
|
|
|
|
return -EINVAL;
|
|
|
|
|
2007-02-13 08:16:18 +08:00
|
|
|
dev = to_iwch_dev(ibdev);
|
|
|
|
memset(props, 0, sizeof *props);
|
|
|
|
memcpy(&props->sys_image_guid, dev->rdev.t3cdev_p->lldev->dev_addr, 6);
|
2008-07-15 14:48:47 +08:00
|
|
|
props->hw_ver = dev->rdev.t3cdev_p->type;
|
|
|
|
props->fw_ver = fw_vers_string_to_u64(dev);
|
2007-02-13 08:16:18 +08:00
|
|
|
props->device_cap_flags = dev->device_cap_flags;
|
2008-07-15 14:48:49 +08:00
|
|
|
props->page_size_cap = dev->attr.mem_pgsizes_bitmask;
|
2007-02-13 08:16:18 +08:00
|
|
|
props->vendor_id = (u32)dev->rdev.rnic_info.pdev->vendor;
|
|
|
|
props->vendor_part_id = (u32)dev->rdev.rnic_info.pdev->device;
|
2008-04-30 04:46:52 +08:00
|
|
|
props->max_mr_size = dev->attr.max_mr_size;
|
2007-02-13 08:16:18 +08:00
|
|
|
props->max_qp = dev->attr.max_qps;
|
|
|
|
props->max_qp_wr = dev->attr.max_wrs;
|
|
|
|
props->max_sge = dev->attr.max_sge_per_wr;
|
|
|
|
props->max_sge_rd = 1;
|
|
|
|
props->max_qp_rd_atom = dev->attr.max_rdma_reads_per_qp;
|
2007-11-09 23:21:58 +08:00
|
|
|
props->max_qp_init_rd_atom = dev->attr.max_rdma_reads_per_qp;
|
2007-02-13 08:16:18 +08:00
|
|
|
props->max_cq = dev->attr.max_cqs;
|
|
|
|
props->max_cqe = dev->attr.max_cqes_per_cq;
|
|
|
|
props->max_mr = dev->attr.max_mem_regs;
|
|
|
|
props->max_pd = dev->attr.max_pds;
|
|
|
|
props->local_ca_ack_delay = 0;
|
2008-07-15 14:48:45 +08:00
|
|
|
props->max_fast_reg_page_list_len = T3_MAX_FASTREG_DEPTH;
|
2007-02-13 08:16:18 +08:00
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int iwch_query_port(struct ib_device *ibdev,
|
|
|
|
u8 port, struct ib_port_attr *props)
|
|
|
|
{
|
2009-05-28 05:42:36 +08:00
|
|
|
struct iwch_dev *dev;
|
|
|
|
struct net_device *netdev;
|
|
|
|
struct in_device *inetdev;
|
|
|
|
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s ibdev %p\n", __func__, ibdev);
|
2008-10-01 05:51:19 +08:00
|
|
|
|
2009-05-28 05:42:36 +08:00
|
|
|
dev = to_iwch_dev(ibdev);
|
|
|
|
netdev = dev->rdev.port_info.lldevs[port-1];
|
|
|
|
|
2008-10-01 05:51:19 +08:00
|
|
|
memset(props, 0, sizeof(struct ib_port_attr));
|
2007-02-13 08:16:18 +08:00
|
|
|
props->max_mtu = IB_MTU_4096;
|
2009-05-28 05:42:36 +08:00
|
|
|
if (netdev->mtu >= 4096)
|
|
|
|
props->active_mtu = IB_MTU_4096;
|
|
|
|
else if (netdev->mtu >= 2048)
|
|
|
|
props->active_mtu = IB_MTU_2048;
|
|
|
|
else if (netdev->mtu >= 1024)
|
|
|
|
props->active_mtu = IB_MTU_1024;
|
|
|
|
else if (netdev->mtu >= 512)
|
|
|
|
props->active_mtu = IB_MTU_512;
|
|
|
|
else
|
|
|
|
props->active_mtu = IB_MTU_256;
|
|
|
|
|
|
|
|
if (!netif_carrier_ok(netdev))
|
|
|
|
props->state = IB_PORT_DOWN;
|
|
|
|
else {
|
|
|
|
inetdev = in_dev_get(netdev);
|
2009-10-08 06:51:07 +08:00
|
|
|
if (inetdev) {
|
|
|
|
if (inetdev->ifa_list)
|
|
|
|
props->state = IB_PORT_ACTIVE;
|
|
|
|
else
|
|
|
|
props->state = IB_PORT_INIT;
|
|
|
|
in_dev_put(inetdev);
|
|
|
|
} else
|
2009-05-28 05:42:36 +08:00
|
|
|
props->state = IB_PORT_INIT;
|
|
|
|
}
|
|
|
|
|
2007-02-13 08:16:18 +08:00
|
|
|
props->port_cap_flags =
|
|
|
|
IB_PORT_CM_SUP |
|
|
|
|
IB_PORT_SNMP_TUNNEL_SUP |
|
|
|
|
IB_PORT_REINIT_SUP |
|
|
|
|
IB_PORT_DEVICE_MGMT_SUP |
|
|
|
|
IB_PORT_VENDOR_CLASS_SUP | IB_PORT_BOOT_MGMT_SUP;
|
|
|
|
props->gid_tbl_len = 1;
|
|
|
|
props->pkey_tbl_len = 1;
|
|
|
|
props->active_width = 2;
|
2012-02-29 00:49:50 +08:00
|
|
|
props->active_speed = IB_SPEED_DDR;
|
2007-02-13 08:16:18 +08:00
|
|
|
props->max_msg_sz = -1;
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2008-02-22 07:13:36 +08:00
|
|
|
static ssize_t show_rev(struct device *dev, struct device_attribute *attr,
|
|
|
|
char *buf)
|
2007-02-13 08:16:18 +08:00
|
|
|
{
|
2008-02-22 07:13:36 +08:00
|
|
|
struct iwch_dev *iwch_dev = container_of(dev, struct iwch_dev,
|
|
|
|
ibdev.dev);
|
|
|
|
PDBG("%s dev 0x%p\n", __func__, dev);
|
|
|
|
return sprintf(buf, "%d\n", iwch_dev->rdev.t3cdev_p->type);
|
2007-02-13 08:16:18 +08:00
|
|
|
}
|
|
|
|
|
2008-02-22 07:13:36 +08:00
|
|
|
static ssize_t show_fw_ver(struct device *dev, struct device_attribute *attr, char *buf)
|
2007-02-13 08:16:18 +08:00
|
|
|
{
|
2008-02-22 07:13:36 +08:00
|
|
|
struct iwch_dev *iwch_dev = container_of(dev, struct iwch_dev,
|
|
|
|
ibdev.dev);
|
2007-02-13 08:16:18 +08:00
|
|
|
struct ethtool_drvinfo info;
|
2008-02-22 07:13:36 +08:00
|
|
|
struct net_device *lldev = iwch_dev->rdev.t3cdev_p->lldev;
|
2007-02-13 08:16:18 +08:00
|
|
|
|
2008-02-22 07:13:36 +08:00
|
|
|
PDBG("%s dev 0x%p\n", __func__, dev);
|
2007-02-13 08:16:18 +08:00
|
|
|
lldev->ethtool_ops->get_drvinfo(lldev, &info);
|
|
|
|
return sprintf(buf, "%s\n", info.fw_version);
|
|
|
|
}
|
|
|
|
|
2008-02-22 07:13:36 +08:00
|
|
|
static ssize_t show_hca(struct device *dev, struct device_attribute *attr,
|
|
|
|
char *buf)
|
2007-02-13 08:16:18 +08:00
|
|
|
{
|
2008-02-22 07:13:36 +08:00
|
|
|
struct iwch_dev *iwch_dev = container_of(dev, struct iwch_dev,
|
|
|
|
ibdev.dev);
|
2007-02-13 08:16:18 +08:00
|
|
|
struct ethtool_drvinfo info;
|
2008-02-22 07:13:36 +08:00
|
|
|
struct net_device *lldev = iwch_dev->rdev.t3cdev_p->lldev;
|
2007-02-13 08:16:18 +08:00
|
|
|
|
2008-02-22 07:13:36 +08:00
|
|
|
PDBG("%s dev 0x%p\n", __func__, dev);
|
2007-02-13 08:16:18 +08:00
|
|
|
lldev->ethtool_ops->get_drvinfo(lldev, &info);
|
|
|
|
return sprintf(buf, "%s\n", info.driver);
|
|
|
|
}
|
|
|
|
|
2008-02-22 07:13:36 +08:00
|
|
|
static ssize_t show_board(struct device *dev, struct device_attribute *attr,
|
|
|
|
char *buf)
|
2007-02-13 08:16:18 +08:00
|
|
|
{
|
2008-02-22 07:13:36 +08:00
|
|
|
struct iwch_dev *iwch_dev = container_of(dev, struct iwch_dev,
|
|
|
|
ibdev.dev);
|
|
|
|
PDBG("%s dev 0x%p\n", __func__, dev);
|
|
|
|
return sprintf(buf, "%x.%x\n", iwch_dev->rdev.rnic_info.pdev->vendor,
|
|
|
|
iwch_dev->rdev.rnic_info.pdev->device);
|
2007-02-13 08:16:18 +08:00
|
|
|
}
|
|
|
|
|
IB/core: Make device counter infrastructure dynamic
In practice, each RDMA device has a unique set of counters that the
hardware implements. Having a central set of counters that they must
all adhere to is limiting and causes many useful counters to not be
available.
Therefore we create a dynamic counter registration infrastructure.
The driver must implement a stats structure allocation routine, in
which the driver must place the directory name it wants, a list of
names for all of the counters, an array of u64 counters themselves,
plus a few generic configuration options.
We then implement a core routine to create a sysfs file for each
of the named stats elements, and a core routine to retrieve the
stats when any of the sysfs attribute files are read.
To avoid excessive beating on the stats generation routine in the
drivers, the core code also caches the stats for a short period of
time so that someone attempting to read all of the stats in a
given device's directory will not result in a stats generation
call per file read.
Future work will attempt to standardize just the shared stats
elements, and possibly add a method to get the stats via netlink
in addition to sysfs.
Signed-off-by: Christoph Lameter <cl@linux.com>
Signed-off-by: Mark Bloch <markb@mellanox.com>
Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Doug Ledford <dledford@redhat.com>
[ Add caching, make structure names more informative, add i40iw support,
other significant rewrites from the original patch ]
2016-05-17 01:49:33 +08:00
|
|
|
enum counters {
|
|
|
|
IPINRECEIVES,
|
|
|
|
IPINHDRERRORS,
|
|
|
|
IPINADDRERRORS,
|
|
|
|
IPINUNKNOWNPROTOS,
|
|
|
|
IPINDISCARDS,
|
|
|
|
IPINDELIVERS,
|
|
|
|
IPOUTREQUESTS,
|
|
|
|
IPOUTDISCARDS,
|
|
|
|
IPOUTNOROUTES,
|
|
|
|
IPREASMTIMEOUT,
|
|
|
|
IPREASMREQDS,
|
|
|
|
IPREASMOKS,
|
|
|
|
IPREASMFAILS,
|
|
|
|
TCPACTIVEOPENS,
|
|
|
|
TCPPASSIVEOPENS,
|
|
|
|
TCPATTEMPTFAILS,
|
|
|
|
TCPESTABRESETS,
|
|
|
|
TCPCURRESTAB,
|
|
|
|
TCPINSEGS,
|
|
|
|
TCPOUTSEGS,
|
|
|
|
TCPRETRANSSEGS,
|
|
|
|
TCPINERRS,
|
|
|
|
TCPOUTRSTS,
|
|
|
|
TCPRTOMIN,
|
|
|
|
TCPRTOMAX,
|
|
|
|
NR_COUNTERS
|
|
|
|
};
|
|
|
|
|
|
|
|
static const char * const names[] = {
|
|
|
|
[IPINRECEIVES] = "ipInReceives",
|
|
|
|
[IPINHDRERRORS] = "ipInHdrErrors",
|
|
|
|
[IPINADDRERRORS] = "ipInAddrErrors",
|
|
|
|
[IPINUNKNOWNPROTOS] = "ipInUnknownProtos",
|
|
|
|
[IPINDISCARDS] = "ipInDiscards",
|
|
|
|
[IPINDELIVERS] = "ipInDelivers",
|
|
|
|
[IPOUTREQUESTS] = "ipOutRequests",
|
|
|
|
[IPOUTDISCARDS] = "ipOutDiscards",
|
|
|
|
[IPOUTNOROUTES] = "ipOutNoRoutes",
|
|
|
|
[IPREASMTIMEOUT] = "ipReasmTimeout",
|
|
|
|
[IPREASMREQDS] = "ipReasmReqds",
|
|
|
|
[IPREASMOKS] = "ipReasmOKs",
|
|
|
|
[IPREASMFAILS] = "ipReasmFails",
|
|
|
|
[TCPACTIVEOPENS] = "tcpActiveOpens",
|
|
|
|
[TCPPASSIVEOPENS] = "tcpPassiveOpens",
|
|
|
|
[TCPATTEMPTFAILS] = "tcpAttemptFails",
|
|
|
|
[TCPESTABRESETS] = "tcpEstabResets",
|
|
|
|
[TCPCURRESTAB] = "tcpCurrEstab",
|
|
|
|
[TCPINSEGS] = "tcpInSegs",
|
|
|
|
[TCPOUTSEGS] = "tcpOutSegs",
|
|
|
|
[TCPRETRANSSEGS] = "tcpRetransSegs",
|
|
|
|
[TCPINERRS] = "tcpInErrs",
|
|
|
|
[TCPOUTRSTS] = "tcpOutRsts",
|
|
|
|
[TCPRTOMIN] = "tcpRtoMin",
|
|
|
|
[TCPRTOMAX] = "tcpRtoMax",
|
|
|
|
};
|
|
|
|
|
|
|
|
static struct rdma_hw_stats *iwch_alloc_stats(struct ib_device *ibdev,
|
|
|
|
u8 port_num)
|
|
|
|
{
|
|
|
|
BUILD_BUG_ON(ARRAY_SIZE(names) != NR_COUNTERS);
|
|
|
|
|
|
|
|
/* Our driver only supports device level stats */
|
|
|
|
if (port_num != 0)
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
return rdma_alloc_hw_stats_struct(names, NR_COUNTERS,
|
|
|
|
RDMA_HW_STATS_DEFAULT_LIFESPAN);
|
|
|
|
}
|
|
|
|
|
|
|
|
static int iwch_get_mib(struct ib_device *ibdev, struct rdma_hw_stats *stats,
|
|
|
|
u8 port, int index)
|
2008-07-15 14:48:48 +08:00
|
|
|
{
|
|
|
|
struct iwch_dev *dev;
|
|
|
|
struct tp_mib_stats m;
|
|
|
|
int ret;
|
|
|
|
|
IB/core: Make device counter infrastructure dynamic
In practice, each RDMA device has a unique set of counters that the
hardware implements. Having a central set of counters that they must
all adhere to is limiting and causes many useful counters to not be
available.
Therefore we create a dynamic counter registration infrastructure.
The driver must implement a stats structure allocation routine, in
which the driver must place the directory name it wants, a list of
names for all of the counters, an array of u64 counters themselves,
plus a few generic configuration options.
We then implement a core routine to create a sysfs file for each
of the named stats elements, and a core routine to retrieve the
stats when any of the sysfs attribute files are read.
To avoid excessive beating on the stats generation routine in the
drivers, the core code also caches the stats for a short period of
time so that someone attempting to read all of the stats in a
given device's directory will not result in a stats generation
call per file read.
Future work will attempt to standardize just the shared stats
elements, and possibly add a method to get the stats via netlink
in addition to sysfs.
Signed-off-by: Christoph Lameter <cl@linux.com>
Signed-off-by: Mark Bloch <markb@mellanox.com>
Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Doug Ledford <dledford@redhat.com>
[ Add caching, make structure names more informative, add i40iw support,
other significant rewrites from the original patch ]
2016-05-17 01:49:33 +08:00
|
|
|
if (port != 0 || !stats)
|
|
|
|
return -ENOSYS;
|
|
|
|
|
2008-07-15 14:48:48 +08:00
|
|
|
PDBG("%s ibdev %p\n", __func__, ibdev);
|
|
|
|
dev = to_iwch_dev(ibdev);
|
|
|
|
ret = dev->rdev.t3cdev_p->ctl(dev->rdev.t3cdev_p, RDMA_GET_MIB, &m);
|
|
|
|
if (ret)
|
|
|
|
return -ENOSYS;
|
|
|
|
|
IB/core: Make device counter infrastructure dynamic
In practice, each RDMA device has a unique set of counters that the
hardware implements. Having a central set of counters that they must
all adhere to is limiting and causes many useful counters to not be
available.
Therefore we create a dynamic counter registration infrastructure.
The driver must implement a stats structure allocation routine, in
which the driver must place the directory name it wants, a list of
names for all of the counters, an array of u64 counters themselves,
plus a few generic configuration options.
We then implement a core routine to create a sysfs file for each
of the named stats elements, and a core routine to retrieve the
stats when any of the sysfs attribute files are read.
To avoid excessive beating on the stats generation routine in the
drivers, the core code also caches the stats for a short period of
time so that someone attempting to read all of the stats in a
given device's directory will not result in a stats generation
call per file read.
Future work will attempt to standardize just the shared stats
elements, and possibly add a method to get the stats via netlink
in addition to sysfs.
Signed-off-by: Christoph Lameter <cl@linux.com>
Signed-off-by: Mark Bloch <markb@mellanox.com>
Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Doug Ledford <dledford@redhat.com>
[ Add caching, make structure names more informative, add i40iw support,
other significant rewrites from the original patch ]
2016-05-17 01:49:33 +08:00
|
|
|
stats->value[IPINRECEIVES] = ((u64)m.ipInReceive_hi << 32) + m.ipInReceive_lo;
|
|
|
|
stats->value[IPINHDRERRORS] = ((u64)m.ipInHdrErrors_hi << 32) + m.ipInHdrErrors_lo;
|
|
|
|
stats->value[IPINADDRERRORS] = ((u64)m.ipInAddrErrors_hi << 32) + m.ipInAddrErrors_lo;
|
|
|
|
stats->value[IPINUNKNOWNPROTOS] = ((u64)m.ipInUnknownProtos_hi << 32) + m.ipInUnknownProtos_lo;
|
|
|
|
stats->value[IPINDISCARDS] = ((u64)m.ipInDiscards_hi << 32) + m.ipInDiscards_lo;
|
|
|
|
stats->value[IPINDELIVERS] = ((u64)m.ipInDelivers_hi << 32) + m.ipInDelivers_lo;
|
|
|
|
stats->value[IPOUTREQUESTS] = ((u64)m.ipOutRequests_hi << 32) + m.ipOutRequests_lo;
|
|
|
|
stats->value[IPOUTDISCARDS] = ((u64)m.ipOutDiscards_hi << 32) + m.ipOutDiscards_lo;
|
|
|
|
stats->value[IPOUTNOROUTES] = ((u64)m.ipOutNoRoutes_hi << 32) + m.ipOutNoRoutes_lo;
|
|
|
|
stats->value[IPREASMTIMEOUT] = m.ipReasmTimeout;
|
|
|
|
stats->value[IPREASMREQDS] = m.ipReasmReqds;
|
|
|
|
stats->value[IPREASMOKS] = m.ipReasmOKs;
|
|
|
|
stats->value[IPREASMFAILS] = m.ipReasmFails;
|
|
|
|
stats->value[TCPACTIVEOPENS] = m.tcpActiveOpens;
|
|
|
|
stats->value[TCPPASSIVEOPENS] = m.tcpPassiveOpens;
|
|
|
|
stats->value[TCPATTEMPTFAILS] = m.tcpAttemptFails;
|
|
|
|
stats->value[TCPESTABRESETS] = m.tcpEstabResets;
|
|
|
|
stats->value[TCPCURRESTAB] = m.tcpOutRsts;
|
|
|
|
stats->value[TCPINSEGS] = m.tcpCurrEstab;
|
|
|
|
stats->value[TCPOUTSEGS] = ((u64)m.tcpInSegs_hi << 32) + m.tcpInSegs_lo;
|
|
|
|
stats->value[TCPRETRANSSEGS] = ((u64)m.tcpOutSegs_hi << 32) + m.tcpOutSegs_lo;
|
|
|
|
stats->value[TCPINERRS] = ((u64)m.tcpRetransSeg_hi << 32) + m.tcpRetransSeg_lo,
|
|
|
|
stats->value[TCPOUTRSTS] = ((u64)m.tcpInErrs_hi << 32) + m.tcpInErrs_lo;
|
|
|
|
stats->value[TCPRTOMIN] = m.tcpRtoMin;
|
|
|
|
stats->value[TCPRTOMAX] = m.tcpRtoMax;
|
|
|
|
|
|
|
|
return stats->num_counters;
|
2008-07-15 14:48:48 +08:00
|
|
|
}
|
|
|
|
|
2008-02-22 07:13:36 +08:00
|
|
|
static DEVICE_ATTR(hw_rev, S_IRUGO, show_rev, NULL);
|
|
|
|
static DEVICE_ATTR(fw_ver, S_IRUGO, show_fw_ver, NULL);
|
|
|
|
static DEVICE_ATTR(hca_type, S_IRUGO, show_hca, NULL);
|
|
|
|
static DEVICE_ATTR(board_id, S_IRUGO, show_board, NULL);
|
2007-02-13 08:16:18 +08:00
|
|
|
|
2008-02-22 07:13:36 +08:00
|
|
|
static struct device_attribute *iwch_class_attributes[] = {
|
|
|
|
&dev_attr_hw_rev,
|
|
|
|
&dev_attr_fw_ver,
|
|
|
|
&dev_attr_hca_type,
|
2008-07-15 14:48:48 +08:00
|
|
|
&dev_attr_board_id,
|
2007-02-13 08:16:18 +08:00
|
|
|
};
|
|
|
|
|
2015-05-14 08:02:58 +08:00
|
|
|
static int iwch_port_immutable(struct ib_device *ibdev, u8 port_num,
|
|
|
|
struct ib_port_immutable *immutable)
|
|
|
|
{
|
|
|
|
struct ib_port_attr attr;
|
|
|
|
int err;
|
|
|
|
|
|
|
|
err = iwch_query_port(ibdev, port_num, &attr);
|
|
|
|
if (err)
|
|
|
|
return err;
|
|
|
|
|
|
|
|
immutable->pkey_tbl_len = attr.pkey_tbl_len;
|
|
|
|
immutable->gid_tbl_len = attr.gid_tbl_len;
|
2015-05-14 08:02:59 +08:00
|
|
|
immutable->core_cap_flags = RDMA_CORE_PORT_IWARP;
|
2015-05-14 08:02:58 +08:00
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2007-02-13 08:16:18 +08:00
|
|
|
int iwch_register_device(struct iwch_dev *dev)
|
|
|
|
{
|
|
|
|
int ret;
|
|
|
|
int i;
|
|
|
|
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s iwch_dev %p\n", __func__, dev);
|
2007-02-13 08:16:18 +08:00
|
|
|
strlcpy(dev->ibdev.name, "cxgb3_%d", IB_DEVICE_NAME_MAX);
|
|
|
|
memset(&dev->ibdev.node_guid, 0, sizeof(dev->ibdev.node_guid));
|
|
|
|
memcpy(&dev->ibdev.node_guid, dev->rdev.t3cdev_p->lldev->dev_addr, 6);
|
|
|
|
dev->ibdev.owner = THIS_MODULE;
|
2008-08-05 02:08:37 +08:00
|
|
|
dev->device_cap_flags = IB_DEVICE_LOCAL_DMA_LKEY |
|
|
|
|
IB_DEVICE_MEM_WINDOW |
|
|
|
|
IB_DEVICE_MEM_MGT_EXTENSIONS;
|
2008-07-15 14:48:53 +08:00
|
|
|
|
|
|
|
/* cxgb3 supports STag 0. */
|
|
|
|
dev->ibdev.local_dma_lkey = 0;
|
2007-02-13 08:16:18 +08:00
|
|
|
|
|
|
|
dev->ibdev.uverbs_cmd_mask =
|
|
|
|
(1ull << IB_USER_VERBS_CMD_GET_CONTEXT) |
|
|
|
|
(1ull << IB_USER_VERBS_CMD_QUERY_DEVICE) |
|
|
|
|
(1ull << IB_USER_VERBS_CMD_QUERY_PORT) |
|
|
|
|
(1ull << IB_USER_VERBS_CMD_ALLOC_PD) |
|
|
|
|
(1ull << IB_USER_VERBS_CMD_DEALLOC_PD) |
|
|
|
|
(1ull << IB_USER_VERBS_CMD_REG_MR) |
|
|
|
|
(1ull << IB_USER_VERBS_CMD_DEREG_MR) |
|
|
|
|
(1ull << IB_USER_VERBS_CMD_CREATE_COMP_CHANNEL) |
|
|
|
|
(1ull << IB_USER_VERBS_CMD_CREATE_CQ) |
|
|
|
|
(1ull << IB_USER_VERBS_CMD_DESTROY_CQ) |
|
|
|
|
(1ull << IB_USER_VERBS_CMD_REQ_NOTIFY_CQ) |
|
|
|
|
(1ull << IB_USER_VERBS_CMD_CREATE_QP) |
|
|
|
|
(1ull << IB_USER_VERBS_CMD_MODIFY_QP) |
|
|
|
|
(1ull << IB_USER_VERBS_CMD_POLL_CQ) |
|
|
|
|
(1ull << IB_USER_VERBS_CMD_DESTROY_QP) |
|
|
|
|
(1ull << IB_USER_VERBS_CMD_POST_SEND) |
|
|
|
|
(1ull << IB_USER_VERBS_CMD_POST_RECV);
|
|
|
|
dev->ibdev.node_type = RDMA_NODE_RNIC;
|
|
|
|
memcpy(dev->ibdev.node_desc, IWCH_NODE_DESC, sizeof(IWCH_NODE_DESC));
|
|
|
|
dev->ibdev.phys_port_cnt = dev->rdev.port_info.nports;
|
2007-05-03 18:48:47 +08:00
|
|
|
dev->ibdev.num_comp_vectors = 1;
|
2007-02-13 08:16:18 +08:00
|
|
|
dev->ibdev.dma_device = &(dev->rdev.rnic_info.pdev->dev);
|
|
|
|
dev->ibdev.query_device = iwch_query_device;
|
|
|
|
dev->ibdev.query_port = iwch_query_port;
|
|
|
|
dev->ibdev.query_pkey = iwch_query_pkey;
|
|
|
|
dev->ibdev.query_gid = iwch_query_gid;
|
|
|
|
dev->ibdev.alloc_ucontext = iwch_alloc_ucontext;
|
|
|
|
dev->ibdev.dealloc_ucontext = iwch_dealloc_ucontext;
|
|
|
|
dev->ibdev.mmap = iwch_mmap;
|
|
|
|
dev->ibdev.alloc_pd = iwch_allocate_pd;
|
|
|
|
dev->ibdev.dealloc_pd = iwch_deallocate_pd;
|
|
|
|
dev->ibdev.create_ah = iwch_ah_create;
|
|
|
|
dev->ibdev.destroy_ah = iwch_ah_destroy;
|
|
|
|
dev->ibdev.create_qp = iwch_create_qp;
|
|
|
|
dev->ibdev.modify_qp = iwch_ib_modify_qp;
|
|
|
|
dev->ibdev.destroy_qp = iwch_destroy_qp;
|
|
|
|
dev->ibdev.create_cq = iwch_create_cq;
|
|
|
|
dev->ibdev.destroy_cq = iwch_destroy_cq;
|
|
|
|
dev->ibdev.resize_cq = iwch_resize_cq;
|
|
|
|
dev->ibdev.poll_cq = iwch_poll_cq;
|
|
|
|
dev->ibdev.get_dma_mr = iwch_get_dma_mr;
|
|
|
|
dev->ibdev.reg_user_mr = iwch_reg_user_mr;
|
|
|
|
dev->ibdev.dereg_mr = iwch_dereg_mr;
|
|
|
|
dev->ibdev.alloc_mw = iwch_alloc_mw;
|
|
|
|
dev->ibdev.dealloc_mw = iwch_dealloc_mw;
|
2015-07-30 15:32:45 +08:00
|
|
|
dev->ibdev.alloc_mr = iwch_alloc_mr;
|
2015-10-14 00:11:29 +08:00
|
|
|
dev->ibdev.map_mr_sg = iwch_map_mr_sg;
|
2007-02-13 08:16:18 +08:00
|
|
|
dev->ibdev.attach_mcast = iwch_multicast_attach;
|
|
|
|
dev->ibdev.detach_mcast = iwch_multicast_detach;
|
|
|
|
dev->ibdev.process_mad = iwch_process_mad;
|
|
|
|
dev->ibdev.req_notify_cq = iwch_arm_cq;
|
|
|
|
dev->ibdev.post_send = iwch_post_send;
|
|
|
|
dev->ibdev.post_recv = iwch_post_receive;
|
IB/core: Make device counter infrastructure dynamic
In practice, each RDMA device has a unique set of counters that the
hardware implements. Having a central set of counters that they must
all adhere to is limiting and causes many useful counters to not be
available.
Therefore we create a dynamic counter registration infrastructure.
The driver must implement a stats structure allocation routine, in
which the driver must place the directory name it wants, a list of
names for all of the counters, an array of u64 counters themselves,
plus a few generic configuration options.
We then implement a core routine to create a sysfs file for each
of the named stats elements, and a core routine to retrieve the
stats when any of the sysfs attribute files are read.
To avoid excessive beating on the stats generation routine in the
drivers, the core code also caches the stats for a short period of
time so that someone attempting to read all of the stats in a
given device's directory will not result in a stats generation
call per file read.
Future work will attempt to standardize just the shared stats
elements, and possibly add a method to get the stats via netlink
in addition to sysfs.
Signed-off-by: Christoph Lameter <cl@linux.com>
Signed-off-by: Mark Bloch <markb@mellanox.com>
Reviewed-by: Steve Wise <swise@opengridcomputing.com>
Signed-off-by: Doug Ledford <dledford@redhat.com>
[ Add caching, make structure names more informative, add i40iw support,
other significant rewrites from the original patch ]
2016-05-17 01:49:33 +08:00
|
|
|
dev->ibdev.alloc_hw_stats = iwch_alloc_stats;
|
|
|
|
dev->ibdev.get_hw_stats = iwch_get_mib;
|
2010-10-21 20:37:06 +08:00
|
|
|
dev->ibdev.uverbs_abi_ver = IWCH_UVERBS_ABI_VERSION;
|
2015-05-14 08:02:58 +08:00
|
|
|
dev->ibdev.get_port_immutable = iwch_port_immutable;
|
2007-02-13 08:16:18 +08:00
|
|
|
|
2007-07-10 11:12:26 +08:00
|
|
|
dev->ibdev.iwcm = kmalloc(sizeof(struct iw_cm_verbs), GFP_KERNEL);
|
|
|
|
if (!dev->ibdev.iwcm)
|
|
|
|
return -ENOMEM;
|
|
|
|
|
2007-02-13 08:16:18 +08:00
|
|
|
dev->ibdev.iwcm->connect = iwch_connect;
|
|
|
|
dev->ibdev.iwcm->accept = iwch_accept_cr;
|
|
|
|
dev->ibdev.iwcm->reject = iwch_reject_cr;
|
|
|
|
dev->ibdev.iwcm->create_listen = iwch_create_listen;
|
|
|
|
dev->ibdev.iwcm->destroy_listen = iwch_destroy_listen;
|
|
|
|
dev->ibdev.iwcm->add_ref = iwch_qp_add_ref;
|
|
|
|
dev->ibdev.iwcm->rem_ref = iwch_qp_rem_ref;
|
|
|
|
dev->ibdev.iwcm->get_qp = iwch_get_qp;
|
2016-04-12 21:55:01 +08:00
|
|
|
memcpy(dev->ibdev.iwcm->ifname, dev->rdev.t3cdev_p->lldev->name,
|
|
|
|
sizeof(dev->ibdev.iwcm->ifname));
|
2007-02-13 08:16:18 +08:00
|
|
|
|
2010-05-07 08:03:25 +08:00
|
|
|
ret = ib_register_device(&dev->ibdev, NULL);
|
2007-02-13 08:16:18 +08:00
|
|
|
if (ret)
|
|
|
|
goto bail1;
|
|
|
|
|
|
|
|
for (i = 0; i < ARRAY_SIZE(iwch_class_attributes); ++i) {
|
2008-02-22 07:13:36 +08:00
|
|
|
ret = device_create_file(&dev->ibdev.dev,
|
|
|
|
iwch_class_attributes[i]);
|
2007-02-13 08:16:18 +08:00
|
|
|
if (ret) {
|
|
|
|
goto bail2;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
return 0;
|
|
|
|
bail2:
|
|
|
|
ib_unregister_device(&dev->ibdev);
|
|
|
|
bail1:
|
2009-09-06 11:22:36 +08:00
|
|
|
kfree(dev->ibdev.iwcm);
|
2007-02-13 08:16:18 +08:00
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
|
|
|
void iwch_unregister_device(struct iwch_dev *dev)
|
|
|
|
{
|
|
|
|
int i;
|
|
|
|
|
2008-04-17 12:01:10 +08:00
|
|
|
PDBG("%s iwch_dev %p\n", __func__, dev);
|
2007-02-13 08:16:18 +08:00
|
|
|
for (i = 0; i < ARRAY_SIZE(iwch_class_attributes); ++i)
|
2008-02-22 07:13:36 +08:00
|
|
|
device_remove_file(&dev->ibdev.dev,
|
|
|
|
iwch_class_attributes[i]);
|
2007-02-13 08:16:18 +08:00
|
|
|
ib_unregister_device(&dev->ibdev);
|
2009-09-06 11:22:36 +08:00
|
|
|
kfree(dev->ibdev.iwcm);
|
2007-02-13 08:16:18 +08:00
|
|
|
return;
|
|
|
|
}
|