For multipage BIOs, we were always using sg[0] instead of advancing
through the list. Oops :-)
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>
If POISON_POINTER_DELTA isn't defined, ensure they're in page 0 which
should never be mapped.
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>
In the bio completion handler, check for bios on the congestion list
for this NVM queue. Also, lock the congestion list in the make_request
function as the queue may end up being shared between multiple CPUs.
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>
In addition to recording the completion data for each command, record
the anticipated completion time. Choose a timeout of 5 seconds for
normal I/Os and 60 seconds for admin I/Os.
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>
If we're sharing a queue between multiple CPUs and we cancel a sync I/O,
we must have the queue locked to avoid corrupting the stack of the thread
that submitted the I/O. It turns out this is the same locking that's needed
for the threaded irq handler, so share that code.
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>
If the adapter completes a command ID that is outside the bounds of
the array, return CMD_CTX_INVALID instead of random data, and print a
message in the sync_completion handler (which is rapidly becoming the
misc completion handler :-)
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>
Set the context value to CMD_CTX_COMPLETED, and print a message in the
sync_completion handler if we see it.
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>
I have plans for other special values in sync_completion. Plus, this
is more self-documenting, and lets us detect bogus usages.
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>
We're currently calling bio_endio from hard interrupt context. This is
not a good idea for preemptible kernels as it will cause longer latencies.
Using a threaded interrupt will run the entire queue processing mechanism
(including bio_endio) in a thread, which can be preempted. Unfortuantely,
it also adds about 7us of latency to the single-I/O case, so make it a
module parameter for the moment.
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>
We can't have preemption disabled when we call schedule(). Accept the
possibility that we'll get preempted, and it'll cost us some cacheline
bounces.
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>
If the user sends a fatal signal, sleeping in the TASK_KILLABLE state
permits the task to be aborted. The only wrinkle is making sure that
if/when the command completes later that it doesn't upset anything.
Handle this by setting the data pointer to 0, and checking the value
isn't NULL in the sync completion path. Eventually, bios can be cancelled
through this path too. Note that the cmdid isn't freed to prevent reuse.
We should also abort the command in the future, but this is a good start.
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>
Because I wasn't setting driverfs_dev, the devices were showing up under
/sys/devices/virtual/block. Now they appear underneath the PCI device
which they belong to.
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>
In case the card has been left in a partially-configured state,
write 0 to the Enable bit.
Signed-off-by: Shane Michael Matthews <shane.matthews@intel.com>
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>
Call pci_enable_device_mem() at initialisation and pci_disable_device
at exit.
Signed-off-by: Shane Michael Matthews <shane.matthews@intel.com>
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>
Factor out most of nvme_identify() into a new nvme_submit_user_admin_command()
function. Change nvme_get_range_type() to call it and change nvme_ioctl to
realise that it's getting back all 64 ranges.
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>
Generalise the code from nvme_identify() that sets PRP1 & PRP2 so that
it's usable for commands sent by nvme_submit_bio_queue().
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>
The admin IRQ is supposed to use the pin-based (or single message MSI)
interrupt. Accomplish this by filling in entry[0]'s vector with the
INTx irq number.
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>
Two callers with an almost identical long string of arguments, and
introducing a third soon. Time to factor out the commonalities.
Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com>
