Some commands to the xHCI hardware cannot be allowed to fail due to out of
memory issues or the command ring being full.
Add a way to reserve a TRB on the command ring, and make all command
queueing functions indicate whether they are using a reserved TRB.
Add a way to pre-allocate all the memory a command might need. A command
needs an input context, a variable to store the status, and (optionally) a
completion for the caller to wait on. Change all code that assumes the
input device context, status, and completion for a command is stored in
the xhci virtual USB device structure (xhci_virt_device).
Store pending completions in a FIFO in xhci_virt_device. Make the event
handler for a configure endpoint command check to see whether a pending
command in the list has completed. We need to use separate input device
contexts for some configure endpoint commands, since multiple drivers can
submit requests at the same time that require a configure endpoint
command.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The xhci_ring structure contained information that is really related to an
endpoint, not a ring. This will cause problems later when endpoint
streams are supported and there are multiple rings per endpoint.
Move the endpoint state and cancellation information into a new virtual
endpoint structure, xhci_virt_ep. The list of TRBs to be cancelled should
be per endpoint, not per ring, for easy access. There can be only one TRB
that the endpoint stopped on after a stop endpoint command (even with
streams enabled); move the stopped TRB information into the new virtual
endpoint structure. Also move the 31 endpoint rings and temporary ring
storage from the virtual device structure (xhci_virt_device) into the
virtual endpoint structure (xhci_virt_ep).
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Interrupt transfers are submitted to the xHCI hardware using the same TRB
type as bulk transfers. Re-use the bulk transfer enqueueing code to
enqueue interrupt transfers.
Interrupt transfers are a bit different than bulk transfers. When the
interrupt endpoint is to be serviced, the xHC will consume (at most) one
TD. A TD (comprised of sg list entries) can take several service
intervals to transmit. The important thing for device drivers to note is
that if they use the scatter gather interface to submit interrupt
requests, they will not get data sent from two different scatter gather
lists in the same service interval.
For now, the xHCI driver will use the service interval from the endpoint's
descriptor (bInterval). Drivers will need a hook to poll at a more
frequent interval. Set urb->interval to the interval that the xHCI
hardware will use.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The xHCI hardware reports the number of bytes untransferred for a given
transfer buffer. If the hardware reports a bytes untransferred value
greater than the submitted buffer size, we want to play it safe and say no
data was transferred. If the driver considers a short packet to be an
error, remember to set -EREMOTEIO.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Make sure that the driver that submitted the URB considers a short packet
an error before setting -EREMOTEIO during a short control transfer.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Make sure that the amount of data the xHC says was transmitted is less
than or equal to the size of the requested transfer buffer. Before, if
the host controller erroneously reported that the number of bytes
untransferred was bigger than the buffer in the URB, urb->actual_length
could be set to a very large size.
Make sure urb->actual_length <= urb->transfer_buffer_length.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
On a successful transfer, urb->td is freed before the URB is ready to be
given back to the driver. Don't touch urb->td after it's freed. This bug
would have only shown up when xHCI debugging was turned on, and the freed
memory was quickly reused for something else.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The 0.95 xHCI spec says that non-control endpoints will be halted if a
babble is detected on a transfer. The 0.96 xHCI spec says all types of
endpoints will be halted when a babble is detected. Some hardware that
claims to be 0.95 compliant halts the control endpoint anyway.
When a babble is detected on a control endpoint, check the hardware's
output endpoint context to see if the endpoint is marked as halted. If
the control endpoint is halted, a reset endpoint command must be issued
and the transfer ring dequeue pointer needs to be moved past the stopped
transfer. Basically, we treat it as if the control endpoint had stalled.
Handle bulk babbling endpoints as if we got a completion event with a
stall completion code.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Use trb_comp_code instead of getting the completion code from the transfer
event every time.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
This Fresco Logic xHCI host controller chip revision puts bad data into
the output endpoint context after a Reset Endpoint command. It needs a
Configure Endpoint command (instead of a Set TR Dequeue Pointer command)
after the reset endpoint command.
Set up the input context before issuing the Reset Endpoint command so we
don't copy bad data from the output endpoint context. The HW also can't
handle two commands queued at once, so submit the TRB for the Configure
Endpoint command in the event handler for the Reset Endpoint command.
Devices that stall on control endpoints before a configuration is selected
will not work under this Fresco Logic xHCI host controller revision.
This patch is for prototype hardware that will be given to other companies
for evaluation purposes only, and should not reach consumer hands. Fresco
Logic's next chip rev should have this bug fixed.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When a control endpoint stalls, the next control transfer will clear the
stall. The USB core doesn't call down to the host controller driver's
endpoint_reset() method when control endpoints stall, so the xHCI driver
has to do all its stall handling for internal state in its interrupt handler.
When the host stalls on a control endpoint, it may stop on the data phase
or status phase of the control transfer. Like other stalled endpoints,
the xHCI driver needs to queue a Reset Endpoint command and move the
hardware's control endpoint ring dequeue pointer past the failed control
transfer (with a Set TR Dequeue Pointer or a Configure Endpoint command).
Since the USB core doesn't call usb_hcd_reset_endpoint() for control
endpoints, we need to do this in interrupt context when we get notified of
the stalled transfer. URBs may be queued to the hardware before these two
commands complete. The endpoint queue will be restarted once both
commands complete.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Full speed devices have varying max packet sizes (8, 16, 32, or 64) for
endpoint 0. The xHCI hardware needs to know the real max packet size
that the USB core discovers after it fetches the first 8 bytes of the
device descriptor.
In order to fix this without adding a new hook to host controller drivers,
the xHCI driver looks for an updated max packet size for control
endpoints. If it finds an updated size, it issues an evaluate context
command and waits for that command to finish. This should only happen in
the initialization and device descriptor fetching steps in the khubd
thread, so blocking should be fine.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Refactor out the code issue, wait for, and parse the event completion code
for a configure endpoint command. Modify it to support the evaluate
context command, which has a very similar submission process. Add
functions to copy parts of the output context into the input context
(which will be used in the evaluate context command).
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Different sections of the xHCI 0.95 specification had opposing
requirements for the chain bit in a link transaction request buffer (TRB).
The chain bit is used to designate that adjacent TRBs are all part of the
same scatter gather list that should be sent to the device. Link TRBs can
be in the middle, or at the beginning or end of these chained TRBs.
Sections 4.11.5.1 and 6.4.4.1 both stated the link TRB "shall have the
chain bit set to 1", meaning it is always chained to the next TRB.
However, section 4.6.9 on the stop endpoint command has specific cases for
what the hardware must do for a link TRB with the chain bit set to 0. The
0.96 specification errata later cleared up this issue by fixing the
4.11.5.1 and 6.4.4.1 sections to state that a link TRB can have the chain
bit set to 1 or 0.
The problem is that the xHCI cancellation code depends on the chain bit of
the link TRB being cleared when it's at the end of a TD, and some 0.95
xHCI hardware simply stops processing the ring when it encounters a link
TRB with the chain bit cleared.
Allow users who are testing 0.95 xHCI prototypes to set a module parameter
(link_quirk) to turn on this link TRB work around. Cancellation may not
work if the ring is stopped exactly on a link TRB with chain bit set, but
cancellation should be a relatively uncommon case.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Correct the xHCI code to handle stalls on USB endpoints. We need to move
the endpoint ring's dequeue pointer past the stalled transfer, or the HW
will try to restart the transfer the next time the doorbell is rung.
Don't attempt to clear a halt on an endpoint if we haven't seen a stalled
transfer for it. The USB core will attempt to clear a halt on all
endpoints when it selects a new configuration.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Adds support for controllers that use 64-byte contexts. The following context
data structures are affected by this: Device, Input, Input Control, Endpoint,
and Slot. To accommodate the use of either 32 or 64-byte contexts, a Device or
Input context can only be accessed through functions which look-up and return
pointers to their contained contexts.
Signed-off-by: John Youn <johnyoun@synopsys.com>
Acked-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Pass back a babble error when this error code is seen in the transfer event TRB.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Add more debugging to the irq handler, slot context initialization, ring
operations, URB cancellation, and MMIO writes.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The Event Handler Busy bit in the event ring dequeue pointer is write 1 to
clear. Fix the interrupt service routine to clear that bit after the
event handler has run.
xhci_set_hc_event_deq() is designed to update the event ring dequeue pointer
without changing any of the four reserved bits in the lower nibble. The event
handler busy (EHB) bit is write one to clear, so the new value must always
contain a zero in that bit in order to preserve the EHB value.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When there is a short packet on a control transfer, the xHCI host controller
hardware will generate two events. The first event will be for the data stage
TD with a completion code for a short packet. The second event will be for the
status stage with a successful completion code. Before this patch, the xHCI
driver would giveback the short control URB when it received the event for the
data stage TD. Then it would become confused when it saw a status stage event
for the endpoint for an URB it had already finished processing.
Change the xHCI host controller driver to wait for the status stage event when
it receives a short transfer completion code for a data stage TD.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
There are several xHCI data structures that use two 32-bit fields to
represent a 64-bit address. Since some architectures don't support 64-bit
PCI writes, the fields need to be written in two 32-bit writes. The xHCI
specification says that if a platform is incapable of generating 64-bit
writes, software must write the low 32-bits first, then the high 32-bits.
Hardware that supports 64-bit addressing will wait for the high 32-bit
write before reading the revised value, and hardware that only supports
32-bit writes will ignore the high 32-bit write.
Previous xHCI code represented 64-bit addresses with two u32 values. This
lead to buggy code that would write the 32-bits in the wrong order, or
forget to write the upper 32-bits. Change the two u32s to one u64 and
create a function call to write all 64-bit addresses in the proper order.
This new function could be modified in the future if all platforms support
64-bit writes.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When an endpoint on a device under an xHCI host controller stalls, the
host controller driver must let the hardware know that the USB core has
successfully cleared the halt condition. The HCD submits a Reset Endpoint
Command, which will clear the toggle bit for USB 2.0 devices, and set the
sequence number to zero for USB 3.0 devices.
The xHCI urb_enqueue will accept new URBs while the endpoint is halted,
and will queue them to the hardware rings. However, the endpoint doorbell
will not be rung until the Reset Endpoint Command completes.
Don't queue a reset endpoint command for root hubs. khubd clears halt
conditions on the roothub during the initialization process, but the roothub
isn't a real device, so the xHCI host controller doesn't need to know about the
cleared halt.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The 0.95 xHCI specification requires software to set the "TD size" field
in each transaction request block (TRB). This field gives the host
controller an indication of how much data is remaining in the TD
(including the buffer in the current TRB). Set this field in bulk TRBs
and data stage TRBs for control transfers.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Greg KH introduced a bug into xhci_trb_virt_to_dma() when he changed the
type of offset to dma_addr_t from unsigned int and dropped the casts to
unsigned int around the virtual address pointer subtraction.
trb and seg->trbs are both valid pointers to virtual addresses, so the
compiler will mod the subtraction by the size of union trb (16 bytes).
segment_offset is an unsigned long, which is guaranteed to be at least as
big as a void *.
Drop the void * casts in the first if statement because trb and seg->trbs
are both pointers of the same type (pointers to union trb).
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Force the compiler to write the cycle bit of the Link TRB last. This
ensures that the hardware doesn't think it owns the Link TRB before we set
the chain bit. Reported by Oliver in this thread:
http://marc.info/?l=linux-usb&m=124091532410219&w=2
Reported-by: Oliver Neukum <oliver@neukum.org>
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Make all globally visible functions start with xhci_ and mark functions as
static if they're only called within the same C file. Fix some long lines
while we're at it.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Make sure to preserve all bits *except* the TRB_CHAIN bit when giving a
Link TRB to the hardware. We need to save things like TRB type and the
toggle bit in the control dword.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Turns out someone never built this code on a 64bit platform.
Someone owes me a beer...
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Cc: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The former is way to generic for a global symbol.
Fixes this build error:
drivers/usb/built-in.o: In function `.handle_event': (.text+0x67dd0): multiple definition of `.handle_event'
drivers/pcmcia/built-in.o:(.text+0xcfcc): first defined here
drivers/usb/built-in.o: In function `handle_event': (.opd+0x5bc8): multiple definition of `handle_event'
drivers/pcmcia/built-in.o:(.opd+0xed0): first defined here
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>
Cc: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Add URB cancellation support to the xHCI host controller driver. This
currently supports cancellation for endpoints that do not have streams
enabled.
An URB is represented by a number of Transaction Request Buffers (TRBs),
that are chained together to make one (or more) Transaction Descriptors
(TDs) on an endpoint ring. The ring is comprised of contiguous segments,
linked together with Link TRBs (which may or may not be chained into a TD).
To cancel an URB, we must stop the endpoint ring, make the hardware skip
over the TDs in the URB (either by turning them into No-op TDs, or by
moving the hardware's ring dequeue pointer past the last TRB in the last
TD), and then restart the ring.
There are times when we must drop the xHCI lock during this process, like
when we need to complete cancelled URBs. We must ensure that additional
URBs can be marked as cancelled, and that new URBs can be enqueued (since
the URB completion handlers can do either). The new endpoint ring
variables cancels_pending and state (which can only be modified while
holding the xHCI lock) ensure that future cancellation and enqueueing do
not interrupt any pending cancellation code.
To facilitate cancellation, we must keep track of the starting ring
segment, first TRB, and last TRB for each URB. We also need to keep track
of the list of TDs that have been marked as cancelled, separate from the
list of TDs that are queued for this endpoint. The new variables and
cancellation list are stored in the xhci_td structure.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Add support for bulk URBs that pass scatter gather lists to xHCI. This allows
xHCI to more efficiently enqueue these transfers, and allows the host
controller to take advantage of USB 3.0 "bursts" for bulk endpoints.
Use requested length to calculate the number of TRBs needed for a scatter gather
list transfer, instead of using the number of sglist entries. The application
can pass down a scatter gather list that is bigger than it needs for the
requested transfer.
Scatter gather entries can cross 64KB boundaries, so be careful to setup TRBs
such that no buffer crosses a 64KB boundary.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Allow device drivers to submit URBs to bulk endpoints on devices under an
xHCI host controller. Share code between the control and bulk enqueueing
functions when it makes sense.
To get the best performance out of bulk transfers, SuperSpeed devices must
have the bMaxBurst size copied from their endpoint companion controller
into the xHCI device context. This allows the host controller to "burst"
up to 16 packets before it has to wait for the device to acknowledge the
first packet.
The buffers in Transfer Request Blocks (TRBs) can cross page boundaries,
but they cannot cross 64KB boundaries. The buffer must be broken into
multiple TRBs if a 64KB boundary is crossed.
The sum of buffer lengths in all the TRBs in a Transfer Descriptor (TD)
cannot exceed 64MB. To work around this, the enqueueing code must enqueue
multiple TDs. The transfer event handler may incorrectly give back the
URB in this case, if it gets a transfer event that points somewhere in the
first TD. FIXME later.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Since the xHCI host controller hardware (xHC) has an internal schedule, it
needs a better representation of what devices are consuming bandwidth on
the bus. Each device is represented by a device context, with data about
the device, endpoints, and pointers to each endpoint ring.
We need to update the endpoint information for a device context before a
new configuration or alternate interface setting is selected. We setup an
input device context with modified endpoint information and newly
allocated endpoint rings, and then submit a Configure Endpoint Command to
the hardware.
The host controller can reject the new configuration if it exceeds the bus
bandwidth, or the host controller doesn't have enough internal resources
for the configuration. If the command fails, we still have the older
device context with the previous configuration. If the command succeeds,
we free the old endpoint rings.
The root hub isn't a real device, so always say yes to any bandwidth
changes for it.
The USB core will enable, disable, and then enable endpoint 0 several
times during the initialization sequence. The device will always have an
endpoint ring for endpoint 0 and bandwidth allocated for that, unless the
device is disconnected or gets a SetAddress 0 request. So we don't pay
attention for when xhci_check_bandwidth() is called for a re-add of
endpoint 0.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Allow device drivers to enqueue URBs to control endpoints on devices under
an xHCI host controller. Each control transfer is represented by a
series of Transfer Descriptors (TDs) written to an endpoint ring. There
is one TD for the Setup phase, (optionally) one TD for the Data phase, and
one TD for the Status phase.
Enqueue these TDs onto the endpoint ring that represents the control
endpoint. The host controller hardware will return an event on the event
ring that points to the (DMA) address of one of the TDs on the endpoint
ring. If the transfer was successful, the transfer event TRB will have a
completion code of success, and it will point to the Status phase TD.
Anything else is considered an error.
This should work for control endpoints besides the default endpoint, but
that hasn't been tested.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
xHCI needs to get a "Slot ID" from the host controller and allocate other
data structures for every USB device. Make usb_alloc_dev() and
usb_release_dev() allocate and free these device structures. After
setting up the xHC device structures, usb_alloc_dev() must wait for the
hardware to respond to an Enable Slot command. usb_alloc_dev() fires off
a Disable Slot command and does not wait for it to complete.
When the USB core wants to choose an address for the device, the xHCI
driver must issue a Set Address command and wait for an event for that
command.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Add functionality for getting port status and hub descriptor for xHCI root
hubs. This is WIP because the USB 3.0 hub descriptor is different from
the USB 2.0 hub descriptor. For now, we lie about the root hub descriptor
because the changes won't effect how the core talks to the root hub.
Later we will need to add the USB 3.0 hub descriptor for real hubs, and
this code might change.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
xHCI host controllers can optionally implement a no-op test. This
simple test ensures the OS has correctly setup all basic data structures
and can correctly respond to interrupts from the host controller
hardware.
There are two rings exercised by the no-op test: the command ring, and
the event ring.
The host controller driver writes a no-op command TRB to the command
ring, and rings the doorbell for the command ring (the first entry in
the doorbell array). The hardware receives this event, places a command
completion event on the event ring, and fires an interrupt.
The host controller driver sees the interrupt, and checks the event ring
for TRBs it can process, and sees the command completion event. (See
the rules in xhci-ring.c for who "owns" a TRB. This is a simplified set
of rules, and may not contain all the details that are in the xHCI 0.95
spec.)
A timer fires every 60 seconds to debug the state of the hardware and
command and event rings. This timer only runs if
CONFIG_USB_XHCI_HCD_DEBUGGING is 'y'.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
