Some implementation of UFS host controller HW might have some non-standard
behaviours (quirks) when compared to behaviour specified by UFSHCI
specification. This patch add support to allow specifying all such quirks
to standard UFS host controller driver so standard driver takes them into
account.
In this change a UFSHCD_QUIRK_DELAY_BEFORE_DME_CMDS is introduced,
where a minimum delay of 1ms is required before DME commands for
stability purposes.
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
Reviewed-by: Gilad Broner <gbroner@codeaurora.org>
Signed-off-by: James Bottomley <JBottomley@Odin.com>
UFS driver adds three well known LUs in the initialization, but those
reference counts are not decremented, so it makes ufshcd module
impossible to unload.
This fixes it by putting scsi_device_put() in the initalization, and in
order to protect concurrent access to hba->sdev_ufs_device (UFS Device
W-LU) from manual delete, increment the reference count while requesting
device power mode setting.
The rest of W-LUs (hba->sdev_boot and hba->sdev_rpmb) are not directly
used from driver, so these references in struct ufs_hba are removed.
Signed-off-by: Akinobu Mita <mita@fixstars.com>
Reviewed-by: Maya Erez <merez@codeaurora.org>
Reviewed-by: Dolev Raviv <draviv@codeaurora.org>
Signed-off-by: Christoph Hellwig <hch@lst.de>
- Adding some of the definitions missing in unipro.h, including power
enumeration.
- Read Modify Write Line helper function
- Indication for the type of suspend
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Add capability to control the auto bkops during suspend.
If host explicitly enables the auto bkops (background operation) on device
then only device would perform the bkops on its own. If auto bkops is not
enabled explicitly and if the device reaches to state where it must do
background operation, device would raise the urgent bkops exception event
to host and then host will enable the auto bkops on device. This patch
adds the option to choose whether auto bkops should be enabled during
runtime suspend or not. Since we don't want to keep the device active to
perform the non critical bkops, host will enable urgent bkops only.
Keep auto-bkops enabled after resume if urgent bkops needed.
If device bkops status shows that its in critical need of executing
background operations, host should allow the device to continue doing
background operations.
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Signed-off-by: Christoph Hellwig <hch@lst.de>
The clocks for UFS device will be managed by generic DVFS (Dynamic
Voltage and Frequency Scaling) framework within kernel. This devfreq
framework works with different governors to scale the clocks. By default,
UFS devices uses simple_ondemand governor which scales the clocks up if
the load is more than upthreshold and scales down if the load is less than
downthreshold.
Signed-off-by: Sahitya Tummala <stummala@codeaurora.org>
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Signed-off-by: Christoph Hellwig <hch@lst.de>
The UFS controller clocks can be gated after certain period of
inactivity, which is typically less than runtime suspend timeout.
In addition to clocks the link will also be put into Hibern8 mode
to save more power.
The clock gating can be turned on by enabling the capability
UFSHCD_CAP_CLK_GATING. To enable entering into Hibern8 mode as part of
clock gating, set the capability UFSHCD_CAP_HIBERN8_WITH_CLK_GATING.
The tracing events for clock gating can be enabled through debugfs as:
echo 1 > /sys/kernel/debug/tracing/events/ufs/ufshcd_clk_gating/enable
cat /sys/kernel/debug/tracing/trace_pipe
Signed-off-by: Sahitya Tummala <stummala@codeaurora.org>
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Sometimes, the device shall report its maximum power and speed
capabilities, but we might not wish to configure it to use those
maximum capabilities.
This change adds support for the vendor specific host driver to
implement power change notify callback.
To enable configuring different power modes (number of lanes,
gear number and fast/slow modes) it is necessary to split the
configuration stage from the stage that reads the device max power mode.
In addition, it is not required to read the configuration more than
once, thus the configuration is stored after reading it once.
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: Christoph Hellwig <hch@lst.de>
This patch adds support for UFS device and UniPro link power management
during runtime/system PM.
Main idea is to define multiple UFS low power levels based on UFS device
and UFS link power states. This would allow any specific platform or pci
driver to choose the best suited low power level during runtime and
system suspend based on their power goals.
bkops handlig:
To put the UFS device in sleep state when bkops is disabled, first query
the bkops status from the device and enable bkops on device only if
device needs time to perform the bkops.
START_STOP handling:
Before sending START_STOP_UNIT to the device well-known logical unit
(w-lun) to make sure that the device w-lun unit attention condition is
cleared.
Write protection:
UFS device specification allows LUs to be write protected, either
permanently or power on write protected. If any LU is power on write
protected and if the card is power cycled (by powering off VCCQ and/or
VCC rails), LU's write protect status would be lost. So this means those
LUs can be written now. To ensures that UFS device is power cycled only
if the power on protect is not set for any of the LUs, check if power on
write protect is set and if device is in sleep/power-off state & link in
inactive state (Hibern8 or OFF state).
If none of the Logical Units on UFS device is power on write protected
then all UFS device power rails (VCC, VCCQ & VCCQ2) can be turned off if
UFS device is in power-off state and UFS link is in OFF state. But current
implementation would disable all device power rails even if UFS link is
not in OFF state.
Low power mode:
If UFS link is in OFF state then UFS host controller can be power collapsed
to avoid leakage current from it. Note that if UFS host controller is power
collapsed, full UFS reinitialization will be required on resume to
re-establish the link between host and device.
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Signed-off-by: Sujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: Christoph Hellwig <hch@lst.de>
UFS device may have standard LUs and LUN id could be from 0x00 to 0x7F.
UFS device specification use "Peripheral Device Addressing Format"
(SCSI SAM-5) for standard LUs.
UFS device may also have the Well Known LUs (also referred as W-LU) which
again could be from 0x00 to 0x7F. For W-LUs, UFS device specification only
allows the "Extended Addressing Format" (SCSI SAM-5) which means the W-LUNs
would start from 0xC100 onwards.
This means max. LUN number reported from UFS device could be 0xC17F hence
this patch advertise the "max_lun" as 0xC17F which will allow SCSI mid
layer to detect the W-LUs as well.
But once the W-LUs are detected, UFSHCD driver may get the commands with
SCSI LUN id upto 0xC17F but UPIU LUN id field is only 8-bit wide so it
requires the mapping of SCSI LUN id to UPIU LUN id. This patch also add
support for this mapping.
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Signed-off-by: Sujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: Christoph Hellwig <hch@lst.de>
UFS device specification requires the UFS devices to support 4 well known
logical units:
"REPORT_LUNS" (address: 01h)
"UFS Device" (address: 50h)
"RPMB" (address: 44h)
"BOOT" (address: 30h)
UFS device's power management needs to be controlled by "POWER CONDITION"
field of SSU (START STOP UNIT) command. But this "power condition" field
will take effect only when its sent to "UFS device" well known logical unit
hence we require the scsi_device instance to represent this logical unit in
order for the UFS host driver to send the SSU command for power management.
We also require the scsi_device instance for "RPMB" (Replay Protected
Memory Block) LU so user space process can control this LU. User space may
also want to have access to BOOT LU.
This patch adds the scsi device instances for each of all well known LUs
(except "REPORT LUNS" LU).
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Signed-off-by: Christoph Hellwig <hch@lst.de>
The maximum power consumption in active is determined by bActiveICCLevel.
The configuration is done by reading max current supported by the
regulators connected to VCC, VCCQ and VCCQ2 rails on the boards, and
reading the current consumption levels from the device for each rails
(vcc/vccq/vccq2) using power descriptor.
We configure the bActiveICCLevel attribute, with the max value that
correspond to the minimum-of(VCC-current-level,VCCQ-current-level,
VCCQ2-current-level).
In order to minimize resume latency, pre-fetch icc levels and reference
clock during initialization and avoid reading them each link startup
during resume.
Signed-off-by: Raviv Shvili <rshvili@codeaurora.org>
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Signed-off-by: Christoph Hellwig <hch@lst.de>
In ->hce_enable_notify() callback the vendor specific initialization
may carry out additional DME configuration using UIC commands and
hence the UIC command completion interrupt enable bit should be set
before the post reset notification.
Add retries if the link-startup fails. This is required since due to
hardware timing issues, the Uni-Pro link-startup might fail. The UFS
HCI recovery procedure contradicts the Uni-Pro sequence. The UFS HCI
specifies to resend DME_LINKSTARTUP command after IS.ULLS (link-lost
interrupt) is received. The Uni-Pro specifies that if link-startup
fails the link is in "down" state. The link-lost is indicated to the
DME user only when the link is up. Hence, the UFS HCI recovery procedure
of waiting for IS.ULLS and retrying link-startup may not work properly.
At the end, if detection fails, power off (disable clocks, regulators,
phy) if the UFS device detection fails. This saves power while UFS device
is not embedded into the system.
Signed-off-by: Sujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Add generic clock initialization support for UFSHCD platform
driver. The clock info is read from device tree using standard
clock bindings. A generic max-clock-frequency-hz property is
defined to save information on maximum operating clock frequency
the h/w supports.
Signed-off-by: Sujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Signed-off-by: Christoph Hellwig <hch@lst.de>
UFS devices are powered by at most three external power supplies -
- VCC - The flash memory core power supply, 2.7V to 3.6V or 1.70V to 1.95V
- VCCQ - The controller and I/O power supply, 1.1V to 1.3V
- VCCQ2 - Secondary controller and/or I/O power supply, 1.65V to 1.95V
For some devices VCCQ or VCCQ2 are optional as they can be
generated using internal LDO inside the UFS device.
Add DT bindings for voltage regulators that can be controlled
from host driver.
Signed-off-by: Sujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Some vendor specific controller versions might need to configure
vendor specific - registers, clocks, voltage regulators etc. to
initialize the host controller UTP layer and Uni-Pro stack.
Provide some common initialization operations that can be used
to configure vendor specifics. The methods can be extended in
future, for example, for power mode transitions.
The operations are vendor/board specific and hence determined with
the help of compatible property in device tree.
Signed-off-by: Sujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Error handling in UFS driver is broken and resets the host controller
for fatal errors without re-initialization. Correct the fatal error
handling sequence according to UFS Host Controller Interface (HCI)
v1.1 specification.
o Processed requests which are completed w/wo error are reported to
SCSI layer and any pending commands that are not started are aborted
in the controller and re-queued into scsi mid-layer queue.
o Upon determining fatal error condition the host controller may hang
forever until a reset is applied. Block SCSI layer for sending new
requests and apply reset in a separate error handling work.
o SCSI is informed about the expected Unit-Attention exception from the
device for the immediate command after a reset so that the SCSI layer
take necessary steps to establish communication with the device.
Signed-off-by: Sujit Reddy Thumma <sthumma@codeaurora.org>
Reviewed-by: Yaniv Gardi <ygardi@codeaurora.org>
Tested-by: Dolev Raviv <draviv@codeaurora.org>
Acked-by: Vinayak Holikatti <vinholikatti@gmail.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
As of now SCSI initiated error handling is broken because,
the reset APIs don't try to bring back the device initialized and
ready for further transfers.
In case of timeouts, the scsi error handler takes care of handling aborts
and resets. Improve the error handling in such scenario by resetting the
device and host and re-initializing them in proper manner.
Signed-off-by: Sujit Reddy Thumma <sthumma@codeaurora.org>
Reviewed-by: Yaniv Gardi <ygardi@codeaurora.org>
Tested-by: Dolev Raviv <draviv@codeaurora.org>
Acked-by: Vinayak Holikatti <vinholikatti@gmail.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Currently, sending Task Management (TM) command to the card might
be broken in some scenarios as listed below:
Problem: If there are more than 8 TM commands the implementation
returns error to the caller.
Fix: Wait for one of the slots to be emptied and send the command.
Problem: Sometimes it is necessary for the caller to know the TM service
response code to determine the task status.
Fix: Propogate the service response to the caller.
Problem: If the TM command times out no proper error recovery is
implemented.
Fix: Clear the command in the controller door-bell register, so that
further commands for the same slot don't fail.
Problem: While preparing the TM command descriptor, the task tag used
should be unique across SCSI/NOP/QUERY/TM commands and not the
task tag of the command which the TM command is trying to manage.
Fix: Use a unique task tag instead of task tag of SCSI command.
Problem: Since the TM command involves H/W communication, abruptly ending
the request on kill interrupt signal might cause h/w malfunction.
Fix: Wait for hardware completion interrupt with TASK_UNINTERRUPTIBLE
set.
Signed-off-by: Sujit Reddy Thumma <sthumma@codeaurora.org>
Reviewed-by: Yaniv Gardi <ygardi@codeaurora.org>
Tested-by: Dolev Raviv <draviv@codeaurora.org>
Acked-by: Vinayak Holikatti <vinholikatti@gmail.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Setting PA_PWRMode using DME_SET triggers the power mode
change. And then the result will be given by the HCS.UPMCRS.
This operation should be done atomically.
Signed-off-by: Seungwon Jeon <tgih.jun@samsung.com>
Reviewed-by: Subhash Jadavani <subhashj@codeaurora.org>
Tested-by: Yaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: Santosh Y <santoshsy@gmail.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
Implements to support GET and SET operations of the DME.
These operations are used to configure the behavior of
the UNIPRO. Along with basic operation, {Peer/AttrSetType}
can be mixed.
Signed-off-by: Seungwon Jeon <tgih.jun@samsung.com>
Reviewed-by: Subhash Jadavani <subhashj@codeaurora.org>
Tested-by: Yaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: Santosh Y <santoshsy@gmail.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
Background operations in the UFS device can be disabled by
the host to reduce the response latency of transfer requests.
Add support for enabling/disabling the background operations
during runtime suspend/resume of the device.
If the device is in critical need of BKOPS it will raise an
URGENT_BKOPS exception which should be handled by the host to
make sure the device performs as expected.
During bootup, the BKOPS is enabled in the device by default.
The disable of BKOPS is supported only when the driver supports
runtime suspend/resume operations as the runtime PM framework
provides a way to determine the device idleness and hence BKOPS
can be managed effectively. During runtime resume the BKOPS is
disabled to reduce latency and during runtime suspend the BKOPS
is enabled to allow device to carry out idle time BKOPS.
In some cases where the BKOPS is disabled during runtime resume
and due to continuous data transfers the runtime suspend is not
triggered, the BKOPS is enabled when the device raises a level-2
exception (outstanding operations - performance impact).
Signed-off-by: Sujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: Santosh Y <santoshsy@gmail.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
Allow UFS device to complete its initialization and accept
SCSI commands by setting fDeviceInit flag. The device may take
time for this operation and hence the host should poll until
fDeviceInit flag is toggled to zero. This step is mandated by
UFS device specification for device initialization completion.
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Signed-off-by: Sujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: Santosh Y <santoshsy@gmail.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
As part of device initialization sequence, sending NOP OUT UPIU and
waiting for NOP IN UPIU response is mandatory. This confirms that the
device UFS Transport (UTP) layer is functional and the host can configure
the device with further commands. Add support for sending NOP OUT UPIU to
check the device connection path and test whether the UTP layer on the
device side is functional during initialization.
A tag is acquired from the SCSI tag map space in order to send the device
management command. When the tag is acquired by internal command the scsi
command is rejected with host busy flag in order to requeue the request.
To avoid frequent collisions between internal commands and scsi commands
the device management command tag is allocated in the opposite direction
w.r.t block layer tag allocation.
Signed-off-by: Sujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Signed-off-by: Santosh Y <santoshsy@gmail.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
Link start-up requires long time with multiphase handshakes
between UFS host and device. This affects driver's probe time.
This patch let link start-up run asynchronously. Link start-up
will be executed at the end of prove separately.
Along with this change, the following is worked.
Defined completion time of uic command to avoid a permanent wait.
Added mutex to guarantee of uic command at a time.
Adapted some sequence of controller initialization after link statup
according to HCI standard.
Signed-off-by: Seungwon Jeon <tgih.jun@samsung.com>
Signed-off-by: Sujit Reddy Thumma <sthumma@codeaurora.org>
Tested-by: Maya Erez <merez@codeaurora.org>
Signed-off-by: Santosh Y <santoshsy@gmail.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
It makes interrupt setting more flexible especially
for disabling. And wrong bit mask is fixed for ver 1.0.
[17:16] is added for mask.
Signed-off-by: Seungwon Jeon <tgih.jun@samsung.com>
Tested-by: Maya Erez <merez@codeaurora.org>
Signed-off-by: Santosh Y <santoshsy@gmail.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
This patch separates PCI code from ufshcd.c and makes it as a
core driver module and adds a new file ufshcd-pci.c as PCI glue
driver.
[jejb: strip __devinit and devexit_p()]
Reviewed-by: Arnd Bergmann <arnd@arndb.de>
Reviewed-by: Namjae Jeon <linkinjeon@gmail.com>
Reviewed-by: Subhash Jadavani <subhashj@codeaurora.org>
Tested-by: Maya Erez <merez@codeaurora.org>
Signed-off-by: Vinayak Holikatti <vinholikatti@gmail.com>
Signed-off-by: Santosh Yaraganavi <santoshsy@gmail.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
