aboutsummaryrefslogtreecommitdiff
path: root/drivers/scsi/ufs/ufshcd.h
AgeCommit message (Collapse)Author
2021-09-21scsi: ufs: core: Revert "scsi: ufs: Synchronize SCSI and UFS error handling"Adrian Hunter
This reverts commit a113eaaf86373362b053279049907ff82b5df6c8. There are a couple of issues with the commit: 1. It causes deadlocks. 2. It causes the shost->eh_cmd_q list of failed requests not to be processed, ever. So revert it. 1. Deadlocks The SCSI error handler runs with requests blocked beginning when scsi_schedule_eh() sets SHOST_RECOVERY state, continuing through scsi_error_handler() callback ->eh_strategy_handler() until scsi_restart_operations() is called. By setting eh_strategy_handler to ufshcd_err_handler, the patch changed the UFS error handler to run with requests blocked, including PM requests, for the entire run of the error handler. That conflicts with UFS error handler existing synchronization with UFS device PM operations. The UFS error handler synchronizes with runtime PM by doing pm_runtime_get_sync() prior to blocking requests itself. It synchronizes with system PM by use of hba->host_sem, again before blocking requests itself. However, if requests are already blocked, then PM operations will block. So: the UFS error handler blocks waiting on PM + PM blocks waiting on SCSI PM requests to process or fail + PM requests are blocked waiting on error handling to finish = deadlock This happens both for runtime PM and system PM. Prior to the patch, these deadlocks could not happen even if SCSI error handling was running, because the presence of requests in shost->eh_cmd_q would mean the queues could not be suspended, which would mean that, should the UFS error handler run at the same time, it would not need to wait for PM or vice versa. Please note these scenarios are not just theoretical, they were found during testing on a Samsung Galaxy Book S. 2. ->eh_strategy_handler() must process shost->eh_cmd_q list of failed requests, as all other eh_strategy_handler's do except UFS error handler. Refer for example: scsi_unjam_host(), ata_scsi_error() and sas_scsi_recover_host(). Link: https://lore.kernel.org/r/20210917144349.14058-1-adrian.hunter@intel.com Fixes: a113eaaf8637 ("scsi: ufs: Synchronize SCSI and UFS error handling") Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-09-13scsi: ufs: ufs-pci: Fix Intel LKF link stabilityAdrian Hunter
Intel LKF can experience link errors. Make fixes to increase link stability, especially when switching to high speed modes. Link: https://lore.kernel.org/r/20210831145317.26306-1-adrian.hunter@intel.com Fixes: b2c57925df1f ("scsi: ufs: ufs-pci: Add support for Intel LKF") Cc: stable@vger.kernel.org Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-02scsi: ufs: Synchronize SCSI and UFS error handlingBart Van Assche
Use the SCSI error handler instead of a custom error handling strategy. This change reduces the number of potential races in the UFS drivers since the UFS error handler and the SCSI error handler no longer run concurrently. Link: https://lore.kernel.org/r/20210722033439.26550-17-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Cc: Avri Altman <avri.altman@wdc.com> Tested-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-02scsi: ufs: Optimize SCSI command processingBart Van Assche
Use a spinlock to protect hba->outstanding_reqs instead of using atomic operations to update this member variable. This patch is a performance improvement because it reduces the number of atomic operations in the hot path (test_and_clear_bit()) and because it reduces the lock contention on the SCSI host lock. On my test setup this patch improves IOPS by about 1%. Link: https://lore.kernel.org/r/20210722033439.26550-14-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Cc: Avri Altman <avri.altman@wdc.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-02scsi: ufs: Optimize serialization of setup_xfer_req() callsBart Van Assche
Reduce the number of times the host lock is taken in the hot path. Additionally, inline ufshcd_vops_setup_xfer_req() because that function is too short to keep it. Link: https://lore.kernel.org/r/20210722033439.26550-13-bvanassche@acm.org Fixes: a45f937110fa ("scsi: ufs: Optimize host lock on transfer requests send/compl paths") Cc: Jaegeuk Kim <jaegeuk@kernel.org> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Bean Huo <beanhuo@micron.com> Cc: Asutosh Das <asutoshd@codeaurora.org> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-02scsi: ufs: Revert "Utilize Transfer Request List Completion Notification ↵Bart Van Assche
Register" Using the UTRLCNR register involves two MMIO accesses in the hot path while using the doorbell register only involves a single MMIO access. Since MMIO accesses take time, do not use the UTRLCNR register. The spinlock contention on the SCSI host lock that is reintroduced by this commit will be addressed later. This reverts commit 6f7151729647e58ac7c522081255fd0c07b38105. Link: https://lore.kernel.org/r/20210722033439.26550-12-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Cc: Avri Altman <avri.altman@wdc.com> Tested-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-02scsi: ufs: Improve static type checking for the host controller stateBart Van Assche
Assign a name to the enumeration type for UFS host controller states and remove the default clause from switch statements on this enumeration type to make the compiler warn about unhandled enumeration labels. Link: https://lore.kernel.org/r/20210722033439.26550-9-bvanassche@acm.org Cc: Can Guo <cang@codeaurora.org> Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Keoseong Park <keosung.park@samsung.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-02scsi: ufs: Verify UIC locking requirements at runtimeBart Van Assche
Instead of documenting the locking requirements of the UIC code as comments, use lockdep_assert_held() such that lockdep verifies the lockdep requirements at runtime if lockdep is enabled. Link: https://lore.kernel.org/r/20210722033439.26550-8-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-02scsi: ufs: Only include power management code if necessaryBart Van Assche
This patch slightly reduces the UFS driver size if built with power management support disabled. Link: https://lore.kernel.org/r/20210722033439.26550-4-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-02scsi: ufs: Reduce power management code duplicationBart Van Assche
Move the dev_get_drvdata() calls into the ufshcd_{system,runtime}_*() functions. Remove ufshcd_runtime_idle() since it is empty. This patch does not change any functionality. Link: https://lore.kernel.org/r/20210722033439.26550-3-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-01scsi: ufs: ufshpb: Cache HPB Control mode on initAvri Altman
We will use control_mode later when we need to differentiate between device and host control modes. Link: https://lore.kernel.org/r/20210712095039.8093-2-avri.altman@wdc.com Reviewed-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Avri Altman <avri.altman@wdc.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-01scsi: ufs: ufshpb: Add HPB 2.0 supportDaejun Park
Version 2.0 of HBP supports reads of varying sizes from 4KB to 1MB. A read operation <= 32KB is supported as single HPB read. A read between 36KB and 1MB is supported by a combination of write buffer command and HPB read command to deliver more PPN. The write buffer commands may not be issued immediately due to busy tags. To use HPB read more aggressively, the driver can requeue the write buffer command. The requeue threshold is implemented as timeout and can be modified with requeue_timeout_ms entry in sysfs. [mkp: REQ_OP_DRV_* and blk_rq_is_passthrough()] Link: https://lore.kernel.org/r/20210712090025epcms2p3b3d94f6f1b2cfa394e3d9ba130ca0fa7@epcms2p3 Tested-by: Can Guo <cang@codeaurora.org> Tested-by: Stanley Chu <stanley.chu@mediatek.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by: Can Guo <cang@codeaurora.org> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-01scsi: ufs: ufshpb: Introduce Host Performance Buffer featureDaejun Park
Implement Host Performance Buffer (HPB) initialization and add function calls to UFS core driver. NAND flash-based storage devices, including UFS, have mechanisms to translate logical addresses of I/O requests to the corresponding physical addresses of the flash storage. In UFS, logical-to-physical-address (L2P) map data, which is required to identify the physical address for the requested I/Os, can only be partially stored in SRAM from NAND flash. Due to this partial loading, accessing the flash address area, where the L2P information for that address is not loaded in the SRAM, can result in serious performance degradation. The basic concept of HPB is to cache L2P mapping entries in host system memory so that both physical block address (PBA) and logical block address (LBA) can be delivered in HPB read command. The HPB read command allows to read data faster than a regular read command in UFS since it provides the physical address (HPB Entry) of the desired logical block in addition to its logical address. The UFS device can access the physical block in NAND directly without searching and uploading L2P mapping table. This improves read performance because the NAND read operation for uploading L2P mapping table is removed. In HPB initialization, the host checks if the UFS device supports HPB feature and retrieves related device capabilities. Then, HPB parameters are configured in the device. Total start-up time of popular applications was measured and the difference observed between HPB being enabled and disabled. Popular applications are 12 game apps and 24 non-game apps. Each test cycle consists of running 36 applications in sequence. We repeated the cycle for observing performance improvement by L2P mapping cache hit in HPB. The following is the test environment: - kernel version: 4.4.0 - RAM: 8GB - UFS 2.1 (64GB) Results: +-------+----------+----------+-------+ | cycle | baseline | with HPB | diff | +-------+----------+----------+-------+ | 1 | 272.4 | 264.9 | -7.5 | | 2 | 250.4 | 248.2 | -2.2 | | 3 | 226.2 | 215.6 | -10.6 | | 4 | 230.6 | 214.8 | -15.8 | | 5 | 232.0 | 218.1 | -13.9 | | 6 | 231.9 | 212.6 | -19.3 | +-------+----------+----------+-------+ We also measured HPB performance using iozone: $ iozone -r 4k -+n -i2 -ecI -t 16 -l 16 -u 16 -s $IO_RANGE/16 -F \ mnt/tmp_1 mnt/tmp_2 mnt/tmp_3 mnt/tmp_4 mnt/tmp_5 mnt/tmp_6 mnt/tmp_7 \ mnt/tmp_8 mnt/tmp_9 mnt/tmp_10 mnt/tmp_11 mnt/tmp_12 mnt/tmp_13 \ mnt/tmp_14 mnt/tmp_15 mnt/tmp_16 Results: +----------+--------+---------+ | IO range | HPB on | HPB off | +----------+--------+---------+ | 1 GB | 294.8 | 300.87 | | 4 GB | 293.51 | 179.35 | | 8 GB | 294.85 | 162.52 | | 16 GB | 293.45 | 156.26 | | 32 GB | 277.4 | 153.25 | +----------+--------+---------+ Link: https://lore.kernel.org/r/20210712085830epcms2p8c1288b7f7a81b044158a18232617b572@epcms2p8 Reported-by: kernel test robot <lkp@intel.com> Tested-by: Bean Huo <beanhuo@micron.com> Tested-by: Can Guo <cang@codeaurora.org> Tested-by: Stanley Chu <stanley.chu@mediatek.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by: Bart Van Assche <bvanassche@acm.org> Reviewed-by: Can Guo <cang@codeaurora.org> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Acked-by: Avri Altman <Avri.Altman@wdc.com> Signed-off-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-07-18scsi: ufs: Refactor ufshcd_is_intr_aggr_allowed()Keoseong Park
Simplify if-else statement to return statement and remove code related to CONFIG_SCSI_UFS_DWC that is not in use. Link: https://lore.kernel.org/r/1891546521.01624860001810.JavaMail.epsvc@epcpadp3 Cc: Joao Pinto <jpinto@synopsys.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Keoseong Park <keosung.park@samsung.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-07-12scsi: ufs: core: Add missing host_lock in ufshcd_vops_setup_xfer_req()Jaegeuk Kim
This patch adds a host_lock which existed before on ufshcd_vops_setup_xfer_req(). Link: https://lore.kernel.org/r/20210701005117.3846179-1-jaegeuk@kernel.org Fixes: a45f937110fa ("scsi: ufs: Optimize host lock on transfer requests send/compl paths") Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Bean Huo <beanhuo@micron.com> Cc: Bart Van Assche <bvanassche@acm.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-06-07scsi: ufs: Utilize Transfer Request List Completion Notification RegisterCan Guo
By reading the UTP Transfer Request List Completion Notification Register, which is added in UFSHCI Ver 3.0, SW can easily get the compeleted transfer requests. Thus, SW can get rid of host lock, which is used to synchronize the tr_doorbell and outstanding_reqs, on transfer requests dispatch and completion paths. This can further benefit random read/write performance. Link: https://lore.kernel.org/r/1621845419-14194-4-git-send-email-cang@codeaurora.org Cc: Stanley Chu <stanley.chu@mediatek.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Co-developed-by: Asutosh Das <asutoshd@codeaurora.org> Signed-off-by: Asutosh Das <asutoshd@codeaurora.org> Signed-off-by: Can Guo <cang@codeaurora.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-06-07scsi: ufs: Optimize host lock on transfer requests send/compl pathsCan Guo
Current UFS IRQ handler is completely wrapped by host lock, and because ufshcd_send_command() is also protected by host lock, when IRQ handler fires, not only the CPU running the IRQ handler cannot send new requests, the rest CPUs can neither. Move the host lock wrapping the IRQ handler into specific branches, i.e., ufshcd_uic_cmd_compl(), ufshcd_check_errors(), ufshcd_tmc_handler() and ufshcd_transfer_req_compl(). Meanwhile, to further reduce occpuation of host lock in ufshcd_transfer_req_compl(), host lock is no longer required to call __ufshcd_transfer_req_compl(). As per test, the optimization can bring considerable gain to random read/write performance. Link: https://lore.kernel.org/r/1621845419-14194-3-git-send-email-cang@codeaurora.org Cc: Stanley Chu <stanley.chu@mediatek.com> Reported-by: kernel test robot <lkp@intel.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Co-developed-by: Asutosh Das <asutoshd@codeaurora.org> Signed-off-by: Asutosh Das <asutoshd@codeaurora.org> Signed-off-by: Can Guo <cang@codeaurora.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-06-02scsi: ufs: core: Export ufshcd_hba_stop()Alice.Chao
Export ufshcd_hba_stop() to allow vendors to disable HCI in variant ops. Link: https://lore.kernel.org/r/20210528033624.12170-2-alice.chao@mediatek.com Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Alice.Chao <alice.chao@mediatek.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-05-21scsi: ufs: core: Clean up whitespaceKeoseong Park
checkpatch reports the following errors: ERROR: space prohibited before that ',' (ctx:WxW) #945: FILE: drivers/scsi/ufs/ufshcd.h:945: +int ufshcd_init(struct ufs_hba * , void __iomem * , unsigned int); ^ ERROR: space prohibited before that ',' (ctx:WxW) #945: FILE: drivers/scsi/ufs/ufshcd.h:945: +int ufshcd_init(struct ufs_hba * , void __iomem * , unsigned int); ^ Remove unnecessary whitespace in ufshcd.h. Link: https://lore.kernel.org/r/2038148563.21621340102306.JavaMail.epsvc@epcpadp3 Signed-off-by: Keoseong Park <keosung.park@samsung.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-05-21scsi: ufs: Use designated initializers in ufs_pm_lvl_states[]Bart Van Assche
The comments in the enum ufs_pm_level definition are redundant. Remove the comments from the ufs_pm_level enum and use designated initializers in the ufs_pm_lvl_states[] definition instead. Link: https://lore.kernel.org/r/20210519202058.12634-3-bvanassche@acm.org Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Bean Huo <beanhuo@micron.com> Cc: Jaegeuk Kim <jaegeuk@kernel.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-05-15scsi: ufs: core: Remove usfhcd_is_*_pm() macrosBart Van Assche
Remove these macros to make the UFS driver source code easier to read. These macros were introduced by commit 57d104c153d3 ("ufs: add UFS power management support"). Link: https://lore.kernel.org/r/20210513171229.7439-1-bvanassche@acm.org Cc: Can Guo <cang@codeaurora.org> Cc: Alim Akhtar <alim.akhtar@samsung.com> Cc: Avri Altman <avri.altman@wdc.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Bean Huo <beanhuo@micron.com> Cc: Adrian Hunter <adrian.hunter@intel.com> Acked-by: Avri Altman <avri.altman@wdc.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-05-14scsi: ufs: core: Introduce HBA performance monitor sysfs nodesCan Guo
Add a new sysfs group which has nodes to monitor data/request transfer performance. This sysfs group has nodes showing total sectors/requests transferred, total busy time spent and max/min/avg/sum latencies. This group can be enhanced later to show more UFS driver layer performance statistics data during runtime. Link: https://lore.kernel.org/r/1619058521-35307-2-git-send-email-cang@codeaurora.org Reviewed-by: Daejun Park <daejun7.park@samsung.com> Acked-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Can Guo <cang@codeaurora.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-05-10scsi: ufs: core: Enable power management for wlunAsutosh Das
During runtime-suspend of ufs host, the SCSI devices are already suspended and so are the queues associated with them. However, the ufs host sends SSU (START_STOP_UNIT) to the wlun during runtime-suspend. During the process blk_queue_enter() checks if the queue is not in suspended state. If so, it waits for the queue to resume, and never comes out of it. Commit 52abca64fd94 ("scsi: block: Do not accept any requests while suspended") adds the check to see if the queue is in suspended state in blk_queue_enter(). Call trace: __switch_to+0x174/0x2c4 __schedule+0x478/0x764 schedule+0x9c/0xe0 blk_queue_enter+0x158/0x228 blk_mq_alloc_request+0x40/0xa4 blk_get_request+0x2c/0x70 __scsi_execute+0x60/0x1c4 ufshcd_set_dev_pwr_mode+0x124/0x1e4 ufshcd_suspend+0x208/0x83c ufshcd_runtime_suspend+0x40/0x154 ufshcd_pltfrm_runtime_suspend+0x14/0x20 pm_generic_runtime_suspend+0x28/0x3c __rpm_callback+0x80/0x2a4 rpm_suspend+0x308/0x614 rpm_idle+0x158/0x228 pm_runtime_work+0x84/0xac process_one_work+0x1f0/0x470 worker_thread+0x26c/0x4c8 kthread+0x13c/0x320 ret_from_fork+0x10/0x18 Fix this by registering ufs device wlun as a SCSI driver and registering it for block runtime-pm. Also make this a supplier for all other LUNs. This way the wlun device suspends after all the consumers and resumes after HBA resumes. This also registers a new SCSI driver for rpmb wlun. This new driver is mostly used to clear rpmb uac. [mkp: resolve merge conflict with 5.13-rc1 and fix doc warning] Fixed smatch warnings: Reported-by: kernel test robot <lkp@intel.com> Reported-by: Dan Carpenter <dan.carpenter@oracle.com> Link: https://lore.kernel.org/r/4662c462e79e3e7f541f54f88f8993f421026d83.1619223249.git.asutoshd@codeaurora.org Reviewed-by: Adrian Hunter <adrian.hunter@intel.com> Co-developed-by: Can Guo <cang@codeaurora.org> Signed-off-by: Can Guo <cang@codeaurora.org> Signed-off-by: Asutosh Das <asutoshd@codeaurora.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-03-24scsi: ufs: core: Correct status type in ufshcd_vops_pwr_change_notify()Yue Hu
The status parameter's type should be enum ufs_notify_change_status. Link: https://lore.kernel.org/r/20210311040210.1315-1-zbestahu@gmail.com Signed-off-by: Yue Hu <huyue2@yulong.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-03-18scsi: ufs: core: Tidy up WB configuration codeYue Hu
There are similar code implementations for WB configuration in ufshcd_wb_{ctrl, toggle_flush_during_h8, toggle_flush}. Extract the common parts to create a new helper with a flag parameter to reduce code duplication. Meanwhile, rename ufshcd_wb_ctrl() to ufshcd_wb_toggle() for better readability. And remove unnecessary log messages from ufshcd_wb_config() since relevant toggle function will emit messages. Also change ufshcd_wb_toggle_flush{__during_h8} to void type accordingly. Link: https://lore.kernel.org/r/20210318095536.2048-1-zbestahu@gmail.com Reviewed-by: Avri Altman <avri.altman@wdc.com> Signed-off-by: Yue Hu <huyue2@yulong.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-03-04scsi: ufs: ufs-debugfs: Add user-defined exception event rate limitingAdrian Hunter
An enabled user-specified exception event that does not clear quickly will repeatedly cause the handler to run. That could unduly disturb the driver behaviour being tested or debugged. To prevent that add debugfs file exception_event_rate_limit_ms. When a exception event happens, it is disabled, and then after a period of time (default 20ms) the exception event is enabled again. Note that if the driver also has that exception event enabled, it will not be disabled. Link: https://lore.kernel.org/r/20210209062437.6954-5-adrian.hunter@intel.com Acked-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-03-04scsi: ufs: ufs-debugfs: Add user-defined exception_event_maskAdrian Hunter
Allow users to enable specific exception events via debugfs. The bits enabled by the driver ee_drv_ctrl are separated from the bits enabled by the user ee_usr_ctrl. The control mask ee_mask_ctrl is the logical-or of those two. A mutex is needed to ensure that the masks match what was written to the device. Link: https://lore.kernel.org/r/20210209062437.6954-4-adrian.hunter@intel.com Acked-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-02-22scsi: ufs: Fix a duplicate dev quirk numberAvri Altman
Fixes: 2b2bfc8aa519 ("scsi: ufs: Introduce a quirk to allow only page-aligned sg entries") Link: https://lore.kernel.org/r/20210211104638.292499-1-avri.altman@wdc.com Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Avri Altman <avri.altman@wdc.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-01-20scsi: ufs: Refactor ufshcd_init/exit_clk_scaling/gating()Can Guo
ufshcd_hba_exit() is always called after ufshcd_exit_clk_scaling() and ufshcd_exit_clk_gating(). Move ufshcd_exit_clk_scaling/gating() to ufshcd_hba_exit(). Meanwhile, add dedicated functions to initialize and remove sysfs nodes of clock scaling/gating to make the code more readable. Overall functionality remains same. Link: https://lore.kernel.org/r/1611137065-14266-3-git-send-email-cang@codeaurora.org Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Can Guo <cang@codeaurora.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-01-20scsi: ufs: Protect some contexts from unexpected clock scalingCan Guo
In contexts like suspend, shutdown, and error handling we need to suspend devfreq to make sure these contexts won't be disturbed by clock scaling. However, suspending devfreq is not enough since users can still trigger a clock scaling by manipulating the devfreq sysfs nodes like min/max_freq and governor even after devfreq is suspended. Moreover, mere suspending devfreq cannot synchroinze a clock scaling which has already been invoked through these sysfs nodes. Add one more flag in struct clk_scaling and wrap the entire func ufshcd_devfreq_scale() with the clk_scaling_lock, so that we can use this flag and clk_scaling_lock to control and synchronize clock scaling invoked through devfreq sysfs nodes. Link: https://lore.kernel.org/r/1611137065-14266-2-git-send-email-cang@codeaurora.org Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Can Guo <cang@codeaurora.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-01-20scsi: ufs: Group UFS WB related flags in struct ufs_dev_infoBean Huo
UFS device-related flags should be grouped in ufs_dev_info. Move wb_enabled and wb_buf_flush_enabled out from struct ufs_hba, group them in struct ufs_dev_info, and align the names of the structure members vertically. Link: https://lore.kernel.org/r/20210119163847.20165-6-huobean@gmail.com Reviewed-by: Can Guo <cang@codeaurora.org> Acked-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-01-20scsi: ufs: Add "wb_on" sysfs node to control WB on/offBean Huo
Currently UFS WriteBooster driver uses clock scaling up/down to set WB on/off. For the platforms which don't support UFSHCD_CAP_CLK_SCALING, WB will be always on. Provide a sysfs attribute to enable/disable WB during runtime. Write 1/0 to "wb_on" sysfs node to enable/disable UFS WB. Link: https://lore.kernel.org/r/20210119163847.20165-2-huobean@gmail.com Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-01-20scsi: ufs: Introduce a quirk to allow only page-aligned sg entriesKiwoong Kim
Some SoCs require a single scatterlist entry for smaller than page size, i.e. 4KB. When dispatching commands with more than one scatterlist entry under 4KB in size the following behavior is observed: A command to read a block range is dispatched with two scatterlist entries that are named AAA and BBB. After dispatching, the host builds two PRDT entries and during transmission, device sends just one DATA IN because device doesn't care about host DMA. The host then transfers the combined amount of data from start address of the area named AAA. As a consequence, the area that follows AAA in memory would be corrupted. |<------------->| +-------+------------ +-------+ + AAA + (corrupted) ... + BBB + +-------+------------ +-------+ To avoid this we need to enforce page size alignment for sg entries. Link: https://lore.kernel.org/r/56dddef94f60bd9466fd77e69f64bbbd657ed2a1.1611026909.git.kwmad.kim@samsung.com Signed-off-by: Kiwoong Kim <kwmad.kim@samsung.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-01-14scsi: ufs: Protect PM ops and err_handler from user access through sysfsCan Guo
User layer may access sysfs nodes when system PM ops or error handling is running. This can cause various problems. Rename eh_sem to host_sem and use it to protect PM ops and error handling from user layer intervention. Link: https://lore.kernel.org/r/1610594010-7254-3-git-send-email-cang@codeaurora.org Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Acked-by: Avri Altman <avri.altman@wdc.com> Signed-off-by: Can Guo <cang@codeaurora.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-01-12scsi: ufs: ufs-debugfs: Add error countersAdrian Hunter
People testing have a need to know how many errors might be occurring over time. Add error counters and expose them via debugfs. A module initcall is used to create a debugfs root directory for ufshcd-related items. In the case that modules are built-in, then initialization is done in link order, so move ufshcd-core to the top of the Makefile. Link: https://lore.kernel.org/r/20210107072538.21782-1-adrian.hunter@intel.com Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Can Guo <cang@codeaurora.org> Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-01-05scsi: ufs: Add a quirk to permit overriding UniPro defaultsKiwoong Kim
The UniPro specification states that attribute IDs of the following parameters are vendor-specific so some SoCs could have no regions at the defined addresses: - DME_LocalFC0ProtectionTimeOutVal - DME_LocalTC0ReplayTimeOutVal - DME_LocalAFC0ReqTimeOutVal In addition, the following parameters should be set considering the compatibility between host and device. - PA_PWRMODEUSERDATA0 - PA_PWRMODEUSERDATA1 - PA_PWRMODEUSERDATA2 - PA_PWRMODEUSERDATA3 - PA_PWRMODEUSERDATA4 - PA_PWRMODEUSERDATA5 Introduce a quirk to allow vendor drivers to override the UniPro defaults. Link: https://lore.kernel.org/r/1fedd3dea0ccc980913a5995a10510d86a5b01b9.1608513782.git.kwmad.kim@samsung.com Acked-by: Avri Altman <Avri.Altman@wdc.com> Signed-off-by: Kiwoong Kim <kwmad.kim@samsung.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-01-01Merge tag 'scsi-fixes' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi Pull SCSI fixes from James Bottomley: "This is a load of driver fixes (12 ufs, 1 mpt3sas, 1 cxgbi). The big core two fixes are for power management ("block: Do not accept any requests while suspended" and "block: Fix a race in the runtime power management code") which finally sorts out the resume problems we've occasionally been having. To make the resume fix, there are seven necessary precursors which effectively renames REQ_PREEMPT to REQ_PM, so every "special" request in block is automatically a power management exempt one. All of the non-PM preempt cases are removed except for the one in the SCSI Parallel Interface (spi) domain validation which is a genuine case where we have to run requests at high priority to validate the bus so this becomes an autopm get/put protected request" * tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (22 commits) scsi: cxgb4i: Fix TLS dependency scsi: ufs: Un-inline ufshcd_vops_device_reset function scsi: ufs: Re-enable WriteBooster after device reset scsi: ufs-mediatek: Use correct path to fix compile error scsi: mpt3sas: Signedness bug in _base_get_diag_triggers() scsi: block: Do not accept any requests while suspended scsi: block: Remove RQF_PREEMPT and BLK_MQ_REQ_PREEMPT scsi: core: Only process PM requests if rpm_status != RPM_ACTIVE scsi: scsi_transport_spi: Set RQF_PM for domain validation commands scsi: ide: Mark power management requests with RQF_PM instead of RQF_PREEMPT scsi: ide: Do not set the RQF_PREEMPT flag for sense requests scsi: block: Introduce BLK_MQ_REQ_PM scsi: block: Fix a race in the runtime power management code scsi: ufs-pci: Enable UFSHCD_CAP_RPM_AUTOSUSPEND for Intel controllers scsi: ufs-pci: Fix recovery from hibernate exit errors for Intel controllers scsi: ufs-pci: Ensure UFS device is in PowerDown mode for suspend-to-disk ->poweroff() scsi: ufs-pci: Fix restore from S4 for Intel controllers scsi: ufs-mediatek: Keep VCC always-on for specific devices scsi: ufs: Allow regulators being always-on scsi: ufs: Clear UAC for RPMB after ufshcd resets ...
2020-12-16Merge tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsiLinus Torvalds
Pull SCSI updates from James Bottomley: "This consists of the usual driver updates (ufs, qla2xxx, smartpqi, target, zfcp, fnic, mpt3sas, ibmvfc) plus a load of cleanups, a major power management rework and a load of assorted minor updates. There are a few core updates (formatting fixes being the big one) but nothing major this cycle" * tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (279 commits) scsi: mpt3sas: Update driver version to 36.100.00.00 scsi: mpt3sas: Handle trigger page after firmware update scsi: mpt3sas: Add persistent MPI trigger page scsi: mpt3sas: Add persistent SCSI sense trigger page scsi: mpt3sas: Add persistent Event trigger page scsi: mpt3sas: Add persistent Master trigger page scsi: mpt3sas: Add persistent trigger pages support scsi: mpt3sas: Sync time periodically between driver and firmware scsi: qla2xxx: Update version to 10.02.00.104-k scsi: qla2xxx: Fix device loss on 4G and older HBAs scsi: qla2xxx: If fcport is undergoing deletion complete I/O with retry scsi: qla2xxx: Fix the call trace for flush workqueue scsi: qla2xxx: Fix flash update in 28XX adapters on big endian machines scsi: qla2xxx: Handle aborts correctly for port undergoing deletion scsi: qla2xxx: Fix N2N and NVMe connect retry failure scsi: qla2xxx: Fix FW initialization error on big endian machines scsi: qla2xxx: Fix crash during driver load on big endian machines scsi: qla2xxx: Fix compilation issue in PPC systems scsi: qla2xxx: Don't check for fw_started while posting NVMe command scsi: qla2xxx: Tear down session if FW say it is down ...
2020-12-09scsi: ufs: Un-inline ufshcd_vops_device_reset functionStanley Chu
More and more statements are being added to ufshcd_vops_device_reset() and this function is being called from multiple locations in the driver. Un-inline the function to allow the compiler to make better decisions. Link: https://lore.kernel.org/r/20201208135635.15326-3-stanley.chu@mediatek.com Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-12-09scsi: ufs: Re-enable WriteBooster after device resetStanley Chu
UFS 3.1 specification mentions that the WriteBooster flags listed below will be set to their default values, i.e. disabled, after power cycle or any type of reset event. Thus we need to reset the flag variables kept in struct hba to align with the device status and ensure that WriteBooster-related functions are configured properly after device reset. Without this fix, WriteBooster will not be enabled successfully after by ufshcd_wb_ctrl() after device reset because hba->wb_enabled remains true. Flags required to be reset to default values: - fWriteBoosterEn: hba->wb_enabled - fWriteBoosterBufferFlushEn: hba->wb_buf_flush_enabled - fWriteBoosterBufferFlushDuringHibernate: No variable mapped Link: https://lore.kernel.org/r/20201208135635.15326-2-stanley.chu@mediatek.com Fixes: 3d17b9b5ab11 ("scsi: ufs: Add write booster feature support") Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-12-07scsi: ufs: Fix a race condition between ufshcd_abort() and eh_work()Can Guo
In current task abort routine, if task abort happens to the device W-LUN, the code directly jumps to ufshcd_eh_host_reset_handler() to perform a full reset and restore then returns FAIL or SUCCESS. Commands sent to the device W-LUN are most likely the SSU cmds sent during UFS PM operations. If such SSU cmd enters task abort routine when ufshcd_eh_host_reset_handler() flushes eh_work, it will get stuck there since err_handler is serialized with PM operations. In order to unblock above call path, we merely clean up the lrb taken by this cmd, queue the eh_work and return SUCCESS. Once the cmd is aborted, the PM operation which sends out the cmd just errors out, then err_handler shall be able to proceed with the full reset and restore. In this scenario, the cmd is aborted even before it is actually cleared by HW, set the lrb->in_use flag to prevent subsequent cmds, including SCSI cmds and dev cmds, from taking the lrb released from abort. The flag shall evetually be cleared in __ufshcd_transfer_req_compl() invoked by the full reset and restore from err_handler. [mkp: conflict with event logging series] Link: https://lore.kernel.org/r/1606910644-21185-3-git-send-email-cang@codeaurora.org Reviewed-by: Asutosh Das <asutoshd@codeaurora.org> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Can Guo <cang@codeaurora.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-12-07scsi: ufs: Serialize eh_work with system PM events and async scanCan Guo
Serialize eh_work with system PM events and async scan to make sure eh_work does not run in parallel with them. Link: https://lore.kernel.org/r/1606910644-21185-2-git-send-email-cang@codeaurora.org Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Reviewed-by: Asutosh Das <asutoshd@codeaurora.org> Reviewed-by: Hongwu Su <hongwus@codeaurora.org> Signed-off-by: Can Guo <cang@codeaurora.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-12-07scsi: ufs: Introduce phy_initialization helperStanley Chu
Introduce phy_initialization helper since this is the only one variant function without helper. Link: https://lore.kernel.org/r/20201205120041.26869-3-stanley.chu@mediatek.com Reviewed-by: Avri Altman <avri.altman@wdc.com> Signed-off-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-12-07scsi: ufs: Remove unused setup_regulators variant functionStanley Chu
Since setup_regulators variant function is not used by any vendors, simply remove it. Link: https://lore.kernel.org/r/20201205120041.26869-2-stanley.chu@mediatek.com Reviewed-by: Avri Altman <avri.altman@wdc.com> Signed-off-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-12-07scsi: ufs: Introduce event_notify variant functionStanley Chu
Introduce event_notify variant function to allow vendor to get notification of important events and connect to any proprietary debugging facilities. Link: https://lore.kernel.org/r/20201205115901.26815-4-stanley.chu@mediatek.com Reviewed-by: Asutosh Das <asutoshd@codeaurora.org> Reviewed-by: Can Guo <cang@codeaurora.org> Signed-off-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-12-07scsi: ufs: Refine error history functionsStanley Chu
The UFS error history does not only have "history of errors" but also a log of some other events which are not defined as errors. This patch fixes the confused naming of related functions and changes the approach for updating and printing history in preparation of next patch. This patch does not change any functionality. Link: https://lore.kernel.org/r/20201205115901.26815-3-stanley.chu@mediatek.com Reviewed-by: Asutosh Das <asutoshd@codeaurora.org> Reviewed-by: Can Guo <cang@codeaurora.org> Signed-off-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-12-01scsi: ufs: Stop hardcoding the scale down gearCan Guo
Instead of hardcoding the scale down gear, make it a member of the ufs_clk_scaling struct. Link: https://lore.kernel.org/r/1606442334-22641-1-git-send-email-cang@codeaurora.org Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Asutosh Das <asutoshd@codeaurora.org> Signed-off-by: Can Guo <cang@codeaurora.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-12-01scsi: ufs: Refactor ufshcd_setup_clocks() to remove skip_ref_clkCan Guo
Remove the param skip_ref_clk from __ufshcd_setup_clocks(), but keep a flag in struct ufs_clk_info to tell whether a clock can be disabled or not while the link is active. Link: https://lore.kernel.org/r/1606356063-38380-2-git-send-email-cang@codeaurora.org Reviewed-by: Hongwu Su <hongwus@codeaurora.org> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Can Guo <cang@codeaurora.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-11-19scsi: ufs: Clear UAC for FFU and RPMB LUNsJaegeuk Kim
In order to conduct FFU or RPMB operations, UFS needs to clear UNIT ATTENTION condition. Clear it explicitly so that we get no failures during initialization. Link: https://lore.kernel.org/r/20201117165839.1643377-4-jaegeuk@kernel.org Signed-off-by: Jaegeuk Kim <jaegeuk@google.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-11-17scsi: ufs: Refactor ADAPT configuration functionStanley Chu
Several vendors are using same code to configure ADAPT settings for HS-G4. Simply refactor it as common function. Link: https://lore.kernel.org/r/20201116065054.7658-8-stanley.chu@mediatek.com Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-12 06:41:14 +0000