summaryrefslogtreecommitdiff
path: root/drivers/scsi/ufs/ufshcd.c
AgeCommit message (Collapse)AuthorFilesLines
2022-04-27scsi: ufs: core: scsi_get_lba() error fixPeter Wang1-2/+3
commit 2bd3b6b75946db2ace06e145d53988e10ed7e99a upstream. When ufs initializes without scmd->device->sector_size set, scsi_get_lba() will get a wrong shift number and trigger an ubsan error. The shift exponent 4294967286 is too large for the 64-bit type 'sector_t' (aka 'unsigned long long'). Call scsi_get_lba() only when opcode is READ_10/WRITE_10/UNMAP. Link: https://lore.kernel.org/r/20220307111752.10465-1-peter.wang@mediatek.com Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Peter Wang <peter.wang@mediatek.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2022-04-08scsi: ufs: Fix runtime PM messages never-ending cycleAdrian Hunter1-2/+19
commit 71bb9ab6e3511b7bb98678a19eb8cf1ccbf3ca2f upstream. Kernel messages produced during runtime PM can cause a never-ending cycle because user space utilities (e.g. journald or rsyslog) write the messages back to storage, causing runtime resume, more messages, and so on. Messages that tell of things that are expected to happen, are arguably unnecessary, so suppress them. UFS driver messages are changes to from dev_err() to dev_dbg() which means they will not display unless activated by dynamic debug of building with -DDEBUG. sdev->silence_suspend is set to skip messages from sd_suspend_common() "Synchronizing SCSI cache", "Stopping disk" and scsi_report_sense() "Power-on or device reset occurred" message (Note, that message appears when the LUN is accessed after runtime PM, not during runtime PM) Example messages from Ubuntu 21.10: $ dmesg | tail [ 1620.380071] ufshcd 0000:00:12.5: ufshcd_print_pwr_info:[RX, TX]: gear=[1, 1], lane[1, 1], pwr[SLOWAUTO_MODE, SLOWAUTO_MODE], rate = 0 [ 1620.408825] ufshcd 0000:00:12.5: ufshcd_print_pwr_info:[RX, TX]: gear=[4, 4], lane[2, 2], pwr[FAST MODE, FAST MODE], rate = 2 [ 1620.409020] ufshcd 0000:00:12.5: ufshcd_find_max_sup_active_icc_level: Regulator capability was not set, actvIccLevel=0 [ 1620.409524] sd 0:0:0:0: Power-on or device reset occurred [ 1622.938794] sd 0:0:0:0: [sda] Synchronizing SCSI cache [ 1622.939184] ufs_device_wlun 0:0:0:49488: Power-on or device reset occurred [ 1625.183175] ufshcd 0000:00:12.5: ufshcd_print_pwr_info:[RX, TX]: gear=[1, 1], lane[1, 1], pwr[SLOWAUTO_MODE, SLOWAUTO_MODE], rate = 0 [ 1625.208041] ufshcd 0000:00:12.5: ufshcd_print_pwr_info:[RX, TX]: gear=[4, 4], lane[2, 2], pwr[FAST MODE, FAST MODE], rate = 2 [ 1625.208311] ufshcd 0000:00:12.5: ufshcd_find_max_sup_active_icc_level: Regulator capability was not set, actvIccLevel=0 [ 1625.209035] sd 0:0:0:0: Power-on or device reset occurred Note for stable: depends on patch "scsi: core: sd: Add silence_suspend flag to suppress some PM messages". Link: https://lore.kernel.org/r/20220228113652.970857-3-adrian.hunter@intel.com Cc: stable@vger.kernel.org Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2022-02-23scsi: ufs: Fix a deadlock in the error handlerBart Van Assche1-39/+14
commit 945c3cca05d78351bba29fa65d93834cb7934c7b upstream. The following deadlock has been observed on a test setup: - All tags allocated - The SCSI error handler calls ufshcd_eh_host_reset_handler() - ufshcd_eh_host_reset_handler() queues work that calls ufshcd_err_handler() - ufshcd_err_handler() locks up as follows: Workqueue: ufs_eh_wq_0 ufshcd_err_handler.cfi_jt Call trace: __switch_to+0x298/0x5d8 __schedule+0x6cc/0xa94 schedule+0x12c/0x298 blk_mq_get_tag+0x210/0x480 __blk_mq_alloc_request+0x1c8/0x284 blk_get_request+0x74/0x134 ufshcd_exec_dev_cmd+0x68/0x640 ufshcd_verify_dev_init+0x68/0x35c ufshcd_probe_hba+0x12c/0x1cb8 ufshcd_host_reset_and_restore+0x88/0x254 ufshcd_reset_and_restore+0xd0/0x354 ufshcd_err_handler+0x408/0xc58 process_one_work+0x24c/0x66c worker_thread+0x3e8/0xa4c kthread+0x150/0x1b4 ret_from_fork+0x10/0x30 Fix this lockup by making ufshcd_exec_dev_cmd() allocate a reserved request. Link: https://lore.kernel.org/r/20211203231950.193369-10-bvanassche@acm.org Tested-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Adrian Hunter <adrian.hunter@intel.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2022-02-23scsi: ufs: Remove dead codeBart Van Assche1-6/+1
commit d77ea8226b3be23b0b45aa42851243b62a27bda1 upstream. Commit 7252a3603015 ("scsi: ufs: Avoid busy-waiting by eliminating tag conflicts") guarantees that 'tag' is not in use by any SCSI command. Remove the check that returns early if a conflict occurs. Link: https://lore.kernel.org/r/20211203231950.193369-6-bvanassche@acm.org Tested-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Acked-by: Avri Altman <avri.altman@wdc.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2022-02-16scsi: ufs: Use generic error code in ufshcd_set_dev_pwr_mode()Kiwoong Kim1-3/+6
[ Upstream commit ad6c8a426446873febc98140d81d5353f8c0825b ] The return value of ufshcd_set_dev_pwr_mode() is passed to device PM core. However, the function currently returns a SCSI result which the PM core doesn't understand. This might lead to unexpected behaviors in userland; a platform reset was observed in Android. Use a generic error code for SSU failures. Link: https://lore.kernel.org/r/1642743182-54098-1-git-send-email-kwmad.kim@samsung.com Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Kiwoong Kim <kwmad.kim@samsung.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
2022-01-27scsi: ufs: Fix a kernel crash during shutdownBart Van Assche1-5/+10
[ Upstream commit 3489c34bd02b73a72646037d673a122a53cee174 ] Fix the following kernel crash: Unable to handle kernel paging request at virtual address ffffffc91e735000 Call trace: __queue_work+0x26c/0x624 queue_work_on+0x6c/0xf0 ufshcd_hold+0x12c/0x210 __ufshcd_wl_suspend+0xc0/0x400 ufshcd_wl_shutdown+0xb8/0xcc device_shutdown+0x184/0x224 kernel_restart+0x4c/0x124 __arm64_sys_reboot+0x194/0x264 el0_svc_common+0xc8/0x1d4 do_el0_svc+0x30/0x8c el0_svc+0x20/0x30 el0_sync_handler+0x84/0xe4 el0_sync+0x1bc/0x1c0 Fix this crash by ungating the clock before destroying the work queue on which clock gating work is queued. Link: https://lore.kernel.org/r/20211203231950.193369-15-bvanassche@acm.org Tested-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
2022-01-27scsi: ufs: Fix race conditions related to driver dataBart Van Assche1-0/+7
[ Upstream commit 21ad0e49085deb22c094f91f9da57319a97188e4 ] The driver data pointer must be set before any callbacks are registered that use that pointer. Hence move the initialization of that pointer from after the ufshcd_init() call to inside ufshcd_init(). Link: https://lore.kernel.org/r/20211203231950.193369-7-bvanassche@acm.org Fixes: 3b1d05807a9a ("[SCSI] ufs: Segregate PCI Specific Code") Reported-by: Alexey Dobriyan <adobriyan@gmail.com> Tested-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
2021-12-22scsi: ufs: core: Retry START_STOP on UNIT_ATTENTIONJaegeuk Kim1-3/+9
commit af21c3fd5b3ec540a97b367a70b26616ff7e0c55 upstream. Commit 57d104c153d3 ("ufs: add UFS power management support") made the UFS driver submit a REQUEST SENSE command before submitting a power management command to a WLUN to clear the POWER ON unit attention. Instead of submitting a REQUEST SENSE command before submitting a power management command, retry the power management command until it succeeds. This is the preparation to get rid of all UNIT ATTENTION code which should be handled by users. Link: https://lore.kernel.org/r/20211001182015.1347587-2-jaegeuk@kernel.org Cc: Adrian Hunter <adrian.hunter@intel.com> Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com> Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-11-25scsi: ufs: core: Fix another task management completion raceAdrian Hunter1-2/+1
[ Upstream commit 5cb37a26355d79ab290220677b1b57d28e99a895 ] hba->outstanding_tasks, which is read under host_lock spinlock, tells the interrupt handler what task management tags are in use by the driver. The doorbell register bits indicate which tags are in use by the hardware. A doorbell bit that is 0 is because the bit has yet to be set by the driver, or because the task is complete. It is only possible to disambiguate the 2 cases, if reading/writing the doorbell register is synchronized with reading/writing hba->outstanding_tasks. For that reason, reading REG_UTP_TASK_REQ_DOOR_BELL must be done under spinlock. Link: https://lore.kernel.org/r/20211108064815.569494-3-adrian.hunter@intel.com Fixes: f5ef336fd2e4 ("scsi: ufs: core: Fix task management completion") Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
2021-11-25scsi: ufs: core: Fix task management completion timeout raceAdrian Hunter1-5/+0
[ Upstream commit 886fe2915cce6658b0fc19e64b82879325de61ea ] __ufshcd_issue_tm_cmd() clears req->end_io_data after timing out, which races with the completion function ufshcd_tmc_handler() which expects req->end_io_data to have a value. Note __ufshcd_issue_tm_cmd() and ufshcd_tmc_handler() are already synchronized using hba->tmf_rqs and hba->outstanding_tasks under the host_lock spinlock. It is also not necessary (nor typical) to clear req->end_io_data because the block layer does it before allocating out requests e.g. via blk_get_request(). So fix by not clearing it. Link: https://lore.kernel.org/r/20211108064815.569494-2-adrian.hunter@intel.com Fixes: f5ef336fd2e4 ("scsi: ufs: core: Fix task management completion") Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
2021-11-25scsi: ufs: core: Improve SCSI abort handlingBart Van Assche1-0/+1
[ Upstream commit 3ff1f6b6ba6f97f50862aa50e79959cc8ddc2566 ] The following has been observed on a test setup: WARNING: CPU: 4 PID: 250 at drivers/scsi/ufs/ufshcd.c:2737 ufshcd_queuecommand+0x468/0x65c Call trace: ufshcd_queuecommand+0x468/0x65c scsi_send_eh_cmnd+0x224/0x6a0 scsi_eh_test_devices+0x248/0x418 scsi_eh_ready_devs+0xc34/0xe58 scsi_error_handler+0x204/0x80c kthread+0x150/0x1b4 ret_from_fork+0x10/0x30 That warning is triggered by the following statement: WARN_ON(lrbp->cmd); Fix this warning by clearing lrbp->cmd from the abort handler. Link: https://lore.kernel.org/r/20211104181059.4129537-1-bvanassche@acm.org Fixes: 7a3e97b0dc4b ("[SCSI] ufshcd: UFS Host controller driver") Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
2021-11-18scsi: ufs: core: Stop clearing UNIT ATTENTIONSBart Van Assche1-183/+1
[ Upstream commit edc0596cc04bf0ac3a69c66e994d3ff8b650ff71 ] Commit aa53f580e67b ("scsi: ufs: Minor adjustments to error handling") introduced a ufshcd_clear_ua_wluns() call in ufshcd_err_handling_unprepare(). As explained in detail by Adrian Hunter, this can trigger a deadlock. Avoid that deadlock by removing the code that clears the unit attention. This is safe because the only software that relies on clearing unit attentions is the Android Trusty software and because support for handling unit attentions has been added in the Trusty software. See also https://lore.kernel.org/linux-scsi/20210930124224.114031-2-adrian.hunter@intel.com/ Note that "scsi: ufs: Retry START_STOP on UNIT_ATTENTION" is a prerequisite for this commit. Link: https://lore.kernel.org/r/20211001182015.1347587-3-jaegeuk@kernel.org Fixes: aa53f580e67b ("scsi: ufs: Minor adjustments to error handling") Cc: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Bart Van Assche <bvanassche@google.com> Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
2021-11-18scsi: ufs: core: Fix ufshcd_probe_hba() prototype to match the definitionBean Huo1-1/+1
[ Upstream commit 68444d73d6a5864ede12df6366bd6602e022ae5b ] Since commit 568dd9959611 ("scsi: ufs: Rename the second ufshcd_probe_hba() argument"), the second ufshcd_probe_hba() argument has been changed to init_dev_params. Link: https://lore.kernel.org/r/20210929200640.828611-3-huobean@gmail.com Fixes: 568dd9959611 ("scsi: ufs: Rename the second ufshcd_probe_hba() argument") Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
2021-10-30scsi: ufs: ufshpb: Remove HPB2.0 flowsAvri Altman1-6/+1
The Host Performance Buffer feature allows UFS read commands to carry the physical media addresses along with the LBAs, thus allowing less internal L2P-table switches in the device. HPB1.0 allowed a single LBA, while HPB2.0 increases this capacity up to 255 blocks. Carrying more than a single record, the read operation is no longer purely of type "read" but a "hybrid" command: Writing the physical address to the device in one operation and reading back the required payload in another. The JEDEC HPB spec defines two commands for this operation: HPB-WRITE-BUFFER (0x2) to write the physical addresses to device, and HPB-READ to read the payload. With the current HPB design the UFS driver has no alternative but to divide the READ request into 2 separate commands: HPB-WRITE-BUFFER and HPB-READ. This causes a great deal of aggravation to the block layer guys who demanded that we completely revert the entire HPB driver regardless of the huge amount of corporate effort already invested in it. As a compromise, remove only the pieces that implement the 2.0 specification. This is done as a matter of urgency for the final 5.15 release. Link: https://lore.kernel.org/r/20211030062301.248-1-avri.altman@wdc.com Tested-by: Avri Altman <avri.altman@wdc.com> Tested-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Bart Van Assche <bvanassche@acm.org> Reviewed-by: Bean Huo <beanhuo@micron.com> Co-developed-by: James Bottomley <James.Bottomley@HansenPartnership.com> Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com> Signed-off-by: Avri Altman <avri.altman@wdc.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-10-05scsi: ufs: core: Fix task management completionAdrian Hunter1-30/+22
The UFS driver uses blk_mq_tagset_busy_iter() when identifying task management requests to complete, however blk_mq_tagset_busy_iter() doesn't work. blk_mq_tagset_busy_iter() only iterates requests dispatched by the block layer. That appears as if it might have started since commit 37f4a24c2469 ("blk-mq: centralise related handling into blk_mq_get_driver_tag") which removed 'data->hctx->tags->rqs[rq->tag] = rq' from blk_mq_rq_ctx_init() which gets called: blk_get_request blk_mq_alloc_request __blk_mq_alloc_request blk_mq_rq_ctx_init Since UFS task management requests are not dispatched by the block layer, hctx->tags->rqs[rq->tag] remains NULL, and since blk_mq_tagset_busy_iter() relies on finding requests using hctx->tags->rqs[rq->tag], UFS task management requests are never found by blk_mq_tagset_busy_iter(). By using blk_mq_tagset_busy_iter(), the UFS driver was relying on internal details of the block layer, which was fragile and subsequently got broken. Fix by removing the use of blk_mq_tagset_busy_iter() and having the driver keep track of task management requests. Link: https://lore.kernel.org/r/20210922091059.4040-1-adrian.hunter@intel.com Fixes: 1235fc569e0b ("scsi: ufs: core: Fix task management request completion timeout") Fixes: 69a6c269c097 ("scsi: ufs: Use blk_{get,put}_request() to allocate and free TMFs") Cc: stable@vger.kernel.org Tested-by: Bart Van Assche <bvanassche@acm.org> Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-09-29scsi: ufs: Fix illegal offset in UPIU event traceJonathan Hsu1-2/+1
Fix incorrect index for UTMRD reference in ufshcd_add_tm_upiu_trace(). Link: https://lore.kernel.org/r/20210924085848.25500-1-jonathan.hsu@mediatek.com Fixes: 4b42d557a8ad ("scsi: ufs: core: Fix wrong Task Tag used in task management request UPIUs") Cc: stable@vger.kernel.org Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Jonathan Hsu <jonathan.hsu@mediatek.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-09-22scsi: ufs: core: Unbreak the reset handlerBart Van Assche1-1/+1
A command tag is passed as the second argument of the __ufshcd_transfer_req_compl() call in ufshcd_eh_device_reset_handler() instead of a bitmask. Fix this by passing a bitmask as argument instead of a command tag. Link: https://lore.kernel.org/r/20210916175408.2260084-1-bvanassche@acm.org Fixes: a45f937110fa ("scsi: ufs: Optimize host lock on transfer requests send/compl paths") Cc: Can Guo <cang@codeaurora.org> Reviewed-by: Avri Altman <avri.altman@wdc.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-09-22scsi: ufs: core: Revert "scsi: ufs: Synchronize SCSI and UFS error handling"Adrian Hunter1-57/+54
This reverts commit a113eaaf86373362b053279049907ff82b5df6c8. There are a couple of issues with the commit: 1. It causes deadlocks. 2. It causes the shost->eh_cmd_q list of failed requests not to be processed, ever. So revert it. 1. Deadlocks The SCSI error handler runs with requests blocked beginning when scsi_schedule_eh() sets SHOST_RECOVERY state, continuing through scsi_error_handler() callback ->eh_strategy_handler() until scsi_restart_operations() is called. By setting eh_strategy_handler to ufshcd_err_handler, the patch changed the UFS error handler to run with requests blocked, including PM requests, for the entire run of the error handler. That conflicts with UFS error handler existing synchronization with UFS device PM operations. The UFS error handler synchronizes with runtime PM by doing pm_runtime_get_sync() prior to blocking requests itself. It synchronizes with system PM by use of hba->host_sem, again before blocking requests itself. However, if requests are already blocked, then PM operations will block. So: the UFS error handler blocks waiting on PM + PM blocks waiting on SCSI PM requests to process or fail + PM requests are blocked waiting on error handling to finish = deadlock This happens both for runtime PM and system PM. Prior to the patch, these deadlocks could not happen even if SCSI error handling was running, because the presence of requests in shost->eh_cmd_q would mean the queues could not be suspended, which would mean that, should the UFS error handler run at the same time, it would not need to wait for PM or vice versa. Please note these scenarios are not just theoretical, they were found during testing on a Samsung Galaxy Book S. 2. ->eh_strategy_handler() must process shost->eh_cmd_q list of failed requests, as all other eh_strategy_handler's do except UFS error handler. Refer for example: scsi_unjam_host(), ata_scsi_error() and sas_scsi_recover_host(). Link: https://lore.kernel.org/r/20210917144349.14058-1-adrian.hunter@intel.com Fixes: a113eaaf8637 ("scsi: ufs: Synchronize SCSI and UFS error handling") Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-09-14scsi: ufs: ufs-pci: Fix Intel LKF link stabilityAdrian Hunter1-1/+2
Intel LKF can experience link errors. Make fixes to increase link stability, especially when switching to high speed modes. Link: https://lore.kernel.org/r/20210831145317.26306-1-adrian.hunter@intel.com Fixes: b2c57925df1f ("scsi: ufs: ufs-pci: Add support for Intel LKF") Cc: stable@vger.kernel.org Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-25scsi: ufs: Fix ufshcd_request_sense_async() for Samsung KLUFG8RHDA-B2D1Adrian Hunter1-7/+29
Samsung KLUFG8RHDA-B2D1 does not clear the unit attention condition if the length is zero. So go back to requesting all the sense data, as it was before patch "scsi: ufs: Request sense data asynchronously". That is simpler than creating and maintaining a quirk for affected devices. Link: https://lore.kernel.org/r/20210824114150.2105-1-adrian.hunter@intel.com Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-12scsi: ufs: Use scsi_cmd_to_rq() instead of scsi_cmnd.requestBart Van Assche1-5/+6
Prepare for removal of the request pointer by using scsi_cmd_to_rq() instead. This patch does not change any functionality. Link: https://lore.kernel.org/r/20210809230355.8186-48-bvanassche@acm.org Reviewed-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-10scsi: ufs: ufshpb: Remove redundant initialization of variable 'lba'Colin Ian King1-1/+1
The variable 'lba' is being initialized with a value that is never read, it is being updated later on. The assignment is redundant and can be removed. Link: https://lore.kernel.org/r/20210804133241.113509-1-colin.king@canonical.com Signed-off-by: Colin Ian King <colin.king@canonical.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com> Addresses-Coverity: ("Unused value")
2021-08-10scsi: ufshcd: Fix device links when BOOT WLUN fails to probeAdrian Hunter1-2/+21
Managed device links are deleted by device_del(). However it is possible to add a device link to a consumer before device_add(), and then discovering an error prevents the device from being used. In that case normally references to the device would be dropped and the device would be deleted. However the device link holds a reference to the device, so the device link and device remain indefinitely (unless the supplier is deleted). For UFSHCD, if a LUN fails to probe (e.g. absent BOOT WLUN), the device will not have been registered but can still have a device link holding a reference to the device. The unwanted device link will prevent runtime suspend indefinitely. Amend device link removal to accept removal of a link with an unregistered consumer device (suggested by Rafael), and fix UFSHCD by explicitly deleting the device link when SCSI destroys the SCSI device. Link: https://lore.kernel.org/r/a1c9bac8-b560-b662-f0aa-58c7e000cbbd@intel.com Fixes: b294ff3e3449 ("scsi: ufs: core: Enable power management for wlun") Reviewed-by: Rafael J. Wysocki <rafael@kernel.org> Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-10scsi: ufs: Fix unsigned int compared with less than zeroColin Ian King1-1/+1
Variable 'tag' is currently an unsigned int and is being compared to less than zero, this check is always false. Fix this by making 'tag' an int. Link: https://lore.kernel.org/r/20210806144301.19864-1-colin.king@canonical.com Fixes: 4728ab4a8e64 ("scsi: ufs: Remove ufshcd_valid_tag()") Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Colin Ian King <colin.king@canonical.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com> Addresses-Coverity: ("Macro compares unsigned to 0")
2021-08-06scsi: ufs: core: Add L2P entry swap quirk for Micron UFSBean Huo1-1/+2
For Micron UFS devices the L2P entry need to be byteswapped before sending an HPB READ command to the UFS device. Add the quirk UFS_DEVICE_QUIRK_SWAP_L2P_ENTRY_FOR_HPB_READ to address this. Link: https://lore.kernel.org/r/20210804182128.458356-2-huobean@gmail.com Reviewed-by: Avri Altman <avri.altman@wdc.com> Signed-off-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-06scsi: ufs: core: Remove redundant call in ufshcd_add_command_trace()Bean Huo1-6/+3
ufshcd_add_cmd_upiu_trace() will be called later anyway. Simplify code by moving if-statement. Link: https://lore.kernel.org/r/20210802180803.100033-1-huobean@gmail.com Reviewed-by: Avri Altman <avri.altman@wdc.com> Signed-off-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-03scsi: ufs: Add fault injection supportBart Van Assche1-0/+8
Make it easier to test the UFS error handler and abort handler. Link: https://lore.kernel.org/r/20210722033439.26550-19-bvanassche@acm.org Acked-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-03scsi: ufs: Retry aborted SCSI commands instead of completing these successfullyBart Van Assche1-12/+23
Neither SAM nor the UFS standard require that the UFS controller fills in the completion status of commands that have been aborted (LUN RESET aborts pending commands). Hence do not rely on the completion status provided by the UFS controller for aborted commands but instead ask the SCSI core to retry SCSI commands that have been aborted. Link: https://lore.kernel.org/r/20210722033439.26550-18-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Cc: Avri Altman <avri.altman@wdc.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-03scsi: ufs: Synchronize SCSI and UFS error handlingBart Van Assche1-47/+55
Use the SCSI error handler instead of a custom error handling strategy. This change reduces the number of potential races in the UFS drivers since the UFS error handler and the SCSI error handler no longer run concurrently. Link: https://lore.kernel.org/r/20210722033439.26550-17-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Cc: Avri Altman <avri.altman@wdc.com> Tested-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-03scsi: ufs: Request sense data asynchronouslyBart Van Assche1-31/+33
Clearing a unit attention synchronously from inside the UFS error handler may trigger the following deadlock: - ufshcd_err_handler() calls ufshcd_err_handling_unprepare() and the latter function calls ufshcd_clear_ua_wluns(). - ufshcd_clear_ua_wluns() submits a REQUEST SENSE command and that command activates the SCSI error handler. - The SCSI error handler calls ufshcd_host_reset_and_restore(). - ufshcd_host_reset_and_restore() executes the following code: ufshcd_schedule_eh_work(hba); flush_work(&hba->eh_work); This sequence results in a deadlock (circular wait). Fix this by requesting sense data asynchronously. Link: https://lore.kernel.org/r/20210722033439.26550-16-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Cc: Avri Altman <avri.altman@wdc.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-03scsi: ufs: Fix the SCSI abort handlerBart Van Assche1-35/+19
Make the following changes in ufshcd_abort(): - Return FAILED instead of SUCCESS if the abort handler notices that a SCSI command has already been completed. Returning SUCCESS in this case triggers a use-after-free and may trigger a kernel crash. - Fix the code for aborting SCSI commands submitted to a WLUN. The current approach for aborting SCSI commands that have been submitted to a WLUN and that timed out is as follows: - Report to the SCSI core that the command has completed successfully. Let the block layer free any data buffers associated with the command. - Mark the command as outstanding in 'outstanding_reqs'. - If the block layer tries to reuse the tag associated with the aborted command, busy-wait until the tag is freed. This approach can result in: - Memory corruption if the controller accesses the data buffer after the block layer has freed the associated data buffers. - A race condition if ufshcd_queuecommand() or ufshcd_exec_dev_cmd() checks the bit that corresponds to an aborted command in 'outstanding_reqs' after it has been cleared and before it is reset. - High energy consumption if ufshcd_queuecommand() repeatedly returns SCSI_MLQUEUE_HOST_BUSY. Fix this by reporting to the SCSI error handler that aborting a SCSI command failed if the SCSI command was submitted to a WLUN. Link: https://lore.kernel.org/r/20210722033439.26550-15-bvanassche@acm.org Fixes: 7a7e66c65d41 ("scsi: ufs: Fix a race condition between ufshcd_abort() and eh_work()") Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Cc: Avri Altman <avri.altman@wdc.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-03scsi: ufs: Optimize SCSI command processingBart Van Assche1-11/+18
Use a spinlock to protect hba->outstanding_reqs instead of using atomic operations to update this member variable. This patch is a performance improvement because it reduces the number of atomic operations in the hot path (test_and_clear_bit()) and because it reduces the lock contention on the SCSI host lock. On my test setup this patch improves IOPS by about 1%. Link: https://lore.kernel.org/r/20210722033439.26550-14-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Cc: Avri Altman <avri.altman@wdc.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-03scsi: ufs: Optimize serialization of setup_xfer_req() callsBart Van Assche1-1/+2
Reduce the number of times the host lock is taken in the hot path. Additionally, inline ufshcd_vops_setup_xfer_req() because that function is too short to keep it. Link: https://lore.kernel.org/r/20210722033439.26550-13-bvanassche@acm.org Fixes: a45f937110fa ("scsi: ufs: Optimize host lock on transfer requests send/compl paths") Cc: Jaegeuk Kim <jaegeuk@kernel.org> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Bean Huo <beanhuo@micron.com> Cc: Asutosh Das <asutoshd@codeaurora.org> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-03scsi: ufs: Revert "Utilize Transfer Request List Completion Notification ↵Bart Van Assche1-37/+15
Register" Using the UTRLCNR register involves two MMIO accesses in the hot path while using the doorbell register only involves a single MMIO access. Since MMIO accesses take time, do not use the UTRLCNR register. The spinlock contention on the SCSI host lock that is reintroduced by this commit will be addressed later. This reverts commit 6f7151729647e58ac7c522081255fd0c07b38105. Link: https://lore.kernel.org/r/20210722033439.26550-12-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Cc: Avri Altman <avri.altman@wdc.com> Tested-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-03scsi: ufs: Inline ufshcd_outstanding_req_clear()Bart Van Assche1-11/+1
Inline ufshcd_outstanding_req_clear() since it only has one caller and since its body is only one line long. Link: https://lore.kernel.org/r/20210722033439.26550-11-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-03scsi: ufs: Remove several wmb() callsBart Van Assche1-11/+0
From arch/arm/include/asm/io.h #define __iowmb() wmb() [ ... ] #define writel(v,c) ({ __iowmb(); writel_relaxed(v,c); }) From Documentation/memory-barriers.txt: "Note that, when using writel(), a prior wmb() is not needed to guarantee that the cache coherent memory writes have completed before writing to the MMIO region." In other words, calling wmb() before writel() is not necessary. Hence remove the wmb() calls that precede a writel() call. Remove the wmb() calls that precede a ufshcd_send_command() call since the latter function uses writel(). Remove the wmb() call from ufshcd_wait_for_dev_cmd() since the following chain of events guarantees that the CPU will see up-to-date LRB values: - UFS controller writes to host memory. - UFS controller posts completion interrupt after the memory writes from the previous step are visible to the CPU. - complete(hba->dev_cmd.complete) is called from the UFS interrupt handler. - The wait_for_completion(hba->dev_cmd.complete) call in ufshcd_wait_for_dev_cmd() returns. Link: https://lore.kernel.org/r/20210722033439.26550-10-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Cc: Avri Altman <avri.altman@wdc.com> Tested-by: Avri altman <avri.altman@wdc.com> Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-03scsi: ufs: Improve static type checking for the host controller stateBart Van Assche1-15/+0
Assign a name to the enumeration type for UFS host controller states and remove the default clause from switch statements on this enumeration type to make the compiler warn about unhandled enumeration labels. Link: https://lore.kernel.org/r/20210722033439.26550-9-bvanassche@acm.org Cc: Can Guo <cang@codeaurora.org> Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Keoseong Park <keosung.park@samsung.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-03scsi: ufs: Verify UIC locking requirements at runtimeBart Van Assche1-7/+9
Instead of documenting the locking requirements of the UIC code as comments, use lockdep_assert_held() such that lockdep verifies the lockdep requirements at runtime if lockdep is enabled. Link: https://lore.kernel.org/r/20210722033439.26550-8-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-03scsi: ufs: Remove ufshcd_valid_tag()Bart Van Assche1-32/+10
scsi_add_host() allocates shost->can_queue tags. ufshcd_init() sets shost->can_queue to hba->nutrs. In other words, we know that tag values will less than hba->nutrs. Hence remove the checks that verify that blk_get_request() returns a tag less than hba->nutrs. This check was introduced by commit 14497328b6a6 ("scsi: ufs: verify command tag validity"). Keep the tag >= 0 check because it helps to detect use-after-free issues. Link: https://lore.kernel.org/r/20210722033439.26550-7-bvanassche@acm.org CC: Avri Altman <avri.altman@wdc.com> Cc: Alim Akhtar <alim.akhtar@samsung.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-03scsi: ufs: Use DECLARE_COMPLETION_ONSTACK() where appropriateBart Van Assche1-6/+3
From Documentation/scheduler/completion.rst: "When a completion is declared as a local variable within a function, then the initialization should always use DECLARE_COMPLETION_ONSTACK() explicitly, not just to make lockdep happy, but also to make it clear that limited scope had been considered and is intentional." Link: https://lore.kernel.org/r/20210722033439.26550-6-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Cc: Avri Altman <avri.altman@wdc.com> Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-03scsi: ufs: Rename the second ufshcd_probe_hba() argumentBart Van Assche1-4/+4
Rename the second argument of ufshcd_probe_hba() such that the name of that argument reflects its purpose instead of how the function is called. See also commit 1b9e21412f72 ("scsi: ufs: Split ufshcd_probe_hba() based on its called flow"). Link: https://lore.kernel.org/r/20210722033439.26550-5-bvanassche@acm.org Cc: Asutosh Das <asutoshd@codeaurora.org> Cc: Can Guo <cang@codeaurora.org> Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-03scsi: ufs: Only include power management code if necessaryBart Van Assche1-0/+8
This patch slightly reduces the UFS driver size if built with power management support disabled. Link: https://lore.kernel.org/r/20210722033439.26550-4-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-03scsi: ufs: Reduce power management code duplicationBart Van Assche1-20/+21
Move the dev_get_drvdata() calls into the ufshcd_{system,runtime}_*() functions. Remove ufshcd_runtime_idle() since it is empty. This patch does not change any functionality. Link: https://lore.kernel.org/r/20210722033439.26550-3-bvanassche@acm.org Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-03scsi: ufs: Fix memory corruption by ufshcd_read_desc_param()Bart Van Assche1-3/+5
If param_offset > buff_len then the memcpy() statement in ufshcd_read_desc_param() corrupts memory since it copies 256 + buff_len - param_offset bytes into a buffer with size buff_len. Since param_offset < 256 this results in writing past the bound of the output buffer. Link: https://lore.kernel.org/r/20210722033439.26550-2-bvanassche@acm.org Fixes: cbe193f6f093 ("scsi: ufs: Fix potential NULL pointer access during memcpy") Reviewed-by: Avri Altman <avri.altman@wdc.com> Reviewed-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-01scsi: ufs: ufshpb: Add HPB 2.0 supportDaejun Park1-4/+21
Version 2.0 of HBP supports reads of varying sizes from 4KB to 1MB. A read operation <= 32KB is supported as single HPB read. A read between 36KB and 1MB is supported by a combination of write buffer command and HPB read command to deliver more PPN. The write buffer commands may not be issued immediately due to busy tags. To use HPB read more aggressively, the driver can requeue the write buffer command. The requeue threshold is implemented as timeout and can be modified with requeue_timeout_ms entry in sysfs. [mkp: REQ_OP_DRV_* and blk_rq_is_passthrough()] Link: https://lore.kernel.org/r/20210712090025epcms2p3b3d94f6f1b2cfa394e3d9ba130ca0fa7@epcms2p3 Tested-by: Can Guo <cang@codeaurora.org> Tested-by: Stanley Chu <stanley.chu@mediatek.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by: Can Guo <cang@codeaurora.org> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-01scsi: ufs: ufshpb: Prepare HPB read for cached sub-regionDaejun Park1-0/+2
If the logical address of a read I/O belongs to an active sub-region, the HPB driver modifies the read I/O command to an HPB read. The driver modifies the UFS UPIU instead of modifying the existing SCSI command. In HPB version 1.0, the maximum read I/O size that can be converted to HPB read is 4KB. The dirty map of the active sub-region prevents an incorrect HPB read that has stale physical page number which is updated by previous write I/O. [mkp: REQ_OP_DRV_* and blk_rq_is_passthrough()] Link: https://lore.kernel.org/r/20210712085936epcms2p4b0ec5c8cecdeea6cc043d684363842b6@epcms2p4 Tested-by: Bean Huo <beanhuo@micron.com> Tested-by: Can Guo <cang@codeaurora.org> Tested-by: Stanley Chu <stanley.chu@mediatek.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by: Can Guo <cang@codeaurora.org> Reviewed-by: Bart Van Assche <bvanassche@acm.org> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Acked-by: Avri Altman <Avri.Altman@wdc.com> Signed-off-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-01scsi: ufs: ufshpb: L2P map management for HPB readDaejun Park1-0/+4
Implement L2P map management in HPB. The HPB divides logical addresses into several regions. A region consists of several sub-regions. The sub-region is a basic unit where L2P mapping is managed. The driver loads L2P mapping data of each sub-region. The loaded sub-region is called active-state. The HPB driver unloads L2P mapping data as region unit. The unloaded region is called inactive-state. Sub-region/region candidates to be loaded and unloaded are delivered from the UFS device. The UFS device delivers the recommended active sub-region and inactivate region to the driver using sense data. The HPB module performs L2P mapping management on the host through the delivered information. A pinned region is a preset region on the UFS device that is always in activate-state. The data structures for map data requests and L2P mappings use the mempool API, minimizing allocation overhead while avoiding static allocation. The mininum size of the memory pool used in the HPB is implemented as a module parameter so that it can be configurable by the user. To guarantee a minimum memory pool size of 4MB: ufshpb_host_map_kbytes=4096. The map_work manages active/inactive via 2 "to-do" lists: - hpb->lh_inact_rgn: regions to be inactivated - hpb->lh_act_srgn: subregions to be activated These lists are maintained on I/O completion. [mkp: switch to REQ_OP_DRV_*] Link: https://lore.kernel.org/r/20210712085859epcms2p36e420f19564f6cd0c4a45d54949619eb@epcms2p3 Tested-by: Bean Huo <beanhuo@micron.com> Tested-by: Can Guo <cang@codeaurora.org> Tested-by: Stanley Chu <stanley.chu@mediatek.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by: Bart Van Assche <bvanassche@acm.org> Reviewed-by: Can Guo <cang@codeaurora.org> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Acked-by: Avri Altman <Avri.Altman@wdc.com> Signed-off-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-08-01scsi: ufs: ufshpb: Introduce Host Performance Buffer featureDaejun Park1-0/+48
Implement Host Performance Buffer (HPB) initialization and add function calls to UFS core driver. NAND flash-based storage devices, including UFS, have mechanisms to translate logical addresses of I/O requests to the corresponding physical addresses of the flash storage. In UFS, logical-to-physical-address (L2P) map data, which is required to identify the physical address for the requested I/Os, can only be partially stored in SRAM from NAND flash. Due to this partial loading, accessing the flash address area, where the L2P information for that address is not loaded in the SRAM, can result in serious performance degradation. The basic concept of HPB is to cache L2P mapping entries in host system memory so that both physical block address (PBA) and logical block address (LBA) can be delivered in HPB read command. The HPB read command allows to read data faster than a regular read command in UFS since it provides the physical address (HPB Entry) of the desired logical block in addition to its logical address. The UFS device can access the physical block in NAND directly without searching and uploading L2P mapping table. This improves read performance because the NAND read operation for uploading L2P mapping table is removed. In HPB initialization, the host checks if the UFS device supports HPB feature and retrieves related device capabilities. Then, HPB parameters are configured in the device. Total start-up time of popular applications was measured and the difference observed between HPB being enabled and disabled. Popular applications are 12 game apps and 24 non-game apps. Each test cycle consists of running 36 applications in sequence. We repeated the cycle for observing performance improvement by L2P mapping cache hit in HPB. The following is the test environment: - kernel version: 4.4.0 - RAM: 8GB - UFS 2.1 (64GB) Results: +-------+----------+----------+-------+ | cycle | baseline | with HPB | diff | +-------+----------+----------+-------+ | 1 | 272.4 | 264.9 | -7.5 | | 2 | 250.4 | 248.2 | -2.2 | | 3 | 226.2 | 215.6 | -10.6 | | 4 | 230.6 | 214.8 | -15.8 | | 5 | 232.0 | 218.1 | -13.9 | | 6 | 231.9 | 212.6 | -19.3 | +-------+----------+----------+-------+ We also measured HPB performance using iozone: $ iozone -r 4k -+n -i2 -ecI -t 16 -l 16 -u 16 -s $IO_RANGE/16 -F \ mnt/tmp_1 mnt/tmp_2 mnt/tmp_3 mnt/tmp_4 mnt/tmp_5 mnt/tmp_6 mnt/tmp_7 \ mnt/tmp_8 mnt/tmp_9 mnt/tmp_10 mnt/tmp_11 mnt/tmp_12 mnt/tmp_13 \ mnt/tmp_14 mnt/tmp_15 mnt/tmp_16 Results: +----------+--------+---------+ | IO range | HPB on | HPB off | +----------+--------+---------+ | 1 GB | 294.8 | 300.87 | | 4 GB | 293.51 | 179.35 | | 8 GB | 294.85 | 162.52 | | 16 GB | 293.45 | 156.26 | | 32 GB | 277.4 | 153.25 | +----------+--------+---------+ Link: https://lore.kernel.org/r/20210712085830epcms2p8c1288b7f7a81b044158a18232617b572@epcms2p8 Reported-by: kernel test robot <lkp@intel.com> Tested-by: Bean Huo <beanhuo@micron.com> Tested-by: Can Guo <cang@codeaurora.org> Tested-by: Stanley Chu <stanley.chu@mediatek.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by: Bart Van Assche <bvanassche@acm.org> Reviewed-by: Can Guo <cang@codeaurora.org> Reviewed-by: Bean Huo <beanhuo@micron.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Acked-by: Avri Altman <Avri.Altman@wdc.com> Signed-off-by: Daejun Park <daejun7.park@samsung.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-07-31scsi: ufs: Allow async suspend/resume callbacksVincent Palomares1-0/+1
Allow UFS suspend/resume callbacks to run in parallel with other suspend/resume callbacks. This can recoup dozens of milliseconds on the resume path if UFS hardware needs to be powered back on. Suspending and resuming asynchronously is safe to do so long as the driver callbacks only depend on resources made available by either a) parent devices or b) devices explicitly marked as suppliers with device_link_add. Link: https://lore.kernel.org/r/20210728012743.1063928-1-paillon@google.com Cc: Bjorn Helgaas <bhelgaas@google.com> Cc: Jaegeuk Kim <jaegeuk@kernel.org> Cc: Bart Van Assche <bvanassche@acm.org> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Stanley Chu <stanley.chu@mediatek.com> Cc: Can Guo <cang@codeaurora.org> Cc: Asutosh Das <asutoshd@codeaurora.org> Cc: Avri Altman <avri.altman@wdc.com> Cc: Martin K. Petersen <martin.petersen@oracle.com> Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Vincent Palomares <paillon@google.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-07-21scsi: ufs: core: Use scsi_get_lba() to get LBAMartin K. Petersen1-2/+1
Use the scsi_get_lba() helper instead of a function internal to the SCSI disk driver. Remove #include "sd.h". Link: https://lore.kernel.org/r/20210609033929.3815-16-martin.petersen@oracle.com Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com> Message-Id: <20210609033929.3815-16-martin.petersen@oracle.com>
generated by cgit v1.2.3 (git 2.39.0) at 2024-08-02 14:41:38 +0300