summaryrefslogtreecommitdiff
path: root/drivers/scsi/scsi_debug.c
AgeCommit message (Collapse)AuthorFilesLines
2021-02-22Merge tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsiLinus Torvalds1-1/+1
Pull SCSI updates from James Bottomley: "This series consists of the usual driver updates (ufs, ibmvfc, qla2xxx, hisi_sas, pm80xx) plus the removal of the gdth driver (which is bound to cause conflicts with a trivial change somewhere). The only big major rework of note is the one from Hannes trying to clean up our result handling code in the drivers to make it consistent" * tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (194 commits) scsi: MAINTAINERS: Adjust to reflect gdth scsi driver removal scsi: ufs: Give clk scaling min gear a value scsi: lpfc: Fix 'physical' typos scsi: megaraid_mbox: Fix spelling of 'allocated' scsi: qla2xxx: Simplify the calculation of variables scsi: message: fusion: Fix 'physical' typos scsi: target: core: Change ASCQ for residual write scsi: target: core: Signal WRITE residuals scsi: target: core: Set residuals for 4Kn devices scsi: hisi_sas: Add trace FIFO debugfs support scsi: hisi_sas: Flush workqueue in hisi_sas_v3_remove() scsi: hisi_sas: Enable debugfs support by default scsi: hisi_sas: Don't check .nr_hw_queues in hisi_sas_task_prep() scsi: hisi_sas: Remove deferred probe check in hisi_sas_v2_probe() scsi: lpfc: Add auto select on IRQ_POLL scsi: ncr53c8xx: Fix typos scsi: lpfc: Fix ancient double free scsi: qla2xxx: Fix some memory corruption scsi: qla2xxx: Remove redundant NULL check scsi: megaraid: Fix ifnullfree.cocci warnings ...
2021-02-09scsi: scsi_debug: Fix a memory leakMaurizio Lombardi1-0/+1
The sdebug_q_arr pointer must be freed when the module is unloaded. $ cat /sys/kernel/debug/kmemleak unreferenced object 0xffff888e1cfb0000 (size 4096): comm "modprobe", pid 165555, jiffies 4325987516 (age 685.194s) hex dump (first 32 bytes): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [<00000000458f4f5d>] 0xffffffffc06702d9 [<000000003edc4b1f>] do_one_initcall+0xe9/0x57d [<00000000da7d518c>] do_init_module+0x1d1/0x6f0 [<000000009a6a9248>] load_module+0x36bd/0x4f50 [<00000000ddb0c3ce>] __do_sys_init_module+0x1db/0x260 [<000000009532db57>] do_syscall_64+0xa5/0x420 [<000000002916b13d>] entry_SYSCALL_64_after_hwframe+0x6a/0xdf Fixes: 87c715dcde63 ("scsi: scsi_debug: Add per_host_store option") Link: https://lore.kernel.org/r/20210208111734.34034-1-mlombard@redhat.com Acked-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Maurizio Lombardi <mlombard@redhat.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-01-23scsi: scsi_debug: Do not set COMMAND_COMPLETEHannes Reinecke1-1/+1
COMMAND_COMPLETE is defined as '0', so setting it is quite pointless. Link: https://lore.kernel.org/r/20210113090500.129644-23-hare@suse.de Reviewed-by: Christoph Hellwig <hch@lst.de> Acked-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-01-06scsi: scsi_debug: Fix memleak in scsi_debug_init()Dinghao Liu1-2/+3
When sdeb_zbc_model does not match BLK_ZONED_NONE, BLK_ZONED_HA or BLK_ZONED_HM, we should free sdebug_q_arr to prevent memleak. Also there is no need to execute sdebug_erase_store() on failure of sdeb_zbc_model_str(). Link: https://lore.kernel.org/r/20201226061503.20050-1-dinghao.liu@zju.edu.cn Acked-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Dinghao Liu <dinghao.liu@zju.edu.cn> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-10-15Merge tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsiLinus Torvalds1-11/+72
Pull SCSI updates from James Bottomley: "The usual driver updates (ufs, qla2xxx, tcmu, ibmvfc, lpfc, smartpqi, hisi_sas, qedi, qedf, mpt3sas) and minor bug fixes. There are only three core changes: adding sense codes, cleaning up noretry and adding an option for limitless retries" * tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (226 commits) scsi: hisi_sas: Recover PHY state according to the status before reset scsi: hisi_sas: Filter out new PHY up events during suspend scsi: hisi_sas: Add device link between SCSI devices and hisi_hba scsi: hisi_sas: Add check for methods _PS0 and _PR0 scsi: hisi_sas: Add controller runtime PM support for v3 hw scsi: hisi_sas: Switch to new framework to support suspend and resume scsi: hisi_sas: Use hisi_hba->cq_nvecs for calling calling synchronize_irq() scsi: qedf: Remove redundant assignment to variable 'rc' scsi: lpfc: Remove unneeded variable 'status' in lpfc_fcp_cpu_map_store() scsi: snic: Convert to use DEFINE_SEQ_ATTRIBUTE macro scsi: qla4xxx: Delete unneeded variable 'status' in qla4xxx_process_ddb_changed scsi: sun_esp: Use module_platform_driver to simplify the code scsi: sun3x_esp: Use module_platform_driver to simplify the code scsi: sni_53c710: Use module_platform_driver to simplify the code scsi: qlogicpti: Use module_platform_driver to simplify the code scsi: mac_esp: Use module_platform_driver to simplify the code scsi: jazz_esp: Use module_platform_driver to simplify the code scsi: mvumi: Fix error return in mvumi_io_attach() scsi: lpfc: Drop nodelist reference on error in lpfc_gen_req() scsi: be2iscsi: Fix a theoretical leak in beiscsi_create_eqs() ...
2020-10-06scsi: scsi_debug: Support host tagsetJohn Garry1-18/+10
When host_max_queue is set (> 0), set the Scsi_Host.host_tagset such that blk-mq will use a hostwide tagset over all SCSI host submission queues. This means that we may expose all submission queues and always use the hwq chosen by blk-mq. And since if sdebug_host_max_queue is set, sdebug_max_queue is fixed to the same value, we can simplify how sdebug_driver_template.can_queue is set. Signed-off-by: John Garry <john.garry@huawei.com> Tested-by: Douglas Gilbert <dgilbert@interlog.com> Acked-by: Martin K. Petersen <martin.petersen@oracle.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-09-15Merge branch '5.9/scsi-fixes' into 5.10/scsi-ufsMartin K. Petersen1-2/+2
Resolve UFS discrepancies between fixes and queue. Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-09-03scsi: scsi_debug: Make sdebug_build_parts() respect virtual_gbJohn Pittman1-1/+1
If virtual_gb is passed while using num_parts, when creating the partitions, virtual_gb is not respected. Set num_sectors using get_sdebug_capacity() to pull virtual_gb if set. Link: https://lore.kernel.org/r/20200902211434.9979-3-jpittman@redhat.com Acked-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: John Pittman <jpittman@redhat.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-09-03scsi: scsi_debug: Adjust num_parts to create equally sized partitionsJohn Pittman1-3/+7
Currently when using the num_parts parameter, partitions are aligned and the end sector is one prior to the next start. This creates different sized partitions. Create instead equally sized partitions by trimming the end of each partition to the size of the smallest partition. This aligns better with what one would expect from automatically created partitions and can be helpful with testing things such as raid which often expect legs of the same size. Minimal space is lost as the initial partition starting size is calculated by dividing num_sectors by sdebug_num_parts. Link: https://lore.kernel.org/r/20200902211434.9979-2-jpittman@redhat.com Acked-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: John Pittman <jpittman@redhat.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-09-02scsi: scsi_debug: Implement lun_formatDouglas Gilbert1-7/+64
Implement 'flat space LUN addressing', which allows us to raise the max_lun limitation to 16384. The maximum number of LUNs prior to this patch was 256. Link: https://lore.kernel.org/r/20200821042249.5097-1-dgilbert@interlog.com Suggested-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-08-25scsi: scsi_debug: Remove superfluous close zone in resp_open_zone()Niklas Cassel1-2/+0
resp_open_zone() always calls zbc_open_zone() with parameter explicit set to true. If zbc_open_zone() is called with parameter explicit set to true, and the current zone state is implicit open, it will call zbc_close_zone() on the zone before proceeding. Therefore, there is no need for resp_open_zone() to call zbc_close_zone() on an implicitly open zone before calling zbc_open_zone(). Remove superfluous close zone in resp_open_zone(). Link: https://lore.kernel.org/r/20200821130007.39938-1-niklas.cassel@wdc.com Reviewed-by: Damien Le Moal <damien.lemoal@wdc.com> Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-08-18scsi: scsi_debug: Fix scp is NULL errorsDouglas Gilbert1-0/+2
John Garry reported 'sdebug_q_cmd_complete: scp is NULL' failures that were mainly seen on aarch64 machines (e.g. RPi 4 with four A72 CPUs). The problem was tracked down to a missing critical section on a "short circuit" path. Namely, the time to process the current command so far has already exceeded the requested command duration (i.e. the number of nanoseconds in the ndelay parameter). The random=1 parameter setting was pivotal in finding this error. The failure scenario involved first taking that "short circuit" path (due to a very short command duration) and then taking the more likely hrtimer_start() path (due to a longer command duration). With random=1 each command's duration is taken from the uniformly distributed [0..ndelay) interval. The fio utility also helped by reliably generating the error scenario at about once per minute on a RPi 4 (64 bit OS). Link: https://lore.kernel.org/r/20200813155738.109298-1-dgilbert@interlog.com Reported-by: John Garry <john.garry@huawei.com> Reviewed-by: Lee Duncan <lduncan@suse.com> Signed-off-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-07-29scsi: scsi_debug: Implement tur_ms_to_ready parameterDouglas Gilbert1-17/+93
The current driver responds to TEST UNIT READY (TUR) with a GOOD status immediately after a scsi_debug device (LU) is created. This is unrealistic as even SSDs take some time after power-on before accepting media access commands. Add the tur_ms_to_ready parameter whose unit is milliseconds (default 0) and is the period before which a TUR (or any media access command) will set the CHECK CONDITION status with a sense key of NOT READY and an additional sense of "Logical unit is in process of becoming ready". The period starts when each scsi_debug device is created. This patch was prompted by T10 proposal 20-061r2 which was accepted on 2020716. It adds that a TUR in the situation described in the previous paragraph may set the INFO field (or descriptor) in the sense data to the estimated number in milliseconds before a subsequent TUR will yield a GOOD status. This patch follows that advice. Link: https://lore.kernel.org/r/20200724155531.668144-1-dgilbert@interlog.com Reported-by: kernel test robot <lkp@intel.com> Signed-off-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-07-29scsi: scsi_debug: Fix request senseDouglas Gilbert1-35/+29
The SCSI REQUEST SENSE command emulation was found to be broken. It is a quite complex command so try and make it do a subset of what it should do. Remove the attempt to mimic SCSI-1 REQUEST SENSE (i.e. return the sense data for the previous failed command). Add some reporting of "pollable" sense data [see spc6r02: 5.12.2]. Keep the IEC mode page MRIE=6 TEST=1 predictive failure reporting. Link: https://lore.kernel.org/r/20200723194819.545573-1-dgilbert@interlog.com Signed-off-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-07-14scsi: scsi_debug: Update documentation url and bump versionDouglas Gilbert1-3/+3
This driver maintains a version number which is cross-referenced in the documentation (e.g. to indicate when features are added or changed) and exposed through the responses to various SCSI commands. For example the version number is use as the Product Revision number in standard SCSI INQUIRY responses issued by this driver. The version date string is placed in a vendor specific area in each standard SCSI INQUIRY response. This patch bumps both. Update the driver documentation URL that appears at the top of the driver source file. Link: https://lore.kernel.org/r/20200712182927.72044-3-dgilbert@interlog.com Signed-off-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-07-14scsi: scsi_debug: every_nth triggered error injectionDouglas Gilbert1-130/+101
This patch simplifies, or at least makes more consistent, the way setting the every_nth parameter injects errors. Here is a list of 'opts' flags and in which cases they inject errors when abs(every_nth)%command_count == 0 is reached: - OPT_RECOVERED_ERR: issued on READ(*)s, WRITE(*)s and WRITE_SCATTEREDs - OPT_DIF_ERR: issued on READ(*)s, WRITE(*)s and WRITE_SCATTEREDs - OPT_DIX_ERR: issued on READ(*)s, WRITE(*)s and WRITE_SCATTEREDs - OPT_SHORT_TRANSFER: issued on READ(*)s - OPT_TRANSPORT_ERR: issued on all commands - OPT_CMD_ABORT: issued on all commands The other uses of every_nth were not modified. Previously if, for example, OPT_SHORT_TRANSFER was armed then if (abs(every_nth) % command_count == 0) occurred during a command that was _not_ a READ, then no error injection occurred. This behaviour puzzled several testers. Now a global "inject_pending" flag is set and the _next_ READ will get hit and that flag is cleared. OPT_RECOVERED_ERR, OPT_DIF_ERR and OPT_DIX_ERR have similar behaviour. A downside of this is that there might be a hang-over pending injection that gets triggered by a following test. Also expand the every_nth runtime parameter so that it can take hex value (i.e. with a leading '0x') as well as a decimal value. Now both the 'opts' and the 'every_nth' runtime parameters can take hexadecimal values. Link: https://lore.kernel.org/r/20200712182927.72044-2-dgilbert@interlog.com Signed-off-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-07-14scsi: scsi_debug: Support hostwide tagsJohn Garry1-12/+68
Many SCSI HBAs support a hostwide tagset, whereby each command submitted to the HW from all submission queues must have a unique tag identifier. Normally this unique tag will be in the range [0, max queue], where "max queue" is the depth of each of the submission queues. Add support for this hostwide tag feature, via module parameter "host_max_queue". A non-zero value means that the feature is enabled. In this case, the submission queues are not exposed to upper layer, i.e. from blk-mq prespective, the device has a single hw queue. There are 2 reasons for this: a. It is assumed that the host can support nr_hw_queues * can_queue commands, but this is not true for hostwide tags b. For nr_hw_queues != 0, the request tag is not unique over all HW queues, and some HBA drivers want to use this tag for the hostwide tag However, like many SCSI HBA drivers today - megaraid sas being an example - the full set of HW submission queues are still used in the LLDD driver. So instead of using a complicated "reply_map" to create a per-CPU submission queue mapping like megaraid_sas (as it depends on a PCI device + MSIs) - use a simple algorithm: hwq = cpu % queue count If the host_max_queue param is set non-zero, then the max queue depth is fixed at this value also. If and when hostwide shared tags are supported in blk-mq/scsi mid-layer, then the policy to set nr_hw_queues = 0 for hostwide tags can be revised. Link: https://lore.kernel.org/r/1594297400-24756-3-git-send-email-john.garry@huawei.com Acked-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: John Garry <john.garry@huawei.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-07-14scsi: scsi_debug: Add check for sdebug_max_queue during module initJohn Garry1-0/+6
sdebug_max_queue should not exceed SDEBUG_CANQUEUE, otherwise crashes like this can be triggered by passing an out-of-range value: Hardware name: Huawei D06 /D06, BIOS Hisilicon D06 UEFI RC0 - V1.16.01 03/15/2019 pstate: 20400009 (nzCv daif +PAN -UAO BTYPE=--) pc : schedule_resp+0x2a4/0xa70 [scsi_debug] lr : schedule_resp+0x52c/0xa70 [scsi_debug] sp : ffff800022ab36f0 x29: ffff800022ab36f0 x28: ffff0023a935a610 x27: ffff800008e0a648 x26: 0000000000000003 x25: ffff0023e84f3200 x24: 00000000003d0900 x23: 0000000000000000 x22: 0000000000000000 x21: ffff0023be60a320 x20: ffff0023be60b538 x19: ffff800008e13000 x18: 0000000000000000 x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000 x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000 x11: 0000000000000000 x10: 0000000000000000 x9 : 0000000000000001 x8 : 0000000000000000 x7 : 0000000000000000 x6 : 00000000000000c1 x5 : 0000020000200000 x4 : dead0000000000ff x3 : 0000000000000200 x2 : 0000000000000200 x1 : ffff800008e13d88 x0 : 0000000000000000 Call trace: schedule_resp+0x2a4/0xa70 [scsi_debug] scsi_debug_queuecommand+0x2c4/0x9e0 [scsi_debug] scsi_queue_rq+0x698/0x840 __blk_mq_try_issue_directly+0x108/0x228 blk_mq_request_issue_directly+0x58/0x98 blk_mq_try_issue_list_directly+0x5c/0xf0 blk_mq_sched_insert_requests+0x18c/0x200 blk_mq_flush_plug_list+0x11c/0x190 blk_flush_plug_list+0xdc/0x110 blk_finish_plug+0x38/0x210 blkdev_direct_IO+0x450/0x4d8 generic_file_read_iter+0x84/0x180 blkdev_read_iter+0x3c/0x50 aio_read+0xc0/0x170 io_submit_one+0x5c8/0xc98 __arm64_sys_io_submit+0x1b0/0x258 el0_svc_common.constprop.3+0x68/0x170 do_el0_svc+0x24/0x90 el0_sync_handler+0x13c/0x1a8 el0_sync+0x158/0x180 Code: 528847e0 72a001e0 6b00003f 540018cd (3941c340) In addition, it should not be less than 1. So add checks for these, and fail the module init for those cases. [mkp: changed if condition to match error message] Link: https://lore.kernel.org/r/1594297400-24756-2-git-send-email-john.garry@huawei.com Fixes: c483739430f1 ("scsi_debug: add multiple queue support") Reviewed-by: Ming Lei <ming.lei@redhat.com> Acked-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: John Garry <john.garry@huawei.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-07-03scsi: scsi_debug: Fix in_use bitmap corruptionDouglas Gilbert1-3/+6
Heavy testing indicates the irqsave() spinlock around the __set_bit() is insufficient to stop following clear_bit() calls being rarely applied out-of-order. Also the nearby failed kzalloc() path leading to SCSI_MLQUEUE_HOST_BUSY does not properly undo the in_use bitmap and num_in_q, fix. Link: https://lore.kernel.org/r/20200702145355.522283-1-dgilbert@interlog.com Signed-off-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-05-20scsi: scsi_debug: Parser tables and code interactionDouglas Gilbert1-50/+63
This patch is in response to a static analyser report from Dan Carpenter titled: "[bug report] scsi: scsi_debug: Add per_host_store option". This code may not clear the static analyzer reports, but may shed light on why they occur. Amongst other things this driver has a table driven SCSI command parser which also involves some C code. There are some invariants between the table entries and the corresponding C code (i.e. the resp_*() functions) that, if broken, may lead to a NULL dereference. And the report is valid, at least in the case of the PRE-FETCH command. Alas, that is not one of the cases that the static analyzer reported. In this particular corner case: when the fake_rw flag is set and the table entry for a "store"-accessing command does not have the required F_FAKE_RW flag set, do the following. Call BUG_ON() in the devip2sip() very close to a comment block explaining why it was called and how to fix it. checkpatch.pl complains about the BUG_ON() but there is no reasonable remedial action that can be taken at run time. This change allows the code reported by the static analyzer to be simplified. Comments were also added to the table flags (e.g. F_FAKE_RW) so developers who add commands might be more inclined to use them (properly). Link: https://lore.kernel.org/r/20200513013943.25285-1-dgilbert@interlog.com Reported-by: Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-05-12scsi: scsi_debug: Fix an error handling bug in sdeb_zbc_model_str()Dan Carpenter1-1/+1
This test is checking the wrong variable. It should be testing "res". The "sdeb_zbc_model" variable is an enum (unsigned in this situation) and we never assign negative values to it. [mkp: fixed commit desc issue reported by Doug] Link: https://lore.kernel.org/r/20200509100408.GA5555@mwanda Fixes: 9267e0eb41fe ("scsi: scsi_debug: Add ZBC module parameter") Acked-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-05-08scsi: scsi_debug: Disallow zone sizes that are not powers of 2Damien Le Moal1-10/+7
Allowing a non-power-of-2 zone size forces the use of direct division operations of 64-bit sector values to obtain a zone number or number of zones. Doing so without using do_div() leads to compilation errors on 32-bit architectures. Devices with a zone size that is not a power of 2 do not exist today so allowing their emulation is of limited interest as the sd driver will not support them anyway. To fix this compilation error, instead of using do_div() for sector values divisions, simply disallow zone size values that are not a power of 2. [mkp: commit desc] Link: https://lore.kernel.org/r/20200507023526.221574-1-damien.lemoal@wdc.com Fixes: 98e0a689868c ("scsi: scsi_debug: Add zone_size_mb module parameter") Fixes: f0d1cf9378bd ("scsi: scsi_debug: Add ZBC zone commands") Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com> Acked-by: Geert Uytterhoeven <geert@linux-m68k.org> Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-05-05scsi: scsi_debug: Implement ZBC host-aware emulationDamien Le Moal1-45/+103
Implement ZBC host-aware device model emulation. The main changes from the host-managed emulation are the device type (TYPE_DISK is used), relaxation of access checks for read and write operations and different handling of a sequential write preferred zone write pointer as mandated by the ZBC r05 specifications. To facilitate the implementation and avoid a lot of "if" statement, the zmodel field is added to the device information and the z_type field to the zone state data structure. Link: https://lore.kernel.org/r/20200422104221.378203-8-damien.lemoal@wdc.com Tested-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-05-05scsi: scsi_debug: Add zone_size_mb module parameterDamien Le Moal1-5/+7
Add the zone_size_mb module parameters to control the zone size of a ZBC device. If the zone size specified is not a divisor of the device capacity, the last zone of the device will be created as a smaller "runt" zone. This parameter is ignored for device types other than 0x14 (zbc=2 case). Note: for testing purposes, zone sizes that are not a power of 2 are accepted but will result in the drive being rejected by the sd driver. Link: https://lore.kernel.org/r/20200422104221.378203-7-damien.lemoal@wdc.com Signed-off-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-05-05scsi: scsi_debug: Add zone_nr_conv module parameterDamien Le Moal1-1/+12
Allow controlling the number of conventional zones of a ZBC device with the new zone_nr_conv module parameter. The default value is 1 and the specified value must be less than the total number of zones of the device. This parameter is ignored for device types other than 0x14 (zbc=2 case). Link: https://lore.kernel.org/r/20200422104221.378203-6-damien.lemoal@wdc.com Signed-off-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-05-05scsi: scsi_debug: Add zone_max_open module parameterDamien Le Moal1-3/+5
Add the zone_max_open module parameters to control the maximum number of open zones of a ZBC device. This parameter is ignored for device types other than 0x14 (zbc=2 case). Link: https://lore.kernel.org/r/20200422104221.378203-5-damien.lemoal@wdc.com Signed-off-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-05-05scsi: scsi_debug: Add ZBC module parameterDouglas Gilbert1-3/+78
Add the zbc module parameter to take either: 0: none (probably a conventional disk) 1: host-aware 2: host-managed These values are chosen to match 'enum blk_zoned_model' found in include/linux/blkdev.h . Instead of "none", "no" or "0" can be given. Instead of "host-aware", "aware or "1" can be given. Instead of "host-managed", "managed" or "2" can be given. Note: the zbc parameter can only be given at driver/module load time; it cannot be changed via sysfs thereafter. At this time there is no ZBC "host-aware" implementation so that string (or the value '1') results in a modprobe error. Link: https://lore.kernel.org/r/20200422104221.378203-4-damien.lemoal@wdc.com Signed-off-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-05-05scsi: scsi_debug: Add ZBC zone commandsDouglas Gilbert1-15/+804
Add support for the 5 ZBC commands and enough functionality to emulate a host-managed device with one conventional zone and a set of sequential write-required zones up to the disk capacity. Link: https://lore.kernel.org/r/20200422104221.378203-3-damien.lemoal@wdc.com Signed-off-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-05-05scsi: scsi_debug: Add ZBC mode and VPD pagesDouglas Gilbert1-11/+44
The ZBC standard "piggy-backs" on many, but not all, of the facilities in SBC. Add those ZBC mode pages (plus mode parameter block descriptors (e.g. "WP")) and VPD pages in common with SBC. Add ZBC specific VPD page for the host-managed ZBC device type (ptype=0x14). Link: https://lore.kernel.org/r/20200422104221.378203-2-damien.lemoal@wdc.com Signed-off-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-05-05scsi: scsi_debug: Bump to version 1.89Douglas Gilbert1-3/+3
The scsi_debug driver version is visible in: /sys/modules/scsi_debug/version and can thus be used by user space programs to alter the features they try to use. Since the per_host_store and zbc/zone options are significant additions, bump the version number to 1.89 . Link: https://lore.kernel.org/r/20200421151424.32668-9-dgilbert@interlog.com Signed-off-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-05-05scsi: scsi_debug: Re-arrange parameters alphabeticallyDouglas Gilbert1-16/+18
This module has a lot of parameters and when searching for one, the author prefers them in alphabetical order. This can lead to somewhat illogical ordering (e.g. inq_product before inq_vendor). However it is not clear what another sensible total logical ordering would be. Link: https://lore.kernel.org/r/20200421151424.32668-8-dgilbert@interlog.com Signed-off-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-05-05scsi: scsi_debug: Implement PRE-FETCH commandsDouglas Gilbert1-5/+70
Many disks implement the SCSI PRE-FETCH commands. One use case might be a disk-to-disk compare, say between disks A and B. Then this sequence of commands might be used: PRE-FETCH(from B, IMMED), READ(from A), VERIFY (BYTCHK=1 on B with data returned from READ). The PRE-FETCH (which returns quickly due to the IMMED) fetches the data from the media into B's cache which should speed the trailing VERIFY command. The next chunk of the compare might be done in parallel, with A and B reversed. The implementation tries to bring the specified range in main memory into the cache(s) associated with this machine's CPU(s) using the prefetch_range() function. Link: https://lore.kernel.org/r/20200421151424.32668-7-dgilbert@interlog.com Signed-off-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-05-05scsi: scsi_debug: Improve command duration calculationDouglas Gilbert1-2/+29
Previously the code did the work implied by the given SCSI command and after that it waited for a timer based on the user specified command duration to be exhausted before informing the mid-level that the command was complete. For short command durations, the time to complete the work implied by the SCSI command could be significant compared to the user specified command duration. For example a WRITE of 128 blocks (say 512 bytes each) on a machine that can copy from main memory to main memory at a rate of 10 GB/sec will take around 6.4 microseconds to do that copy. If the user specified a command duration of 5 microseconds (ndelay=5000), should the driver do a further delay of 5 microseconds after the copy or return immediately because 6.4 > 5 ? The action prior to this patch was to always do the timer based delay. After this patch, for ndelay values less than 1 millisecond, this driver will complete the command immediately. And in the case where the user specified delay was 7 microseconds, a timer delay of 600 nanoseconds will be set ((7 - 6.4) * 1000). Link: https://lore.kernel.org/r/20200421151424.32668-6-dgilbert@interlog.com Signed-off-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-05-05scsi: scsi_debug: Weaken rwlock around ramdisk accessDouglas Gilbert1-29/+18
The design of this driver is to do any ramdisk access on the same thread that invoked the queuecommand() call. That is assumed to be user space context. The command duration is implemented by setting the delay with a high resolution timer. The hr timer's callback may well be in interrupt context, but it doesn't touch the ramdisk. So try removing the _irqsave()/_irqrestore() portion on the read-write lock that protects ramdisk access. Link: https://lore.kernel.org/r/20200421151424.32668-5-dgilbert@interlog.com Signed-off-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-05-05scsi: scsi_debug: Implement VERIFY(10), add VERIFY(16)Douglas Gilbert1-7/+94
With the addition of the per_host_store option, the ability to check whether two different ramdisk images are the same or not becomes practical. Prior to this patch VERIFY(10) always returned true (i.e. the SCSI GOOD status) without checking. This option adds support for BYTCHK equal to 0, 1 and 3. If the comparison fails, then a sense key of MISCOMPARE is returned as per the T10 standards. Also add support for the VERIFY(16) command. Link: https://lore.kernel.org/r/20200421151424.32668-4-dgilbert@interlog.com Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-05-05scsi: scsi_debug: Add per_host_store optionDouglas Gilbert1-214/+494
The scsi_debug driver has always been restricted to using one ramdisk image (or none) for its storage. This means that thousands of scsi_debug devices can be created without exhausting the host machine's RAM. The downside is that all scsi_debug devices share the same ramdisk image. This option changes the way a following write to the add_host parameter (or an add_host in the module/driver invocation) operates. For each new host that is created while per_host_store is true, a new store (of dev-size_mb MiB) is created and associated with all the LUs that belong to that new host. The user (who will need root permissions) needs to take care not to exhaust all the machine's available RAM. One reason for doing this is to check that (partial) disk to disk copies based on scsi_debug devices have actually copied accurately. To test this the add_host=<n> parameter where <n> is 2 or greater can be used when the scsi_debug module is loaded. Let us assume that /dev/sdb and /dev/sg1 are the same scsi_debug device, while /dev/sdc and /dev/sg2 are the same scsi_debug device. With per_host_store=1 add_host=2 they will have different ramdisk images. Then the following pseudocode could be executed to check if the sgh_dd copy worked: dd if=/dev/urandom of=/dev/sdb sgh_dd if=/dev/sg1 of=/dev/sg2 [plus option(s) to test] cmp /dev/sdb /dev/sdc If the cmp fails then the copy has failed (or some other mechanism wrote to /dev/sdb or /dev/sdc in the interim). [mkp: use kstrtobool()] Link: https://lore.kernel.org/r/20200421151424.32668-3-dgilbert@interlog.com Signed-off-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-05-05scsi: scsi_debug: Randomize command completion timeDouglas Gilbert1-3/+39
Add a new command line option (e.g. random=1) and sysfs attribute that causes subsequent command completion times to be between the current command delay setting and 0. A uniformly distributed 32 bit, kernel provided integer is used for this purpose. Since the existing 'delay' whose units are jiffies (typically milliseconds) and 'ndelay' (units: nanoseconds) options (and sysfs attributes) span a range greater than 32 bits, some scaling is required. The purpose of this patch is to widen the range of testing cases that are visited in long running tests. Put simply: rarely struct race conditions are more likely to be found when this facility is used. The default is the previous case in which all command completions were roughly equal to (if not, slightly longer) than the value given by the 'delay' or 'ndelay' settings (or their defaults). This option's default is equivalent to setting 'random=0' . [mkp: use kstrtobool()] Link: https://lore.kernel.org/r/20200421151424.32668-2-dgilbert@interlog.com Reviewed-by: Hannes Reinecke <hare@suse.de> Signed-off-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-03-24block: move struct partition out of genhd.hChristoph Hellwig1-2/+3
struct partition is the on-disk format of a MSDOS partition table entry. Move it out of genhd.h into a new msdos_partition.h header and give it a msdos_ prefix to avoid confusion. Also move the magic number from block/partitions/msdos.h to the new header so that it can be used by the SCSI drivers looking at the DOS partition tables. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-11-20scsi: scsi_debug: num_tgts must be >= 0Maurizio Lombardi1-0/+5
Passing the parameter "num_tgts=-1" will start an infinite loop that exhausts the system memory Link: https://lore.kernel.org/r/20191115163727.24626-1-mlombard@redhat.com Signed-off-by: Maurizio Lombardi <mlombard@redhat.com> Acked-by: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2019-11-09scsi: core: Fix scsi_get/set_resid() interfaceDamien Le Moal1-2/+2
struct scsi_cmnd cmd->req.resid_len which is returned and set respectively by the helper functions scsi_get_resid() and scsi_set_resid() is an unsigned int. Reflect this fact in the interface of these helper functions. Also fix compilation errors due to min() and max() type mismatch introduced by this change in scsi debug code, usb transport code and in the USB ENE card reader driver. Link: https://lore.kernel.org/r/20191030090847.25650-1-damien.lemoal@wdc.com Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2019-05-24treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 45Thomas Gleixner1-6/+1
Based on 1 normalized pattern(s): this program is free software you can redistribute it and or modify it under the terms of the gnu general public license as published by the free software foundation either version 2 or at your option any later version extracted by the scancode license scanner the SPDX license identifier GPL-2.0-or-later has been chosen to replace the boilerplate/reference in 11 file(s). Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org> Reviewed-by: Allison Randal <allison@lohutok.net> Cc: linux-spdx@vger.kernel.org Link: https://lkml.kernel.org/r/20190520170858.370933192@linutronix.de Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-03-10Merge tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsiLinus Torvalds1-134/+52
Pull SCSI updates from James Bottomley: "This is mostly update of the usual drivers: arcmsr, qla2xxx, lpfc, hisi_sas, target/iscsi and target/core. Additionally Christoph refactored gdth as part of the dma changes. The major mid-layer change this time is the removal of bidi commands and with them the whole of the osd/exofs driver and filesystem. This is a major simplification for block and mq in particular" * tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (240 commits) scsi: cxgb4i: validate tcp sequence number only if chip version <= T5 scsi: cxgb4i: get pf number from lldi->pf scsi: core: replace GFP_ATOMIC with GFP_KERNEL in scsi_scan.c scsi: mpt3sas: Add missing breaks in switch statements scsi: aacraid: Fix missing break in switch statement scsi: kill command serial number scsi: csiostor: drop serial_number usage scsi: mvumi: use request tag instead of serial_number scsi: dpt_i2o: remove serial number usage scsi: st: osst: Remove negative constant left-shifts scsi: ufs-bsg: Allow reading descriptors scsi: ufs: Allow reading descriptor via raw upiu scsi: ufs-bsg: Change the calling convention for write descriptor scsi: ufs: Remove unused device quirks Revert "scsi: ufs: disable vccq if it's not needed by UFS device" scsi: megaraid_sas: Remove a bunch of set but not used variables scsi: clean obsolete return values of eh_timed_out scsi: sd: Optimal I/O size should be a multiple of physical block size scsi: MAINTAINERS: SCSI initiator and target tweaks scsi: fcoe: make use of fip_mode enum complete ...
2019-02-12scsi: scsi_debug: Implement support for write protectMartin K. Petersen1-45/+31
Teach scsi_debug to honor SWP in the Control Mode Page and report the resulting WP state in the Device-Specific Parameter field. In check_device_access_params() verify that commands that will write the medium are permitted to do so. Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com> Acked-by: Douglas Gilbert <dgilbert@interlog.com>
2019-02-12scsi: scsi_debug: Use scsi_[gs]et_resid() where appropriateBart Van Assche1-4/+5
This patch does not change any functionality. Cc: Douglas Gilbert <dgilbert@interlog.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Acked-by: Douglas Gilbert <dgilbert@interlog.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2019-02-12scsi: scsi_debug: Fix a recently introduced regressionBart Van Assche1-8/+7
A recent commit removed an element from opcode_info_arr[] but did not modify opcode_ind_arr[] nor was SDEB_I_XDWRITEREAD removed. Remove SDEB_I_XDWRITEREAD and bring the two arrays again in sync. This patch avoids that the following is reported: BUG: KASAN: null-ptr-deref in scsi_debug_queuecommand+0x60f/0xc90 [scsi_debug] Read of size 1 at addr 0000000000000001 by task iscsi-test-cu/683 CPU: 3 PID: 683 Comm: iscsi-test-cu Not tainted 5.0.0-rc5-dbg+ #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014 Call Trace: dump_stack+0x86/0xca kasan_report.cold.3+0x5/0x3e __asan_load1+0x47/0x50 scsi_debug_queuecommand+0x60f/0xc90 [scsi_debug] scsi_queue_rq+0xc17/0x12e0 blk_mq_dispatch_rq_list+0x5fc/0xb10 blk_mq_sched_dispatch_requests+0x2f7/0x300 __blk_mq_run_hw_queue+0xd6/0x180 __blk_mq_delay_run_hw_queue+0x25c/0x290 blk_mq_run_hw_queue+0x119/0x1b0 blk_mq_sched_insert_request+0x274/0x350 blk_execute_rq_nowait+0x78/0x90 blk_execute_rq+0xcc/0x140 sg_io+0x30f/0x700 scsi_cmd_ioctl+0x4d4/0x540 scsi_cmd_blk_ioctl+0x7b/0x8b sd_ioctl+0xba/0x150 blkdev_ioctl+0x6e1/0xea0 block_ioctl+0x79/0x90 do_vfs_ioctl+0x12b/0x9b0 ksys_ioctl+0x41/0x80 __x64_sys_ioctl+0x43/0x50 do_syscall_64+0x71/0x210 entry_SYSCALL_64_after_hwframe+0x49/0xbe Cc: Christoph Hellwig <hch@lst.de> Cc: Douglas Gilbert <dgilbert@interlog.com> Fixes: ae3d56d81507 ("scsi: remove bidirectional command support") Signed-off-by: Bart Van Assche <bvanassche@acm.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2019-02-09scsi: ata: Use unsigned int for cmd's type in ioctls in scsi_host_templateNathan Chancellor1-1/+2
Clang warns several times in the scsi subsystem (trimmed for brevity): drivers/scsi/hpsa.c:6209:7: warning: overflow converting case value to switch condition type (2147762695 to 18446744071562347015) [-Wswitch] case CCISS_GETBUSTYPES: ^ drivers/scsi/hpsa.c:6208:7: warning: overflow converting case value to switch condition type (2147762694 to 18446744071562347014) [-Wswitch] case CCISS_GETHEARTBEAT: ^ The root cause is that the _IOC macro can generate really large numbers, which don't fit into type 'int', which is used for the cmd parameter in the ioctls in scsi_host_template. My research into how GCC and Clang are handling this at a low level didn't prove fruitful. However, looking at the rest of the kernel tree, all ioctls use an 'unsigned int' for the cmd parameter, which will fit all of the _IOC values in the scsi/ata subsystems. Make that change because none of the ioctls expect a negative value for any command, it brings the ioctls inline with the reset of the kernel, and it removes ambiguity, which is never good when dealing with compilers. Link: https://github.com/ClangBuiltLinux/linux/issues/85 Link: https://github.com/ClangBuiltLinux/linux/issues/154 Link: https://github.com/ClangBuiltLinux/linux/issues/157 Signed-off-by: Nathan Chancellor <natechancellor@gmail.com> Acked-by: Bradley Grove <bgrove@attotech.com> Acked-by: Don Brace <don.brace@microsemi.com> Reviewed-by: Bart Van Assche <bvanassche@acm.org> Tested-by: Nick Desaulniers <ndesaulniers@google.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2019-02-06scsi: remove bidirectional command supportChristoph Hellwig1-77/+8
No real need for bidi support once the OSD code is gone. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2019-01-29scsi: scsi_debug: fix write_same with virtual_gb problemDouglas Gilbert1-20/+21
The WRITE SAME(10) and (16) implementations didn't take account of the buffer wrap required when the virtual_gb parameter is greater than 0. Fix that and rename the fake_store() function to lba2fake_store() to lessen confusion with the global fake_storep pointer. Bump version date. Signed-off-by: Douglas Gilbert <dgilbert@interlog.com> Reported-by: Bart Van Assche <bvanassche@acm.org> Tested by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-12-29Merge tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsiLinus Torvalds1-4/+4
Pull SCSI updates from James Bottomley: "This is mostly update of the usual drivers: smarpqi, lpfc, qedi, megaraid_sas, libsas, zfcp, mpt3sas, hisi_sas. Additionally, we have a pile of annotation, unused variable and minor updates. The big API change is the updates for Christoph's DMA rework which include removing the DISABLE_CLUSTERING flag. And finally there are a couple of target tree updates" * tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (259 commits) scsi: isci: request: mark expected switch fall-through scsi: isci: remote_node_context: mark expected switch fall-throughs scsi: isci: remote_device: Mark expected switch fall-throughs scsi: isci: phy: Mark expected switch fall-through scsi: iscsi: Capture iscsi debug messages using tracepoints scsi: myrb: Mark expected switch fall-throughs scsi: megaraid: fix out-of-bound array accesses scsi: mpt3sas: mpt3sas_scsih: Mark expected switch fall-through scsi: fcoe: remove set but not used variable 'port' scsi: smartpqi: call pqi_free_interrupts() in pqi_shutdown() scsi: smartpqi: fix build warnings scsi: smartpqi: update driver version scsi: smartpqi: add ofa support scsi: smartpqi: increase fw status register read timeout scsi: smartpqi: bump driver version scsi: smartpqi: add smp_utils support scsi: smartpqi: correct lun reset issues scsi: smartpqi: correct volume status scsi: smartpqi: do not offline disks for transient did no connect conditions scsi: smartpqi: allow for larger raid maps ...
2018-12-19scsi: remove the use_clustering flagChristoph Hellwig1-1/+2
The same effects can be achieved by setting the dma_boundary to PAGE_SIZE - 1 and the max_segment_size to PAGE_SIZE, so shift those settings into the drivers. Note that in many cases the setting might be bogus, but this keeps the status quo. [mkp: fix myrs and myrb] Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>