diff options
| author | Pavel Begunkov <asml.silence@gmail.com> | 2021-10-13 11:57:13 +0300 |
|---|---|---|
| committer | Jens Axboe <axboe@kernel.dk> | 2021-10-18 15:17:35 +0300 |
| commit | 8971a3b7f1bf2b3096e558a0362a469dee77f426 (patch) | |
| tree | d208ac362578e56a8848ebcab6eaaab5c567b74f /block/blk-mq.c | |
| parent | 4f7ab09a1ca0bc955635705c359ab3021b405fd0 (diff) | |
| download | linux-8971a3b7f1bf2b3096e558a0362a469dee77f426.tar.xz | |
blk-mq: optimise *end_request non-stat path
We already have a blk_mq_need_time_stamp() check in
__blk_mq_end_request() to get a timestamp, hide all the statistics
accounting under it. It cuts some cycles for requests that don't need
stats, and is free otherwise.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://lore.kernel.org/r/e0f2ea812e93a8adcd07101212e7d7e70ca304e7.1634115360.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Diffstat (limited to 'block/blk-mq.c')
| -rw-r--r-- | block/blk-mq.c | 18 |
1 files changed, 8 insertions, 10 deletions
diff --git a/block/blk-mq.c b/block/blk-mq.c index b7b8437f0a04..568773de9a7f 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -579,20 +579,18 @@ void blk_mq_free_plug_rqs(struct blk_plug *plug) inline void __blk_mq_end_request(struct request *rq, blk_status_t error) { - u64 now = 0; + if (blk_mq_need_time_stamp(rq)) { + u64 now = ktime_get_ns(); - if (blk_mq_need_time_stamp(rq)) - now = ktime_get_ns(); + if (rq->rq_flags & RQF_STATS) { + blk_mq_poll_stats_start(rq->q); + blk_stat_add(rq, now); + } - if (rq->rq_flags & RQF_STATS) { - blk_mq_poll_stats_start(rq->q); - blk_stat_add(rq, now); + blk_mq_sched_completed_request(rq, now); + blk_account_io_done(rq, now); } - blk_mq_sched_completed_request(rq, now); - - blk_account_io_done(rq, now); - if (rq->end_io) { rq_qos_done(rq->q, rq); rq->end_io(rq, error); |