diff options
| author | Pavel Begunkov <asml.silence@gmail.com> | 2021-08-09 15:04:04 +0300 |
|---|---|---|
| committer | Jens Axboe <axboe@kernel.dk> | 2021-08-23 22:07:56 +0300 |
| commit | c97d8a0f68b30960e9c8089bc37cc3b96a96f84d (patch) | |
| tree | fbad0bb9fb30d9295d296a4fbe40f84720accf4b /fs | |
| parent | b191e2dfe5955b392bc8c0ae546dfa5a13649c38 (diff) | |
| download | linux-c97d8a0f68b30960e9c8089bc37cc3b96a96f84d.tar.xz | |
io_uring: avoid touching inode in rw prep
If we use fixed files, we can be sure (almost) that REQ_F_ISREG is set.
However, for non-reg files io_prep_rw() still will look into inode to
double check, and that's expensive and can be avoided.
The only caveat is that it only currently works with 64+ bit
architectures, see FFS_ISREG, so we should consider that.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/0a62780c491ca2522cd52db4ae3f16e03aafed0f.1628471125.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Diffstat (limited to 'fs')
| -rw-r--r-- | fs/io_uring.c | 25 |
1 files changed, 15 insertions, 10 deletions
diff --git a/fs/io_uring.c b/fs/io_uring.c index 7027c7e1f562..69e5a2102f94 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -1231,6 +1231,20 @@ static bool req_need_defer(struct io_kiocb *req, u32 seq) return false; } +#define FFS_ASYNC_READ 0x1UL +#define FFS_ASYNC_WRITE 0x2UL +#ifdef CONFIG_64BIT +#define FFS_ISREG 0x4UL +#else +#define FFS_ISREG 0x0UL +#endif +#define FFS_MASK ~(FFS_ASYNC_READ|FFS_ASYNC_WRITE|FFS_ISREG) + +static inline bool io_req_ffs_set(struct io_kiocb *req) +{ + return IS_ENABLED(CONFIG_64BIT) && (req->flags & REQ_F_FIXED_FILE); +} + static void io_req_track_inflight(struct io_kiocb *req) { if (!(req->flags & REQ_F_INFLIGHT)) { @@ -2682,7 +2696,7 @@ static int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe) unsigned ioprio; int ret; - if (!(req->flags & REQ_F_ISREG) && S_ISREG(file_inode(file)->i_mode)) + if (!io_req_ffs_set(req) && S_ISREG(file_inode(file)->i_mode)) req->flags |= REQ_F_ISREG; kiocb->ki_pos = READ_ONCE(sqe->off); @@ -6319,15 +6333,6 @@ static void io_wq_submit_work(struct io_wq_work *work) } } -#define FFS_ASYNC_READ 0x1UL -#define FFS_ASYNC_WRITE 0x2UL -#ifdef CONFIG_64BIT -#define FFS_ISREG 0x4UL -#else -#define FFS_ISREG 0x0UL -#endif -#define FFS_MASK ~(FFS_ASYNC_READ|FFS_ASYNC_WRITE|FFS_ISREG) - static inline struct io_fixed_file *io_fixed_file_slot(struct io_file_table *table, unsigned i) { |