diff options
| author | Almog Khaikin <almogkh@gmail.com> | 2022-03-21 12:00:59 +0300 |
|---|---|---|
| committer | Jens Axboe <axboe@kernel.dk> | 2022-03-21 15:33:29 +0300 |
| commit | 649bb75d19c93f5459f450191953dff4825fda3e (patch) | |
| tree | 6fbdce114ff684f42607f4145e086bbb9a2d3fa9 /fs/io_uring.c | |
| parent | f63cf5192fe3418ad5ae1a4412eba5694b145f79 (diff) | |
| download | linux-649bb75d19c93f5459f450191953dff4825fda3e.tar.xz | |
io_uring: fix memory ordering when SQPOLL thread goes to sleep
Without a full memory barrier between the store to the flags and the
load of the SQ tail the two operations can be reordered and this can
lead to a situation where the SQPOLL thread goes to sleep while the
application writes to the SQ tail and doesn't see the wakeup flag.
This memory barrier pairs with a full memory barrier in the application
between its store to the SQ tail and its load of the flags.
Signed-off-by: Almog Khaikin <almogkh@gmail.com>
Link: https://lore.kernel.org/r/20220321090059.46313-1-almogkh@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Diffstat (limited to 'fs/io_uring.c')
| -rw-r--r-- | fs/io_uring.c | 7 |
1 files changed, 7 insertions, 0 deletions
diff --git a/fs/io_uring.c b/fs/io_uring.c index 1a65d7880440..48f4540d7dd5 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -8042,6 +8042,13 @@ static int io_sq_thread(void *data) needs_sched = false; break; } + + /* + * Ensure the store of the wakeup flag is not + * reordered with the load of the SQ tail + */ + smp_mb(); + if (io_sqring_entries(ctx)) { needs_sched = false; break; |