diff options
| author | Dylan Yudaken <dylany@fb.com> | 2022-01-21 15:38:56 +0300 |
|---|---|---|
| committer | Greg Kroah-Hartman <gregkh@linuxfoundation.org> | 2022-02-01 19:27:11 +0300 |
| commit | 82cc3382e34e2254453cdfc0e0c3190357e27eed (patch) | |
| tree | 24d8dbe87a8adbcceaffd1a0ce5e8866e3c1e6f6 /fs/io_uring.c | |
| parent | fa005a5c5d4dc1eb1bcf99761fc1f678bfd46a08 (diff) | |
| download | linux-82cc3382e34e2254453cdfc0e0c3190357e27eed.tar.xz | |
io_uring: fix bug in slow unregistering of nodes
[ Upstream commit b36a2050040b2d839bdc044007cdd57101d7f881 ]
In some cases io_rsrc_ref_quiesce will call io_rsrc_node_switch_start,
and then immediately flush the delayed work queue &ctx->rsrc_put_work.
However the percpu_ref_put does not immediately destroy the node, it
will be called asynchronously via RCU. That ends up with
io_rsrc_node_ref_zero only being called after rsrc_put_work has been
flushed, and so the process ends up sleeping for 1 second unnecessarily.
This patch executes the put code immediately if we are busy
quiescing.
Fixes: 4a38aed2a0a7 ("io_uring: batch reap of dead file registrations")
Signed-off-by: Dylan Yudaken <dylany@fb.com>
Link: https://lore.kernel.org/r/20220121123856.3557884-1-dylany@fb.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Sasha Levin <sashal@kernel.org>
Diffstat (limited to 'fs/io_uring.c')
| -rw-r--r-- | fs/io_uring.c | 7 |
1 files changed, 6 insertions, 1 deletions
diff --git a/fs/io_uring.c b/fs/io_uring.c index f713b91537f4..993913c585fb 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -7718,10 +7718,15 @@ static void io_rsrc_node_ref_zero(struct percpu_ref *ref) struct io_ring_ctx *ctx = node->rsrc_data->ctx; unsigned long flags; bool first_add = false; + unsigned long delay = HZ; spin_lock_irqsave(&ctx->rsrc_ref_lock, flags); node->done = true; + /* if we are mid-quiesce then do not delay */ + if (node->rsrc_data->quiesce) + delay = 0; + while (!list_empty(&ctx->rsrc_ref_list)) { node = list_first_entry(&ctx->rsrc_ref_list, struct io_rsrc_node, node); @@ -7734,7 +7739,7 @@ static void io_rsrc_node_ref_zero(struct percpu_ref *ref) spin_unlock_irqrestore(&ctx->rsrc_ref_lock, flags); if (first_add) - mod_delayed_work(system_wq, &ctx->rsrc_put_work, HZ); + mod_delayed_work(system_wq, &ctx->rsrc_put_work, delay); } static struct io_rsrc_node *io_rsrc_node_alloc(struct io_ring_ctx *ctx) |