diff options
| author | Pavel Begunkov <asml.silence@gmail.com> | 2023-09-12 16:57:07 +0300 |
|---|---|---|
| committer | Greg Kroah-Hartman <gregkh@linuxfoundation.org> | 2023-09-19 13:27:54 +0300 |
| commit | 9704cfcf1fa9bd7b7a2e7d3adf892baa106d771d (patch) | |
| tree | 6a05a2d4054c2cc106f3594183c60c57d60c04df /io_uring/io-wq.c | |
| parent | 605d055452e7abed4aca012953aecc79c09baf12 (diff) | |
| download | linux-9704cfcf1fa9bd7b7a2e7d3adf892baa106d771d.tar.xz | |
io_uring/sqpoll: fix io-wq affinity when IORING_SETUP_SQPOLL is used
From: Jens Axboe <axboe@kernel.dk>
[ upstream commit ebdfefc09c6de7897962769bd3e63a2ff443ebf5 ]
If we setup the ring with SQPOLL, then that polling thread has its
own io-wq setup. This means that if the application uses
IORING_REGISTER_IOWQ_AFF to set the io-wq affinity, we should not be
setting it for the invoking task, but rather the sqpoll task.
Add an sqpoll helper that parks the thread and updates the affinity,
and use that one if we're using SQPOLL.
Fixes: fe76421d1da1 ("io_uring: allow user configurable IO thread CPU affinity")
Cc: stable@vger.kernel.org # 5.10+
Link: https://github.com/axboe/liburing/discussions/884
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Diffstat (limited to 'io_uring/io-wq.c')
| -rw-r--r-- | io_uring/io-wq.c | 7 |
1 files changed, 5 insertions, 2 deletions
diff --git a/io_uring/io-wq.c b/io_uring/io-wq.c index dc3d4b835622..98ac9dbcec2f 100644 --- a/io_uring/io-wq.c +++ b/io_uring/io-wq.c @@ -1350,13 +1350,16 @@ static int io_wq_cpu_offline(unsigned int cpu, struct hlist_node *node) return __io_wq_cpu_online(wq, cpu, false); } -int io_wq_cpu_affinity(struct io_wq *wq, cpumask_var_t mask) +int io_wq_cpu_affinity(struct io_uring_task *tctx, cpumask_var_t mask) { int i; + if (!tctx || !tctx->io_wq) + return -EINVAL; + rcu_read_lock(); for_each_node(i) { - struct io_wqe *wqe = wq->wqes[i]; + struct io_wqe *wqe = tctx->io_wq->wqes[i]; if (mask) cpumask_copy(wqe->cpu_mask, mask); |