diff options
| author | Ming Lei <ming.lei@redhat.com> | 2024-05-15 04:31:57 +0300 |
|---|---|---|
| committer | Greg Kroah-Hartman <gregkh@linuxfoundation.org> | 2024-06-12 12:12:45 +0300 |
| commit | 714e59b5456e4d6e4295a9968c564abe193f461c (patch) | |
| tree | 9f9f933f487f3be2303a04122be1c35930f0eee2 /block | |
| parent | d4a60298ac34f027a09f8f893fdbd9e06279bb24 (diff) | |
| download | linux-714e59b5456e4d6e4295a9968c564abe193f461c.tar.xz | |
blk-cgroup: fix list corruption from reorder of WRITE ->lqueued
[ Upstream commit d0aac2363549e12cc79b8e285f13d5a9f42fd08e ]
__blkcg_rstat_flush() can be run anytime, especially when blk_cgroup_bio_start
is being executed.
If WRITE of `->lqueued` is re-ordered with READ of 'bisc->lnode.next' in
the loop of __blkcg_rstat_flush(), `next_bisc` can be assigned with one
stat instance being added in blk_cgroup_bio_start(), then the local
list in __blkcg_rstat_flush() could be corrupted.
Fix the issue by adding one barrier.
Cc: Tejun Heo <tj@kernel.org>
Cc: Waiman Long <longman@redhat.com>
Fixes: 3b8cc6298724 ("blk-cgroup: Optimize blkcg_rstat_flush()")
Signed-off-by: Ming Lei <ming.lei@redhat.com>
Link: https://lore.kernel.org/r/20240515013157.443672-3-ming.lei@redhat.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Sasha Levin <sashal@kernel.org>
Diffstat (limited to 'block')
| -rw-r--r-- | block/blk-cgroup.c | 10 |
1 files changed, 10 insertions, 0 deletions
diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index 9359b57545d2..8102a23b2487 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -1036,6 +1036,16 @@ static void __blkcg_rstat_flush(struct blkcg *blkcg, int cpu) struct blkg_iostat cur; unsigned int seq; + /* + * Order assignment of `next_bisc` from `bisc->lnode.next` in + * llist_for_each_entry_safe and clearing `bisc->lqueued` for + * avoiding to assign `next_bisc` with new next pointer added + * in blk_cgroup_bio_start() in case of re-ordering. + * + * The pair barrier is implied in llist_add() in blk_cgroup_bio_start(). + */ + smp_mb(); + WRITE_ONCE(bisc->lqueued, false); /* fetch the current per-cpu values */ |