From 84567995612258c23bc55795575babe7ef605dd9 Mon Sep 17 00:00:00 2001 From: Boqun Feng Date: Wed, 26 Aug 2015 19:52:46 +0800 Subject: locking/atomics, cmpxchg: Privatize the inclusion of asm/cmpxchg.h After commit: 654672d4ba1a ("locking/atomics: Add _{acquire|release|relaxed}() variants of some atomic operations") Architectures may only provide {cmp,}xchg_relaxed definitions in asm/cmpxchg.h. Other variants, such as {cmp,}xchg, may be built in linux/atomic.h, which means simply including asm/cmpxchg.h may not get the definitions of all the{cmp,}xchg variants. Therefore, we should privatize the inclusions of asm/cmpxchg.h to keep it only included in arch/* and replace the inclusions outside with linux/atomic.h Signed-off-by: Boqun Feng Signed-off-by: Peter Zijlstra (Intel) Acked-by: Greg Kroah-Hartman Acked-by: Will Deacon Cc: Andrew Morton Cc: Aybuke Ozdemir Cc: Chris Brannon Cc: Davidlohr Bueso Cc: Greg Kroah-Hartman Cc: Jonathan Corbet Cc: Kirk Reiser Cc: Kishon Vijay Abraham I Cc: Linus Torvalds Cc: Paul E. McKenney Cc: Peter Zijlstra Cc: Samuel Thibault Cc: Shradha Shah Cc: Thomas Gleixner Cc: William Hubbs Cc: devel@driverdev.osuosl.org Cc: linux-net-drivers@solarflare.com Cc: speakup@linux-speakup.org Link: http://lkml.kernel.org/r/1440589966-26280-1-git-send-email-boqun.feng@gmail.com Signed-off-by: Ingo Molnar --- Documentation/atomic_ops.txt | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'Documentation') diff --git a/Documentation/atomic_ops.txt b/Documentation/atomic_ops.txt index b19fc34efdb1..c9d1cacb4395 100644 --- a/Documentation/atomic_ops.txt +++ b/Documentation/atomic_ops.txt @@ -542,6 +542,10 @@ The routines xchg() and cmpxchg() must provide the same exact memory-barrier semantics as the atomic and bit operations returning values. +Note: If someone wants to use xchg(), cmpxchg() and their variants, +linux/atomic.h should be included rather than asm/cmpxchg.h, unless +the code is in arch/* and can take care of itself. + Spinlocks and rwlocks have memory barrier expectations as well. The rule to follow is simple: -- cgit v1.2.3 From e58cdf585a38412f9f56c512d20e8e12637d9892 Mon Sep 17 00:00:00 2001 From: Stephen Boyd Date: Wed, 9 Sep 2015 16:46:38 -0700 Subject: locking/Documentation/lockstat: Fix typo - lokcing -> locking Signed-off-by: Stephen Boyd Signed-off-by: Peter Zijlstra (Intel) Cc: Andrew Morton Cc: Jiri Kosina Cc: Jonathan Corbet Cc: Linus Torvalds Cc: Paul E. McKenney Cc: Peter Zijlstra Cc: Thomas Gleixner Link: http://lkml.kernel.org/r/1441842398-25063-1-git-send-email-sboyd@codeaurora.org Signed-off-by: Ingo Molnar --- Documentation/locking/lockstat.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'Documentation') diff --git a/Documentation/locking/lockstat.txt b/Documentation/locking/lockstat.txt index 568bbbacee91..5786ad2cd5e6 100644 --- a/Documentation/locking/lockstat.txt +++ b/Documentation/locking/lockstat.txt @@ -12,7 +12,7 @@ Because things like lock contention can severely impact performance. - HOW Lockdep already has hooks in the lock functions and maps lock instances to -lock classes. We build on that (see Documentation/lokcing/lockdep-design.txt). +lock classes. We build on that (see Documentation/locking/lockdep-design.txt). The graph below shows the relation between the lock functions and the various hooks therein. -- cgit v1.2.3 From e3e72ab80a3fac0b88e07d358a2c75724ccd66b4 Mon Sep 17 00:00:00 2001 From: Peter Zijlstra Date: Fri, 18 Sep 2015 13:22:52 +0200 Subject: atomic: Implement atomic_read_ctrl() Provide atomic_read_ctrl() to mirror READ_ONCE_CTRL(), such that we can more conveniently use atomics in control dependencies. Since we can assume atomic_read() implies a READ_ONCE(), we must only emit an extra smp_read_barrier_depends() in order to upgrade to READ_ONCE_CTRL() semantics. Requested-by: Dmitry Vyukov Signed-off-by: Peter Zijlstra (Intel) Acked-by: Will Deacon Cc: Andrew Morton Cc: Linus Torvalds Cc: Paul E. McKenney Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: linux-kernel@vger.kernel.org Cc: oleg@redhat.com Link: http://lkml.kernel.org/r/20150918115637.GM3604@twins.programming.kicks-ass.net Signed-off-by: Ingo Molnar --- Documentation/memory-barriers.txt | 17 +++++++++-------- include/asm-generic/atomic-long.h | 3 ++- include/linux/atomic.h | 18 ++++++++++++++++++ 3 files changed, 29 insertions(+), 9 deletions(-) (limited to 'Documentation') diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt index 2ba8461b0631..41ffd7e9cdcf 100644 --- a/Documentation/memory-barriers.txt +++ b/Documentation/memory-barriers.txt @@ -637,7 +637,8 @@ as follows: b = p; /* BUG: Compiler and CPU can both reorder!!! */ Finally, the READ_ONCE_CTRL() includes an smp_read_barrier_depends() -that DEC Alpha needs in order to respect control depedencies. +that DEC Alpha needs in order to respect control depedencies. Alternatively +use one of atomic{,64}_read_ctrl(). So don't leave out the READ_ONCE_CTRL(). @@ -796,9 +797,9 @@ site: https://www.cl.cam.ac.uk/~pes20/ppcmem/index.html. In summary: - (*) Control dependencies must be headed by READ_ONCE_CTRL(). - Or, as a much less preferable alternative, interpose - smp_read_barrier_depends() between a READ_ONCE() and the + (*) Control dependencies must be headed by READ_ONCE_CTRL(), + atomic{,64}_read_ctrl(). Or, as a much less preferable alternative, + interpose smp_read_barrier_depends() between a READ_ONCE() and the control-dependent write. (*) Control dependencies can order prior loads against later stores. @@ -820,10 +821,10 @@ In summary: and WRITE_ONCE() can help to preserve the needed conditional. (*) Control dependencies require that the compiler avoid reordering the - dependency into nonexistence. Careful use of READ_ONCE_CTRL() - or smp_read_barrier_depends() can help to preserve your control - dependency. Please see the Compiler Barrier section for more - information. + dependency into nonexistence. Careful use of READ_ONCE_CTRL(), + atomic{,64}_read_ctrl() or smp_read_barrier_depends() can help to + preserve your control dependency. Please see the Compiler Barrier + section for more information. (*) Control dependencies pair normally with other types of barriers. diff --git a/include/asm-generic/atomic-long.h b/include/asm-generic/atomic-long.h index c7a5c1ad2ed3..8942cdc676a7 100644 --- a/include/asm-generic/atomic-long.h +++ b/include/asm-generic/atomic-long.h @@ -35,7 +35,7 @@ typedef atomic_t atomic_long_t; #endif #define ATOMIC_LONG_READ_OP(mo) \ -static inline long atomic_long_read##mo(atomic_long_t *l) \ +static inline long atomic_long_read##mo(const atomic_long_t *l) \ { \ ATOMIC_LONG_PFX(_t) *v = (ATOMIC_LONG_PFX(_t) *)l; \ \ @@ -43,6 +43,7 @@ static inline long atomic_long_read##mo(atomic_long_t *l) \ } ATOMIC_LONG_READ_OP() ATOMIC_LONG_READ_OP(_acquire) +ATOMIC_LONG_READ_OP(_ctrl) #undef ATOMIC_LONG_READ_OP diff --git a/include/linux/atomic.h b/include/linux/atomic.h index 29dafa184aeb..e326469bb9d6 100644 --- a/include/linux/atomic.h +++ b/include/linux/atomic.h @@ -4,6 +4,15 @@ #include #include +#ifndef atomic_read_ctrl +static inline int atomic_read_ctrl(const atomic_t *v) +{ + int val = atomic_read(v); + smp_read_barrier_depends(); /* Enforce control dependency. */ + return val; +} +#endif + /* * Relaxed variants of xchg, cmpxchg and some atomic operations. * @@ -455,6 +464,15 @@ static inline int atomic_dec_if_positive(atomic_t *v) #include #endif +#ifndef atomic64_read_ctrl +static inline long long atomic64_read_ctrl(const atomic64_t *v) +{ + long long val = atomic64_read(v); + smp_read_barrier_depends(); /* Enforce control dependency. */ + return val; +} +#endif + #ifndef atomic64_andnot static inline void atomic64_andnot(long long i, atomic64_t *v) { -- cgit v1.2.3