summaryrefslogtreecommitdiff
path: root/include/linux/rcutiny.h
diff options
context:
space:
mode:
authorBoqun Feng <boqun.feng@gmail.com>2015-07-31 02:55:38 +0300
committerPaul E. McKenney <paulmck@linux.vnet.ibm.com>2015-10-06 21:08:23 +0300
commitbb73c52bad3666997ed2ec83c0c80c3f8ef55008 (patch)
tree5dd6150075147c23984f6c4902ba5fcde4612338 /include/linux/rcutiny.h
parentdb3e8db45e1cbf8cc22f1083a559d78eb91ccd63 (diff)
downloadlinux-bb73c52bad3666997ed2ec83c0c80c3f8ef55008.tar.xz
rcu: Don't disable preemption for Tiny and Tree RCU readers
Because preempt_disable() maps to barrier() for non-debug builds, it forces the compiler to spill and reload registers. Because Tree RCU and Tiny RCU now only appear in CONFIG_PREEMPT=n builds, these barrier() instances generate needless extra code for each instance of rcu_read_lock() and rcu_read_unlock(). This extra code slows down Tree RCU and bloats Tiny RCU. This commit therefore removes the preempt_disable() and preempt_enable() from the non-preemptible implementations of __rcu_read_lock() and __rcu_read_unlock(), respectively. However, for debug purposes, preempt_disable() and preempt_enable() are still invoked if CONFIG_PREEMPT_COUNT=y, because this allows detection of sleeping inside atomic sections in non-preemptible kernels. However, Tiny and Tree RCU operates by coalescing all RCU read-side critical sections on a given CPU that lie between successive quiescent states. It is therefore necessary to compensate for removing barriers from __rcu_read_lock() and __rcu_read_unlock() by adding them to a couple of the RCU functions invoked during quiescent states, namely to rcu_all_qs() and rcu_note_context_switch(). However, note that the latter is more paranoia than necessity, at least until link-time optimizations become more aggressive. This is based on an earlier patch by Paul E. McKenney, fixing a bug encountered in kernels built with CONFIG_PREEMPT=n and CONFIG_PREEMPT_COUNT=y. Signed-off-by: Boqun Feng <boqun.feng@gmail.com> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Diffstat (limited to 'include/linux/rcutiny.h')
-rw-r--r--include/linux/rcutiny.h1
1 files changed, 1 insertions, 0 deletions
diff --git a/include/linux/rcutiny.h b/include/linux/rcutiny.h
index c8a0722f77ea..4c1aaf9cce7b 100644
--- a/include/linux/rcutiny.h
+++ b/include/linux/rcutiny.h
@@ -216,6 +216,7 @@ static inline bool rcu_is_watching(void)
static inline void rcu_all_qs(void)
{
+ barrier(); /* Avoid RCU read-side critical sections leaking across. */
}
#endif /* __LINUX_RCUTINY_H */