[tip:core/rcu] rcu: Improve srcu_readers_active_idx()' s cache locality

[tip-bot for Lai Jiangshan <laijs@xxxxxxxxxxxxxx>]

Commit-ID:  dc87917501e324701dbfb249def44054b5220187
Gitweb:     http://git.kernel.org/tip/dc87917501e324701dbfb249def44054b5220187
Author:     Lai Jiangshan <laijs@xxxxxxxxxxxxxx>
AuthorDate: Tue, 6 Mar 2012 17:57:34 +0800
Committer:  Paul E. McKenney <paulmck@xxxxxxxxxxxxxxxxxx>
CommitDate: Mon, 30 Apr 2012 10:48:24 -0700

rcu: Improve srcu_readers_active_idx()'s cache locality

Expand the calls to srcu_readers_active_idx() from srcu_readers_active()
inline.  This change improves cache locality by interating over the CPUs
once rather than twice.

Signed-off-by: Lai Jiangshan <laijs@xxxxxxxxxxxxxx>
Signed-off-by: Paul E. McKenney <paulmck@xxxxxxxxxxxxxxxxxx>
---
 kernel/srcu.c |    9 ++++++++-
 1 files changed, 8 insertions(+), 1 deletions(-)

diff --git a/kernel/srcu.c b/kernel/srcu.c
index e0139a2..a43211c 100644
--- a/kernel/srcu.c
+++ b/kernel/srcu.c
@@ -193,7 +193,14 @@ static bool srcu_readers_active_idx_check(struct srcu_struct *sp, int idx)
  */
 static int srcu_readers_active(struct srcu_struct *sp)
 {
-	return srcu_readers_active_idx(sp, 0) + srcu_readers_active_idx(sp, 1);
+	int cpu;
+	unsigned long sum = 0;
+
+	for_each_possible_cpu(cpu) {
+		sum += ACCESS_ONCE(per_cpu_ptr(sp->per_cpu_ref, cpu)->c[0]);
+		sum += ACCESS_ONCE(per_cpu_ptr(sp->per_cpu_ref, cpu)->c[1]);
+	}
+	return sum;
 }
 
 /**
--
To unsubscribe from this list: send the line "unsubscribe linux-tip-commits" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html