diff options
| author | Peter Zijlstra | 2020-10-01 15:54:14 +0200 |
|---|---|---|
| committer | Peter Zijlstra | 2020-11-10 18:39:00 +0100 |
| commit | 14e292f8d45380c519a83d9b0f37089a17eedcdf (patch) | |
| tree | 50b8a4826a5fa1a806304a497930b46e8282d0ae /lib/cpumask.c | |
| parent | 3015ef4b98f53fe7eba4f5f82f562c0e074d213c (diff) | |
sched,rt: Use cpumask_any*_distribute()
Replace a bunch of cpumask_any*() instances with
cpumask_any*_distribute(), by injecting this little bit of random in
cpu selection, we reduce the chance two competing balance operations
working off the same lowest_mask pick the same CPU.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>
Reviewed-by: Daniel Bristot de Oliveira <bristot@redhat.com>
Link: https://lkml.kernel.org/r/20201023102347.190759694@infradead.org
Diffstat (limited to 'lib/cpumask.c')
| -rw-r--r-- | lib/cpumask.c | 18 |
1 files changed, 18 insertions, 0 deletions
diff --git a/lib/cpumask.c b/lib/cpumask.c index 85da6ab4fbb5..35924025097b 100644 --- a/lib/cpumask.c +++ b/lib/cpumask.c @@ -267,3 +267,21 @@ int cpumask_any_and_distribute(const struct cpumask *src1p, return next; } EXPORT_SYMBOL(cpumask_any_and_distribute); + +int cpumask_any_distribute(const struct cpumask *srcp) +{ + int next, prev; + + /* NOTE: our first selection will skip 0. */ + prev = __this_cpu_read(distribute_cpu_mask_prev); + + next = cpumask_next(prev, srcp); + if (next >= nr_cpu_ids) + next = cpumask_first(srcp); + + if (next < nr_cpu_ids) + __this_cpu_write(distribute_cpu_mask_prev, next); + + return next; +} +EXPORT_SYMBOL(cpumask_any_distribute); |