diff options
| author | Peter Zijlstra | 2023-08-16 15:40:59 +0200 |
|---|---|---|
| committer | Peter Zijlstra | 2023-08-17 17:07:07 +0200 |
| commit | 63304558ba5dcaaff9e052ee43cfdcc7f9c29e85 (patch) | |
| tree | 3a8ca75f5776357159eed23696024bc5a4f448b7 /kernel | |
| parent | 7170509cadbb76e5fa7d7b090d2cbdb93d56a2de (diff) | |
sched/eevdf: Curb wakeup-preemption
Mike and others noticed that EEVDF does like to over-schedule quite a
bit -- which does hurt performance of a number of benchmarks /
workloads.
In particular, what seems to cause over-scheduling is that when lag is
of the same order (or larger) than the request / slice then placement
will not only cause the task to be placed left of current, but also
with a smaller deadline than current, which causes immediate
preemption.
[ notably, lag bounds are relative to HZ ]
Mike suggested we stick to picking 'current' for as long as it's
eligible to run, giving it uninterrupted runtime until it reaches
parity with the pack.
Augment Mike's suggestion by only allowing it to exhaust it's initial
request.
One random data point:
echo NO_RUN_TO_PARITY > /debug/sched/features
perf stat -a -e context-switches --repeat 10 -- perf bench sched messaging -g 20 -t -l 5000
3,723,554 context-switches ( +- 0.56% )
9.5136 +- 0.0394 seconds time elapsed ( +- 0.41% )
echo RUN_TO_PARITY > /debug/sched/features
perf stat -a -e context-switches --repeat 10 -- perf bench sched messaging -g 20 -t -l 5000
2,556,535 context-switches ( +- 0.51% )
9.2427 +- 0.0302 seconds time elapsed ( +- 0.33% )
Suggested-by: Mike Galbraith <umgwanakikbuti@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20230816134059.GC982867@hirez.programming.kicks-ass.net
Diffstat (limited to 'kernel')
| -rw-r--r-- | kernel/sched/fair.c | 12 | ||||
| -rw-r--r-- | kernel/sched/features.h | 1 |
2 files changed, 13 insertions, 0 deletions
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index f496cef90ce7..0b7445cd5af9 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -873,6 +873,13 @@ static struct sched_entity *pick_eevdf(struct cfs_rq *cfs_rq) if (curr && (!curr->on_rq || !entity_eligible(cfs_rq, curr))) curr = NULL; + /* + * Once selected, run a task until it either becomes non-eligible or + * until it gets a new slice. See the HACK in set_next_entity(). + */ + if (sched_feat(RUN_TO_PARITY) && curr && curr->vlag == curr->deadline) + return curr; + while (node) { struct sched_entity *se = __node_2_se(node); @@ -5167,6 +5174,11 @@ set_next_entity(struct cfs_rq *cfs_rq, struct sched_entity *se) update_stats_wait_end_fair(cfs_rq, se); __dequeue_entity(cfs_rq, se); update_load_avg(cfs_rq, se, UPDATE_TG); + /* + * HACK, stash a copy of deadline at the point of pick in vlag, + * which isn't used until dequeue. + */ + se->vlag = se->deadline; } update_stats_curr_start(cfs_rq, se); diff --git a/kernel/sched/features.h b/kernel/sched/features.h index 61bcbf5e46a4..f770168230ae 100644 --- a/kernel/sched/features.h +++ b/kernel/sched/features.h @@ -6,6 +6,7 @@ */ SCHED_FEAT(PLACE_LAG, true) SCHED_FEAT(PLACE_DEADLINE_INITIAL, true) +SCHED_FEAT(RUN_TO_PARITY, true) /* * Prefer to schedule the task we woke last (assuming it failed |