sched: Support POSIX's SCHED_RR scheduling policy #1338

nmeum · 2024-11-11T15:58:35Z

Previously, pull request #1223 added support for the SCHED_FIFO but didn't implement the SCHED_RR policy. This PR attempts to follow-up on that by proposing an implementation of SCHED_RR.

Time slicing support, as mandated by SCHED_RR, is implemented though the set_realtime_time_slice(duration) API added in the aforementioned pull request. Within the scheduler, the amount of time the thread ran so far is tracked if it exceeds the set duration, the thread is preempted (if there is a runnable thread of same or higher priority). The thread's run time is reset on preemption.

This is my first OSv pull request, I added some basic tests which I can expand further. Additionally, I checked that the tests added #1223 still pass. Further testing is definitely needed, hence this is a draft PR for now.

Let me know what you think.

Previously, pull request cloudius-systems#1223 added support for the SCHED_FIFO but didn't implement SCHED_RR. This PR attempts to follow-up on that by proposing an implementation of SCHED_RR. Time slicing support, as mandated by SCHED_RR, is implemented though the `set_realtime_time_slice(duration)` API added in the aforementioned pull request. Within the scheduler, the amount of time the thread ran so far is tracked if it exceeds the set duration, the thread is preempted (if there is a runnable thread of same or higher priority). The thread's run time is reset on preemption. Signed-off-by: Sören Tempel <tempel@ibr.cs.tu-bs.de>

core/sched.cc

nmeum · 2024-11-11T16:00:49Z

core/sched.cc

+        // p is no longer running, if it has a realtime slice reset it.
+        if (p->_realtime.has_slice()) {
+            p->_realtime.reset_slice();
+        }


It is not entirely clear to me if the slice should be reset if the thread is no longer runnable (e.g. because of blocking I/O). POSIX does not explicitly describe when the slice should be reset.

Yes, I'm also not sure, but think this if is right. I think the idea of the time slice is to make sure that a single thread never in its priority group never runs more than 1ms (for example) without letting other threads in its group run. But if the thread blocks or yields voluntarily (I believe this if covers both cases, right?), then it gives some other thread a chance to run and it too has a chance to run for a whole time-slice, so it's only fair that this thread's time slice is reset to zero. I think.
I tried searching if anybody mentions this question, and couldn't find such a discussion.

Unless I am missing something, Linux only seems to reset the time slice once it expires not when the thread is no longer runnable. AFAIK This is the only place in the rt scheduler where it is reset: https://github.com/torvalds/linux/blob/v6.16/kernel/sched/rt.c#L2580-L2583

wkozaczuk

Your code change looks good. I did ask some questions though.

wkozaczuk · 2024-11-22T21:09:43Z

core/sched.cc

+                // If the threads have the same realtime priority, then only reschedule
+                // if the currently executed thread has exceeded its time slice (if any).
+                if (t._realtime._priority == p->_realtime._priority &&
+                    ((!p->_realtime.has_slice() || p->_realtime.has_remaining()))) {


So all this means we are going to keep the current thread p running should stay running if _time_slice is 0 (run 'forever' until yields or waits) OR there is still time left to run per its _time_slice, right?

[…] should stay running if _time_slice is 0 (run 'forever' until yields or waits) OR there is still time left to run per its _time_slice, right?

Yes p->_realtime.has_slice() checks if it has a time slice (i.e. _time_slice != 0), p->_realtime.has_remaining() checks if there is still time remaining on the slice (if it has one).

Note that, even if the thread has exceeded its time slice it may still be selected to run again if there is no thread with a higher priority. Hence, the priority comparison in the if condition.

wkozaczuk · 2024-11-22T21:13:59Z

core/sched.cc

+                enqueue(*p);
+                p->_realtime.reset_slice();
+            } else {
+                // POSIX requires that if a real-time thread doesn't yield but


So this means that if the current thread p _time_slice is 0 OR p still has some remaining time to run, we will call enqueue_first_equal(). Is this correct?

Yes, I think it's correct. If we got here it means p was preempted. If it still has remaining time, it means it was preempted by a higher-priority realtime thread but when that higher-priority thread doesn't want to run, this thread p should continue running and continue its current time slice. The documentation says: "A SCHED_RR thread that has been preempted by a higher priority thread and subsequently resumes execution as a running thread will complete the unexpired portion of its round-robin time quantum.". It should be the first one in its priority group to run (and therefore enqueue_first_equal()) just like when no time slices existed.

core/sched.cc

wkozaczuk · 2024-11-22T22:15:33Z

@nyh What do you think?

nyh

Looks very nice to me, and appears correct although I'm a bit worried that I'm rusty in this code and might have missed something. I only left a few minor comments/requests.

I just realized that we never actually implemented the POSIX API for these features ;-) I we think we have such patches in #386 and maybe it will be nice to revive them.

core/sched.cc

nyh · 2024-11-24T15:46:42Z

core/sched.cc

+                enqueue(*p);
+                p->_realtime.reset_slice();
+            } else {
+                // POSIX requires that if a real-time thread doesn't yield but


Yes, I think it's correct. If we got here it means p was preempted. If it still has remaining time, it means it was preempted by a higher-priority realtime thread but when that higher-priority thread doesn't want to run, this thread p should continue running and continue its current time slice. The documentation says: "A SCHED_RR thread that has been preempted by a higher priority thread and subsequently resumes execution as a running thread will complete the unexpired portion of its round-robin time quantum.". It should be the first one in its priority group to run (and therefore enqueue_first_equal()) just like when no time slices existed.

nyh · 2024-11-24T16:17:13Z

core/sched.cc

+        // p is no longer running, if it has a realtime slice reset it.
+        if (p->_realtime.has_slice()) {
+            p->_realtime.reset_slice();
+        }


Yes, I'm also not sure, but think this if is right. I think the idea of the time slice is to make sure that a single thread never in its priority group never runs more than 1ms (for example) without letting other threads in its group run. But if the thread blocks or yields voluntarily (I believe this if covers both cases, right?), then it gives some other thread a chance to run and it too has a chance to run for a whole time-slice, so it's only fair that this thread's time slice is reset to zero. I think.
I tried searching if anybody mentions this question, and couldn't find such a discussion.

core/sched.cc

tests/tst-thread-realtime.cc

nmeum · 2024-12-04T13:37:18Z

Thanks a lot to both of you for the detailed comments and feedback! I made some minor changes and left further comments above. I think the main two things that remain to be sorted out are:

Do we want want to support setting a realtime slice without a priority? If not, there are a few places where we should check if priority == 0 as pointed out above.
I believe the tests should be expanded a bit, if you have ideas regarding useful test cases for SCHED_RR let me know. Maybe there are also tests for this scheduling policy in some POSIXy open source operating system of choice that we can use as a source of inspiration.

nmeum · 2025-09-04T13:39:44Z

Rebased, removed the flaky test, made the other test a little more useful and added the priority == 0 check to has_slice() so threads without priority == 0 are never subject to the time slice.

nmeum commented Nov 11, 2024

View reviewed changes

core/sched.cc Show resolved Hide resolved

nmeum commented Nov 11, 2024

View reviewed changes

wkozaczuk reviewed Nov 22, 2024

View reviewed changes

nyh requested changes Nov 24, 2024

View reviewed changes

nmeum force-pushed the pr/sched-rr branch from 3c27113 to 72db4be Compare September 4, 2025 13:37

nmeum mentioned this pull request Sep 4, 2025

Build mode detection via NDEBUG has unintended side effects #1386

Closed

nmeum force-pushed the pr/sched-rr branch from 72db4be to a8eab20 Compare October 13, 2025 07:04

nmeum marked this pull request as ready for review October 13, 2025 07:06

Uh oh!

sched: Support POSIX's SCHED_RR scheduling policy #1338

Are you sure you want to change the base?

sched: Support POSIX's SCHED_RR scheduling policy #1338

Uh oh!

Conversation

nmeum commented Nov 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wkozaczuk left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

wkozaczuk commented Nov 22, 2024

Uh oh!

nyh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nmeum commented Dec 4, 2024

Uh oh!

nmeum commented Sep 4, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

nmeum commented Nov 11, 2024 •

edited

Loading