Haiku's Scheduler: Priorities, Real-Time Work, and Responsive Multimedia

Haiku’s scheduler decides which runnable thread receives a CPU, but it cannot make blocked code runnable or manufacture processing capacity. That simple boundary explains many misleading performance diagnoses. A media callback that waits behind a lock is not fixed by adding cores, and a worker asleep on I/O does not consume a time slice. Conversely, a busy polling loop remains runnable and competes for CPU even when it accomplishes nothing useful.

The public Kernel Kit exposes thread priorities and states so applications can communicate intent and diagnostic tools can report what occurred. The kernel’s current scheduler implementation adds multiprocessor placement, load tracking, latency policy, and power considerations behind that API. Applications should rely on the documented contract rather than guessing a private queue algorithm from an old BeOS description.

Runnable, running, and waiting are different states

A thread executes only while it is running on a CPU. A ready thread can run but is waiting for the scheduler to select it. A thread waiting for a semaphore, port message, condition, timer, or I/O completion should not compete as ready work until the event that can satisfy it occurs. Haiku’s thread_info API reports documented states including running, ready, receiving, asleep, suspended, and waiting.

This is why total CPU percentage is insufficient evidence. A desktop can feel frozen at modest aggregate utilization if a window thread waits on a lock held by another blocked thread. A single infinite loop can saturate one core while the rest of the machine appears lightly loaded. A real-time producer may wake at the right moment but miss its deadline because a dependency is late.

Blocking is desirable when a thread has no work. A port read, semaphore acquisition, timed wait, or message loop lets the kernel remove the thread from runnable competition and wake it when relevant state changes. Polling a flag repeatedly wastes CPU, heats the system, and can delay the work the polling thread hopes to observe. If periodic behavior is required, use a timer or bounded sleep with a clearly stated timing tolerance rather than a zero-delay loop.

Priorities express urgency, not importance

Haiku’s public OS.h defines named priority levels. B_LOW_PRIORITY is intended for background work, B_NORMAL_PRIORITY for ordinary threads, and B_DISPLAY_PRIORITY for user-interface work. Higher named levels include urgent display and real-time display priorities, then urgent and real-time priorities. The header also marks the beginning of the real-time range.

A numeric priority tells the scheduler which runnable work is more urgent; it does not declare that one application’s data matters more to the user. A backup is important but usually not latency-sensitive. An audio callback may handle only a small buffer yet have a narrow deadline. Choosing priorities by deadline and interaction requirements produces better behavior than marking every substantial feature “real time.”

The Kernel Kit’s spawn_thread() requires a priority for a newly created native thread, and set_thread_priority() can change an existing thread. That power creates responsibility. Raising a CPU-bound thread above display work can make the interface unresponsive. Raising many competing threads together changes little about their relative order while increasing the work they can starve.

Use the lowest priority that meets a measured requirement. Start ordinary workers at normal priority, background indexing or maintenance below interactive work, and reserve real-time levels for short, bounded sections with demonstrated deadlines. Priority changes should be documented beside the timing reason and tested under load, not copied from another component by name.

Real-time code must bound every dependency

A real-time priority is not a guarantee that an operation finishes on time. The selected thread can still encounter cache misses, faults, device latency, kernel work, or synchronization. If it acquires a mutex held by a normal-priority worker that is itself descheduled or blocked on storage, the high-priority thread inherits the entire dependency chain as latency.

For audio and video pipelines, the deadline-facing callback should perform predictable work: transfer already prepared buffers, update small state, and signal downstream processing. Decoding a large frame, allocating unpredictable amounts of memory, reading from disk, resolving a hostname, or writing logs synchronously belongs outside that path. Producer workers can fill bounded queues ahead of time; the callback consumes without waiting on slow resources.

Queue depth is a latency trade-off. Too little buffering makes brief scheduling or I/O delays audible or visible. Too much buffering increases response delay and memory use. Measure underruns, callback execution time, and end-to-end latency together. A system with no underruns but several seconds of interaction lag is not a successful real-time design.

Locks need the same scrutiny. Keep critical sections short, avoid calling unknown or blocking code while holding a lock, and define a consistent acquisition order. If a high-priority path must share state with ordinary work, prefer snapshots, lock-free handoff where correctness permits, or a narrowly scoped synchronization protocol whose worst case is known.

Multiprocessor scheduling adds placement decisions

On a multiprocessor system, the scheduler chooses both a thread and a CPU. Moving a thread can make another core available sooner, but migration may discard warm caches and add coordination overhead. Keeping a thread near the data it recently used can improve efficiency; balancing load prevents one CPU from carrying a long ready queue while another sits idle.

Haiku’s current scheduler source is organized into common scheduling code, per-CPU and per-thread structures, load tracking, run queues, tracing, and named low-latency and power-saving policy files. That source is the authoritative place to study implementation behavior at a particular revision. It should not be compressed into “strict priority queue” or “round robin” without qualifying the code revision and policy mode.

CPU affinity can be a measurement tool or specialized constraint, but pinning is not a default cure. An unnecessary affinity restriction removes placement choices from the scheduler and may concentrate related work on an overloaded CPU. Use it only when hardware access, reproducible benchmarking, or a verified cache/topology requirement justifies the loss of flexibility.

Power policy also interacts with latency. Keeping more CPUs active can reduce wake-up delay but consume more energy; consolidating work can let cores remain idle. A scheduler policy must negotiate those goals while respecting runnable priorities. Application code should state genuine deadlines and block when idle, giving the kernel accurate information rather than trying to micromanage every CPU.

Priority inversion and starvation

Priority inversion occurs when urgent work depends on a resource held by less urgent work. The visible symptom may be an audio glitch or stuck animation even though the high-priority thread exists and is ready to continue. The forensic question is not only “what priority is it?” but “what is it waiting for, who owns that object, and what is that owner waiting for?”

Starvation is the opposite scheduling risk: sufficiently favored runnable work prevents lower-priority work from making timely progress. A high-priority loop that never blocks is especially dangerous because it continuously presents eligible work. Even if the desktop retains some CPU on another core, shared locks or services can transmit the starvation across the system.

Changing all priorities upward obscures both conditions. It can move the bottleneck, change timing enough to hide a race, or starve a service outside the test. Preserve the original states, stacks, ownership, and timestamps before experimenting. Then change one relationship at a time and verify the deadline that motivated it.

Diagnose timing with thread-level evidence

Start with the exact symptom and time scale. A constant freeze, periodic stutter, single missed frame, and gradual drift imply different causes. Capture which team and thread should have produced the missing result. Haiku’s Kernel Debugging Land documentation describes teams, threads, semaphore inspection, and stack crawling; userland tools and debugger facilities can provide analogous evidence without entering the kernel debugger for every case.

For each relevant thread, record state, priority, CPU time, and stack. A waiting thread’s wait object matters more than its CPU percentage. Follow ownership until reaching the runnable or blocked thread at the end of the chain. For a busy thread, sample stacks to learn whether it performs useful work, spins on a condition, or repeatedly retries an error.

Measure callback duration and wake-up lateness separately. A callback that starts late points toward scheduling or an upstream signal. One that starts on time but finishes late points toward its own work or a dependency acquired after wake-up. Trace queue fill levels and underruns so the observed glitch can be aligned with producer, consumer, and device timing.

Finally, reproduce under controlled load and retain the scheduler mode, hardware topology, Haiku revision, and application priority settings. Scheduler internals evolve; evidence from one revision should not be presented as an eternal algorithm. The durable application guidance remains consistent: block when idle, keep deadline paths bounded, use priorities sparingly, and inspect actual wait chains before blaming CPU capacity.

Related:

Sources: