Chapter 56 Coroutines (Part 2): The Scheduler and Timers

Chapter Orientation

The middle coroutine chapter upgrades the first volume's manual three beats (Create/Resume/read-final-state) to scheduler custody: xrtCoSchedCreate builds the scheduler, xrtCoGo (or xrtCoSchedPost to submit a function) starts a coroutine, xrtCoSleep is the in-coroutine timed yield, xrtCoSchedRun drives the pump (until all coroutines finish) with PollFor/PollUntil stepping on event waits; the timer wheel is one with the scheduler (a Sleep's yield is woken by the timing wheel); one scheduler instance per thread is the standard model — the coroutine's thread affinity is satisfied naturally by the scheduler. After this chapter, business code almost never touches the first volume's primitives.

56.1引入

The first volume's scheduler prototype (three tasks alternating) left a question: who calls Resume? If the answer is "the main loop rotating by hand", then "which coroutine runs first, who wakes the one sleeping seconds, how new coroutines jump the queue" all must be hand-written — that "driving policy" code is the scheduler. Its three core decisions: the ready queue (Post enqueues, Run dequeues and drives — FIFO or by policy); the timer wheel (a sleeping coroutine hangs on the timing structure, expiry returns it to the ready queue — the answer to "who wakes it"); the pump loop (RunUntil drives with a deadline — the answer to "when to stop": return when the queue is empty and no timers pend, or the deadline hits).

The standard shape assembled from these mechanisms is one scheduler per thread + a pump loop: the thread's main loop is just while ( running ) SchedRunUntil(sched, deadline) — submitted coroutines get driven, Sleeps wake on time, external events (IO completion callbacks) Post new coroutines. Volume 7's network engine's event loop is this shape industrialized: IO event → wake the corresponding connection's coroutine → the pump loop continues.

56.2概念

The scheduler's core three

xrtCoGo is the standard entrance: creation and enqueueing in one step (most scenes need no Create-then-submit); xrtCoSchedPost submits ordinary functions (signature xcoschedpostproc — a light event, executed on submission); Post carries the backpressure cap (submissions fail when the CreateLimit cap is full — Chapter 21's backpressure valve, scheduler edition).Sleep's semantic points: called inside a coroutine (undefined on an ordinary thread); yields to the timing wheel, woken by the pump at expiry — the thread keeps driving others.The Park/Wake pair: Park suspends the current coroutine until Woken (waking from external threads is safe), cancelled, or the deadline — the standard channel for external events (IO completion) to wake coroutines.

The pump loop: Run and RunUntil

xrtCoSchedRun drives until all coroutines finish (the batch-task waiting shape); xrtCoSchedStep single-steps (drives at most one ready coroutine — the shape for embedding an existing event loop); PollFor/PollUntil wait for events and step within a deadline (waking as soon as an external Post arrives — the resident pump loop's standard beat: while(running) PollFor(sched, 10ms)). xrtCoSchedAlive reports the live-coroutine count (a monitoring metric); xrtCoSchedClose requests cancel-all and stops accepting (the scheduler's shutdown entrance — Chapter 54's cancellation tree's upper button). The pump loop's two host shapes: resident thread — the while(running) PollFor(sched, 10ms) loop, external events waking via Post (the network-engine shape); embedded — an outer event loop manually calling Step/PollFor (a GUI framework's idle time slices — embedding the scheduler into an existing loop).

The timer family

Timing capability arrives in three families: xrtCoSleep (in-coroutine sleep), xrtCoParkFor/ParkUntil (suspend until a deadline or Wake — external event and timer, whichever first), and delayed execution of submissions (a Posted function executes on the next pump beat — a natural "next frame" semantics). All run on the same pump — timed wakeups and coroutine resumption serialize on the same thread (the single-threaded scheduler's zero-lock dividend).

One per thread: the model and its reasons

The standard model is one scheduler instance per thread. Three layers of reasons: the coroutine's thread affinity (the first volume's hard law) is satisfied automatically by "coroutines run only in their owning thread's scheduler"; a scheduler within one thread is zero-lock (queues and the timing wheel are touched only by the owning thread — Chapter 53's "when not to use locks" largest exemption zone); multi-thread scaling = multiple independent schedulers + cross-thread Post (submission is thread-safe — external events Post from any thread into the right thread's scheduler). This and Chapter 59's executor/task-pool "work-stealing multithreading" are two models — each with its home turf (the comparison in the next section).

Scheduler vs task pool: choosing between two models

The selection mnemonic: wait a lot — scheduler (suspension is cheap); compute a lot — task pool (throughput first); mixed loads use both (a CPU task running in the pool Posts back to the scheduler-side coroutine on completion — Chapter 58's continuation across models).

56.3示例

Complete program: three tasks alternating and the pump

From the repository sample examples/concurrency/coroutine_scheduler/main.c:

#include <xrt.h>
#include <stdio.h>



/* 示例任务异步等待后打印编号。 */
static ptr exampleTask(ptr pData)
{
	int iTask = (int)(intptr_t)pData;

	(void)xrtCoSleep((uint64)iTask * 1000u);
	printf("task %d\n", iTask);
	return pData;
}



/* 调度器 post 在所属线程中创建真正的协程任务。 */
static void examplePost(xcosched* pSched, ptr pData)
{
	(void)xrtCoGo(pSched, exampleTask, pData, NULL);
}





















 */


/* 先投递多个轻量任务，再运行到投递和协程全部完成。 */
int main(void)
{
	xcosched* pSched = xrtCoSchedCreate();

	if ( pSched == NULL ) {
		return 1;
	}
	for ( int i = 1; i <= 3; i++ ) {
		if ( !xrtCoSchedPost(
			pSched,
			examplePost,
			(ptr)(intptr_t)i
		) ) {
			return 2;
		}
	}
	if ( !xrtCoSchedRun(pSched) ) {
		return 3;
	}
	(void)xrtCoSchedDestroy(pSched);
	(void)xrtCoThreadDetach();
	return 0;
}

What just happened. (1) Three coroutines xrtCoGo into the scheduler — created and enqueued. (2) The pump (Run or PollFor stepping) drives: each driven to its suspension point, rotated, driven to completion — the output order of task 1/2/3 is decided jointly by the queue and the yield points (the full scheduling-policy tour is in the tour sample). (3) After all finish, xrtCoSchedRun returns; Alive reaching zero is assertable; the three coroutines' final states are each checkable. (4) Against the first volume's "manual three beats": this sample has no explicit Create/Resume — separating starting from driving is the scheduler's whole abstraction.

Complete program: timing and the pump loop

From examples/concurrency/coroutine_event/main.c — coroutine events and timed wakeups:

#include <stdio.h>
#include <xrt.h>



/* 生产者完成异步工作后置位事件。 */
static ptr producer(ptr pData)
{
	xcoevent* pEvent = (xcoevent*)pData;

	if ( xrtCoSleep(1) != XWAIT_OK ) {
		return NULL;
	}
	return xrtCoEventSet(pEvent) ? pEvent : NULL;
}



/* 消费者不阻塞调度线程地等待生产者。 */
static ptr consumer(ptr pData)
{
	xcoevent* pEvent = (xcoevent*)pData;

	if ( xrtCoEventAwait(pEvent) != XWAIT_OK ) {
		return NULL;
	}
	printf("event received\n");
	return pEvent;
}



















 */


/* 展示同一调度器中生产者和消费者的事件同步。 */
int main(void)
{
	xcoevent tEvent;
	xcosched* pSched;
	xcoro* pConsumer;
	bool bOkay;

	if ( !xrtCoEventInit(&tEvent, false, false) ) {
		return 1;
	}
	pSched = xrtCoSchedCreate();
	if ( pSched == NULL ) {
		(void)xrtCoEventUnit(&tEvent);
		return 1;
	}
	pConsumer = xrtCoSpawn(pSched, consumer, &tEvent, NULL);
	bOkay =
		(pConsumer != NULL) &&
		xrtCoGo(pSched, producer, &tEvent, NULL) &&
		xrtCoSchedRun(pSched) &&
		(xrtCoResult(pConsumer) == &tEvent) &&
		xrtCoDestroy(pConsumer);
	(void)xrtCoSchedDestroy(pSched);
	(void)xrtCoEventUnit(&tEvent);
	(void)xrtCoThreadDetach();
	return bOkay ? 0 : 1;
}

What just happened. (1) A coroutine suspends with xrtCoPark (waiting for an external wake); on timer expiry or a Wake call the coroutine returns to the ready queue — the timed/event wakeup path walked end to end (Park's return value distinguishes natural wake/cancel/deadline). (2) During the wait the pump loop drives other coroutines (or returns empty) — "concurrent waiting" on a single thread is the combination of yield + wakeup. (3) Timed wakeups and coroutine resumption serialize on the same thread — data passed between coroutines needs no lock (empirical proof of the single-thread scheduling dividend; cross-thread, only Post and Wake are the two safe entrances). The coroutine_tour sample is left as full-interface reading: scheduler configuration, policy variants, statistics — one assertion group each.

56.4契约

• Start/drive separation: Go/Post only enqueue, only the pump drives — submission and Wake are thread-safe (legal cross-thread); driving belongs solely to the scheduler's thread.
• Sleep/Park semantics: called inside coroutines (undefined on ordinary threads); Sleep on the timing wheel, Park waiting for Wake/cancel/deadline — neither sleeps the thread.
• Pump shapes: Run to all-finished; PollFor/Until time-bounded event stepping (the resident loop beat); Step single-step embedding.
• Zero-lock dividend: within a single-threaded scheduler, queues/timing wheel/callbacks are lock-free — only cross-instance cooperation needs synchronization (via Post or Chapter 21's queues).
• Model selection: wait a lot (IO-dense) — scheduler; compute a lot (CPU-dense) — task pool; mixed loads run the two models in concert.

From examples to engineering: three hosts of the scheduler

Network engine (Volume 7's home turf): a resident thread's PollFor(10ms) pump + IO completion callbacks Wake-ing connection coroutines — one coroutine per connection, straight-line read-write handling; the engine's event layer (from Chapter 63) shares the thread with the scheduler pump — event driving and coroutine resuming on one thread, zero locks.Async business orchestration: multi-step flows (check cache → query DB → write back) with each step Parking for an external event (DB callback Wakes) — straight-line code, asynchronous execution; against Future chains: coroutines have a stack and locals, state stays out of closures.The Actor model: one scheduler coroutine per Actor, messages submitted via Post (submission is "sending a letter") — single-coroutine handling guarantees lock-free Actors. The three hosts share one shape: the pump is the heart, Post/Wake the arteries, coroutines the cells — understand this circulation and you understand every use of the scheduler.

The shutdown path: the scheduler's drain

The scheduler's shutdown is where Chapter 49's signals and Chapter 54's cancellation meet in the coroutine world. The full path: external trigger (signal or management command) → xrtCoSchedClose (requests cancel-all coroutines and stops accepting — Chapter 54's cancellation tree's big button) → the pump keeps running until Alive reaches zero (each coroutine senses cancellation at its Park/Sleep/yield point and drains via its cleanup stack) → xrtCoSchedDestroy tears down. Three disciplines: after Close, the pump keeps running (cancellation is cooperative — no pump, no one drains); Alive at zero before Destroy (same root as the first volume's "Destroy succeeds only when finished"); time-bounded waiting (the pump waits for the drain under a deadline — on timeout, abandon grace, record the undrained count, force-quit — Chapter 49's five-second convention, scheduler edition).

An observation reminder: the scheduler's health metrics

The scheduler itself needs observation (Chapter 50's pipeline, sentry post in the concurrent world): Alive count (live coroutines — the yardstick of steady-state ceilings and leak drift), Post queue depth (backpressure level — CreateLimit overflow counts are overload signals), pump cycle time (one PollFor beat's actual duration — lengthening cycles mean some coroutine does heavy work before yielding — Pitfall 1's runtime fingerprint), Sleep/Park wakeup latency (timing-precision drift — evidence the pump rhythm is being dragged). Four metrics into structured logs (Chapter 38's fields), and the scheduler's behavior turns from a black box into curves — for concurrency's "feels wrong", read the curves before guessing at code.

Common Mistakes

static ptr worker(ptr pData)
{
	BlockingRead(Fd, Buf, Size);   /* blocking IO - not a Yield! */
	return NULL;                    /* the pump is stuck: every other coroutine waits */
}

static ptr worker(ptr pData)
{
	/* move blocking operations out of the scheduler: submit to the task pool (Chapter 59), Post back on completion */
	xrtTaskSubmit(gPool, blockingTask, pData, NULL);   /* CPU/IO task into the pool */
	xrtCoPark();   /* suspend - the pool's completion callback Wakes */
	/* the pool's callback Posts a new coroutine back to this scheduler (Chapter 58's continuation shape) */
	return NULL;
}

/* thread B */
/* directly manipulating scheduler internals - racing the pump loop, undefined */

/* thread B */
xrtCoSchedPost(gSched, shutdownProc, NULL);   /* Post a "shutdown function" */
/* the shutdown coroutine performs cleanup inside the scheduler's thread - the right thread doing the right thing */

Programming Exercises

Summary