Chapter 135 Embedding into a Host Program

Chapter Orientation

"Embedding" is a first-class viewpoint in XRT's design — every chapter's APIs live under the assumption of "being part of a host program". This chapter gathers the integration points scattered across chapters into one manual: memory (xrtSetAllocator takes over as soon as main enters — afterwards every library allocation goes to the host; first-allocation locking keeps it stable); errors (mapping the thread error slot into the host's exception system — every "return value announcing failure" is a candidate source of host exceptions; error objects can travel across layers); trimming (the host selects only the root modules it needs — Chapter 131's trim semantics applied host-side; the minimal integration never silently grows); observability (memory_stats' near-zero-cost always-on counters — the host's health panel reads them directly); form selection (single-header/static/dynamic — Chapter 132's three consumer kinds as the host's decision). Embedding's highest goal: the library is the host's component, not a parallel kingdom — memory under your governance, errors translated by you, size decided by you.

135.1引入

The most common failure shape of host integration is "the library wants to be the master": malloc on its own initiative (the host's memory budget slips out of control), errors only printed (the host's error system can't catch them), the full payload dragged in (embedded can't hold it), global state deciding for itself (multi-instance/multi-version coexistence crashes). Every XRT integration point is a direct answer to these failures — and each is pinned by tests (allocator replacement has allocator_tour's dual-track verification, error mapping has _threads' slot-isolation verification, trimming has the size-profile assertions, stats has the overhead baseline).

First build the timeline view of takeover: main's first act swaps the allocator (before locking) → initialize host facilities → enable stats as needed → during business runtime the error slot travels with the thread → at teardown, a stats snapshot + Drain of all library objects. Wrong order (swapping after the first allocation) is exactly the rejection scenario of Chapter 5 — this chapter strings them into a timeline.

135.2概念

Memory takeover: the allocator and the lock

Common host allocator shapes: arena (request-scoped bump allocation — the whole block frees at request end, Chapter 7's host application), budget counting (a quota allocator — the library's memory ceiling belongs to host policy), proxy accounting (forwarding to the system allocator + bookkeeping — the host panel reads the library's consumption directly). The lock's meaning, stressed again: two library modules (or two places in host code) both try to swap the allocator — first come first served, the later one rejected — integration stability over flexibility.

Error mapping: from the slot into the host's system

XRT's failure report is "return value + thread error slot" (Chapter 4) — the host-side mapping asks three questions: where to intercept (check return values at every XRT call site — a macro/wrapper function intercepts uniformly); how to translate (xrtErrorKind → host exception type/error code; xrtErrorMessage → human message; the cause chain → log expansion); how to carry (across threads, first TakeError — Chapter 4's carrying semantics honored inside the host's task queue). Slot isolation: host threads each hold their own slot — the isolation verified by _threads tests is exactly the host's concurrency-safety guarantee.

Trimming and size: the host's right to choose

The host declares root modules (Chapter 131) — a "containers + strings only" host artifact contains no network symbol whatsoever (asserted by forbid_symbols). Evolution safety: no selection = Core — XRT adding modules in the future never silently enlarges the host artifact (the contract states this rule's purpose explicitly). Multi-TU consistency: all the host's TUs use a consistent module set (at modular link time) — the build system defines it in one place (CMake's target_compile_definitions declared once).

Observability: stats always on in production

memory_stats's (introduced in Chapter 6) host-side role is the health panel's data source: xrtMemStatsEnable(true) once, periodic Snapshot reads — malloc_calls (business allocation count) / pooled (pool hits) / direct (pass-through) / backing (calls reaching the system) — four readings forming memory health's minimal metric set. Near-zero overhead is the premise of always-on (counting only, no records — contrast memory_debug's full-record high cost); leak detection (live never returning to zero) goes into long-running services' alarm thresholds.

The host's three-form decision

Single header (scripts/prototypes — one include) / static library (real projects — link-frozen) / dynamic library (plugin hosts — independent upgrades) — Chapter 132's consumer choice restated in host context as an integration decision: the host build system's hookup cost and the distribution form decide; behavioral consistency (asserted by verify) makes switching imperceptible.

135.3示例

First complete program: the always-on stats health panel

The program below is from examples/memory/stats — an observability sample in production form:

 */

#include <stdio.h>

#include <xrt.h>



int main(void)
{
	xmemstats tStats;
	ptr pSmall;
	ptr pLarge;

	/* 开关可查询：先确认当前状态，再打开并清零。 */
	printf("before=%d\n", xrtMemStatsEnabled() ? 1 : 0);
	xrtMemStatsEnable(true);
	printf("after=%d\n", xrtMemStatsEnabled() ? 1 : 0);
	xrtMemStatsReset();

	/* 一次小块（走池）+ 一次大块（直通）。 */
	pSmall = xrtMalloc(48);
	pLarge = xrtMalloc(4096);
	if ( (pSmall == NULL) || (pLarge == NULL) ) {
		xrtFree(pSmall);
		xrtFree(pLarge);
		return 1;
	}
	xrtFree(pSmall);
	xrtFree(pLarge);

	/* 读取窗口内计数（字段语义见文件头读数解释）。 */
	xrtMemStatsGet(&tStats);
	printf("malloc_calls=%llu pooled=%llu direct=%llu backing=%llu\n",
		(unsigned long long)tStats.MallocCalls,
		(unsigned long long)tStats.PooledAllocCalls,
		(unsigned long long)tStats.DirectAllocCalls,
		(unsigned long long)tStats.BackingAllocCalls);

	/* 用完关闭，回到零开销模式。 */
	xrtMemStatsEnable(false);
	return 0;
}

What just happened. (1) xrtMemStatsEnable(true) + Reset — query the switch (before/after) then zero it: the observability discipline of confirm first, then act. (2) After two business allocations (a 48B small block + a 4096B large block), the snapshot reads: pooled=1 (small block hit the pool — zero system calls), direct=1 (large block over the threshold, passed through), backing=3 (3 actual on the system side — 1 pass-through + 2 pool-internal warm-ups) — the reconciliation of the business view with the system view is exactly the raw material of capacity planning. (3) The host panel periodically running "snapshot + read" is the entirety of library memory health monitoring — always-on costs nearly nothing (counter increments).

Second complete program: full verification of allocator takeover

The second program is from examples/core/allocator_tour — host memory takeover acceptance:

 */

#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <xrt.h>

/* 计数分配器：仍路由系统堆，便于验证替换生效。 */
typedef struct examplecount {
	size_t iAllocs;
	size_t iFrees;
} examplecount;

static ptr exampleAlloc(ptr pContext, size_t iSize)
{
	++((examplecount*)pContext)->iAllocs;
	return (ptr)malloc(iSize);
}

static ptr exampleRealloc(ptr pContext, ptr pMemory, size_t iSize)
{
	(void)pContext;
	return (ptr)realloc(pMemory, iSize);
}

static void exampleFree(ptr pContext, ptr pMemory)
{
	++((examplecount*)pContext)->iFrees;
	free(pMemory);
}

int main(void)
{
	xallocator Default;
	xallocator Custom;
	examplecount Count;
	uint8* pBlock;
	uint8* pDup;
	int iResult = 1;

	/* ---- 必须在任何 XRT 分配前完成替换。 ---- */
	xrtGetAllocator(&Default);
	memset(&Count, 0, sizeof(Count));
	Custom.Context = (ptr)&Count;
	Custom.Alloc = exampleAlloc;
	Custom.Realloc = exampleRealloc;
	Custom.Free = exampleFree;
	if ( !xrtSetAllocator(&Custom) ) {
		goto Cleanup;
	}

	/* ---- At 族五接口：位置参数不影响堆行为。 ---- */
	pBlock = (uint8*)xrtMallocAt(4u, __FILE__, __LINE__);
	if ( (pBlock == NULL) ||
		((pDup = (uint8*)xrtMemDupAt("hi", 2u, __FILE__,
			__LINE__)) == NULL) ||
		(memcmp(pDup, "hi", 2u) != 0) ||
		((pBlock = (uint8*)xrtReallocAt(pBlock, 16u, __FILE__,
			__LINE__)) == NULL) ||
		(xrtCallocAt(2u, 4u, __FILE__, __LINE__) ==
			NULL) ) {
		goto Cleanup;
	}
	/* Calloc 清零验证单独做——保留指针用于释放。 */
	{
		uint8* pZero = (uint8*)xrtCallocAt(2u, 4u, __FILE__,
			__LINE__);

		if ( (pZero == NULL) || (pZero[0] != 0u) ||
			(pZero[7] != 0u) ) {
			goto Cleanup;
		}
		xrtFreeAt(pZero, __FILE__, __LINE__);
	}
	xrtFreeAt(pDup, __FILE__, __LINE__);
	xrtFreeAt(pBlock, __FILE__, __LINE__);
	printf("allocator: swap -> at-family alloc/calloc/dup/realloc ok\n");

	
	 * 且 GetAllocator 仍报告已安装的计数分配器。 ---- */
	
	 * 分配才经过自定义 Alloc；计数只需 >= 1 即证明替换生效。 */
	if ( xrtSetAllocator(&Default) ||
		(Count.iAllocs < 1u) ) {
		goto Cleanup;
	}
	{
		xallocator Current;

		xrtGetAllocator(&Current);
		if ( (Current.Alloc != Custom.Alloc) ||
			(Current.Free != Custom.Free) ) {
			goto Cleanup;
		}
	}
	printf("allocator: locked after first alloc -> swap rejected ok\n");
	iResult = 0;

Cleanup:
	return iResult;
}

What just happened. (1) After the swap, all four at-family entrances verified (Alloc/Calloc/Dup/Realloc) — the complete face a host allocator must implement (not just Alloc — Realloc's semantics are often forgotten). (2) The second line's locking verification: a second SetAllocator rejected — the host integration's timing contract (must precede the first allocation) turned from error into tested fact. (3) These two samples together are host integration's "Hello World+": takeover (this one) + observability (the previous) — memory sovereignty and visibility arriving together.

The host checklist: the acceptance order of integration

Condense this chapter into a pre-launch checklist — tick items in dependency order: (1) allocator taken over at main's first line (all four at-family entrances implemented, locking semantics understood); (2) module trim declared (root-module list into the build system, size profile's forbidden symbols verified); (3) error-mapping layer in place (Kind translation table, cause chain into logs, the cross-thread TakeError path); (4) observability always on (stats enable + periodic snapshots + four-reading alarm thresholds); (5) diagnostic form decided (production EXCLUDE applied, debug builds switchable); (6) artifact form chosen (single-header/static/dynamic — hookup with the host build system verified); (7) teardown path rehearsed (all library objects Drained, final stats snapshot, error slots cleared). All seven ticked, and XRT merges into the host as a "component" — memory visible, errors translatable, size controllable, behavior verifiable — embedding's complete state is not "it compiles" but "sovereignty is clear".

135.4契约

• Takeover timing: SetAllocator before any XRT allocation (main's-first-line level); locked after the first allocation; all four at-family entrances implemented.
• Memory sovereignty: after takeover, every library allocation (pool internals included) goes to the host — budget/stats/arena belong to the host.
• Error mapping: check return values at call sites → translate Kind → display Message → cause chain into logs; across threads, TakeError first.
• Slot isolation: host threads each hold their own error slot — concurrency safety locked by _threads tests.
• Trim sovereignty: the host declares root modules; no selection = Core without growth; multi-TU sets consistent (one definition in the build system).
• Observability always on: stats counts only, near-zero cost; the four readings (calls/pooled/direct/backing) are health's minimal set.
• Form consistency: single-header/static/dynamic behavior asserted by verify — host switching at zero cost.
• Diagnostics removable: the MEMORY_DEBUG exclusion semantics (Chapter 131) — full production functionality minus diagnostics.

Common Mistakes

int main(void) {
	init_something_using_xrt();   /* already allocated inside */
	xrtSetAllocator(&MyAlloc);    /* rejected: the first allocation happened */
}

int main(void) {
	xrtSetAllocator(&MyAlloc);    /* the first act */
	init_something_using_xrt();   /* everything goes through the host allocator */
}

bool my_json_parse(str s, xvalue** out) {
	if ( !xrtJsonParse(...) ) { return false; }   /* all detail lost */
}

bool my_json_parse(str s, xvalue** out, my_error* e) {
	if ( !xrtJsonParse(...) ) {
		fill_host_error(e, xrtTakeError());  /* Kind/domain/chain fully translated */
		return false;
	}
}

#define XRT_MODULE_ALL   /* includes memory_debug - full detail recording in production */

#define XRT_MODULE_ALL
#define XRT_EXCLUDE_MEMORY_DEBUG   /* production: keep stats, drop detail */
/* debugging builds add XRT_MODULE_MEMORY_DEBUG explicitly */

Programming Exercises

Summary