Chapter 33 XLON Extended Serialization

Chapter Orientation

JSON can't express far too much: binary needs hand-rolled Base64, timestamps need string conventions, sets and integer-keyed maps degrade into arrays — every gap is one more layer of custom convention, and every convention layer is an interoperability mine. XLON is a strict superset of JSON: legal JSON is automatically legal XLON, plus four additional first-class types — bytes("...") (inline Base64), time("...") (ISO timestamps), set[...] (deduplicated sets), and intmap{...} (integer-keyed maps). The parser is the same one as Chapter 31's (event callbacks distinguish by type field; extended types no longer degrade into strings), and serialization writes the four types back as-is — full-type round trips at zero convention cost. When to use XLON and when to stick with JSON: this chapter gives the decision rules.

33.1引入

The true shape of one internal config: the certificate key is binary (in JSON, only a Base64 string — the consumer must know "this string needs decoding"); the expiry is a timestamp (string or number? whose timezone?); the allowed ports are a set (what about duplicates in a JSON array? does the consumer deduplicate?); the connection parameters are a port-number-to-description map (JSON object keys can only be strings — who converts between "80" and 80?). Four fields, four conventions, two pages of documentation — and someone still got it wrong.

XLON's answer is to make these four shapes first-class types: bytes("AAEC...") carries its own encoding declaration, time("2026-07-31T00:00:00Z") unifies on UTC milliseconds, set[80, 443] deduplicates by semantics, intmap{80: "http"} maps with native integer keys — the value tree's xvalue (Chapter 30) already has these four types; XLON simply stops serialization from flattening them. The superset design guarantees smoothness: existing JSON toolchains keep working, and the JSON portions of an XLON document are legal to any JSON parser.

33.2概念

The four extended types

All four types have counterparts in the value tree (Chapter 30): Bytes/Time values, set/intmap containers — XLON serialization is simply the complete mapping between "value tree ↔ text", losing no type information. The event stream (SAX) likewise distinguishes: xxlonevent's Type field marks extended types directly, so consumers never have to guess "is this string secretly a time".

What strict superset means

Three properties, worth stating separately. Forward compatible: any legal JSON document is legal XLON — renaming existing .json configs to .xlon costs nothing. Not backward compatible: an XLON document with extended types is illegal to a pure JSON parser (bytes( is not JSON syntax) — XLON belongs only to channels where "both ends are XRT or have agreed on XLON". Non-interfering: one parser suite handles both, parsing by actual document content — a JSON document's value tree contains no extended types, an XLON document's may; the config system maintains no second code path for the second format.

Selection rules against JSON

The rule's core variable is whether the receiver is controllable: controllable → XLON for full types; not → JSON for interoperability. Mixed shapes are common too: the public API's fields use JSON types while the internal processing representation uses value trees (which always carry full types) — convert at the boundary, don't compromise inside.

Error location and strictness

XLON inherits all of Chapter 31's strictness: resource-limit three gates, duplicate-key rejection, errors with line/column positions. Extended-type parse errors are likewise located to the character — the illegal month in time("2026-13-01...") is pinpointed on the spot.

33.3示例

Complete program: a full-type round trip

From the repository sample examples/data/xlon/main.c — the four extended types written and read in one pass:

 */

#include <inttypes.h>
#include <stdio.h>

#include <xrt.h>







 */
static xxlonvisitaction printXlonEvent(
	const xxlonevent* pEvent,
	ptr pUserData
)
{
	(void)pUserData;
	if ( pEvent->Type == XXLON_EVENT_BYTES ) {
		printf("bytes = %u\n", (unsigned)pEvent->Value.Bytes.Size);
	} else if ( pEvent->Type == XXLON_EVENT_TIME ) {
		printf("time = %" PRId64 "\n", (int64)pEvent->Value.Time);
	}
	return XXLON_VISIT_NEXT;
}



int main(void)
{
	/* 四种扩展类型各出现一次的输入文档。 */
	static const char sInput[] =
		"{\"blob\":bytes(\"AAEC/w==\"),"
		"\"updated\":time(\"2026-07-31T08:00:00+08:00\"),"
		"\"roles\":set[\"reader\",\"writer\"],"
		"\"ports\":intmap{80:\"http\",443:\"https\"}}";
	xxlonreadconfig ReadConfig;
	xxlonwriteconfig WriteConfig;
	xxlonwriter* pWriter;
	xvalue* pRoot;
	str sText;
	size_t iSize;

	/* ---- 1) DOM 往返：解析 → 美化输出（time 已归一为 UTC Z 后缀）---- */
	pRoot = xrtXlonParse((xstrview){ sInput, sizeof(sInput) - 1u });
	if ( pRoot == NULL ) {
		return 1;
	}
	sText = xrtXlonStringify(pRoot, true, &iSize);
	xrtValueRelease(pRoot);
	if ( sText == NULL ) {
		return 2;
	}
	printf("%.*s\n", (int)iSize, sText);
	xrtFree(sText);

	/* ---- 2) SAX：同一输入走事件流，只处理扩展类型 ---- */
	xrtXlonReadConfigInit(&ReadConfig);
	if (
		xrtXlonVisit(
			(xstrview){ sInput, sizeof(sInput) - 1u },
			&ReadConfig,
			printXlonEvent,
			NULL
		) != XXLON_VISIT_DONE
	) {
		return 3;
	}

	




	 */
	xrtXlonWriteConfigInit(&WriteConfig);
	pWriter = xrtXlonWriterCreate(&WriteConfig);
	if (
		(pWriter == NULL) ||
		!xrtXlonWriterObject(pWriter) ||
		!xrtXlonWriterName(pWriter, XRT_STR_LITERAL("code")) ||
		!xrtXlonWriterInt(pWriter, 200) ||
		!xrtXlonWriterName(pWriter, XRT_STR_LITERAL("tags")) ||
		!xrtXlonWriterSet(pWriter) ||                     /* 开 set */
		!xrtXlonWriterString(pWriter, XRT_STR_LITERAL("xrt")) ||
		!xrtXlonWriterEnd(pWriter) ||                     /* 闭 set */
		!xrtXlonWriterEnd(pWriter) ||                     /* 闭对象 */
		!xrtXlonWriterFinish(pWriter)
	) {
		xrtXlonWriterFree(pWriter);
		return 4;
	}
	sText = xrtXlonWriterTake(pWriter, &iSize);
	xrtXlonWriterFree(pWriter);
	if ( sText == NULL ) {
		return 5;
	}
	printf("%.*s\n", (int)iSize, sText);
	xrtFree(sText);
	return 0;
}

What just happened. (1) Serialization segment: the value tree's four types written out as-is — bytes("AAEC/w==") with its encoding declaration, time(...) as an ISO string, set[...] and intmap{...} each with their own bracket semantics. Type information is visible in the text — readers of the document need no convention table. (2) Parsing segment: bytes = 4 — Base64 decoded back to 4 binary bytes; time = 1785456000000000 — the ISO string normalized to a UTC millisecond integer (Chapter 3's xtime measure). (3) The last line is compact serialization (not pretty) — the shape of set["xrt"] on one line; machine channels use compact, on-disk review uses pretty, the same choice as JSON's.

Complete program: file round trip and error location

From examples/data/xlon_tour/main.c:

 */

#include <stdio.h>
#include <string.h>
#include <xrt.h>

/* 输出回调：累计字节数。 */
typedef struct examplesink {
	size_t iBytes;
	bool bOk;
} examplesink;

static bool exampleSink(xbytesview Data, ptr pUserData)
{
	examplesink* pSink = (examplesink*)pUserData;

	pSink->iBytes += Data.Size;
	return pSink->bOk;
}

int main(void)
{
	static const char* sFile = "xlon_tour_tmp.xlon";
	static const char* sBad = "{\"a\":}";
	xxlonreadconfig ReadConfig;
	xxlonwriteconfig WriteConfig;
	xxlonlocation Location;
	xvalue* pDom = NULL;
	xvalue* pFileDom = NULL;
	xvalue* pSub = NULL;
	xxlonwriter* pWriter = NULL;
	examplesink Sink;
	str sText = NULL;
	FILE* pOut = NULL;
	int iResult = 1;

	/* ---- Valid 正反 + Read 带配置。 ---- */
	xrtXlonReadConfigInit(&ReadConfig);
	if ( xrtXlonValid((xstrview) { sBad, 6u }) ||
		!xrtXlonValid(XRT_STR_LITERAL("[1,2,3]")) ||
		((pDom = xrtXlonRead(XRT_STR_LITERAL(
			"{\"n\":7}"), &ReadConfig)) == NULL) ) {
		goto Cleanup;
	}
	printf("xlon: valid +/- and read(config) ok\n");

	/* ---- 错误定位：坏文档的 xrt.xlon 错误带文本位置。 ---- */
	{
		const xerror* pError;

		if ( (xrtXlonRead((xstrview) { sBad, 6u },
				&ReadConfig) != NULL) ) {
			goto Cleanup;
		}
		pError = xrtGetError();
		if ( (pError == NULL) ||
			!xrtXlonErrorLocation(pError, &Location) ||
			(Location.Line != 1u) ||
			(Location.Column == 0u) ) {
			goto Cleanup;
		}
	}
	printf("xlon: error location line=1 ok\n");

	/* ---- 文件双形态：手写落盘 → ParseFile/ReadFile 读回。 ---- */
	pOut = fopen(sFile, "wb");
	if ( (pOut == NULL) ||
		(fwrite("{\"f\":1.5}", 1u, 9u, pOut) != 9u) ) {
		goto Cleanup;
	}
	fclose(pOut);
	pOut = NULL;
	pFileDom = xrtXlonParseFile(sFile);
	if ( (pFileDom == NULL) ||
		(xrtXlonReadFile(sFile, &ReadConfig) == NULL) ) {
		goto Cleanup;
	}
	/* WriteFile/StringifyFile：原子替换后可再读回。 */
	xrtXlonWriteConfigInit(&WriteConfig);
	if ( !xrtXlonWriteFile(sFile, pDom, &WriteConfig) ||
		!xrtXlonStringifyFile(sFile, pDom, true) ||
		(xrtXlonParseFile(sFile) == NULL) ) {
		goto Cleanup;
	}
	printf("xlon: file roundtrip parse/read -> write/stringify ok\n");

	/* ---- 同步写出：Write 把 Value 树提交给回调。 ---- */
	memset(&Sink, 0, sizeof(Sink));
	Sink.bOk = true;
	if ( !xrtXlonWrite(pDom, &WriteConfig, exampleSink,
			(ptr)&Sink) ||
		(Sink.iBytes != 7u) ) { /* {"n":7} 恰七字节。 */
		goto Cleanup;
	}
	printf("xlon: sync write via callback ok\n");

	/* ---- 增量写入器：CreateSink + 全值域方法。 ---- */
	memset(&Sink, 0, sizeof(Sink));
	Sink.bOk = true;
	pWriter = xrtXlonWriterCreateSink(&WriteConfig, exampleSink,
		(ptr)&Sink);
	pSub = xrtXlonParse(XRT_STR_LITERAL("[true]"));
	if ( (pWriter == NULL) || (pSub == NULL) ||
		/* 顶层数组：混排七种值域 + 子树 + 标签。 */
		!xrtXlonWriterArray(pWriter) ||
		!xrtXlonWriterNull(pWriter) ||
		!xrtXlonWriterBool(pWriter, true) ||
		!xrtXlonWriterUInt(pWriter, 12u) ||
		!xrtXlonWriterFloat(pWriter, 0.5) ||
		!xrtXlonWriterBytes(pWriter,
			(xbytesview) { (const uint8*)"xy", 2u }) ||
		!xrtXlonWriterTime(pWriter, (xtime)1000000) ||
		!xrtXlonWriterTag(pWriter, XRT_STR_LITERAL("base64"),
			XRT_STR_LITERAL("eHl6")) ||
		!xrtXlonWriterValue(pWriter, pSub) ||
		!xrtXlonWriterEnd(pWriter) ||
		!xrtXlonWriterFinish(pWriter) ) {
		goto Cleanup;
	}
	/* 嵌套 intmap：整数键映射容器。 */
	pWriter = (xrtXlonWriterFree(pWriter), NULL);
	pWriter = xrtXlonWriterCreateSink(&WriteConfig, exampleSink,
		(ptr)&Sink);
	if ( (pWriter == NULL) ||
		!xrtXlonWriterIntMap(pWriter) ||
		!xrtXlonWriterKey(pWriter, 7) ||
		!xrtXlonWriterBool(pWriter, false) ||
		!xrtXlonWriterEnd(pWriter) ||
		!xrtXlonWriterFinish(pWriter) ||
		(Sink.iBytes < 20u) ) {
		goto Cleanup;
	}
	printf("xlon: sink writer full value domain ok\n");
	iResult = 0;

Cleanup:
	if ( pOut != NULL ) {
		fclose(pOut);
	}
	xrtXlonWriterFree(pWriter);
	xrtValueRelease(pSub);
	xrtValueRelease(pFileDom);
	xrtValueRelease(pDom);
	xrtFree(sText);
	(void)remove(sFile);
	return iResult;
}

What just happened. (1) Validation and reading: legality validation of an XLON document and config-field reading in one pass. (2) Error location: feed input with a missing comma and line=1 reports precisely — Chapter 31 JSON's locating ability inherited verbatim. (3) File round trip: parse → read fields → write back → stringify, a closed loop verifying no type is lost. (4) Callback-style writing: serialization output is handed over chunk by chunk through a callback — the shape of streaming to disk or wiring straight to a sink, identical to Chapter 31's Writer sink. (5) The sink write covers the whole value domain: scalars and all four extended types each take a turn through the sink — full types are expressible not only in DOM; streaming writes lose no type either. The five "ok"s together prove: "XLON has the same engineering capability set as JSON, only with more types".

33.4契约

• Strict superset: legal JSON is necessarily legal XLON; one parser suite; JSON documents contain no extended types.
• Four-type round trip: bytes/time/set/intmap serialize and parse back as-is, at zero convention cost.
• Time normalization: time(...) parses to normalized UTC milliseconds (the xtime measure); serialization writes ISO 8601.
• Selection rule: receiver controllable → XLON for full types; not → JSON for interoperability; convert at boundaries, don't compromise inside.
• Inherited strictness: three gates, duplicate-key rejection, line/column error location — fully consistent with JSON.
• Compact and pretty: compact for machine channels, pretty for on-disk review — the same selection rule as JSON's.

From examples to engineering: three hosts of XLON

Internal configs (the main battleground): load at startup → value tree → Chapter 30's merging — binary keys, expiry times, permission sets, and port maps all first-class types, zero conversions in consuming code. This is XLON's direct target scenario. Internal service bus: message serialization between services — both ends are XRT, and the full-type round trip deletes the codec layers on both sides; combined with Chapter 21's queues, choose per channel between passing value trees directly (no serialization) and transmitting XLON-serialized (across processes). The downgrade exit at outer boundaries: when JSON output is required, downgrade by convention (Pitfall 1's good form) — downgrade logic concentrates in the boundary module while internal value trees keep full types. The three hosts share one point: type information is always first-class internally, downgrading only at explicitly marked boundaries.

A comparison: the same config in two spellings

Write the "certificate config" in both formats and the differences leap out. JSON shape: {"key": "AAEC...", "expire": "2026-07-31T00:00:00Z", "ports": [80, 443, 80]} — the key needs documentation saying it's Base64, expire needs an ISO-and-timezone convention, the ports array carries duplicates and the consumer must deduplicate. XLON shape: {"key": bytes("AAEC..."), "expire": time("2026-07-31T00:00:00Z"), "ports": set[80, 443]} — three self-explanatory fields, one line of exact reading each in consuming code. The comparison's point is not "XLON is shorter" (it may be longer) but that conventions moved from documents into syntax — documents forget and get lost; syntax is enforced by the parser. That is the whole meaning of "first-class types".

Common Mistakes

/* return XLON serialization directly on an external API */
str sBody = xrtXlonStringify(pResponse, false, NULL);
HttpReply(200, sBody);   /* client JSON parser: syntax error */

/* JSON only on external channels: extended types convert explicitly at the boundary */
str sBody = xrtJsonStringify(pResponse, false, NULL);
/* bytes/time converted to the agreed JSON forms before the exit (field docs spell it out) */

xstrview When;
xrtValueGetString(pUpdated, &When);   /* the type is Time - the exact read fails */

xtime When;
xrtValueGetTime(pUpdated, &When);     /* GetTime reads the millisecond integer */
/* for display formatting: format it yourself (Chapter 42's time module) */

Programming Exercises

Summary