Chapter 93 Body Decoding: Streaming Content-Encoding Restoration

Chapter Orientation

Chapter 91 solved "the body's boundaries"; this chapter solves "the body's content encoding". What the framing layer reads out of a Content-Encoding: gzip response is compressed bytes — the business wants plaintext. xrtHttpDecodeCreate builds a streaming decoder directly from the response head fields (auto-chaining multi-level encodings — declarations like gzip, br), xrtHttpDecodeWrite feeds segment by segment with decoded plaintext pushed to a callback, and xrtHttpDecodeDone verifies the ending (gzip's CRC and length trailer — against silent corruption). The design posture matches Chapter 91: segments decoded as they arrive, plaintext pushed straight to the consumer — no full-response buffering anywhere; memory usage is independent of body size. Paired with the send side's content negotiation (Accept-Encoding's q-values), this completes the compression loop and closes Volume 9's HTTP core.

93.1引入

Downloading a 50 MB gzip-compressed JSON: the naive implementation is "receive the entire response → decompress in one block → get 80 MB of plaintext" — 130 MB peak memory, and processing starts only after the last byte arrives. The streaming implementation is "each compressed segment that arrives → immediately decode a stretch of plaintext → push to the parser" — constant memory (on the order of one sliding window), first-byte latency roughly the network's first segment. The difference is not the gzip algorithm (Chapter 29's decompressor is already streaming) but the wiring: who connects "the segments the HTTP framing layer emits" to "the decompressor's input", who handles the chaining of multi-level encodings, and who verifies the ending?

The http_decode module is that wire: its input end is the Data view Chapter 91's BodyRead emits, its output end your callback, and in between it auto-chains every level the Content-Encoding declares. Whatever the head field declares is what gets decoded — including identity (no decode) and multi-level chains; a declared-but-not-compiled encoding (like br) fails creation explicitly — never a silent skip (silent = handing compressed data to the business as plaintext; everything downstream is wrong).

93.2概念

Built from the head: Create's contract

xrtHttpDecodeCreate(字段数组, 数量, 可选配置) (field array, count, optional config) scans the Content-Encoding field (multiple entries chained in order of appearance) and builds the decoder: gzip (Chapter 29's Deflate family's gzip container), deflate (zlib wrapping), identity (pass-through), and multi-level chains (even the pathological but legal declaration of gzip over gzip is handled correctly). No Content-Encoding field = pass-through.Failure semantics: an unknown or not-compiled encoding makes creation return NULL — the caller decides on degradation (discard the response) or an error; never "guess something close and decode anyway".

The configuration (optional; NULL takes defaults) distinguishes compat and safe modes: legacy servers (old deflate implementations especially) produce some non-conformant output; compat mode accepts it through a leniency window (browser-compatible behavior), while safe mode strictly rejects — the default takes the safe side, relaxed explicitly when talking to legacy backends. This is the standard compromise of the "strict parsing" discipline facing historical baggage: strict by default, leniency must be explicit.

Streaming feed: Write and the callback

The bFinal parameter marks "this is the last segment" — the gzip stream's terminating block (with CRC32 and original length) is verified here; Done may also be called separately after Write (some transports separate "feeding is done" from "the last segment").The callback shape's meaning: the decoder does zero allocation of any buffer for you — plaintext is pushed to you, and the buffering strategy (parse directly? write to a pipe? flush to disk?) is entirely yours. In the same lineage as Chapter 29's decompression callbacks and Chapter 91's Body views.

The loop with content negotiation

Decoding is the response side; the request side declares "what I can decode": Accept-Encoding: gzip;q=1.0, deflate (xrtHttpAcceptEncodingQuality queries the q-value — the thousandths-integer model, same as Chapters 91/92). The server picks a mutually acceptable encoding from the request head and writes it into Content-Encoding. The client's discipline: declare only what you can truly decode — writing br into Accept-Encoding without a compiled brotli decoder means the server picks br and your Create fails. The http_encoding module provides the request side's parsing and construction.

The composition panorama: one complete response's consumption pipeline

Four layers, each at its post, with only views and callbacks between layers — the complete skeleton Volume 9's first five chapters assemble; Chapter 98's xhttp runtime wraps it into high-level APIs.

93.3示例

First complete program: from response head to plaintext callback

The program below is from examples/http/decode/main.c — a 42-byte gzip frame stream decoded whole:

#include <xrt/http_decode.h>

#include <stdio.h>



/* 把解码片段直接写到标准输出，实际网络代码可改为业务消费回调。 */
static bool printBody(xbytesview Data, ptr pData)
{
	FILE* pOutput = (FILE*)pData;

	return fwrite(Data.Data, 1, Data.Size, pOutput) == Data.Size;
}





















 */


/* 展示从 HTTP/1 Header 建立流式 gzip 解码器。 */
int main(void)
{
	static const uint8 Gzip[] = {
		0x1F, 0x8B, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00,
		0x02, 0xFF, 0xCB, 0x48, 0xCD, 0xC9, 0xC9, 0x57,
		0x48, 0xCE, 0xCF, 0x2D, 0x28, 0x4A, 0x2D, 0x2E,
		0x4E, 0x4D, 0x51, 0x28, 0xCF, 0x2F, 0xCA, 0x49,
		0x01, 0x00, 0xA1, 0x2D, 0x94, 0x53, 0x16, 0x00,
		0x00, 0x00
	};
	static const xhttpfield Fields[] = {
		{
			XRT_STR_INIT("Content-Encoding"),
			XRT_STR_INIT("gzip")
		}
	};
	xhttpdecode* pDecode = xrtHttpDecodeCreate(Fields, 1, NULL);
	bool bSuccess;

	if ( pDecode == NULL ) {
		return 1;
	}
	bSuccess = xrtHttpDecodeWrite(
		pDecode,
		(xbytesview){ Gzip, sizeof(Gzip) },
		true,
		printBody,
		stdout
	) && xrtHttpDecodeDone(pDecode);
	xrtHttpDecodeDestroy(pDecode);
	return bSuccess ? 0 : 1;
}

What just happened. (1) The input is an embedded 42-byte gzip stream (starting with the 1F 8B magic) plus a Content-Encoding: gzip field — mocking "head array + compressed body" already emitted by Chapter 91. (2) DecodeCreate scans the field and builds the decoder: one level of gzip. (3) DecodeWrite feeds once (bFinal=true — this sample arrives in a single segment; streaming scenes pass false per segment, true on the last), and the decoded plaintext is pushed straight to the printBody callback — on the network you would swap in a JSON parser or pipe writer here, memory not growing with the body. (4) DecodeDone verifies the ending: a gzip stream whose trailing CRC32 and length fields don't match fails here — compressed data corrupted in transit: better an error than wrong data (silent corruption is the compression chain's sneakiest failure). (5) Destroy releases — the decoder is this module's only owning object.

Second complete program: the decoder full-feature tour

The second program is from examples/http/decode_tour/main.c — three lines covering modes, content mode, and reset:

 */

#include <stdio.h>
#include <string.h>
#include <xrt.h>

/* 一段 gzip 压缩的 "identity-passthrough-check"。 */
static const uint8 arrGzip[46] = {
	0x1F, 0x8B, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x0A,
	0xCB, 0x4C, 0x49, 0xCD, 0x2B, 0xC9, 0x2C, 0xA9, 0xD4, 0x2D,
	0x48, 0x2C, 0x2E, 0x2E, 0xC9, 0x28, 0xCA, 0x2F, 0x4D, 0xCF,
	0xD0, 0x4D, 0xCE, 0x48, 0x4D, 0xCE, 0x06, 0x00, 0x9A, 0xFB,
	0x0B, 0xA2, 0x1A, 0x00, 0x00, 0x00
};

/* 输出回调：累计并比对内容。 */
typedef struct exampleout {
	size_t iBytes;
	char arrText[64];
} exampleout;

static bool exampleOutput(xbytesview Data, ptr pUserData)
{
	exampleout* pOut = (exampleout*)pUserData;

	if ( pOut->iBytes + Data.Size > sizeof(pOut->arrText) ) {
		return false;
	}
	memcpy(pOut->arrText + pOut->iBytes, Data.Data, Data.Size);
	pOut->iBytes += Data.Size;
	return true;
}

int main(void)
{
	xhttpdecodeconfig Compat;
	xhttpdecodeconfig Safe;
	xhttpdecode* pDecode = NULL;
	xhttpfield Fields[2];
	exampleout Out;
	int iResult = 1;

	/* ---- 两套配置 + gzip 字段建解码器。 ---- */
	xrtHttpDecodeConfigInit(&Compat);
	xrtHttpDecodeConfigInitSafe(&Safe);
	Fields[0].Name = XRT_STR_LITERAL("Content-Encoding");
	Fields[0].Value = XRT_STR_LITERAL("gzip");
	pDecode = xrtHttpDecodeCreate(Fields, 1u, &Compat);
	if ( (pDecode == NULL) ||
		/* 兼容配置不设输出上限；安全配置有显式上限。 */
		(Compat.OutputLimit != XHTTP_DECODE_OUTPUT_UNLIMITED) ||
		(Safe.OutputLimit == Compat.OutputLimit) ) {
		goto Cleanup;
	}
	printf("decode: config compat vs safe + create ok\n");

	/* ---- gzip 模式 + 输入输出计量。 ---- */
	memset(&Out, 0, sizeof(Out));
	if ( (xrtHttpDecodeMode(pDecode) != XHTTP_DECODE_CONTENT) ||
		!xrtHttpDecodeWrite(pDecode,
			(xbytesview) { arrGzip, sizeof(arrGzip) }, true,
			exampleOutput, (ptr)&Out) ||
		!xrtHttpDecodeDone(pDecode) ||
		(Out.iBytes != 26u) ||
		(memcmp(Out.arrText, "identity-passthrough-check",
			26u) != 0) ||
		(xrtHttpDecodeInputSize(pDecode) != 46u) ||
		(xrtHttpDecodeOutputSize(pDecode) != 26u) ) {
		goto Cleanup;
	}
	printf("decode: gzip content mode + input/output sizes ok\n");

	/* ---- Reset 换 identity 字段：窗口复用 + 透传模式。 ---- */
	Fields[0].Value = XRT_STR_LITERAL("identity");
	memset(&Out, 0, sizeof(Out));
	if ( !xrtHttpDecodeReset(pDecode, Fields, 1u, &Compat) ||
		(xrtHttpDecodeMode(pDecode) != XHTTP_DECODE_IDENTITY) ||
		!xrtHttpDecodeWrite(pDecode,
			(xbytesview) { arrGzip, 4u }, true,
			exampleOutput, (ptr)&Out) ||
		!xrtHttpDecodeDone(pDecode) ||
		(Out.iBytes != 4u) ||
		(xrtHttpDecodeInputSize(pDecode) != 4u) ) {
		goto Cleanup;
	}
	printf("decode: reset to identity passthrough ok\n");
	iResult = 0;

Cleanup:
	xrtHttpDecodeDestroy(pDecode);
	return iResult;
}

What just happened. (1) Line 1 contrasts the two configuration modes and verifies creation — the compat/safe divide is Chapter 91's "strict by default, leniency explicit". (2) Line 2 verifies the gzip content mode and input/output size accounting — input/output sizes give directly usable data for compression-ratio computation and monitoring metrics. (3) Line 3 resets to identity pass-through — the next response on the same connection may be uncompressed: the decoder state zeroes and switches modes, the object reused without rebuilding (one more notch down in per-response cost for high-connection services).

93.4契约

• Built from the head: DecodeCreate scans Content-Encoding (multiple entries chained in order); no field = pass-through; unknown/not-compiled encodings fail creation — no guessing, no silent skipping.
• Modes: safe default (strict); compat relaxes explicitly (a leniency window for legacy servers); a NULL config takes the safe default.
• Streaming: Write per segment (bFinal marks the last); plaintext pushed to the callback immediately; a failed callback fails that segment's processing — the decoder buffers nothing for you.
• Ending verification: Done verifies the gzip CRC and length trailer; failure = corrupted content; the correct action is discarding the whole response, not using a part.
• Reset: Reset zeroes and switches to identity — one object reused across multiple responses on a connection.
• Content negotiation: the Accept-Encoding declaration must match decoding capability; the http_encoding module (AcceptEncodingParse/Quality/Select) provides the request side's parsing/construction; q thousandths integers.
• Object model: Create/Destroy owning (this module's only allocation point); Write/Done allocate nothing extra.
• Trimming: XRT_MODULE_HTTP_DECODE depends on COMPRESS and the HTTP foundation — non-decompressing clients trim it at zero cost.
• Composition boundary: upstream connects to Body's Data view (Chapter 91); downstream to your callback — zero-copy runs through the layers.

Common Mistakes

while ( read_chunk(&Data) ) {
	DecodeWrite(pDec, Data, true, cb, ctx);  /* every segment treated as the last */
}

while ( (St = xrtHttp1BodyRead(&Body, In, bEof,
		&iUsed, &Data, NULL)) != XHTTP1_BODY_DONE ) {
	if ( St != XHTTP1_BODY_DATA ) { return false; }
	/* last segment: the Body layer is about to DONE (next round is DONE) */
	DecodeWrite(pDec, Data, false /* except the true last */, cb, ctx);
}
xrtHttpDecodeDone(pDec);   /* verify the decode layer after the framing layer closes */

/* the callback writes plaintext straight into the database */
static bool onPlain(xbytesview Data, ptr pCtx) {
	db_append((db*)pCtx, Data);   /* when Done fails, it's already written: dirty data */
	return true;
}

/* the callback writes into voidable staging (file/memory segment); commit once Done passes */
static bool onPlain(xbytesview Data, ptr pCtx) {
	staging_append((staging*)pCtx, Data);
	return true;
}
/* ... */
if ( !xrtHttpDecodeDone(pDec) ) {
	staging_discard(&Staging);   /* void the whole: a clean failure */
} else {
	staging_commit(&Staging);
}

/* a "standard header" copied from somewhere */
send_header("Accept-Encoding: gzip, deflate, br, zstd");
/* the server picks br -> Create fails */

/* declaration matches capability: this build supports gzip/deflate */
send_header("Accept-Encoding: gzip, deflate");
/* br not declared -> the server won't pick it */

Programming Exercises

Summary