Chapter 90 HTTP/1 Messages: Parsing and Packing

Chapter Orientation

On Chapter 89's foundation, this chapter assembles the complete HTTP/1 message layer: xrtHttp1RequestParse/xrtHttp1ResponseParse parse on-the-wire request/response heads into a "start line + field array" structure (three-state return, borrowed views), xrtHttp1ResponseWrite packs the status line and fields in one shot, and field lookup uses the foundation's xrtHttpFieldFind/FieldNext directly (case-insensitive search over the parsed field array). The design's keyword is the bound array: xhttp1head binds a caller-provided field array — one Head structure per thread, reused forever, zero per-request allocation; this is the foundation path of the xhttp runtime and homegrown gateways. Bodies (chunked and friends) are Chapter 91's framing topic; this chapter stops at "the empty line ending the head block".

90.1引入

An HTTP/1 server handles tens of thousands of requests per second, each head about 1 KB with a dozen-plus fields. If every parse allocates a field array and frees it afterward — the allocator's locks and fragmentation become the bottleneck first, and the post-OOM recovery path twists the knife. XRT's answer is visible in the API shape: xrtHttp1HeadInit(&Head, Fields, 8) binds the caller's stack array to the Head — parsing merely fills the array with views; the next request reuses the same Head and array, allocation count zero.

The second design decision is the three-state return: XHTTP1_READY (a complete head has arrived), XHTTP1_MORE (insufficient input, wait for more), XHTTP1_ERROR (illegal structure). With the "failure does not advance" convention, a streaming server can mindlessly "receive a stretch, try parsing, on MORE receive again" — Chapter 72's framing loop, HTTP edition.

90.2概念

Parsing: from bytes to structure

xhttp1head after parsing holds: Method/Target (requests) or Status/Reason (responses), Version, the field array (Fields/FieldCount — the bound array's fill area), and Bytes (total head-block bytes including the terminating empty line — the body starts there). The three-state return model:

Requests and responses share the Head structure (the role is decided by the call entrance: RequestParse/ResponseParse). ResponseParse takes a method parameter — because response semantics depend on the request method (a HEAD response has no body; this context becomes a hard rule in Chapter 91's Plan).

The parser's transport bindings (http1_net / http1_tls)

The input of RequestParse/ResponseParse is contiguous memory — the shape of test vectors and whole buffers. A real receive path hands the engine's buffer chain instead (the head may be split across two blocks, Chapter 65), and a TLS connection hands a plaintext stream spanning records (Chapter 87). The binding layer plugs the parser into these two real sources: xrtHttp1RequestParseBuffer/xrtHttp1ResponseParseBuffer consume an xnetbuf (the Head's views borrow the buffer; only on a straddling head is a contiguous prefix allocated on demand — "the function does not consume the input; the caller consumes Head.Bytes after processing"); xrtHttp1RequestParseTls/xrtHttp1ResponseParseTls eat an xtlsstream directly (no plaintext consumed; on record straddling only the Header is merged — no connection-level fixed buffer). Each binding has two functions, but the signature and three-state semantics are identical to the in-memory versions: the binding layer is thin because the parser left "where bytes come from" to its parameters by design — limits defense, three-state returns, and the zero-allocation discipline all carry over. Chapter 104's xhttp server and Chapter 108's streaming service run on exactly these two bindings underneath.

Lookup and traversal

The common operations after parsing: on the Head.Fields array, use Chapter 89's xrtHttpFieldFind/FieldNext directly (case-insensitive — HTTP field names are semantically case-insensitive; same-named multi-value fields traverse in order, Set-Cookie style), xrtHttpFieldGet/FieldGetUnique (value views and uniqueness semantics). Combined with Chapter 89's token iterator, these cover composite judgments like "does Connection contain upgrade" — Chapter 94's HTTP upgrade (the WebSocket handshake) uses exactly this combination.

Packing: status line + fields in one write

xrtHttp1ResponseWrite(版本, 状态码, 短语, 字段数组, 数量, 输出, 容量, &长度) (version, status code, phrase, field array, count, output, capacity, &length) — the status line and field block are written in one shot: "HTTP/1.1 200 OK\r\nContent-Type: ...\r\n\r\n". Capacity atomicity as in Chapter 89: insufficient means zero writes. The request counterpart is xrtHttp1RequestWrite.Why not assemble with printf: every step of an assembly path can fail midway (half a message), and format details (status-line spaces, field terminators) scatter everywhere — one-shot packing collects them into a single fuzz-tested entrance.

Limits and defense

xhttp1limits (optionally passed to Parse) controls: the field-count cap (the bound array's capacity is natively one), the head-block total-length cap, and method/target length caps.Why they are mandatory: a malicious client can send a 2 GB head block — a parser without limits is a memory amplifier. Defaults are conservative; proxy scenarios relax as needed — limits are a deployment decision, not a global constant.

A preview of the Message and Body layering

This chapter's APIs stop at "the head block ends". Chapter 91's MessageParse combines "head + body" in one pass (fits small messages), and the Body family streams the body (fits a proxy forwarding as it receives) — both layers share this chapter's head parsing.

90.3示例

First complete program: request parsing and response packing

The program below is from examples/http/http1/main.c — the shortest path of "receive a request, send a response":

#include <stdio.h>

#include <xrt.h>
























 */


/* 展示零拷贝解析与直接响应封包的基础路径。 */
int main(void)
{
	static const char Request[] =
		"GET /health HTTP/1.1\r\nHost: localhost\r\n\r\n";
	static const xhttpfield ResponseFields[] = {
		{ XRT_STR_INIT("Content-Type"), XRT_STR_INIT("application/json") },
		{ XRT_STR_INIT("Content-Length"), XRT_STR_INIT("11") }
	};
	xhttpfield RequestFields[8];
	xhttp1head Head;
	xbytesview Input;
	char Response[256];
	size_t iSize;

	Input.Data = (cbytes)Request;
	Input.Size = sizeof(Request) - 1u;
	xrtHttp1HeadInit(&Head, RequestFields, 8);
	if ( xrtHttp1RequestParse(
		Input, &Head, NULL, NULL
	) != XHTTP1_READY ) {
		return 1;
	}
	printf("%.*s %.*s\n",
		(int)Head.Method.Size, Head.Method.Data,
		(int)Head.Target.Size, Head.Target.Data);
	if ( !xrtHttp1ResponseWrite(
		XHTTP_VERSION_1_1, 200, XRT_STR_LITERAL("OK"),
		ResponseFields, 2, Response, sizeof(Response), &iSize
	) ) {
		return 2;
	}
	fwrite(Response, 1, iSize, stdout);
	fwrite("{\"ok\":true}", 1, 11, stdout);
	return 0;
}

What just happened. (1) The input is complete request bytes (a static array mocking one arrival); HeadInit binds an 8-element stack array — parsing allocates nothing. (2) RequestParse returns READY: Head.Method/Target are borrowed views (pointing inside the input), printing GET /health. (3) ResponseWrite packs in one shot: status line (version/status/phrase) + two fields + the empty line; iSize is the full byte count — the body {"ok":true} follows directly.This is the entirety of a server reply: no per-line printf, no hand-assembled CRLF. In a real service the input comes from Chapter 67's receive buffer (NEXT → keep receiving), and the output goes to the send path (Chapter 68's write budget).

Second complete program: the Head full-feature tour

The second program is from examples/http1/head_tour/main.c — five lines covering limits, lookup, iteration, chunk lines, and Message for you:

 */

#include <stdio.h>
#include <string.h>
#include <xrt.h>

int main(void)
{
	static const char sFixed[] =
		"POST /a HTTP/1.1\r\n"
		"Host: a.test\r\n"
		"Content-Length: 2\r\n"
		"\r\n"
		"ok";
	static const char sChunked[] =
		"POST /b HTTP/1.1\r\n"
		"Host: a.test\r\n"
		"Transfer-Encoding: gzip, chunked\r\n"
		"\r\n";
	xhttp1limits Limits;
	xhttp1bodylimits BodyLimits;
	xhttp1bodyplan Plan;
	xhttp1body Body;
	xhttp1message Message;
	xhttp1errorinfo Error;
	xhttp1head Head;
	xhttpfield Fields[8];
	xhttpfield Trailers[4];
	xhttp1transfercoding Coding;
	const xhttpfield* pField;
	xbytesview View;
	xhttpnext Next;
	size_t iOffset = 0;
	size_t iSize = 0;
	size_t iBytes = 0;
	size_t iCount = 0;
	uint8 arrLine[16];
	int iResult = 1;

	/* ---- 限额默认值 + request-target 合法性正反。 ---- */
	xrtHttp1LimitsInit(&Limits);
	if ( (Limits.MaxStartLine != 8192u) ||
		(Limits.MaxHead != 65536u) ||
		(Limits.MaxFields != 100u) ||
		xrtHttp1TargetValid(XRT_STR_LITERAL("/a b")) ||
		xrtHttp1TargetValid(XRT_STR_LITERAL("/a#f")) ||
		xrtHttp1TargetValid(XRT_STR_LITERAL("")) ||
		!xrtHttp1TargetValid(XRT_STR_LITERAL("/api?v=1")) ) {
		goto Cleanup;
	}
	printf("http1: limits + target valid +/- ok\n");

	/* ---- 字段查找 + Transfer-Encoding 迭代。 ---- */
	xrtHttp1HeadInit(&Head, Fields, 8);
	if ( (xrtHttp1RequestParse(
			(xbytesview) { (const uint8*)sChunked,
				sizeof(sChunked) - 1u },
			&Head, &Limits, &Error) != XHTTP1_READY) ||
		((pField = xrtHttp1Field(&Head,
			XRT_STR_LITERAL("host"))) == NULL) ||
		(pField->Value.Size != 6u) ||
		(memcmp(pField->Value.Data, "a.test", 6u) != 0) ||
		(xrtHttp1Field(&Head, XRT_STR_LITERAL("x-nope")) !=
			NULL) ) {
		goto Cleanup;
	}
	/* TE 迭代须在 Transfer-Encoding 字段值上进行。 */
	if ( (pField = xrtHttp1Field(&Head,
			XRT_STR_LITERAL("transfer-encoding"))) == NULL ) {
		goto Cleanup;
	}
	iOffset = 0;
	if ( ((Next = xrtHttp1TransferCodingNext(
			pField->Value, &iOffset, &Coding)) != XHTTP_NEXT_ITEM) ||
		(Coding.Name.Size != 4u) ||
		(memcmp(Coding.Name.Data, "gzip", 4u) != 0) ||
		(xrtHttp1TransferCodingNext(pField->Value, &iOffset,
			&Coding) != XHTTP_NEXT_ITEM) ||
		(Coding.Name.Size != 7u) ||
		(memcmp(Coding.Name.Data, "chunked", 7u) != 0) ||
		(xrtHttp1TransferCodingNext(pField->Value, &iOffset,
			&Coding) != XHTTP_NEXT_END) ) {
		goto Cleanup;
	}
	printf("http1: field lookup + transfer-coding iteration ok\n");

	/* ---- 分帧计划：chunked 头优先于 Content-Length。 ---- */
	if ( !xrtHttp1RequestBodyPlan(&Head, &Plan) ||
		(Plan.Mode != XHTTP1_BODY_CHUNKED) ) {
		goto Cleanup;
	}
	xrtHttp1HeadInit(&Head, Fields, 8);
	if ( (xrtHttp1RequestParse(
			(xbytesview) { (const uint8*)sFixed,
				sizeof(sFixed) - 1u },
			&Head, &Limits, &Error) != XHTTP1_READY) ||
		!xrtHttp1RequestBodyPlan(&Head, &Plan) ||
		(Plan.Mode != XHTTP1_BODY_FIXED) ||
		(Plan.Length != 2u) ) {
		goto Cleanup;
	}
	printf("http1: request body plan fixed/chunked ok\n");

	/* ---- chunk-size 行：裸 size 与带扩展两种形态。 ---- */
	if ( !xrtHttp1ChunkLineWrite(5u, XRT_STR_LITERAL(""), arrLine,
			sizeof(arrLine), &iSize) ||
		(iSize != 3u) ||
		(memcmp(arrLine, "5\r\n", 3u) != 0) ||
		!xrtHttp1ChunkLineWrite(5u, XRT_STR_LITERAL(";x"),
			arrLine, sizeof(arrLine), &iSize) ||
		(iSize != 5u) ||
		(memcmp(arrLine, "5;x\r\n", 5u) != 0) ) {
		goto Cleanup;
	}
	printf("http1: chunk line 5 -> \"5\\r\\n\" (+ext) ok\n");

	
	 * BodyLimits 必须先初始化——消息解析会按限额约束正文。 ---- */
	xrtHttp1BodyLimitsInit(&BodyLimits);
	
	 * Head 容量不足时解析器返回 FIELDS。 */
	xrtHttp1MessageInit(&Message, Fields, 8, NULL, 0);
	if ( (xrtHttp1RequestMessageParse(
			(xbytesview) { (const uint8*)sFixed,
				sizeof(sFixed) - 1u },
			true, &Message, &Limits, &BodyLimits,
			&Error) != XHTTP1_READY) ||
		((View = xrtHttp1MessageBodyView(&Message)).Size != 2u) ||
		(memcmp(View.Data, "ok", 2u) != 0) ) {
		goto Cleanup;
	}
	printf("http1: message parse -> borrowed body \"ok\" ok\n");

	
	 * 描述符 → 读入 trailer 区；另演示独立 TrailersParse。 ---- */
	{
		static const char sChunkWire[] = "5\r\nhello\r\n0\r\n";
		static const char sTrailer[] = "X-Sum: 3\r\n\r\n";
		xhttp1bodystatus BodyStatus;
		xbytesview Data;

		/* 换回 chunked 请求头重建 Plan。 */
		xrtHttp1HeadInit(&Head, Fields, 8);
		if ( (xrtHttp1RequestParse(
				(xbytesview) { (const uint8*)sChunked,
					sizeof(sChunked) - 1u },
				&Head, &Limits, &Error) != XHTTP1_READY) ||
			!xrtHttp1RequestBodyPlan(&Head, &Plan) ||
			(Plan.Mode != XHTTP1_BODY_CHUNKED) ||
			!xrtHttp1BodyInit(&Body, &Plan, NULL, 0u,
				&BodyLimits) ) {
			goto Cleanup;
		}
		/* 数据块与终止块分两步喂入。 */
		BodyStatus = xrtHttp1BodyRead(&Body,
			(xbytesview) { (const uint8*)sChunkWire,
				sizeof(sChunkWire) - 1u },
			false, &iBytes, &Data, &Error);
		if ( (BodyStatus != XHTTP1_BODY_DATA) ||
			(Data.Size != 5u) ||
			(memcmp(Data.Data, "hello", 5u) != 0) ) {
			goto Cleanup;
		}
		BodyStatus = xrtHttp1BodyRead(&Body,
			(xbytesview) { (const uint8*)sChunkWire + 8u, 5u },
			false, &iBytes, &Data, &Error);
		if ( (BodyStatus != XHTTP1_BODY_MORE) ||
			(iBytes != 5u) ||
			/* 终止块后处于 trailer 态——此刻才能重绑。 */
			!xrtHttp1BodyTrailers(&Body, Trailers, 4u) ) {
			goto Cleanup;
		}
		BodyStatus = xrtHttp1BodyRead(&Body,
			(xbytesview) { (const uint8*)sTrailer,
				sizeof(sTrailer) - 1u },
			true, &iBytes, &Data, &Error);
		if ( (BodyStatus != XHTTP1_BODY_DONE) ||
			(Body.TrailerCount != 1u) ||
			!xrtHttp1BodyDone(&Body) ) {
			goto Cleanup;
		}
		/* 独立 trailer 区解析（不经 Body Reader）。 */
		iCount = 0;
		if ( (xrtHttp1TrailersParse(
				(xbytesview) { (const uint8*)sTrailer,
					sizeof(sTrailer) - 1u },
				Trailers, 4u, &BodyLimits, &iBytes, &iCount,
				&Error) != XHTTP1_READY) ||
			(iCount != 1u) ||
			(Trailers[0].Value.Size != 1u) ) {
			goto Cleanup;
		}
	}
	printf("http1: body trailers rebind + trailer parse ok\n");
	iResult = 0;

Cleanup:
	return iResult;
}

What just happened. (1) Line 1 verifies the limits model: target length within the limit passes, over the limit refuses — both sides of the defense parameter. (2) Line 2 is the lookup combination: find a field by name + transfer-coding token iteration (Chapter 89's TokenNext applied to a Head). (3) Line 3 previews Chapter 91: the request body's Plan (fixed-length/chunked verdict) is available immediately after head parsing. (4) Line 4 verifies chunk size line generation (5\r\n — extensions allowed). (5) Line 5 is the Message layer: one parse of "head + body", the body a borrowed view ok — the thriftiest path for small messages. (6) Line 6 verifies trailer array rebinding and trailer block parsing — a Chapter 91 preview. Six lines together: this chapter's plus the next's API surface.

Complete program: ParseBuffer — request/response parsing on a buffer chain

From the repository example examples/http1/parse_buffer/main.c:

 */

#include <stdio.h>

#include <xrt.h>



int main(void)
{
	static const char sRequest[] =
		"GET /x HTTP/1.1\r\nHost: a.test\r\n\r\n";
	static const char sResponse[] =
		"HTTP/1.1 200 OK\r\nContent-Length: 2\r\n\r\nok";
	xnetbufpool* pPool = xrtNetBufPoolCreate(NULL);
	xnetbuf ReqBuf;
	xnetbuf RspBuf;
	xhttpfield Fields[8];
	xhttp1head Head;
	bool bOk = false;

	if ( pPool == NULL ) {
		return 1;
	}
	/* 请求链：分两块追加（头跨块场景的最小化模拟）。 */
	if ( xrtNetBufInit(&ReqBuf, pPool) &&
		xrtNetBufAppend(&ReqBuf, sRequest, 12u) &&
		xrtNetBufAppend(&ReqBuf, sRequest + 12u, sizeof(sRequest) - 13u) ) {
		xrtHttp1HeadInit(&Head, Fields, 8);
		if ( xrtHttp1RequestParseBuffer(&ReqBuf, &Head, NULL, NULL) ==
			XHTTP1_READY ) {
			printf("request: %.*s %.*s\n",
				(int)Head.Method.Size, Head.Method.Data,
				(int)Head.Target.Size, Head.Target.Data);
			bOk = true;
		}
		xrtNetBufClear(&ReqBuf);
	}
	/* 响应链：单块。 */
	if ( bOk && xrtNetBufInit(&RspBuf, pPool) &&
		xrtNetBufAppend(&RspBuf, sResponse, sizeof(sResponse) - 1u) ) {
		xrtHttp1HeadInit(&Head, Fields, 8);
		if ( xrtHttp1ResponseParseBuffer(&RspBuf, &Head, NULL, NULL) ==
			XHTTP1_READY ) {
			printf("response: %u\n", (unsigned)Head.Status);
			bOk = Head.Status == 200u;
		}
		xrtNetBufClear(&RspBuf);
	}
	xrtNetBufPoolDestroy(pPool);
	return bOk ? 0 : 2;
}

What just happened. (1) The request bytes are deliberately appended in two blocks — a minimal simulation of a straddling head; ParseBuffer proves chain parsing needs no caller-side stitching. (2) The result is isomorphic to the in-memory version: Head.Method/Target borrow views, printed directly. (3) The response side goes through ResponseParseBuffer — request/response entrances, one Head. (4) The ParseTls variant has no separate example: it differs from the Buffer version only in "where the stream comes from" (an xtlsstream replacing an xnetbuf); the contract line says it, and no example is invented.

90.4契约

• Bound array: the Head binds the caller's field array; the field count is natively capped by the array capacity; no cleanup before reuse (every Parse rebuilds).
• Three-state return: READY/NEXT/ERROR; NEXT consumes nothing and reports no error; failure does not advance (retry semantics as in Chapter 72's framing).
• Borrowed views: Method/Target/Status/Reason/fields all borrow the input; Bytes is the head-block length including the terminating empty line (the body's start).
• Lookup semantics: field names case-insensitive; same-named multi-value fields traverse in order; a miss is an explicit failure, not a NULL value.
• Packing atomicity: status line + fields written in one shot; insufficient capacity writes zero; success publishes the total length.
• Limit defense: limits control head-block/method/target lengths; unlimited parsing is a memory amplifier; defaults conservative.
• Method semantics: response parsing carries the request-method context (the HEAD/204 no-body rule is executed by Chapter 91's Plan).
• Zero per-request allocation: the parsing and packing paths allocate no heap (Head/array/buffers all caller-provided).
• Trimming: HTTP1_HEAD/HTTP1_BODY/HTTP1_MESSAGE independent feature macros, depending on the HTTP foundation.

Common Mistakes

while ( recv(buf) > 0 ) {
	if ( Parse(buf) == XHTTP1_MORE ) {
		continue;   /* only the newly received bytes are tried: never complete */
	}
	break;
}

/* append-style buffer (Chapter 65): everything received accumulates in one span */
while ( recv_append(&In) > 0 ) {
	if ( xrtHttp1RequestParse(view(&In), &Head, NULL, NULL) ==
			XHTTP1_READY ) {
		break;   /* bytes after Bytes are the body/next request */
	}
}

xhttpfield Fields[4];   /* rejects most real requests */
xrtHttp1HeadInit(&Head, Fields, 4);

xhttpfield Fields[32];   /* rule of thumb: real requests 25 + forwarding headroom */
xrtHttp1HeadInit(&Head, Fields, 32);

   the proxy enlarges or explicitly rejects (itself an anti-abuse decision) */

xrtHttp1ResponseWrite(XHTTP_VERSION_1_1, 200, OK,
	NoFields, 0, Out, sizeof(Out), &iSize);
send(Out, iSize);
send(Body, BodySize);   /* no delimiting: the client can't tell when it ends */

static const xhttpfield F[] = {
	{ XRT_STR_INIT("Content-Length"), XRT_STR_INIT("11") }
};
xrtHttp1ResponseWrite(XHTTP_VERSION_1_1, 200,
	XRT_STR_LITERAL("OK"), F, 1, Out, sizeof(Out), &iSize);
send(Out, iSize);       /* the head declares 11 bytes */
send(Body, 11);         /* the body is exactly 11: delimiting complete */

Programming Exercises

Summary