Chapter 41 The Universal IO Stream Abstraction

Chapter Orientation

xreader/xwriter are XRT's stream abstraction: the same ReadFull/WriteFull/Seek interface works over memory buffers, files, and network streams — business code is written once; the adapter decides the destination. The Full semantics are its thoughtfulness: read or write exactly the requested number of bytes, and any partial completion is a failure — the short-read/short-write retry-loop boilerplate disappears. This chapter covers Reader/Writer's core contracts, the built-in adapters (memory buffers, files, custom callbacks), and the line reader, a high-frequency composite part.

41.1引入

A piece of serialization code must write its product out: in tests, into a memory buffer (assert on contents); on disk, into a file; over the network, into a socket stream. Calling fwrite directly means three destinations, three copies of code; defining your own "write interface" means you are reinventing the io module. xwriter standardizes that invention: business code programs against the interface (xrtWriterWriteFull(W, 数据, 长度) (W, data, length)), and the destination is a one-line adapter choice — xrtWriterFromBuffer (memory), the file adapter (Chapter 45), or your own callback adapter (demonstrated in the stream_tour sample).

The Full semantics solve other boilerplate written daily: fwrite can write short (3 bytes written when you asked for 10) — textbooks teach a while-retry loop; WriteFull tucks the loop into the library: either all 10 bytes are written, or it fails, and the return value is the final verdict. The read side is symmetric, with ReadFull's EOF semantics made explicit (reading less than requested fails — EOF is not an error, but "didn't read full" is).

41.2概念

The two interfaces and Full semantics

The division between Read and ReadFull: the former "reads best-effort, however much arrives" (streaming consumption — take a chunk, process a chunk); the latter "reads exactly the requested amount, else fails" (structured reads — a 16-byte message header must read the full 16). The write side is symmetric. The selection mnemonic: streaming processing uses the plain version (process whatever arrived); fixed-length structures use the Full version (less than full is failure) — both semantics are crisp; mixing them is the pit.

The memory adapter: the buffer as destination

xrtWriterFromBuffer(&tBuffer) adapts Chapter 15's xbuffer into a Writer — borrowing, not owning: after the adapter is destroyed the buffer remains, and the Take rhythm for extracting content is unchanged. This lets "accumulate in memory first, decide later" share the same serialization code with "write the file directly" — tests assert on the buffer, production swaps in the file adapter, business code unchanged.

The line reader: the standard part for streaming text

xlinereader (the line sample) wraps any Reader into "iterate line by line" — splitting by lines, line views borrowed, ending at EOF. The common foundation of log scanning, config line-by-line, and line-framed protocols (the prelude to Chapter 72). Its division of labor with Chapter 25's string line-splitting: whole text already in memory → string Split; lines arriving from a stream → the line reader (no full load needed).

Custom adapters: the callback is the interface

When your destination has no built-in adapter (encryption pipelines, compression middle layers, network protocols), the callback adapter is the exit: provide read/write callbacks plus user data, and the interface contract stays the same. The stream_tour sample's custom reader/writer demonstrates the full form (including the Close callback's invocation timing — the custom close=1 assertion).

41.3示例

Complete program: a memory Writer and in-place overwrite

From the repository sample examples/io/buffer/main.c:

 */

#include <stdio.h>

#include <xrt.h>



int main(void)
{
	xbuffer Buffer;          /* 底层存储：分段字节缓冲 */
	xwriter* pWriter;        /* 适配器：借用 Buffer，Destroy 不释放它 */
	int iResult = 1;

	if ( !xrtBufferInit(&Buffer) ) {
		return 1;
	}

	





	 */
	pWriter = xrtWriterFromBuffer(&Buffer);
	if ( (pWriter != NULL) &&
		 xrtWriterWriteFull(pWriter, "hello world", 11u, NULL) &&
		 xrtWriterSeek(pWriter, 6, XSEEK_START, NULL) &&
		 xrtWriterWriteFull(pWriter, "xrt", 3u, NULL) ) {
		printf("%.*s\n", (int)Buffer.Size, (const char*)Buffer.Data);
		iResult = 0;
	}

	/* 两级清理：适配器归适配器，缓冲归缓冲。 */
	xrtWriterDestroy(pWriter);
	xrtBufferUnit(&Buffer);
	return iResult;
}

What just happened. (1) The xrtWriterFromBuffer adapter borrows the buffer — every WriteFull lands directly in it (appending), zero intermediate copies. (2) Seek(XSEEK_START, 偏移) (offset) overwrites at a position: going back to modify already-written bytes — the protocol-encoding posture of "write the header, then go back and fill the length field" (the same family as Chapter 15's BufferWrite, but through the unified interface). (3) WriterDestroy returns only the adapter — the buffer itself remains, its contents read out and printed directly. The whole snippet contains no fwrite/memcpy: the destination decision condenses into the adapter line.

Complete program: the line reader

From examples/io/line/main.c — line-by-line consumption of any Reader:

 */

#include <stdio.h>

#include <xrt.h>



int main(void)
{
	/* 混合行尾 + 末行无换行的日志样本。 */
	static const char sLog[] =
		"INFO server started\r\n"
		"WARN queue is busy\n"
		"ERROR request failed";
	xreader* pReader;
	xlinereader* pLines;
	xlineview Line;          /* 借用：到下一次 Next 前有效 */
	xlinenext Next;
	size_t iCount = 0u;
	int iResult = 1;

	



	 */
	pReader = xrtReaderFromMemory(XRT_BYTES_LITERAL(sLog));
	pLines = xrtLineReaderTake(&pReader, 1024u);
	if ( pLines == NULL ) {
		xrtReaderDestroy(pReader);
		return 1;
	}

	/* 逐行消费：每行是借用视图，处理完再取下一行。 */
	while ( (Next = xrtLineReaderNext(pLines, &Line)) == XLINE_NEXT_LINE ) {
		printf("%zu: %.*s\n", ++iCount, (int)Line.Text.Size, Line.Text.Data);
	}
	if ( Next == XLINE_NEXT_END ) {
		iResult = 0;
	}

	/* 接管模式：一个 Destroy 连底层 Reader 一起关闭。 */
	if ( !xrtLineReaderDestroy(pLines) ) {
		iResult = 1;
	}
	return iResult;
}

What just happened. (1) Memory is first adapted into a Reader (xrtReaderFromMemory), then taken over by the line reader (xrtLineReaderTake transfers ownership — the original Reader pointer must never be touched again); 1024 is the initial line capacity, over-long lines grow automatically, and the buffer expands only with actual line length. (2) xrtLineReaderNext returns three states: XLINE_NEXT_LINE (a line), XLINE_NEXT_END (EOF), or the failure state — mixed line endings (CRLF and LF together) and a final line without newline are all handled correctly. (3) Every line is a borrowed view (the Text field of xlineview) — valid until the next Next (Chapter 18's discipline as usual). (4) In takeover mode, xrtLineReaderDestroy closes the underlying Reader too — one Destroy collects the whole set. The stream_tour sample is left as full-form reading: it walks Reader/Writer's copy operations, Seek/Tell, EOF, custom adapters, and Close timing end to end — nine assertions for nine interface groups.

41.4契约

• Full semantics: read full or write full or fail; short reads and short writes are failures, not partial successes; at EOF, ReadFull fails ("didn't read full" is a fact, not an error — the error-slot semantics are Chapter 4's).
• Interface unification: one body of business code over memory/file/network — the destination is decided by one adapter line alone.
• Borrowed adaptation: the Buffer adapter borrows the buffer; Destroy returns the adapter without touching the underlying resource.
• Borrowed lines: the line reader yields line views valid until the next line.
• Custom adapters: callbacks + user data implement any destination; the Close callback fires exactly once, at Destroy.

From examples to engineering: three hosts of the stream abstraction

Serialization across three media (sample one's full form): structured output code faces the Writer — tests attach the memory adapter for assertions, disk attaches the file adapter (Chapter 45), the network attaches the protocol-stream adapter (Volume 7); the destination is a one-line adapter choice, the serialization logic written once. Protocol parsing pipeline: network bytes → Reader → fixed-length header ReadFull → line reader or body-read-full — Volume 7's HTTP parser is this pipeline's industrial version. Processing pipeline: Reader → a decoding layer (Chapter 27's streaming transcoding form) → business consumption — custom adapters wrap "intermediate processing" into a stream, and the pipeline's layers compose through the unified interface. The three hosts verify one sentence together: an interface abstraction's value is not "many destinations connect", but "business code doesn't move when the destination changes".

The relation to Chapter 15's buffers and Chapter 45's files

The division among three layers is easily confused; one comparison settles it. xbuffer (Chapter 15): the byte accumulator — accumulate then take, with its own memory semantics; the io adapter (this chapter): adapts buffers/files/anything into the unified interface — it owns no data, it converts interfaces; the file module (Chapter 45): lifecycle management of files (open/close/attributes/async), whose reads and writes can enter the unified interface through io adapters. The composition: the file adapter uses the file module internally, the buffer adapter uses xbuffer internally — the io layer is the interface layer above them both. Choosing in code: caring about "how content accumulates" → buffers directly; caring about "a unified read/write interface" → this chapter; caring about "the file as a resource" → Chapter 45.

A design view: minimal completeness of an interface

The io layer's interface surface is tiny — read, write, position, flush, close: five actions. Yet it exactly covers streaming data's every need: this is minimal completeness — every interface is high-frequency, with no removable redundancy. Compared with "big and complete" stream libraries (dozens of convenience functions), minimal completeness pays off in: adapters are easy to write (five callbacks suffice — custom adapters cost almost nothing) and behavior is easy to reason about (few semantic combinations, few pits). When writing your own abstraction layers, calibrate against this standard: the balance point between interface count and expressiveness sits where "no interface can be deleted" — extra convenience functions belong to composition at the business layer, not membership in the interface layer.

Common Mistakes

xrtReaderRead(R, Header, 16);           /* best-effort: might read only 9 bytes */
Parse(Header);                           /* parsing 9 bytes of content as a 16-byte structure - misaligned */

if ( !xrtReaderReadFull(R, Header, 16) ) {
	return false;   /* fewer than 16 bytes read: EOF or interruption - treat as failure */
}
Parse(Header);

const char* gFirstLine;
while ( xrtLineReaderNext(pLines, &Line) == XLINE_NEXT_LINE ) {
	if ( gFirstLine == NULL ) {
		gFirstLine = Line.Data;   /* saving the view - overwritten by the next Next */
	}
}

str sFirst = NULL;
while ( xrtLineReaderNext(pLines, &Line) == XLINE_NEXT_LINE ) {
	if ( sFirst == NULL ) {
		sFirst = xrtStrDupView(Line);   /* to live long, copy */
	}
}

Programming Exercises

Summary