Chapter 121 IMAP: The Server-Side Mailbox

Chapter Orientation

IMAP is POP3's (Chapter 118) "server-authoritative" dual: mail stays on the server, and folders/flags/search/FETCH partial retrieval all happen server-side — the cornerstone of multi-device sync. The protocol complexity is an order higher too: the tagged response model (every command carries a unique tag; server responses correlate by tag — enabling pipelining); the literal mechanism (parameters and responses containing arbitrary bytes go through {N} length declarations — synchronous literals wait for a continuation confirmation, LITERAL+ skips the sync); the untagged event stream (EXISTS/RECENT/FETCH updates cut in at any time). ximap layering: protocol primitives (imap: replies/literals/capabilities/commands — builds no mailbox objects and constructs no search conditions; unknown extensions remain reachable as raw text) → client (imap_client: the two-level command model — a low-level explicit-tag pipeline plus a sequential convenience layer) → command convenience layer (imap_command: safe construction of SELECT/LIST/SEARCH/FETCH/IDLE) → specialized layers (auth/body/message/append/compress).

121.1引入

Three IMAP mechanisms deserve the opening. First: streaming literals. FETCH a 20 MB message and the response carries {20971520} followed by raw bytes — the client must read in explicit-length segments: ReadLiteral takes segments into the caller's buffer, and until it finishes you cannot read subsequent responses — "message size never becomes a fixed memory ceiling" is a protocol-layer guarantee (the same family as POP3's line streaming, but by length rather than by line). Second: IDLE push. The IDLE command lets the server push untagged events as new mail arrives — the server-side version of "a long connection waiting for new mail" (the mail edition of SSE). Third: pipelining. The low-level Send takes explicit tags — send several commands, then one Receive loop consumes events in arrival order and correlates completions by tag — IMAP's concurrency primitive (the throughput shape of connection reuse).

The usual injection defense is present: CommandWrite rejects malformed atoms/control characters/line injection/excess; imap_command's mailbox names are "validated and quoted per IMAP string rules — extra commands cannot be injected via CRLF".

121.2概念

The response model and literals

Literal budgets run both ways: the send side (APPEND of a big message — Chapter 120's composition) and the receive side (FETCH of a big literal). The receive-side contract: on a HasLiteral event first check Event.Literal.Size — over your application budget, Abort immediately (no need to consume the body — the memory defense of "read the menu before deciding to eat"); the low-level streaming path has no default ceiling (ceilings belong to the application); the imap_message convenience layer carries its own 64 MiB budget (the two don't share — each layer, each policy).

The two-level command model

• Low level (Send/SendParts/Write/Continue/Receive/ReadLiteral): explicit-tag pipelining — send multiple commands, consume events in order, correlate by tag; SendParts writes argument pieces with inserted spaces straight to the wire (no whole temporary command constructed). Views borrow the line buffer — invalidated after the next Receive/Next/ReadLiteral (the same family discipline as Chapter 118's line views; here literal reads are also invalidation points).
• Sequential layer (Begin/BeginParts/Next): automatic unique tags; Next yields untagged events (XMAIL_NEXT_ITEM) and the target completion (XMAIL_NEXT_END) as separate states — a unified loop shape; you cannot start the next sequential command while one is active.

The command convenience layer: SELECT summary and streaming results

imap_command covers common commands but builds no heavyweight objects: Select/Examine produce an ximapmailboxinfo with zero allocation (a digest of the mailbox facts; the Present bit distinguishes "not returned" from "value zero" — Chapter 102's presence bit, mailbox edition; ReadOnly takes the final completion's access mode; MailboxInfoUpdate merges subsequent EXISTS/RECENT events into the same summary). Management commands (CREATE/DELETE/RENAME/SUBSCRIBE/CHECK/UNSELECT/CLOSE) fully consume their synchronous completions; streaming commands (LIST/STATUS/SEARCH/FETCH/STORE/COPY/MOVE/EXPUNGE) only build safely and start — results stream through Next+ReadLiteral (the convenience layer never cuts the stream). IDLE: after entering, Done ends it — pushed events flow through the same Receive loop.

TLS, compression, and failure semantics

TLS: implicit/STARTTLS (the handshake only after the tagged OK is fully consumed; re-run CAPABILITY after upgrade — the plaintext snapshot is not carried over; a verifier is mandatory) — the same family trio as SMTP/POP3. COMPRESS (imap_compress): after authentication, negotiate a DEFLATE-compressed wire — a bandwidth optimization for high-latency links. Failure semantics in three: line FAILED (network/cancellation/timeout/protocol misordering/parse failure — "the next byte's position at a command boundary cannot be guaranteed"; the connection is not reusable); command NO/BAD (the server's negation of a complete command — the client keeps a recoverable state, returns a structured error — the next command continues); Abort (abort without waiting, repeatable; FAILED and the most recent completion are kept until Destroy for diagnosis).

121.3示例

Run these commands from the repository root. The builder selects modules, public headers and platform libraries from the manifest, and also prints build and dependency-test logs. Terminal blocks show the example output. The second program starts a loopback server within the process and uses fixed demonstration data. See the client library README for real-service host, port, CA and runtime credential configuration.

First complete program: protocol primitives

The program below is from examples/imap/protocol — an offline loop of replies and commands:

#include <stdio.h>
#include <ximap.h>



/* 解析服务器响应并构建一条 IMAP 命令。 */
int main(void)
{
	ximapresponseview Response;
	char arrCommand[64];
	size_t iSize;

	if ( !xrtImapResponseParse(
		XRT_STR_LITERAL("A001 OK authenticated"),
		&Response
	) || !xrtImapCommandWrite(
		XRT_STR_LITERAL("A002"),
		XRT_STR_LITERAL("SELECT"),
		XRT_STR_LITERAL("\"INBOX\""),
		0,
		arrCommand,
		sizeof(arrCommand),
		&iSize
	) ) {
		return 1;
	}
	printf("status=%d command=%.*s",
		(int)Response.Status, (int)iSize, arrCommand);
	return 0;
}

What just happened. (1) The program parses a fixed tagged OK reply with xrtImapResponseParse, then builds SELECT with an explicit tag and mailbox argument through xrtImapCommandWrite, printing the status and command bytes. (2) The mailbox argument already includes double quotes; the builder validates wire syntax and injection boundaries. Callers choose a quoted string or literal according to the argument type. The same suite tests other reply, literal and escaping boundaries; body reads remain the client state machine responsibility.

Second complete program: a read-only mailbox query

The second program is from extlibs/ximap/examples/offline/main.c.

#include <ximap.h>

#include <stdio.h>
#include <string.h>

#include <math.h>
static inline double exampleTimerLimit(int64 Timeout)
{
    return Timeout == XRT_WAIT_FOREVER ? INFINITY : xrtTimer() + (double)Timeout / 1000.0;
}
static inline bool exampleTimerExpired(double Limit)
{
    return xrtTimer() >= Limit;
}
static inline int64 exampleTimerRemaining(double Limit)
{
    double Ms;
    if (Limit == INFINITY) return XRT_WAIT_FOREVER;
    Ms = ceil((Limit - xrtTimer()) * 1000.0);
    return Ms <= 0 ? 0 : Ms >= 0x1p63 ? INT64_MAX : (int64)Ms;
}

/* 本地演示域名只解析到本进程的回环 IMAP 服务。 */
static xnetaddr ExampleAddress;

typedef struct example_server {
	xnetlistener* Listener;
	double Deadline;
	bool Success;
} example_server;

static xnetaddrlist* exampleResolve(cstr sHost, xnetfamily Family, ptr pData)
{
	xnetaddr Address = ExampleAddress;
	(void)Family;
	(void)pData;
	if ( strcmp(sHost, "imap.example.invalid") != 0 ) return NULL;
	Address.Port = 0u;
	return xrtNetAddrListCreate(&Address, 1u);
}

static bool exampleSend(xnetstream* pStream, cstr sText, double Deadline)
{
	size_t iSize = strlen(sText);
	for ( ;; ) {
		xnetresult Result = xrtNetStreamSend(pStream, sText, iSize);
		if ( Result == XNET_RESULT_OK ) return true;
		if ( (Result != XNET_RESULT_AGAIN) || !xrtNetStreamWait(
			pStream, XNET_STREAM_WAIT_WRITE,exampleTimerRemaining(Deadline), NULL) ) return false;
	}
}

static bool exampleExpect(xnetstream* pStream, cstr sExpected,
	double Deadline)
{
	size_t iSize = strlen(sExpected);
	size_t iUsed = 0u;
	while ( iUsed < iSize ) {
		xnetbytes* pBytes = xrtNetStreamRecv(
			pStream, iSize - iUsed,exampleTimerRemaining(Deadline), NULL);
		xbytesview Bytes;
		if ( pBytes == NULL ) return false;
		Bytes = xrtNetBytesView(pBytes);
		if ( (Bytes.Size == 0u) ||
			(memcmp(Bytes.Data, sExpected + iUsed, Bytes.Size) != 0) ) {
			xrtNetBytesDestroy(pBytes);
			return false;
		}
		iUsed += Bytes.Size;
		xrtNetBytesDestroy(pBytes);
	}
	return true;
}

static int32 exampleServer(ptr pData)
{
	example_server* pServer = (example_server*)pData;
	xnetstream* pStream = xrtNetListenerAcceptWait(
		pServer->Listener,exampleTimerRemaining(pServer->Deadline), NULL);
	bool bOk;
	if ( pStream == NULL ) return 1;
	bOk = exampleSend(pStream, "* OK imap.example.invalid ready\r\n",
		pServer->Deadline) &&
		exampleExpect(pStream, "A00000001 CAPABILITY\r\n",
			pServer->Deadline) &&
		exampleSend(pStream,
			"* CAPABILITY IMAP4rev1\r\n"
			"A00000001 OK capability complete\r\n",
			pServer->Deadline) &&
		exampleExpect(pStream,
			"A00000002 LOGIN \"demo\" \"demo\"\r\n",
			pServer->Deadline) &&
		exampleSend(pStream, "A00000002 OK login complete\r\n",
			pServer->Deadline) &&
		exampleExpect(pStream, "A00000003 EXAMINE \"INBOX\"\r\n",
			pServer->Deadline) &&
		exampleSend(pStream,
			"* 2 EXISTS\r\n"
			"* 0 RECENT\r\n"
			"* OK [UIDVALIDITY 42] valid\r\n"
			"A00000003 OK [READ-ONLY] examined\r\n",
			pServer->Deadline) &&
		exampleExpect(pStream, "A00000004 LOGOUT\r\n",
			pServer->Deadline) &&
		exampleSend(pStream,
			"* BYE signing off\r\n"
			"A00000004 OK logout complete\r\n",
			pServer->Deadline) &&
		xrtNetStreamClose(pStream) &&
		xrtNetStreamWait(pStream, XNET_STREAM_WAIT_CLOSE,exampleTimerRemaining(pServer->Deadline), NULL);
	pServer->Success = bOk;
	xrtNetStreamDestroy(pStream);
	return bOk ? 0 : 2;
}

int main(void)
{
	xnetengineconfig EngineConfig;
	xnetresolverconfig ResolverConfig;
	xnetlistenconfig ListenConfig;
	ximapclientconfig ClientConfig;
	ximapauthconfig Auth;
	ximapmailboxinfo Mailbox;
	example_server Server = { 0 };
	xnetengine* pEngine = NULL;
	xnetresolver* pResolver = NULL;
	xnetlistener* pListener = NULL;
	ximapclient* pClient = NULL;
	xthread* pThread = NULL;
	double Deadline = exampleTimerLimit(INT64_C(10000));
	bool bOk = false;

	xrtNetEngineConfigInit(&EngineConfig);
	EngineConfig.Backend = XNET_PORT_SELECT;
	EngineConfig.Workers = 1u;
	pEngine = xrtNetEngineCreate(&EngineConfig);
	if ( (pEngine == NULL) || !xrtNetEngineStart(pEngine) ) goto Done;
	xrtNetListenConfigInit(&ListenConfig);
	if ( !xrtNetAddrLoopback(&ListenConfig.Address,
		XNET_FAMILY_IPV4, 0u) ) goto Done;
	pListener = xrtNetListen(pEngine, &ListenConfig, NULL, NULL, NULL);
	if ( (pListener == NULL) ||
		!xrtNetListenerLocal(pListener, &ExampleAddress) ) goto Done;
	xrtNetResolverConfigInit(&ResolverConfig);
	ResolverConfig.Workers = 1u;
	ResolverConfig.Lookup = exampleResolve;
	ResolverConfig.CacheEntries = 0u;
	pResolver = xrtNetResolverCreate(&ResolverConfig);
	if ( pResolver == NULL ) goto Done;
	Server.Listener = pListener;
	Server.Deadline = Deadline;
	pThread = xrtThreadCreate(exampleServer, &Server, 0u);
	if ( pThread == NULL ) goto Done;
	xrtImapClientConfigInit(&ClientConfig);
	ClientConfig.Net.Engine = pEngine;
	ClientConfig.Net.Resolver = pResolver;
	ClientConfig.Net.Host = "imap.example.invalid";
	ClientConfig.Net.Port = ExampleAddress.Port;
	pClient = xrtImapClientOpen(&ClientConfig,exampleTimerRemaining(Deadline), NULL);
	if ( pClient == NULL ) goto Done;
	xrtImapAuthConfigInit(&Auth);
	Auth.Method = XIMAP_AUTH_LOGIN;
	Auth.Username = XRT_STR_LITERAL("demo");
	Auth.Secret = XRT_STR_LITERAL("demo");
	/* 明文凭据仅供进程内回环演示；真实服务使用 TLS 范例。 */
	Auth.AllowPlaintext = true;
	xrtImapMailboxInfoInit(&Mailbox);
	if ( !xrtImapClientAuth(pClient, &Auth,exampleTimerRemaining(Deadline), NULL) ||
		!xrtImapClientExamine(pClient, XRT_STR_LITERAL("INBOX"),
			&Mailbox,exampleTimerRemaining(Deadline), NULL) ||
		(Mailbox.Exists != 2u) || (Mailbox.Recent != 0u) ||
		(Mailbox.UidValidity != 42u) || !Mailbox.ReadOnly ||
		(xrtImapClientState(pClient) != XIMAP_CLIENT_SELECTED) ||
		!xrtImapClientLogout(pClient,exampleTimerRemaining(Deadline), NULL) ) goto Done;
	bOk = true;

Done:
	if ( (pClient != NULL) && !bOk ) (void)xrtImapClientAbort(pClient);
	xrtImapClientDestroy(pClient);
	if ( pThread != NULL ) {
		bool bThreadDone = xrtThreadWait(pThread) == XWAIT_OK;
		bOk = bOk && bThreadDone && Server.Success &&
			(xrtThreadExitCode(pThread) == 0);
		xrtThreadDestroy(pThread);
	}
	if ( pListener != NULL ) {
		(void)xrtNetListenerClose(pListener);
		while ( xrtNetListenerState(pListener) != XNET_LISTENER_CLOSED )
			xrtThreadYield();
		xrtNetListenerDestroy(pListener);
	}
	if ( pResolver != NULL ) (void)xrtNetResolverDestroy(pResolver);
	if ( pEngine != NULL ) (void)xrtNetEngineDestroy(pEngine);
	if ( !bOk ) {
		fputs("offline IMAP EXAMINE failed\n", stderr);
		return 1;
	}
	puts("offline IMAP EXAMINE: INBOX has 2 messages (read-only)");
	return 0;
}

What just happened. (1) The program creates an in-process loopback IMAP server. The client reads the greeting and establishes a sequential command session. LOGIN uses a fixed demonstration account, with plaintext authentication allowed only for this local fixture. (2) The client calls xrtImapClientExamine, reads the read-only INBOX state and a count of two messages, checks the selected state and mailbox facts, and waits for tagged completion. EXAMINE does not change server message flags and demonstrates a short read-only query. (3) After LOGOUT the program waits for the server thread, then closes and destroys the listener, resolver and network engine. Success means both the command result and server script were checked. Compression requires a separate authenticated negotiation and stream-decoding flow; see the library README for compression tests and real-client configuration.

121.4契约

• Primitives without models: no mailbox objects / no search/FETCH models built — unknown extensions reachable as raw text.
• Three classes, five states: tagged/untagged/continuation; OK/NO/BAD/PREAUTH/BYE stably recognized; all views via borrowing.
• The literal contract: parsing yields only lengths and markers; data is read in segments; until finished you cannot read subsequent responses — message size ≠ memory ceiling.
• Literal budgets: no default ceiling at the low level (the application checks Size itself and Aborts immediately over budget); the message layer's independent 64 MiB budget.
• Two-level model: low-level explicit-tag pipeline (SendParts writes straight to the wire) / sequential layer automatic tag + state-split Next; sequential commands do not run concurrently.
• View invalidation points: any of Receive/Next/ReadLiteral invalidates line-buffer views.
• Command convenience layer: SELECT's zero-allocation summary (Present bit/event merging); management commands complete synchronously; streaming commands never cut the stream.
• Injection defense: command/mailbox-name injection rejected; control data goes via literal.
• TLS: implicit/STARTTLS; re-CAPABILITY after upgrade; a verifier is mandatory.
• Compression: COMPRESS after authentication; configuration Validated first.
• Failure in three: line FAILED not reusable / command NO-BAD recoverable / Abort idempotent with diagnostics kept.

Common Mistakes

if ( Event.HasLiteral ) {
	while ( ReadLiteral(...) ) { consume(); }   /* size unchecked: 500MB pours straight in */
}

if ( Event.HasLiteral ) {
	if ( Event.Literal.Size > MyBudget ) {
		xrtImapClientAbort(pClient);   /* over budget: abort immediately, skip the body */
		return;
	}
	while ( ReadLiteral(...) ) { consume(); }
}

send(A); send(B);
wait_tagged(A);   /* the server may answer B first: hung waiting */
wait_tagged(B);

send(A); send(B);
while ( Receive(&Event) ) {
	if ( Event.kind == TAGGED ) { dispatch_by_tag(Event.Tag); }
	else { handle_untagged(Event); }   /* inserted events handled immediately */
}

Subject = Event.ResponseView;   /* stored a view */
Receive(&Event2);               /* the buffer has changed */
use(Subject);                   /* garbage */

Subject = copy_view(Event.ResponseView);   /* copy before crossing calls */
Receive(&Event2);
use(Subject);

Programming Exercises

Summary