Chapter 120 POP3: Receiving Mail

Chapter Orientation

POP3 is the simplest of the three receiving protocols (POP3/IMAP/Chapter 119): a download model — connect, authenticate, list messages, fetch one by one, (optionally) delete, say goodbye. Simple doesn't mean careless: multiline-response dot transparency (RETR/TOP body line-start dot escaping and single-dot termination — the symmetric reverse of SMTP DATA); streaming receipt (Begin/Next yields line by line, never allocating the whole message — each line borrows the internal buffer, stable only until the next read: three outlets in direct file writes / incremental MIME / your own storage); UIDL's identity semantics (unique IDs underpin the client logic of "fetch only new mail"); the SASL authentication family (USER/PASS plaintext rejected by default, SASL PLAIN plus OAuth mechanisms); STLS upgrade (CAPA declares → +OK → upgrade in place → re-run CAPA). xpop3 layering: protocol primitives (offline) + client (a synchronous state machine: AUTHORIZATION→TRANSACTION→MULTILINE→UPDATE) + an optional message layer (bounded aggregation).

120.1引入

The model difference between POP3 and IMAP decides the choice: POP3 assumes "taken means local" — the server is a temporary delivery point, the client is the authoritative store; IMAP (Chapter 119) assumes "the server is authoritative" — mail stays on the server, folders/flags/searching all server-side. POP3's simplicity brings low resource use and offline friendliness; IMAP's richness brings multi-device sync. Most scenarios today are "receive and archive" (pull into a local database) — POP3's streaming receipt fits exactly.

Begin/Next's per-line borrowing deserves attention: RETR a 20 MB message, and at any moment only the current line is in memory — the line view Next yields is stable only until the next wire read (the next line's arrival overwrites it). That is far more aggressive than "buffer the whole message" — it turns "receiving mail" into the shape of "reading a file" (isomorphic to Chapter 41's IO streams).

120.2概念

Protocol primitives: replies, facts, and commands

• Replies: xrtPop3ReplyParse distinguishes +OK/-ERR and borrows the status text; StatParse (64-bit counts and bytes — big mailboxes don't overflow), ListParse/UidlParse (multiline items; UIDL is neither copied nor truncated).
• The dot family reused: multiline termination and dot un-escaping use mail_net's MailLineRead/MailDotLine/MailDotDecodeWrite directly — no POP3-specific duplicate (two directions of the same primitive family as SMTP DATA).
• Capabilities: CapabilityParse returns names and parameters even for unknown extensions; Capability gives stable bits only to common standards.
• Commands: CommandWrite as the generic fallback (capacity includes the trailing zero; rejects control characters/CR/LF injection/512B — the same discipline as the SMTP command layer).

The client: state machine and layered entrances

Four levels of layered entrances: Send/Line (the lowest wire level — the opening for SASL continuations and unknown extensions) → Receive (+OK/-ERR parsing) → Command (build + status response) → Begin/Next (multiline, line by line). Every standard command is built on the public entrances — STAT/LIST×2/UIDL×2/RETR/TOP/DELE/RSET/NOOP/QUIT are assemblies, not private paths: custom extensions stand on equal footing with official commands.

Streaming receipt and optional aggregation

RETR/TOP's Begin/Next: each line dot-un-escaped, never allocating the whole message, the line view borrowing the internal receive buffer (stable until the next read). Three outlets: write straight to a file (Chapter 45), incremental MIME parsing (Chapter 116's modules fed step by step), your own message store. The optional pop3_message layer: bounded RetrWrite/TopWrite on the same state machine (aggregates into owned bytes within a cap) plus MIME tree entrances — take only the line-by-line path and no aggregation code comes along (trimming freedom).

Authentication: plaintext rejected by default, the SASL family

pop3_auth: traditional USER/PASS plus RFC 5034 SASL (PLAIN and two OAuth mechanisms). Credential discipline: the configuration only borrows credentials; Base64 and plaintext scratch buffers are zeroed before the call returns (Chapter 77's password engineering honored at the protocol layer). Sending credentials in plaintext is rejected by default — AllowPlaintext exists only for explicitly controlled compatibility environments (the same stance as Chapter 106's Basic TLS baseline). A server rejecting credentials stays in AUTHORIZATION (retryable); a line error enters FAILED.

TLS: implicit and STLS

The base client is plaintext on 110; pop3_client_tls adds two forms: implicit TLS (handshake on connect); STLS (CAPA explicitly declares → send STLS → only after the +OK is fully consumed does the handshake run → re-run CAPA after the handshake — pre-upgrade capabilities never leak into the secure session; the same reasoning as SMTP STARTTLS's re-EHLO). A Verifier must be provided.

120.3示例

Run these commands from the repository root. The builder selects modules, public headers and platform libraries from the manifest, and also prints build and dependency-test logs. Terminal blocks show the example output. The second program starts a loopback server within the process and uses fixed demonstration data. See the client library README for real-service host, port, CA and runtime credential configuration.

First complete program: protocol primitives

The program below is from examples/pop3/protocol — an offline loop of replies and commands:

#include <stdio.h>
#include <xpop3.h>



/* 解析 STAT 并构建 RETR 命令。 */
int main(void)
{
	xpop3stat Stat;
	char arrCommand[32];
	size_t iSize;

	if ( !xrtPop3StatParse(XRT_STR_LITERAL("+OK 12 4096"), &Stat) ||
		 !xrtPop3CommandWrite(
			XRT_STR_LITERAL("RETR"),
			XRT_STR_LITERAL("1"),
			arrCommand,
			sizeof(arrCommand),
			&iSize
		) ) {
		return 1;
	}
	printf("messages=%llu command=%.*s",
		(unsigned long long)Stat.Messages, (int)iSize, arrCommand);
	return 0;
}

What just happened. (1) The program parses a fixed STAT reply with xrtPop3StatParse, obtains the message count, then constructs and prints RETR with xrtPop3CommandWrite. Counts and byte sizes retain 64-bit semantics; command arguments pass protocol validation. (2) Parsing and command construction open no connection and can serve custom clients or test servers. The same suite tests other reply, UIDL and multiline boundaries; this short program demonstrates two entry points.

Second complete program: local retrieval

The second program is from extlibs/xpop3/examples/offline/main.c.

#include <xpop3.h>

#include <stdio.h>
#include <string.h>

#include <math.h>
static inline double exampleTimerLimit(int64 Timeout)
{
    return Timeout == XRT_WAIT_FOREVER ? INFINITY : xrtTimer() + (double)Timeout / 1000.0;
}
static inline bool exampleTimerExpired(double Limit)
{
    return xrtTimer() >= Limit;
}
static inline int64 exampleTimerRemaining(double Limit)
{
    double Ms;
    if (Limit == INFINITY) return XRT_WAIT_FOREVER;
    Ms = ceil((Limit - xrtTimer()) * 1000.0);
    return Ms <= 0 ? 0 : Ms >= 0x1p63 ? INT64_MAX : (int64)Ms;
}

/* 只供本进程的回环 POP3 演示服务器使用。 */
static xnetaddr ExampleAddress;

typedef struct example_server {
	xnetlistener* Listener;
	double Deadline;
	bool Success;
} example_server;

static xnetaddrlist* exampleResolve(cstr sHost, xnetfamily Family, ptr pData)
{
	xnetaddr Address = ExampleAddress;
	(void)Family;
	(void)pData;
	if(strcmp(sHost, "pop3.example.invalid") != 0) return NULL;
	Address.Port = 0u;
	return xrtNetAddrListCreate(&Address, 1u);
}

static bool exampleSend(xnetstream* pStream, cstr sText, double Deadline)
{
	size_t iSize = strlen(sText);
	for(;;)
	{
		xnetresult Result = xrtNetStreamSend(pStream, sText, iSize);
		if(Result == XNET_RESULT_OK) return true;
		if((Result != XNET_RESULT_AGAIN) || !xrtNetStreamWait(
			pStream, XNET_STREAM_WAIT_WRITE,exampleTimerRemaining(Deadline), NULL)) return false;
	}
}

static bool exampleExpect(xnetstream* pStream, cstr sExpected,
	double Deadline)
{
	size_t iSize = strlen(sExpected);
	size_t iUsed = 0u;
	while(iUsed < iSize)
	{
		xnetbytes* pBytes = xrtNetStreamRecv(
			pStream, iSize - iUsed,exampleTimerRemaining(Deadline), NULL);
		xbytesview Bytes;
		if(pBytes == NULL) return false;
		Bytes = xrtNetBytesView(pBytes);
		if((Bytes.Size == 0u) ||
			(memcmp(Bytes.Data, sExpected + iUsed, Bytes.Size) != 0))
		{
			xrtNetBytesDestroy(pBytes);
			return false;
		}
		iUsed += Bytes.Size;
		xrtNetBytesDestroy(pBytes);
	}
	return true;
}

static int32 exampleServer(ptr pData)
{
	example_server* pServer = (example_server*)pData;
	xnetstream* pStream = xrtNetListenerAcceptWait(
		pServer->Listener,exampleTimerRemaining(pServer->Deadline), NULL);
	bool bOk;
	if(pStream == NULL) return 1;
	bOk = exampleSend(pStream, "+OK ready\r\n", pServer->Deadline) &&
		exampleExpect(pStream, "USER demo\r\n", pServer->Deadline) &&
		exampleSend(pStream, "+OK user\r\n", pServer->Deadline) &&
		exampleExpect(pStream, "PASS demo\r\n", pServer->Deadline) &&
		exampleSend(pStream, "+OK mailbox\r\n", pServer->Deadline) &&
		exampleExpect(pStream, "RETR 1\r\n", pServer->Deadline) &&
		exampleSend(pStream,
			"+OK message\r\nSubject: offline example\r\n\r\n"
			"Hello from loopback.\r\n.\r\n", pServer->Deadline) &&
		exampleExpect(pStream, "QUIT\r\n", pServer->Deadline) &&
		exampleSend(pStream, "+OK bye\r\n", pServer->Deadline) &&
		xrtNetStreamClose(pStream) &&
		xrtNetStreamWait(pStream, XNET_STREAM_WAIT_CLOSE,exampleTimerRemaining(pServer->Deadline), NULL);
	pServer->Success = bOk;
	xrtNetStreamDestroy(pStream);
	return bOk ? 0 : 2;
}

int main(void)
{
	xnetengineconfig EngineConfig;
	xnetresolverconfig ResolverConfig;
	xnetlistenconfig ListenConfig;
	xpop3clientconfig ClientConfig;
	example_server Server = { 0 };
	xnetengine* pEngine = NULL;
	xnetresolver* pResolver = NULL;
	xnetlistener* pListener = NULL;
	xpop3client* pClient = NULL;
	xthread* pThread = NULL;
	double Deadline = exampleTimerLimit(INT64_C(10000));
	xstrview Line;
	xmailnext Next;
	size_t iLines = 0u;
	bool bContentOk = true;
	bool bOk = false;

	xrtNetEngineConfigInit(&EngineConfig);
	EngineConfig.Backend = XNET_PORT_SELECT;
	EngineConfig.Workers = 1u;
	pEngine = xrtNetEngineCreate(&EngineConfig);
	if((pEngine == NULL) || !xrtNetEngineStart(pEngine)) goto Done;
	xrtNetListenConfigInit(&ListenConfig);
	if(!xrtNetAddrLoopback(&ListenConfig.Address,
			XNET_FAMILY_IPV4, 0u)) goto Done;
	pListener = xrtNetListen(pEngine, &ListenConfig, NULL, NULL, NULL);
	if((pListener == NULL) ||
		!xrtNetListenerLocal(pListener, &ExampleAddress)) goto Done;
	xrtNetResolverConfigInit(&ResolverConfig);
	ResolverConfig.Workers = 1u;
	ResolverConfig.Lookup = exampleResolve;
	ResolverConfig.CacheEntries = 0u;
	pResolver = xrtNetResolverCreate(&ResolverConfig);
	if(pResolver == NULL) goto Done;
	Server.Listener = pListener;
	Server.Deadline = Deadline;
	pThread = xrtThreadCreate(exampleServer, &Server, 0u);
	if(pThread == NULL) goto Done;
	xrtPop3ClientConfigInit(&ClientConfig);
	ClientConfig.Net.Engine = pEngine;
	ClientConfig.Net.Resolver = pResolver;
	ClientConfig.Net.Host = "pop3.example.invalid";
	ClientConfig.Net.Port = ExampleAddress.Port;
	ClientConfig.ReadCapabilities = false;
	pClient = xrtPop3ClientOpen(&ClientConfig,exampleTimerRemaining(Deadline), NULL);
	/* 明文凭据仅用于进程内回环演示；真实服务请用 client 范例的 TLS。 */
	if((pClient == NULL) || !xrtPop3ClientLogin(pClient,
			XRT_STR_LITERAL("demo"), XRT_STR_LITERAL("demo"),
			true,exampleTimerRemaining(Deadline), NULL) ||
		!xrtPop3ClientRetr(pClient, 1u,exampleTimerRemaining(Deadline), NULL)) goto Done;
	for(;;)
	{
		Next = xrtPop3ClientNext(pClient, &Line,exampleTimerRemaining(Deadline), NULL);
		if(Next != XMAIL_NEXT_ITEM) break;
		if(iLines == 0u)
			bContentOk = bContentOk &&
				(Line.Size == strlen("Subject: offline example")) &&
				(memcmp(Line.Data, "Subject: offline example",
					Line.Size) == 0);
		else if(iLines == 1u)
			bContentOk = bContentOk && (Line.Size == 0u);
		else if(iLines == 2u)
			bContentOk = bContentOk &&
				(Line.Size == strlen("Hello from loopback.")) &&
				(memcmp(Line.Data, "Hello from loopback.",
					Line.Size) == 0);
		else bContentOk = false;
		iLines++;
	}
	if((Next != XMAIL_NEXT_END) || !bContentOk || (iLines != 3u) ||
		!xrtPop3ClientQuit(pClient,exampleTimerRemaining(Deadline), NULL)) goto Done;
	bOk = true;

Done:
	if((pClient != NULL) && !bOk) (void)xrtPop3ClientAbort(pClient);
	xrtPop3ClientDestroy(pClient);
	if(pThread != NULL)
	{
		bool bThreadDone = xrtThreadWait(pThread) == XWAIT_OK;
		bOk = bOk && bThreadDone && Server.Success &&
			(xrtThreadExitCode(pThread) == 0);
		xrtThreadDestroy(pThread);
	}
	if(pListener != NULL)
	{
		(void)xrtNetListenerClose(pListener);
		while(xrtNetListenerState(pListener) != XNET_LISTENER_CLOSED)
			xrtThreadYield();
		xrtNetListenerDestroy(pListener);
	}
	if(pResolver != NULL) (void)xrtNetResolverDestroy(pResolver);
	if(pEngine != NULL) (void)xrtNetEngineDestroy(pEngine);
	if(!bOk)
	{
		fputs("offline POP3 retrieval failed\n", stderr);
		return 1;
	}
	puts("offline POP3 retrieval: 3 message lines");
	return 0;
}

What just happened. (1) The program creates a local listener and server thread, then resolves the demonstration host to loopback. This fixture does not implement CAPA, so automatic capability reads are disabled; select that configuration according to real server behavior. (2) The client completes USER/PASS, executes RETR and consumes xrtPop3ClientNext line by line. The program compares the subject, empty line and body byte for byte, requiring exactly three lines and a complete multiline terminator. (3) After QUIT it waits for the server thread and destroys the connection, listener, resolver and engine. Only content checks and successful retirement produce the success line. Fixed plaintext credentials are explicitly allowed only in this loopback fixture; the real-service example uses TLS and runtime credentials. This program does not delete mail: downloading and server deletion are separate operations.

120.4契约

• Protocol/client layering: the primitives depend on no network; the client is a state-machine assembly; standard commands are built on the public entrances.
• 64-bit facts: STAT counts and bytes are 64-bit; UIDL neither copied nor truncated.
• Shared dot family: multiline termination/un-escaping reuse the mail_net primitives — no POP3-specific duplicate.
• Injection discipline: CommandWrite rejects control/CR/LF/512B — same as SMTP.
• Streaming receipt: RETR/TOP borrow line by line (stable until the next read); no whole-message allocation; three outlets (file/incremental MIME/own storage).
• Optional aggregation: the message layer's bounded RetrWrite/TopWrite — the line path carries no aggregation baggage.
• Credential zeroing: borrowed configuration; scratch buffers zeroed before the call returns; sending credentials in plaintext rejected by default.
• Authentication semantics: rejected credentials stay in AUTHORIZATION; a line error enters FAILED, not reusable.
• TLS: implicit/STLS; the handshake only after +OK is fully consumed; re-CAPA after upgrade; a verifier is mandatory.
• UPDATE semantics: DELE marks take effect at Quit's UPDATE — Abort abandons the deletions.
• Ownership: shared objects borrowed, not destroyed; no calling from Worker callbacks; a single Client has no concurrency.

Common Mistakes

while ( Next(&Line) ) {
	rows[n++] = Line;   /* stored views: all invalid at the next Next */
}
process(rows);   /* all the last line */

while ( xrtPop3ClientNext(pClient, &Line) == OK ) {
	process_now(Line);          /* consume immediately */
	/* or copy: append_own_store(Line); */
}

Config.AllowPlaintext = true;   /* convenience: plaintext credentials */
Auth(pClient, User, Pass);

if ( capa_has_stls() ) {
	Stls(pClient, ...);   /* upgrade first */
}
Auth(pClient, User, Pass);   /* inside the encrypted tunnel */

Dele(pClient, 1);
/* crash/Abort: the mail is undeleted - looks like a bug */

Dele(pClient, 1);          /* mark */
...
xrtPop3ClientQuit(...);    /* UPDATE commits - truly deleted */
/* on processing failure Abort: the marks roll back - a safety net */

Programming Exercises

Summary