Chapter 74 Cryptography (Part 1): Hashes and HMAC

Chapter Orientation

This volume starts with the oldest and most used primitives. Hash functions compress arbitrary data into fixed-length digests — file checksums, content addressing, and set fingerprints all rest on them; HMAC gives the hash a key, upgrading "integrity" to "authenticated integrity"; KDFs (key derivation functions) turn passphrases or negotiated secrets into real keys. This chapter covers the digest family from MD5 to SHA-512/256, both the streaming three-step and one-shot call forms, HMAC's precomputed state, and the pair of clearly-divided derivation functions PBKDF2 and HKDF. They are the bricks of everything that follows: AEAD keys come from KDFs, TLS's handshake transcript hash is a hash, certificates' signatures sign digests. After this chapter you should answer for any scenario "which digest, with or without a key, derive or use directly".

74.1引入

Three real scenarios. You must generate a checksum for a multi-gigabyte file for downloaders to compare against — the file won't fit in memory. Your service exposes a Webhook callback and must verify "this request really comes from the partner who configured the key" — a bare hash won't do: anyone can hash any content; what you verify is "only key holders can compute it". Your users log in with passphrases and you need a symmetric encryption key from the passphrase — passphrase entropy is low (humans can't remember 256 random bits), and using it directly as a key falls to dictionary attacks instantly.

The three scenarios map to three primitive classes: digests (keyless, publicly computable, for fingerprints), MACs (keyed, computable only by key holders, for authentication), and KDFs (which derive suitable key material from low-entropy passphrases or high-entropy negotiated secrets). XRT implements the three in a finely trimmable feature family — SHA-256 and HMAC-SHA-256 are two separate macros, so a checksum-only tool need not compile in MAC code.

74.2概念

The digest family panorama

XRT ships seven digest algorithms, with length metadata unified by xrtCryptoHashSize:

First instinct for choosing: new designs start at SHA-256, always; touch MD5/SHA-1 only to talk to old protocols, and only for the fields the protocol requires, never for any security decision. xrtCryptoHashSize(XCRYPTO_HASH_SHA256) returns 32 — note this is a metadata query, independent of whether the algorithm is compiled in; capacity calculations need not depend on build configuration.

The streaming three-step and one-shot forms

Data does not always sit whole in memory — multi-gigabyte files and chunked network arrivals all need to feed a little, hash a little. The whole digest and HMAC family shares one state machine:

Init establishes state; Update may be called any number of times at any length — internally it accumulates by block, bit-for-bit identical to a one-shot feed; Final writes the fixed-length digest. The one-shot entry xrtSha256(数据, 长度, 输出) is the "three steps merged into one" convenience, semantically equivalent. The state structs (xsha256 etc.) are value objects on the caller's stack — no heap allocation, no shared state, any number of concurrent instances.

HMAC: giving the hash a key

xrtHmacSha256(密钥, 密钥长, 数据, 数据长, 输出) differs from the bare hash signature by only one pair of key parameters, yet the semantics leap in kind: only key holders can compute the same MAC. HmacSha256Init precomputes the inner/outer digest states — the key is processed once, and no later Update ever touches it again. This is the standard practice for "API signing, Webhook verification, session tokens": the sender attaches HMAC(密钥, 内容) with the content, the receiver recomputes and compares — with xrtConstTimeEqual, not memcmp (Chapter 77 explains why).

The HMAC family trims independently per underlying digest: the SHA-256 version depends only on SHA-256; SHA-384/512 share an implementation. Key length is arbitrary (overlong keys are pre-hashed); 32 random bytes are the recommendation.

The KDF twins: PBKDF2 and HKDF

Two derivation functions solving opposite problems, with surprisingly symmetric signatures — remember the division and you won't confuse them:

• PBKDF2 (passphrase → key): the input is a low-entropy passphrase, and the work factor (iteration count) raises the attacker's cost — every passphrase guess must also pay the same hundred thousand iterations. xrtPbkdf2Sha256(口令, salt, 迭代数, 输出). The salt must be random per user and stored with the ciphertext — defeating rainbow tables and preventing identical passphrases from deriving identical keys. The derivation performs zero heap allocations and securely clears all intermediate state before returning.
• HKDF (high entropy → key): the input is already high-entropy material (a DH shared secret, a master key) needing no slowness — what it solves is shape and purpose: turning arbitrary-length entropy into arbitrary-length keys, with the info parameter binding purpose. xrtHkdfSha256(salt, IKM, info, 输出). From the same IKM, info values "session" and "cookie" derive unrelated keys — the standard practice of same-origin, different-purpose isolation; all of TLS 1.3's key schedule is HKDF-driven.

HKDF opens in three layers, shallow to deep: Extract (salt+IKM → fixed-length PRK), Expand (PRK+info → arbitrary-length OKM), and a combined function (keeping PRK on the stack, done in one call) — most scenarios use the combined function; only streaming handshakes (like TLS) need the split.

The boundary with Chapter 12's hash library

XRT also has a non-cryptographic hash library (Chapter 12, the SipHash-family non-crypto hashes) — much faster, but with no security guarantees, used only for hash tables, dedup fingerprints, and cache keys. The boundary in one sentence: anything where attacker-controlled input participates and the result feeds a security decision uses this chapter's cryptographic digests; pure performance scenarios use Chapter 12.

74.3示例

First complete program: the SHA-256 streaming three-step

The following program comes from examples/crypto/sha256/main.c, hashing the two input blocks "hello " + "world" — SHA-256 of "hello world" is the most-used self-test vector:

#include <stdio.h>

#include <xrt.h>


















 */


/* 分块计算 SHA-256，并从不消耗状态的快照取得摘要。 */
int main(void)
{
	xsha256 State;
	uint8 arrDigest[XRT_SHA256_SIZE];

	xrtSha256Init(&State);
	if ( !xrtSha256Update(&State, "hello ", 6) ||
		 !xrtSha256Update(&State, "world", 5) ||
		 !xrtSha256Final(&State, arrDigest) ) {
		return 1;
	}
	for ( size_t i = 0; i < sizeof(arrDigest); i++ ) {
		printf("%02x", (unsigned int)arrDigest[i]);
	}
	printf("\n");
	return 0;
}

What just happened. (1) The xsha256 state sits on the stack, with the XRT_SHA256_SIZE constant declaring the output buffer — lengths always come from macros, never hand-written numbers. (2) Two Update calls yield a result bit-for-bit identical to a one-shot xrtSha256("hello world", 11, ...) — this is the redemption of "chunked arrivals need no whole buffer": in Chapter 45's file streams and Chapter 67's receive callbacks, digests accumulate exactly this way. (3) The output is raw bytes; the example prints them as 64 hex characters with a %02x loop — digest APIs produce bytes only; textualization is the caller's business (Chapter 28's HEX codec fits).

Second complete program: HMAC — only key holders can compute it

The second program comes from examples/crypto/hmac_sha256/main.c, the standard MAC of key "secret" over message "message":

#include <stdio.h>

#include <xrt.h>



















 */


/* 用一次性入口计算请求正文的 HMAC-SHA256。 */
int main(void)
{
	uint8 arrMac[XRT_SHA256_SIZE];

	if ( !xrtHmacSha256("secret", 6, "message", 7, arrMac) ) {
		return 1;
	}
	for ( size_t i = 0; i < sizeof(arrMac); i++ ) {
		printf("%02x", (unsigned int)arrMac[i]);
	}
	printf("\n");
	return 0;
}

What just happened. Read side by side with the first program, only two differences: Init takes the key parameter, and the output length is still 32 — but the semantics are a different world. Anyone can compute SHA-256 of "message", so a bare hash cannot prove "who the message came from"; only a party knowing "secret" can compute this MAC. In real systems the key comes from configuration or negotiation and uses a full 32 random bytes; the verifying side compares with xrtConstTimeEqual. Note HMAC also has the streaming three-step (HmacSha256Init/Update/Final); after Init precomputes the key state, multi-message scenarios need only one Update per message — examples/crypto/hash_tour demonstrates the streaming and one-shot self-consistency of every variant.

Third complete program: PBKDF2 — making keys from passphrases safely

The third program comes from examples/crypto/pbkdf2_sha256/main.c, deriving a 32-byte key from a passphrase with a hundred thousand iterations:

#include <stdio.h>

#include <xrt.h>


















 */


/* 使用显式 salt 和工作因子派生 32 字节密码密钥。 */
int main(void)
{
	uint8 arrKey[32];
	static const uint8 Salt[16] = {
		0x8Du, 0xCBu, 0x89u, 0x2Eu, 0xD0u, 0x31u, 0x42u, 0x49u,
		0xA1u, 0x7Eu, 0x1Fu, 0x5Bu, 0x12u, 0x6Du, 0xE9u, 0x75u
	};

	if ( !xrtPbkdf2Sha256(
			"correct horse battery staple", 28,
			Salt, sizeof(Salt), 100000, arrKey, sizeof(arrKey)
		) ) {
		return 1;
	}
	for ( size_t i = 0; i < sizeof(arrKey); i++ ) {
		printf("%02x", (unsigned int)arrKey[i]);
	}
	printf("\n");
	xrtSecureZero(arrKey, sizeof(arrKey));
	return 0;
}

What just happened. (1) The parameter order is (passphrase, salt, iterations, output) — 100000 iterations is the work factor: a legitimate user pays a hundred thousand iterations per login (milliseconds, acceptable), and an attacker pays the same per passphrase guess, raising dictionary-attack cost a hundred-thousand-fold. (2) The salt is an explicit 16-byte constant — fixed in the example for reproducibility; production must generate it randomly per user and store it with the ciphertext (see Pitfall 3). (3) The ending xrtSecureZero(arrKey, ...) wipes the derived key from the stack — key material is cleared the moment it's used up, a fixed gesture throughout this volume. May passphrase and salt overlap? No — output must not overlap input; the contract validates all parameters before the first write.

Fourth complete program: HKDF — same-origin keys isolated by purpose

The fourth program comes from examples/crypto/hkdf_sha256/main.c, one call turning three pieces of material into a session key:

#include <stdio.h>

#include <xrt.h>




















 */


/* 一行派生 32 字节会话密钥。 */
int main(void)
{
	uint8 arrKey[32];

	if ( !xrtHkdfSha256(
			"salt", 4, "input key material", 18,
			"session", 7, arrKey, sizeof(arrKey)
		) ) {
		return 1;
	}
	for ( size_t i = 0; i < sizeof(arrKey); i++ ) {
		printf("%02x", (unsigned int)arrKey[i]);
	}
	printf("\n");
	return 0;
}

What just happened. The three parameters each play their part: salt is a public random value (change the salt and it's a whole new key), IKM is the raw entropy (a demo string here; real scenarios use a DH shared secret — Chapter 76), and info is the purpose binding. Change info to "cookie" and run again — the output is completely different: this is "one entropy derives many keys, keys never affecting each other", the purpose isolation. Chapter 77's session example will put HKDF into a complete chain: x25519 negotiation → HKDF derivation → AEAD encryption.

74.4契约

• Digest family: seven algorithms (MD5/SHA-1/SHA-224/256/384/512/512_256); xrtCryptoHashSize gives length metadata (independent of compilation); streaming three-step and one-shot are bit-for-bit identical; states are stack value objects, no heap allocation, arbitrary concurrency across instances.
• MD5/SHA-1 boundary: historical protocol interop only (HTTP Digest, WebSocket); must not be used in new designs for signatures, certificates, content trustworthiness, or passphrase storage.
• HMAC: Init precomputes the key state; Update never touches the key again; SHA-256 trims independently, SHA-384/512 share; key length arbitrary, 32 random bytes recommended.
• PBKDF2: for low-entropy passphrases only; iterations and output length must exceed zero; output must not overlap passphrase/salt; zero heap allocations, HMAC state and intermediates cleared before returning; maximum output UINT32_MAX × HashLen (block counter is 32-bit).
• HKDF: for high-entropy material only; three entry layers (Extract/Expand/combined); info binds purpose for same-origin isolation; the combined function keeps PRK on the stack.
• Core tools: xrtConstTimeEqual constant-time comparison (no early exit on a differing first byte); xrtSecureZero volatile zeroing (defeating dead-store elimination); the randomness source comes from the separate RANDOM_SECURE feature (Chapter 12) — crypto_core carries no randomness.
• Errors: failures declared by return value with details in the thread error slot; parameter/range/block-limit errors all return before the first output write — failure never pollutes the output buffer.

Common Mistakes

uint8 Digest[16];
xrtMd5(Content, iSize, Digest);   /* "integrity check" of a new design */
store(Digest);                     /* an attacker can craft colliding content */

uint8 Digest[XRT_SHA256_SIZE];
xrtSha256(Content, iSize, Digest);  /* new designs default to SHA-256 */
store(Digest);

   and never feed its result into any security decision */

/* request carries ?sig=sha256(body); the server recomputes and compares */
uint8 Digest[XRT_SHA256_SIZE];
xrtSha256(Body, iSize, Digest);
if ( memcmp(Digest, SentSig, sizeof(Digest)) == 0 ) {
	accept();   /* the attacker edits body, recomputes sig, and forges away */
}

/* HMAC: only key holders can produce a valid MAC */
uint8 Mac[XRT_SHA256_SIZE];
xrtHmacSha256(ApiKey, sizeof(ApiKey), Body, iSize, Mac);
if ( xrtConstTimeEqual(Mac, SentMac, sizeof(Mac)) ) {
	accept();
}

static const uint8 Salt[16] = { /* hard-coded in the source */ };
xrtPbkdf2Sha256(Password, iLen, Salt, sizeof(Salt),
	100000, Key, sizeof(Key));

uint8 Salt[16];
if ( !xrtSecureRandom(Salt, sizeof(Salt)) ) {  /* random per user */
	return false;
}
xrtPbkdf2Sha256(Password, iLen, Salt, sizeof(Salt),
	100000, Key, sizeof(Key));
save_with_ciphertext(Salt);   /* the salt is not secret; store it with the ciphertext */

Programming Exercises

Summary