WAI Extension: Staged Delivery

Mirrored from the canonical text at commit 7b2b1a75 (2026-10-05).

Status: Draft. A base first, refinements after, each its own object, bound to what it refines by content hash. Every stage is presentable, every prefix of a stage set states its determinism tier, and a refinement that never arrives leaves the stages below it intact. Reference impl: wai-rs — staged::{StageChain, StageSink}, staged::{wir1, predict, wir1_arith, tier, schedule}, container::Refines, codecs::BASE_ADMISSION; the wai stage tool; the corpus staged-conformance/. Keywords MUST, MUST NOT, SHOULD, MAY are RFC 2119/8174.

1. Scope and model

A stage set is a base and layers. The base is any envelope a sink decodes under SPEC §4 or §7.1. A layer is its own WAI1 envelope whose capability is refinement-only (SPEC §5, Refinement capabilities) and which names, by content hash, the base and the envelopes it refines.

A sink holds, for every unit of the base’s output, a stage (0 after the base decodes) and a producer: the content hash of the envelope that produced the unit’s current output, the base at stage 0. Layers apply per unit, and every stage is presentable. A base never names its layers, so a layer MAY be produced after its base, by the base’s publisher or by a party the sink accepts (§10).

2. Terms

• Content hash: jwp-receipts §2.1, BLAKE3("moq-jwp:object\x01" ‖ envelope_bytes) over the envelope bytes exactly as received, written as 64 lowercase hexadecimal digits.
• Unit: the smallest presentable piece of a base’s output, fixed by the base’s capability: one frame of video; the whole picture of an image; the whole clip of audio; the whole attribute tensor of a splat cloud.
• Span: a contiguous run of a base’s units.
• Producer: for a unit, the content hash of the envelope that produced its current output.
• Refinement-only capability: a capability registered with that role (SPEC §5, §8).

3. The refines member

A WAI1 manifest whose model_requirement.capability is refinement-only MUST carry refines, and its model_requirement.fallback MUST be null. A manifest whose capability is not refinement-only MUST NOT carry it.

"refines": { "base": "<64 hex>",
             "prev": "<64 hex>" | [ {"producer": "<64 hex>", "count": 4}, ... ],
             "layer": 2,
             "span": { "from": 0, "count": 4 } }

Rules:

• Hashes MUST be 64 lowercase hexadecimal digits (in a WAI2 entry they are component ids; SPEC §7.1, Refinement components).
• When layer is 1, every producer prev names MUST be base.
• A run array MUST NOT contain two adjacent runs with the same producer. A layer whose units have one producer MUST write prev as a string, so an array of one run is malformed.
• With a span, the run counts MUST sum to its count. The span MUST lie within the base’s units, and with no span the run counts MUST sum to the base’s unit count; a sink checks these once it holds the base (§4.1 step 2).
• from + count MUST fit in 32 bits, and so MUST each count.

Any other JSON type (an integer is written as SPEC §2 defines one), or a value outside these rules, makes refines malformed (refines-malformed); the envelope still parses. A member of refines this revision does not define is ignored (SPEC §8).

4. Applying layers

4.1 Algorithm

A sink that implements this extension MUST process each received envelope R whose capability is refinement-only as follows. Steps 1–6 decide for the whole envelope; steps 7–10 decide for each unit u the layer covers. The first step that fails is the reason for the envelope (1–6) or for the unit (7–10).

1. refines is present and well formed (§3) (refines-malformed). R’s capability is refinement-only, and its model_requirement.fallback is null (capability-not-refinement).
2. The sink has decoded a base whose content hash is refines.base. Otherwise it MAY hold R until its bound (§4.3), and then discards it (base-absent). With the base, refines is checked against the base’s unit count (§3) (refines-malformed).
3. The base was decoded through its manifest’s model_requirement.capability, or through the WAI2 component whose exploded envelope (SPEC §7.1) has the content hash refines.base, and not through a fallback (base-via-fallback).
4. The capability the base was decoded through is admitted by R’s capability (SPEC §5, Base admission) (base-not-admitted).
5. Where the sink verified a delivery receipt for the base, it has verified one for R as §10 requires (refinement-unattested).
6. The payload is well formed for R’s capability, in this order: it parses (Appendix A) (payload-malformed); its unit count is the number of units the layer covers (payload-malformed); its declared cost (Appendix A.5) is within the limit the sink applies, if it applies one (cost-over-limit); retaining R keeps the chain within its budget (§4.5) (chain-over-limit); and every unit’s stream decodes to its last symbol and ends canonically (payload-malformed). A payload malformed for one unit is malformed for all. No unit’s shape is read before step 9.
7. Let P(u) be the producer prev gives for u. u is conflicted when it is frozen, or when an envelope with different content, the same base and the same layer, whose producer for u is also P(u), has passed step 8 for u, earlier (before the base’s chain was dropped included, §4.4) or in the same batch (§4.4) (layer-conflict). A conflicted unit is frozen: the sink MUST apply no further layer to it, except the replay of its own record after its base’s chain is dropped and the base decoded again (§4.4); it keeps the output it had (a layer applied to it before the conflict stays applied), and the sink MUST report the conflict.
8. u is ready when its stage is layer − 1 and its producer is P(u). A unit whose stage is below layer − 1 keeps R held for it, and is evaluated again when its stage advances; at the bound R is discarded for that unit (layer-gap). Any other unit is not ready (prev-mismatch).
9. R’s payload reads u’s current output in the shape its staged form needs (geometry-mismatch).
10. Under a live schedule, R arrived at least lead_ms before u’s presentation time (§8) (deadline-missed).

The sink then computes, for every unit that passed, the next stage by R’s staged form (§6), and sets the unit’s stage to layer and its producer to R’s content hash. Application to one unit is atomic. A sink MUST NOT present a unit’s output from a layer that failed for that unit, and MUST NOT present a partly applied unit.

4.2 Reasons

The list is closed. Two conforming sinks that receive the same envelopes in the same order, at the same times, with the same receipts, hold bound, cost limit, chain budget and schedule, report the same reasons per envelope and per unit and reach the same stages and producers. The order of arrival changes the outcome only through the chain budget at step 6 (§4.5), step 7, the hold bound at step 8 (§4.3: what is refused on arrival and what expires) and step 10.

4.3 Holding

A sink MUST declare the bound to which it holds envelopes it cannot yet apply, as a count of envelopes and a duration from arrival. An envelope held past the duration is discarded with its hold reason (base-absent, or layer-gap for each unit it is held for). An envelope that would exceed the count is discarded on arrival with the reason it would have been held for. A unit refined by a held envelope that is released within the bound, and that the chain budget admits, reaches the same output as one refined in order.

An envelope held for its base and received again is held once. If it comes with stronger evidence of its delivery receipt (§10: a receipt verified over its bytes is stronger than one verified by inclusion alone, which is stronger than none), the sink keeps the stronger evidence for steps 5 and 10; its hold still runs from its first arrival.

4.4 Batches

When the base an envelope names is decoded, the envelopes held for it are released together; when a unit’s stage advances, the envelopes held for that unit are released together. Each envelope released, or offered on arrival, passes steps 1–6, and for each unit step 7 against the envelopes that passed step 8 earlier and step 8, alone. The (envelope, unit) pairs that are ready then form one batch: step 7 compares every pair of the batch with the others, and only then do steps 9 and 10 and application run. So two conflicting layers released together are both refused and the unit frozen, whichever arrived first; a layer that conflicts with one that passed step 8 earlier is refused, and the unit frozen at the stage it has. When a unit is frozen, every envelope held for it is refused for it (layer-conflict). Applying a batch can release another, which is evaluated the same way.

A sink that decodes again a base whose chain it holds keeps the chain: re-decoding a base does not reset its units’ stages, producers or frozen state, and releases and reports nothing. If the sink has verified the base’s delivery receipt since, the base is attested from then on (§10). Attestation belongs to the base’s content hash: a base once attested stays attested, held or dropped, and a decode that carries no receipt does not undo it; a receipt the sink verifies before it decodes the base makes the base attested when it is decoded. Only forgetting the base (§14) ends it. A sink MAY drop a base’s chain when it no longer presents the base. Every envelope held for the chain’s units is then discarded (base-absent, reported once for each), and an envelope for that base is held as for any base the sink has not decoded. The sink keeps each unit’s record: the producers of the stages it reached, the envelope that passed step 8 at its last stage without being applied, and whether it is frozen. A record is empty when the unit reached no stage, nothing passed step 8 for it and it is not frozen; the sink keeps only the others, and nothing of a base all of whose records are empty and that is not attested. When it decodes the base again, each unit starts at stage 0 with its record. Step 7 compares a layer with the recorded producers, so a unit advances again only along them: a layer whose key a recorded stage holds with other content conflicts, and the unit freezes. A frozen unit stays frozen: it may replay its own recorded producers, in order, up to its recorded stage and never past it, and every other layer is refused for it (layer-conflict). So it presents its base’s output or the output of one of its recorded producers, never any other.

A sink MUST declare a record budget, in bytes, for what it keeps of the bases it does not hold: their records, and the attestation of a base dropped while attested or attested before it was decoded (§10). A kept base counts 32 bytes, and each record it keeps 8 bytes, 32 for each recorded stage and 32 for an envelope that passed step 8 without being applied. When what it keeps passes the budget, the sink forgets whole bases, the one whose record or attestation it wrote longest ago first, until it is within it. A forgotten base is as one the sink never decoded and never attested (§14).

These counts are the interoperable measure: every conforming sink counts the same bytes, so two sinks with the same record budget keep and forget the same bases. What an implementation allocates for each counted byte is its own, and it states that ratio beside its record budget.

4.5 Chain budget

A sink MUST declare a budget, in bytes, for the chain of each base it decodes. The chain’s state is the sum of:

• for every unit, the bytes of its current output (1 per u8 element, 2 per i16 sample), and 32 bytes for each stage its record holds (the producers step 7 compares: its stages above 0, or, for a base decoded again, the stages it reached before, §4.4);
• for every layer the chain retains, its payload’s bytes and its declared working bytes (Appendix A.5), and, for each unit it covers and has not been applied to, one unit’s output (its declared output over its units) and 32 bytes (the stage it would add).

A chain retains a layer from the moment it passes step 6 to the end of the offer, release or expiry after which no unit is held for it; the sink then drops it. At step 6, after the cost limit, a layer whose payload, declared output and working bytes and 32 bytes for each unit it covers, added to the state, pass the budget is refused (chain-over-limit) before any stream is decoded. An envelope the chain still retains, received again, adds nothing. Applying a layer to a unit replaces its output with one, and adds a stage, that the layer’s retention counted, and that unit’s share leaves the layer’s: an application lowers the state by the unit’s old output (and by 32 bytes more where it replays a recorded stage). The state rises only when a layer is admitted, so it never passes the budget, and the bytes a chain holds never pass the state.

5. The tier of a stage, and the stage hash

• The base step: decode-equivalence when the capability the base was decoded through has the registry class StandardDefined, or NeuralIntegerExact with a pin that resolved (SPEC §3.1) and declares decode-equivalence; entropy-consistency when the class is NeuralEntropyExact; otherwise none.
• A layer step: the tier its registration gives (§6); decode-equivalence for the residual form.

A unit’s tier is the weakest step of its chain, composed as derived-tracks §3 rule 1. A layer acts on reconstructed output, so entropy-consistency does not pass through it (derived-tracks §3 rule 2): a unit over such a base holds no tier once a layer applies.

The stage hash of a unit is the SHA-256 of its output in the canonical buffer of SPEC §7 Digests: RGB8 for a picture or a frame; the samples as signed 16-bit little-endian integers, channels interleaved, for audio; the attribute tensor’s bytes, plane after plane, for a splat. The stage hash of several units is the SHA-256 of their 32-byte stage hashes concatenated in unit order, as that section defines a clip digest. A stage hash MUST NOT be published, compared as a conformance value, or sealed in a receipt unless every unit it covers holds decode-equivalence.

6. Staged forms

A refinement-only capability’s registration fixes its staged form, payload format, admitted bases (SPEC §5, Base admission), tier and truncation points.

6.1 Residual (WIR1)

wai.image.int_refine, wai.video.int_refine, wai.audio.int_refine and wai.splat.int_refine carry a WIR1 payload (Appendix A): one rANS stream per unit. For each element of a unit, the next stage is clamp(pred + r · q). pred is the payload’s predictor applied to the unit’s current output; r is the decoded residual; q is the payload’s step; and the clamp is to the element type’s range. Every operation is on integers of at most 64 bits.

A residual layer reads no parameter set: everything its decode reads is in its payload and in the unit’s current output. The base’s parameter set was checked when the base was decoded (SPEC §3.1, check 5 among them), and nothing a layer reads needs the check again. Tier: decode-equivalence. Every layer boundary is a truncation point.

6.2 Other staged forms

This revision registers the residual form only. A refinement-only capability of another staged form (the continuation of a world’s state, an enhancement layer carried as its own object, a byte prefix of a progressive codestream, the further codebooks of a residual-vector-quantised code) is registered by a revision that fixes its form, payload, admitted bases, tier and truncation points. Where the form decodes a prefix of a base codec’s own codestream, that revision cites the clause of the codec’s standard that defines the decode of such a prefix, and its reference sink decodes one.

6.3 Truncation points

A refinement capability declares truncation points only where its staged form defines the decode of a prefix. For the residual form that is every layer boundary.

7. Deferral

A layer MAY be published after its base, served only on request, or never. A sink MUST present the current stage of a unit without waiting for a layer it has not been told to expect. A later revision of a refinement is a new chain from the same base, distinguished by prev from layer 2 on. Every layer 1 names the base as its producer, so prev cannot distinguish two revisions of layer 1: they are two envelopes of the same base, layer and producer, and conflict (§4.1 step 7).

8. Live deadlines

Under a live schedule, every unit has a presentation time, given by the base’s timing and the target latency the sink applies, and every layer has a lead, lead_ms: the time before a unit’s presentation by which the layer must have arrived, and been verified under §10, to be applied to it. A sink MUST NOT delay presenting a unit’s current stage to wait for any layer. A layer that arrived less than lead_ms before a unit’s presentation time is not applied to that unit (deadline-missed). It MAY be applied to units of its span that are not yet due. A still image MAY be refined after it is presented. A sink MUST count every unit presented below the stage its expected layers would have reached on time. Verifying a layer’s receipt (§10) adds the time to receive and verify that receipt to the layer’s arrival, which sets the least lead_ms that can be met: over a base whose receipt the sink verified, a layer counts as arrived when the sink verified its receipt (§4.3).

A stage set split across bases (a base for each group of frames) has one clock: a base whose units are the set’s units from position k on presents its unit i at the set’s position k + i, and a sink gives each base its first position.

9. Carriage

A layer is a WAI1 envelope, carried and stored as any envelope is, or a WAI2 component with role: "refinement" (SPEC §7.1, Refinement components). A component and its exploded envelope (SPEC §7.1) are the same object for every rule that names an object by content hash, so a layer reaches the same stages whichever way it travelled. This revision binds no other carriage.

10. Attestation of layers

Each layer is an object under jwp-receipts. A sink that verified a delivery receipt for a base MUST NOT apply a layer unless it has verified a delivery receipt for the layer, through that extension’s §3 step 6, over the layer’s envelope bytes as received. That receipt must be signed by the base’s signer, or by a key the sink holds with the authority delivery under the same origin (jwp-receipts §7.5). A receipt verified without the layer’s envelope bytes (§3 steps 1, 2, 3 and 5, step 4 not run) shows that a publisher signed an object with that content hash; it never authorizes an application. A content hash in refines binds a layer to a base, not to its publisher. A sink that verifies a base’s delivery receipt after it decoded the base applies step 5 to every envelope that arrives after it; envelopes it admitted before keep the decision they had. The verification belongs to the base’s content hash and holds across the base’s chain being dropped and the base decoded again (§4.4).

11. Counting bytes

A figure of delivered bytes for a stage set:

• MUST count the bytes of the WAI1 and WAI2 envelopes as delivered (SPEC §2: magic, length prefixes, manifest and payload);
• MUST state separately the receipt, index and transport framing bytes, and the parameter-set bytes a sink fetched;
• MUST NOT count a text re-encoding of a payload (JSON, base64) in place of the payload’s registered binary format.

A figure over a payload in any other form is not a staged-delivery byte figure.

12. Conformance

A conforming sink of this extension implements §4, §5, §8 and §10, and every staged form whose capabilities it advertises. The corpus staged-conformance/ holds the cases: for each, a conforming sink reports the stated reason per envelope and unit, and reaches the stated stage, producer, tier and stage hash per unit. Its README.md states the cases, what each base’s stage 0 is checked against, and the independent verifier staged_verify.py.

13. Earlier layered carriage

A prototype MoQ binding in this repository carries a layer index in an object header (mrl_layer, with mrl_total_layers and a subscriber’s max_mrl_layer). For an object that is a WAI1 envelope with refines, a sink reads the layer from refines.layer, which sits inside the envelope and so inside its content hash, and ignores the header.

14. Security considerations

• A layer is not authenticated by its refines; §10 authenticates it.
• Deferral lets anyone mint a chain for a public base hash. §10 and §4.1 step 7 bound what a sink presents. A conflict between two attested chains is evidence that an authorized signer signed both, and the sink reports it.
• A hold bound (§4.3), a cost limit (Appendix A.5) and a chain budget (§4.5) are resource limits. A layer’s header declares its output, its symbols and its working bytes before any symbol is decoded, so a sink refuses a layer past its limit or its budget without decoding it. A sink’s own work for a layer is linear in the layer’s payload bytes and declared symbols, up to a factor logarithmic in its units: it parses the payload once, at step 6; decides each unit the layer covers once for each offer or release of the layer; finds a unit’s producer in prev by bisection over its runs; and compares a unit’s key at step 7 with the producers of that unit’s stages. Beyond the chain’s state, it keeps a fixed amount per unit of the base and per unit held.
• An encrypted layer (confidentiality) binds to the content hash of its base’s envelope as delivered, ciphertext included, and is decrypted before §4.1 step 6.
• A sink keeps the record of a base whose chain it dropped (§4.4), so a base decoded again cannot be refined along other producers than it had; the record counts in the chain’s state when the base is decoded again (§4.5), and, while the base is not held, in the sink’s record budget (§4.4). A sink that forgets a base, by StageSink::forget_base in the reference or because its record budget is full, treats it as never decoded and never attested: a layer that conflicted with the forgotten history can apply, and step 5 no longer requires a receipt for it. A sink sizes its record budget for the bases it may decode again.
• Where the base is attested (§10), a frozen unit presents only its base’s output or the output of one of the producers it recorded, replayed in order (§4.4): a party that can sign conflicting layers can stop a unit’s refinement, and cannot make it present anything else. Over a base whose receipt the sink did not verify, anyone who learns the base’s hash can mint a layer, and the first one ready for a unit applies to it.

Appendix A. WIR1

A.1 Layout

Every multi-byte field is little-endian; offsets are in bytes.

Each stream is decoded as integer-payloads §2 decodes a rANS stream (precision 16, the bypass escape of 4-bit groups), against the unit’s table at offset.

A.2 The unit kind

A capability fixes what its units hold (§6.1), and so the fields a payload may carry:

• RGB8 (wai.image.int_refine, wai.video.int_refine): elem 1, predictor 0 or 1, d2 = 3;
• PCM (wai.audio.int_refine): elem 2, predictor 0 or 2, d1 = 1;
• attribute tensor (wai.splat.int_refine): elem 1, predictor 0.

A.3 Order of the checks

A parser checks, in this order, and refuses with the first that fails:

1. the magic (bad_magic);
2. the version byte: present (stream_length), and 1 (version);
3. the 44-byte header: present (stream_length);
4. the reserved field (reserved_nonzero);
5. the header’s fields, in the order elem, predictor, table_mode, n_units, d0/d1/d2, the rate fields, q, cdf_len; then the unit kind’s rules (A.2) (field_range);
6. the caps, in this order (too_large): a unit’s d0·d1·d2 ≤ 2^26; n_units ≤ 2^20; n_units·d0·d1·d2 ≤ 2^30;
7. the tables, one at a time, each present (stream_length) then checked (cdf_malformed); the stream lengths, present (stream_length), each a multiple of 4 and at least 8 (stream_length); the streams, present (stream_length);
8. nothing after the last stream (trailing).

Every size is formed in 64-bit arithmetic with overflow checked, and a size past 64 bits is past the caps, so a 32-bit sink refuses exactly what a 64-bit one does. A parser reads tables and lengths from the bytes the payload holds and allocates nothing a payload’s bytes do not justify.

A.4 Decode

For each unit u the layer is applied to, with the unit’s current output of shape (e0, e1, e2):

1. The shape must be the one the predictor reads (geometry_mismatch), before any symbol is decoded: (d0, d1, d2) for identity; (⌈d0/2⌉, ⌈d1/2⌉, d2) for up2; (n′, 1, d2) with ⌊n′·rate_num/rate_den⌋ = d0 for rate.
2. For element i in raster order (i = (a·d1 + b)·d2 + c), decode the residual r from u’s stream. A stream that runs out during a symbol’s decode is refused at that symbol (stream_exhausted), and nothing after it is decoded. The element is clamp(pred(i) + r·q), to [0, 255] for u8 and [−32768, 32767] for i16. r lies in [−2^31, 2^31 − 1] and |r·q| < 2^47, so every step fits i64.
3. After the unit’s last element, the stream ends canonically (integer-payloads §2) (stream_noncanonical).

Predictors, with prev the unit’s current output:

• identity: pred(i) = prev(i).
• up2: prev is h′ × w′ × c with h′ = ⌈d0/2⌉, w′ = ⌈d1/2⌉ and c = d2, so odd sizes are allowed. For output (y, x, ch): iy = y >> 1, fy = y & 1, ix = x >> 1, fx = x & 1, and p(Y, X) = prev[min(Y, h′ − 1), min(X, w′ − 1), ch]. The prediction is p(iy, ix) when both are 0; (p(iy, ix) + p(iy, ix + 1) + 1) >> 1 when only fx is 1; (p(iy, ix) + p(iy + 1, ix) + 1) >> 1 when only fy is 1; and (p(iy, ix) + p(iy, ix + 1) + p(iy + 1, ix) + p(iy + 1, ix + 1) + 2) >> 2 when both are. This is the half-pel arithmetic of integer-payloads §3.
• rate: prev holds n′ samples of c = d2 interleaved channels. For output sample i and channel ch: s = i·rate_den, lo = s div rate_num, ρ = s mod rate_num, hi = min(lo + 1, n′ − 1), a = prev[lo·c + ch], b = prev[hi·c + ch], and the prediction is rdiv(a·rate_num + (b − a)·ρ, rate_num), where rdiv(n, d) = sign(n)·⌊(2|n| + d) / 2d⌋ rounds half away from zero. This is a + (b − a)·ρ/rate_num rounded half away from zero, the linear interpolation at source position i·rate_den/rate_num, in exact integer arithmetic.

A.5 Declared cost

A payload’s header declares, before any symbol is decoded: its output, n_units·d0·d1·d2 elements of 1 byte (u8) or 2 bytes (i16); its symbols, n_units·d0·d1·d2, one entropy-decode step each; and the most bytes its decode holds at once beyond the payload and the stages below, its parsed tables (4·T·cdf_len), the index of its streams (8·n_units) and one unit’s output while it is built. It runs no network. A sink that applies a limit compares this cost with it after the parse and before any stream is decoded (§4.1 step 6, cost-over-limit), and counts it in the chain’s state while it retains the layer (§4.5).

A.6 Refusal codes

The list is closed, and its codes are those integer-payloads §8 uses for the same faults: bad_magic, version, reserved_nonzero, field_range, too_large, cdf_malformed, stream_length, trailing, stream_exhausted, stream_noncanonical and geometry_mismatch. A parser reports the first that applies, in the order of A.3. §4.1 runs A.4 in two parts: step 6 decodes every unit’s stream to its end (A.4 steps 2 and 3, which read no prediction), so stream_exhausted and stream_noncanonical are the envelope’s payload-malformed, reported before any unit’s shape is read; step 9 then checks each unit’s shape (A.4 step 1), so geometry_mismatch is that unit’s geometry-mismatch. A payload whose n_units is not the number of units the layer covers is the envelope’s payload-malformed with the code field_range. Every other code is the envelope’s payload-malformed.