Skip to main content

WAI Extension: Integer Payloads

Mirrored from the canonical text at commit 498e6b88 ().

Status: Draft. Pins the payload formats of the WAI-native integer codecs that SPEC.md §5 registers without one (Payloads of the WAI-native integer codecs): WIV1 for wai.video.int_motion and wai.video.int_hyper, WIH1 for wai.neural.int_hyper, WIA1 for wai.audio.int_codec and WIS1 for wai.splat.int_codec. Defines each layout, its decode, the checks a sink makes and the closed list of refusal codes; names each capability’s parameter-set members; and makes the JSON the encode and export tools write a converter input, never a payload. Writes down the payload formats and parameter sets that SPEC §5 keeps for wai.neural.int_synth (§13) and wai.neural.int_mlicpp (WIM1 and WMB1, §14). Pins WIF1 (§16), one frame of WIV1, the payload of the frame forms wai.video.int_motion.frame and wai.video.int_hyper.frame, which frame-stream carries one frame an object. Pins WIC1 and WCF1 (§17), the clip and the frame of wai.video.int_cond and wai.video.int_cond.frame, conditional inter-frame coding at the integer tier, and their one-file parameter set WCB1. Reference impl: wai-rs — int_payload::{wire, decode_int, resolve}, int_payload::{wiv1_from_json, wih1_from_json, wia1_from_json, wis1_from_json}, codecs::PINNED_INT_PAYLOADS, container::check_int_payload_framing. Keywords MUST, MUST NOT, SHOULD, MAY are RFC 2119/8174.

1. Scope

capabilitypayloadparameter set (§9)
wai.video.int_motionWIV1 with keyframe 0 (§3)none
wai.video.int_hyperWIV1 with keyframe 1, a nested WIH1 (§3)several files: entropy.json, g_s.whm, h_s.whm
wai.neural.int_hyperWIH1 (§4)several files: entropy.json, g_s.whm, h_s.whm
wai.audio.int_codecWIA1 (§5)one file: model.wam
wai.splat.int_codecWIS1 (§6)one file: g_s.whm
wai.video.int_motion.frameWIF1 with keyframe 0 (§16)none
wai.video.int_hyper.frameWIF1 with keyframe 1, an intra frame a nested WIH1 (§16)several files: entropy.json, g_s.whm, h_s.whm
wai.video.int_condWIC1, a clip of WCF1 frames (§17)one file: model.wcb
wai.video.int_cond.frameWCF1 (§17)one file: model.wcb

The last four are registered with their format (SPEC §5): they are not completions of a registration, and §11 lists no envelope for them.

Why this pin is permitted. Until this revision, these capabilities’ SPEC §5 rows named the decoder that read their payload, and no byte layout. The rows of wai.video.int_motion, wai.audio.int_codec and wai.splat.int_codec also named the encode or export tool that wrote a bitstream, a JSON document; those of wai.neural.int_hyper and wai.video.int_hyper named their decoders only. SPEC §5 called every row’s payload the bytes its codec normally emits, which for these rows was the tools’ JSON. So this revision changes what these strings carry. SPEC §8 permits it under Completing a registration: a revision may complete the registration of a WAI-native string whose row named a decoder and fixed no byte layout by pinning one, and it lists the envelopes that become non-conforming (§11; SPEC §5). SPEC §8’s rule that a registered string’s payload format is fixed forever applies to these strings from this revision on. The encode and export tools still write JSON that carries the same fields; it is a converter input (§10).

wai.neural.int_synth and wai.neural.int_mlicpp keep the payload formats SPEC §5 gives them, and §13 and §14 write those formats down. A sink decodes them and the five strings above (wai.video.int_motion, wai.video.int_hyper, wai.neural.int_hyper, wai.audio.int_codec, wai.splat.int_codec) through one path (int_payload::decode_int), and §8 names the codes it reports for those two.

2. Common rules

Encoding. Every multi-byte field is little-endian. Offsets are in bytes.

Framing. A payload starts with its four-byte magic and a version byte. This revision defines version 1 for every format. Every reserved field is zero. Nothing follows the last section.

rANS streams. A stream is a u32 length L followed by L bytes. L is a multiple of 4 and at least 8. It is decoded as follows (precision 16, with a bypass escape of 4-bit groups):

The decode these rules define is integer-operators E1, rans64_decode, with the modes t4097, lenient and canonical.

One clip, several payloads. These rules fix how a payload is read, not the only payload of a decoded output: tables that differ in entries the stream never selects (a symbol the clip does not use, say), or a different escape layout, code the same output in other bytes. A payload’s hash therefore identifies the payload, not what it decodes to. Where one output must have one identity, it is identified by the digest of its decoded output: for a clip, its frame-state-equivalence digests (SPEC §7, a video receipt’s clip_hash); for the other formats, their capabilities’ equivalence digests.

CDF tables. A table is n entries of u32 with 3 ≤ n ≤ 4097, cdf[0] = 0, cdf[n − 1] = 65536 and every entry greater than the one before. These rules are what keep the symbol search and the state update above in range.

Order of the checks. A parser checks, in this order, and refuses with the first that fails:

  1. the magic: the capability’s (§1), or else format_capability_mismatch if it is another of the seven magics (WIV1, WIH1, WIA1, WIS1, WIF1, WCF1, WIC1), or bad_magic;
  2. the version byte: present (stream_length), and 1 (version);
  3. the fixed header: present (stream_length);
  4. the reserved fields (reserved_nonzero);
  5. the header’s fields, in the order the format’s section lists their rules (field_range), then, for WIV1 and WIF1, the keyframe kind (keyframe_capability_mismatch);
  6. the caps of §7 on the sizes the header declares (too_large), in the order the format’s section lists them;
  7. each later section in order: present (stream_length), then its rules (field_range, cdf_malformed). A section of one table per channel (WIA1, WIS1) is read table by table: each table present, then checked, before the next;
  8. nothing after the last section (trailing, or nested_length for the nested keyframe of a WIV1 or a WIF1 intra frame).

A decode then makes the checks of §7, and refuses a stream that runs out or does not end canonically (§2). Two conforming sinks refuse the same bytes with the same code.

3. WIV1 — integer inter-frame video

offsizefieldrule
04magic WIV1
41version1
51mv_precision1 (whole-pel) or 2 (half-pel); 1 when n_frames is 1
61keyframe0 or 1
71reserved0
84n_frames≥ 1
124h≥ 1
164w≥ 1
204block≥ 1; 1 when n_frames is 1
244q1..=65535
284offseti32
322cdf_len3..=4097
342reserved0
36cdf_len·4the residual table§2
…(n_frames − 1)·nb·4motion vectorsi16 dy, i16 dx
…4 + Lthe residual stream§2
…4 + Kkeyframe 1 only: u32 K, then a WIH1 payloadK is exactly the bytes that remain

The header’s field rules are checked in the order mv_precision, keyframe, n_frames, h and w, block, q, cdf_len, then, for a one-frame clip, mv_precision 1 and block 1: such a clip codes no motion vector, so neither field would change its decode, and they are fixed so that it has one header. keyframe 0 is wai.video.int_motion’s and 1 is wai.video.int_hyper’s; the other is keyframe_capability_mismatch. The frame h·w·3, the clip n_frames·h·w·3, the frame count n_frames and the motion-vector field 2·nb·(n_frames − 1) are capped (§7), in that order.

Motion vectors. nb = ⌈h/block⌉·⌈w/block⌉ and nbx = ⌈w/block⌉. The vectors are frames 1 to n_frames − 1 in order, each its block grid row-major: the vector of pixel (y, x) in frame f is entry (f − 1)·nb + (y div block)·nbx + (x div block), in units of 1/mv_precision pel.

Decode. One decoder reads the residual stream for the whole clip, against the residual table at offset. For each frame in order, for each pixel in raster order (y, x), for each channel c of R, G, B: decode the symbol r, and the sample is clamp(pred + r·q, 0, 255). Since |r| < 2^31 and q < 2^16, every step fits i64.

The output is n_frames RGB8 frames of h × w (rows top to bottom, pixels left to right, R, G, B bytes): the buffers of the capability’s frame-state-equivalence digests (SPEC §7 Digests).

This decode is the stage operators of integer-operators §5E, with no graph (§13.5 there): the residual stream is E1 rans64_decode with t4097, lenient and canonical, the prediction of frames f ≥ 1 is E10 mc_predict, and the sample is E9 residual_step on [0, 255]. The reference decoder decodes through them.

4. WIH1 — the integer conditional image codec

offsizefieldrule
04magic WIH1
41version1
53reserved0
84m≥ 1: latent channels
124sh≥ 1: latent height
164sw≥ 1: latent width
204n_z≥ 1: hyper-latent channels
244zh≥ 1: hyper-latent height
284zw≥ 1: hyper-latent width
324 + Lzthe hyper-latent stream§2
…4 + Lythe latent stream§2

The caps (§7) apply to n_z·zh·zw and to 2·m·sh·sw.

Decode, against the set of §9 (g_s.whm and h_s.whm, two integer networks at one fixed-point scale 2^afrac; entropy.json, the tables):

  1. For each hyper-latent channel k and each of its zh·zw elements, decode a symbol s from the hyper-latent stream against channel k’s table and offset; the element is s·2^afrac + median_k.
  2. h_s on the hyper-latent [n_z, zh, zw] gives [2·m, sh, sw]: its first m channels are the scales σ, the rest the means μ. h_s decodes through its lowering whm1/1 (integer-operators §13.2); the split of its output into σ and μ is a step of this decode, after the shape check, not part of the lowered graph (integer-operators §13.5).
  3. For each latent element p in order (channel, then row, then column), its table is b, where, with the scale edges e_0 … e_{N−1} and σ' = max(σ_p, e_0), b = N − 1 − #{ i < N − 1 : σ' ≤ e_i }. Decode a symbol r from the latent stream against table b and its offset; the element is r·2^afrac + μ_p.
  4. g_s on the latent [m, sh, sw] gives three channels; each value v is clamped to [0, 2^afrac] and becomes (v·255 + 2^(afrac−1)) >> afrac, as RGB8. g_s decodes through its lowering whm1/1 (integer-operators §13.2), and this is the RGB8 step that ends its lowered graph (§13.0).

Every value is formed in i64 and must fit it (§7).

5. WIA1 — the integer learned audio codec

offsizefieldrule
04magic WIA1
41version1
51afrac1..=30; equal to the model’s (§7)
62reserved0
84sr≥ 1: the sample rate the PCM is presented at, in hertz
124n_ch≥ 1: latent channels
164lat_len≥ 1: latent length
204t_out≥ 1: samples presented
244offseti32: the symbol offset of every channel’s table
282cdf_len3..=4097
302reserved0
32n_ch·8mediansi64 each, `
…n_ch·cdf_len·4one table per channel§2
…4 + Lthe latent stream§2

The header’s field rules are checked in the order afrac, then sr, n_ch, lat_len and t_out, then cdf_len. The caps (§7) apply to n_ch·lat_len and to t_out.

Decode, against the set of §9 (model.wam, the integer synthesis):

  1. For each channel k and each of its lat_len elements, decode a symbol s against channel k’s table at offset; the element is s·2^afrac + median_k.
  2. The synthesis (one-dimensional integer transposed convolutions and integer inverse GDN) on [n_ch, lat_len] gives one channel of t ≥ t_out samples. It decodes through its lowering wam1/1 (integer-operators §13.3), with t_out as its scalar.
  3. The first t_out samples v become (v·32767 + 2^(afrac−1)) >> afrac, clamped to [−32768, 32767]: the PCM step that ends the lowered graph (integer-operators §13.3). A sample for which v·32767 + 2^(afrac−1) leaves i64 is refused (field_range).

The output is t_out samples of i16 PCM, one channel, at sr hertz: the buffer of the capability’s sample-equivalence digest (signed 16-bit little-endian samples).

6. WIS1 — the integer learned Gaussian-splat attribute codec

offsizefieldrule
04magic WIS1
41version1
51afrac1..=30; equal to the model’s (§7)
62reserved0
84c≥ 1: latent channels
124sh≥ 1: latent height
164sw≥ 1: latent width
204s≥ 1: the attribute planes’ side
244attr≥ 1: attribute planes
284n_gaussians1..=s·s
32attr·8rangesper plane, f32 lo then f32 hi (IEEE 754 binary32): finite, lo ≤ hi
…c·2table lengthsu16 each, 3..=4097
…c·4symbol offsetsi32 each
…c·8mediansi64 each, `
…the lengths’ sum ·4one table per channel§2
…4 + Lthe latent stream§2

The header’s field rules are checked in the order afrac, the dimensions, n_gaussians. The caps (§7) apply to c·sh·sw and to attr·s·s.

Decode, against the set of §9 (g_s.whm, the integer synthesis):

  1. For each channel k and each of its sh·sw elements, decode a symbol s against channel k’s table and offset; the element is s·2^afrac + median_k.
  2. g_s on [c, sh, sw] gives [attr, s, s] (geometry_mismatch otherwise). g_s decodes through its lowering whm1/1 (integer-operators §13.2).
  3. Each value v is clamped to [0, 2^afrac] and becomes (v·255 + 2^(afrac−1)) >> afrac, a byte: the RGB8 step that ends the lowered graph (integer-operators §13.0).

The output, plane after plane, is the buffer of the capability’s splat-equivalence digest. The first n_gaussians cells of the planes, in raster order, hold one Gaussian each. The ranges are presentation data: a sink presents a plane’s byte u at about lo + (hi − lo)·u/255. They are outside the digest, and this extension does not fix presentation.

7. Caps and the checks a decode makes

Caps, the same on every target. No tensor a payload sizes may exceed 2^26 elements: a frame h·w·3, a WIV1 motion-vector field 2·nb·(n_frames − 1), a latent, a hyper-latent, a network’s output, an attribute tensor, a clip’s t_out. A WIV1 clip may not exceed 2^30 bytes, nor 2^20 frames.

Planning the networks. Before a decode allocates a latent, it plans every network on the payload’s shapes from the layers’ geometry and parameter lengths alone. The network’s input must be within the cap (too_large); then, layer by layer in order:

  1. the layer fits the running channel count (model_mismatch): a convolution’s output channels, kernel and stride are non-zero, it has c·out·k·k weights (c·out·k for the audio synthesis) and out multipliers and biases, and its requantisation shift is under 128; an inverse GDN has c·c and c parameters and shifts under 128; a leaky ReLU’s shift is under 64;
  2. every size the layer forms is within 2^26 (too_large): a convolution’s padded input side n + 2·pad, a transposed convolution’s uncropped side (n − 1)·stride + k + opad and uncropped plane, and the output;
  3. the layer leaves an output (model_mismatch): a kernel no larger than the padded input, a crop 2·pad smaller than the uncropped side.

Every size is formed in 64-bit arithmetic with overflow checked, and a size past 64 bits is past the cap, so a 32-bit sink refuses exactly what a 64-bit one does, with the same code; once a network is planned, every size it forms fits a 32-bit index.

Memory and work. A payload’s header declares its decode’s output and the work that output takes, and a sink can read both before it decodes anything. A stream does not bound them: a table can make one symbol cost a small fraction of a bit, so a few kilobytes of stream can code a clip at the 2^30-byte cap, as any codec can describe a large output in a small file. The bound is the header’s, within the caps:

A WIA1’s latent is the one its t_out needs and no longer: the synthesis of lat_len samples gives at least t_out samples and that of lat_len − 1 fewer (geometry_mismatch), so a payload cannot declare a large latent to present a few samples.

A decode refuses a stream that runs out at its first missing symbol (§2), so a short stream stops early, but a stream that carries its symbols is decoded in full: a sink that will not spend what a header declares refuses before decoding, by its own size policy, applied to the declared cost (int_payload::declared_cost and int_payload::check_cost, with the set; neural::declared_cost over the set a registry resolved; the C ABI’s wai_int_payload_declared, given the set’s path; wai-web’s intPayloadCost; the header alone through container::declared_size), cost_over_limit (§8). The reference neural sink applies one when it is given one (neural::ModelRegistry::with_cost_limit); its C ABI does not decode integer video, PCM or splat attributes, which it has no output kind for. This extension fixes the caps; a sink’s policy may be stricter.

Informative. A sink may keep a parameter set lowered and loaded between decodes, keyed by the SHA-256 of its bytes (integer-operators §8.6), so that only the first decode through it pays for the lowering and the bounds pass. What it keeps is the set’s, the same for every payload, and is not part of any decode’s declared working bytes; the sink’s own budget for kept sets bounds it. The reference implementation’s store (int_set_cache::LoadedSets) starts with a budget of 256 MiB, evicts the least recently used set first, keeps no set larger than the whole budget, and keeps nothing with a budget of 0 (LoadedSets::set_budget; wai-web’s setLoadedSetBudget). A decode that finds its set kept makes every other check of this section, in the order below, and gives the same output and the same refusals.

The parameter set must fit the payload (model_mismatch):

The shapes the payload declares must be the ones the decode gives (geometry_mismatch): h_s’s output and [2·m, sh, sw]; a WIV1 keyframe and h × w; the audio synthesis’s length and t_out (it may be longer, by less than one latent sample’s worth: the synthesis of lat_len − 1 samples gives fewer than t_out); g_s’s output and [attr, s, s].

The order of a decode’s checks. After the parse (§2), a decode checks in this order and refuses with the first that fails:

  1. the set’s members, in the order of §9’s list, each present (set_incomplete) and parsed (model_mismatch);
  2. the parameter set against the payload (model_mismatch): one fixed-point scale, the payload’s afrac, the entropy tables, the hyper-latent channels;
  3. every network planned, in decode order (h_s then g_s; the audio synthesis; the splat g_s), each layer by the three rules above (model_mismatch, too_large): the caps come before any shape is compared;
  4. what the networks give against the payload: for WIH1, h_s’s shape (geometry_mismatch), then g_s’s three channels (model_mismatch), then h_s and g_s lowered and loaded, in that order (model_mismatch); for WIV1 with keyframe 1, its keyframe’s shape and channels as for WIH1, then the keyframe’s size against h × w (geometry_mismatch), then the keyframe’s networks lowered and loaded (model_mismatch); for WIA1, one channel (model_mismatch), then at least t_out samples, then a latent no longer than that needs (both geometry_mismatch), then the synthesis lowered and loaded (model_mismatch); for WIS1, g_s’s shape (geometry_mismatch), then g_s lowered and loaded (model_mismatch);
  5. each stream in decode order (WIH1: the hyper-latent’s, then, after h_s runs, the latent’s; WIV1 with keyframe 1: the keyframe’s streams, then the residual stream), symbol by symbol: a symbol that runs its stream out (stream_exhausted) before its value’s range (field_range); after its last symbol, its canonical end (stream_noncanonical);
  6. each network as it runs, layer by layer (field_range, below).

So a run-out is reported before the value it would have given, and a size past the caps before a shape that disagrees. A sink that applies a cost limit (Memory and work) compares the declared cost with it after step 4, once everything the cost reads has been checked, and before any symbol is decoded (cost_over_limit, §8).

Exactness. A latent rebuilt as s·2^afrac + base must fit i64; before each network layer runs, the decode checks on the values present that no product, partial sum, requantised value or inverse-GDN term can leave the type its arithmetic is formed in. A stream whose values fail either check is refused (field_range). So a decode with overflow checks and one without compute the same integers, and neither panics. A network that decodes through its lowering (integer-operators §13; for WIH1, a WIV1’s keyframe and WIS1, §13.2; for WIA1, §13.3) makes the second check as the check ranges of its graph: a value outside one is refused (field_range), and the refusals are the same. So does WIA1’s PCM step (§5 step 3), whose range is its graph’s last check range.

8. Refusal codes

The list is closed: a sink reports exactly one of these, the first that applies in the order of §2 and §7.

A refusal of the payload or of the parameter set against it, bad_magic through model_mismatch in the table below (and payload_malformed), is final: the sink does not decode the payload by another path, and SPEC §4’s fallback is not tried, since it would decode the same bytes. The other codes say the sink cannot decode through the capability at all, and route as SPEC §4 routes such a capability:

codewhen
bad_magicthe payload does not start with any of the seven magics: a tool’s JSON form lands here
format_capability_mismatchthe payload is another of the seven formats than the capability’s
keyframe_capability_mismatcha WIV1 or WIF1 keyframe kind that is not the capability’s
versiona version this revision does not define
reserved_nonzeroa reserved field that is not zero
field_rangea field of any section outside its range (a header field, a WIS1 range or table length, a median); or a value the stream decodes that leaves the exact integer range (§7)
too_largea size past the caps of §7
cdf_malformeda table that breaks the rules of §2
stream_lengththe payload ends before a field or section it declares; or a stream length that is not a multiple of 4, or is under 8
nested_lengtha keyframe length K (of a WIV1, or of a WIF1 intra frame) that is not the rest of the payload
trailingbytes after the last section
stream_exhausteda stream that ran out before its last symbol
stream_noncanonicala stream that does not end canonically after its last symbol: a word left unread, or a state other than 2^31 (§2)
geometry_mismatcha shape the payload declares that the decode does not give (§7); a frame of a WIC1 clip, or an inter frame of a wai.video.int_cond.frame group, whose header is not its first frame’s (§17)
model_mismatcha parameter set that does not fit the payload (§7)
set_incompletea parameter set without a member the decode reads (§9); a pinned decode reports SPEC §3.1’s set_incomplete
not_dispatchablea capability SPEC §4 does not dispatch
unsupported_capabilitya capability the sink has no integer decoder for
payload_malformeda wai.neural.int_synth or wai.neural.int_mlicpp payload its decoder refuses
cost_over_limita payload whose declared cost (§7) exceeds the limit the sink applies before decoding

9. Parameter sets

A capability that decodes against a parameter set reads these members, by the names a set’s listing gives them (prior-carriage §7):

capabilityshapemembers
wai.neural.int_hyper, wai.video.int_hyperseveral filesg_s.whm and h_s.whm (networks in WHM1), entropy.json (the tables)
wai.audio.int_codecone filemodel.wam (the synthesis in WAM1)
wai.splat.int_codecone fileg_s.whm (the synthesis in WHM1)
wai.video.int_cond, wai.video.int_cond.frameone filemodel.wcb (the WCB1 bundle, §17.3)
wai.video.int_motionnone—

A set of several files is pinned through its listing (wai.prior.set); a one-file set MAY be pinned form-less, by its file’s SHA-256 (SPEC §3.1). wai.audio.int_codec and wai.splat.int_codec were registered with sets of several files, because no one-file format had been named for them; this revision registers each as one file, the file its decoder reads, which holds every parameter its output depends on. That widens their registrations, as SPEC §8 allows a MINOR revision to; a set pinned through its listing still verifies.

WHM1 (int_hyper_synth::IntModel::from_bytes): magic, then afrac, gf, shift and the layer count (u32 each), then per layer a tag byte: 0 a transposed convolution and 1 a convolution (input and output channels, kernel, stride, padding and, for 0, output padding as u32; weights as i16; per-output-channel multipliers and biases as i64), 2 an inverse GDN (channels as u32; gamma as i32, beta as i64), 3 a ReLU, 4 a leaky ReLU (slope i64, its shift u32). WAM1 (int_audio::AudioModel::from_bytes): the same header, then layers of tag 0, a one-dimensional transposed convolution, and tag 2, an inverse GDN. In both forms nothing follows the last layer: a file with bytes after it is refused (model_mismatch), so one network has one file and one digest. The layers’ integer arithmetic is integer-operators §13.2 (WHM1) and §13.3 (WAM1).

entropy.json, the tables of wai.neural.int_hyper and wai.video.int_hyper: a JSON object with these members. Other members are ignored. An integer is a JSON number with no fraction and no exponent, and -0 is zero. A file that is not a JSON text (RFC 8259, UTF-8, no member named twice, at most 64 levels, nothing after the value) is refused (model_mismatch). The reference decode reads the text with no float parser.

membervalue
scale_edgesN ≥ 1 integers, each fitting i64: the edges e_0 … e_{N−1} of the ceil bucket rule (integer-operators E3)
cdfsN arrays of integers, each fitting i32: the latent tables, one per bucket
cdf_lengthsN non-negative integers: table b’s length n_b
offsetsN integers, each fitting i32: table b’s offset
z_cdfsK ≥ 1 arrays of integers, each fitting i32: the hyper-latent tables, one per channel
z_cdf_lengthsK non-negative integers
z_offsetsK integers, each fitting i32
z_median_qK integers, each fitting i64: channel k’s median at 2^afrac

Each table, over its first n_b entries, is a table of §2: 3 ≤ n_b ≤ 4097, n_b no more than its array’s length, cdf[0] = 0, cdf[n_b − 1] = 65536, strictly increasing. Entries after n_b are not read. A set whose tables break any of this is refused (model_mismatch). A payload has no more hyper-latent channels than K (§7). Hyper-latent channel k decodes against z_cdfs[k], z_cdf_lengths[k] and z_offsets[k], and its elements are s·2^afrac + z_median_q[k] (§4 step 1). A latent element decodes against table b, the ceil bucket of its σ over scale_edges (§4 step 3).

10. The tools’ JSON forms

The encode and export tools under tools/ write a JSON bitstream. It is a converter input and a debugging aid, never a payload. An encoder MUST NOT carry it as the payload of any capability of §1, and a sink refuses it (bad_magic). The converters (int_payload::{wiv1,wih1,wia1,wis1}_from_json, and the independent int-payload-conformance/int_payload_verify.py) make explicit what the JSON leaves implicit:

A payload and the JSON it is converted from decode to the same bytes; the conformance vectors check that for every committed JSON bitstream.

11. Envelopes this revision makes non-conforming

This revision completes these strings’ registrations with a payload format they did not have (SPEC §8, Completing a registration). It makes non-conforming, and they still parse:

A sink refuses such a payload when it decodes it (§8), with no fallback attempt. An encoder MUST NOT write one; the reference implementation’s emit paths (wai wrap, the C packers and the browser writers) parse the payload as a sink does and refuse to. An encrypted envelope’s payload (SPEC §6) is ciphertext until a sink decrypts it, so the emit paths do not check it; the sink checks the plaintext when it decodes it (SPEC §6.4). No committed envelope or conformance corpus carried such a payload when this revision was made.

12. Reference implementation and conformance

13. The wai.neural.int_synth payload and set

SPEC §5 keeps this capability’s payload format; this section writes it down.

offsizefieldrule
04H≥ 1: the image’s height
44W≥ 1: the image’s width
82C≥ 1: latent channels
101S_log2the latent’s side is S = 2^S_log2
111reserved0
12C·S·Sthe latentsone signed 8-bit value each, [C, S, S] in CHW order

The format has no magic and no version. The payload is exactly 12 + C·S·S bytes, and C·S·S ≤ 2^26.

The set is one file, int_synth.json: a JSON object whose member layers is a non-empty array of layer objects. A layer has in_ch, out_ch, k, stride and pad, each a non-negative integer below 2^32, with out_ch, k and stride at least 1; out_pad, a non-negative integer below 2^32, 0 when absent; weight, in_ch·out_ch·k·k integers each fitting i32, in the layout [in_ch][out_ch][k][k]; bias, out_ch integers each fitting i32; shift, in 0..=127, 0 when absent; and relu, a boolean, false when absent. Other members are ignored.

The decode is integer-operators §13.1’s graph on the latents. Its output must be [3, H, W], and it is presented as row-major HWC bytes. Every refusal of the payload, the set or the decode is payload_malformed (§8).

The reference decoder (int_synth::decode_bytes) decodes through the lowering. It checks the payload first: its header, a reserved byte of 0, C ≥ 1, C·S·S ≤ 2^26 and its length. It then lowers the set and loads the graph under acc64, which makes the set’s checks (integer-operators §13.1: at least one layer, the layers chain, the lengths, fields below 2^32, out_ch, k and stride at least 1, a shift of at most 127, the caps and the certificate), and runs it. A shift from 64 to 127 has its exact meaning. Several of these checks are new refusals of input no committed corpus contains (integer-operators §15.3, SPEC §8 Planned breaks).

14. WIM1 and WMB1 — wai.neural.int_mlicpp

SPEC §5 keeps this capability’s payload format and its one-file parameter set, the WMB1 bundle (read from a set as model.wmb); this section writes both down.

offsizefieldrule
04magic WIM1
44H≥ 1
84W≥ 1
122Cz≥ 1: hyper-latent channels
142Zh≥ 1
162Zw≥ 1
182Cy≥ 1: latent channels, a multiple of K
202Yh≥ 1
222Yw≥ 1
242K≥ 1: channel slices
262reserved0
284Lzthe z stream’s length in bytes
324Lythe y stream’s length in bytes
36Lzthe z stream§2, with no length prefix
36 + LzLythe y stream§2, with no length prefix

A parser checks, in order: at least 36 bytes; the magic; the reserved field; no dimension 0; Cy a multiple of K; Cz·Zh·Zw, 3·Cy·Yh·Yw and 3·H·W each at most 2^26; and the payload exactly 36 + Lz + Ly bytes. Each stream’s length is checked (a multiple of 4, at least 8) when the decode reaches it.

The set is one file, a WMB1 bundle: the magic WMB1, then four sections, each a u32 length followed by that many bytes, in the order h_s, entropy, g_s (each a WHM1 network, §9) and the tables. The tables are a JSON object with the members of entropy.json (§9), and afrac, read as §9 reads entropy.json. Nothing follows the fourth section. The bundle’s rules: the three networks’ afrac are equal, lie in 1..=30, and equal the tables’ afrac; 1 ≤ N ≤ 65536; and every table, over its length, follows integer-operators E1’s t2 rule (2 ≤ n_b, n_b no more than its array’s length, cdf[0] = 0, cdf[n_b − 1] = 65536, strictly increasing), in place of §2’s rule.

The decode:

  1. Plan, before any stream: Cz is at most the number of z tables; h_s maps [Cz, Zh, Zw] to [2·Cy, Yh, Yw]; entropy maps [3·Cy, Yh, Yw] to [2·Cy, Yh, Yw]; g_s maps [Cy, Yh, Yw] to [3, H, W]; and no tensor any of them forms exceeds 2^26 elements.
  2. z. For each channel k, and each of its Zh·Zw elements in raster order, decode a symbol s against z table k, and the element is dequantize(s, afrac, z_median_q[k]) (integer-operators E2). After the last z symbol, the stream must not have run out.
  3. hyper = h_s(z).
  4. y. Let y be [Cy, Yh, Yw], all 0. For each slice j = 0 … K − 1, and for each pass (the anchors, (row + col) even, then the rest): σ‖μ = entropy(concat(y, hyper)), where every element of y not yet decoded is still 0. Then, for each channel of the slice (j·Cy/K to (j + 1)·Cy/K − 1), and each of its positions in raster order that belongs to the pass: b is the ceil bucket of σ over scale_edges (integer-operators E3); decode r against table b; the element is dequantize(r, afrac, μ). One y stream carries every slice’s two passes, in that order. After the last y symbol, the stream must not have run out.
  5. g_s(y), then the RGB8 output step (integer-operators §13.0), as row-major HWC bytes.

The streams decode with t2, lenient and final_only (integer-operators E1). A refusal of the payload, of either stream or of a value the decode forms is payload_malformed; a bundle that breaks its rules, or that does not fit the payload (step 1, and the load checks below), is model_mismatch (§8).

The reference decoder (int_mlicpp::decode) runs the three networks through their lowering, wmb1/1 (integer-operators §13.4: each network by whm1/1, g_s with the RGB8 step), each graph loaded under acc64. Step 1 checks the header and plans the networks from their layer geometry, then lowers and loads them, which makes the set’s checks of integer-operators §13.2 (a layer of tag 0, 1 or 2 in each network; each layer fits the channels the layer before it gives; header shifts under 128; a leaky shift under 64; no bias of magnitude 2^63 − 1 or more; the certificate), then plans each graph on the payload’s shapes (integer-operators §4: its input’s channel count, Cz, 3·Cy or Cy, and the caps, a convolution’s padded side included), all before either stream. A value outside a graph’s check range is refused (payload_malformed): the ranges refuse exactly the values a layer’s arithmetic could leave i64 on. Some of these checks are new refusals of input no committed corpus contains, and some now come before the streams where an earlier decoder made them when the network ran (integer-operators §15.3, SPEC §8 Planned breaks).

15. Receipts of wai.video.int_motion

Two signed receipts describe a WIV1 payload of wai.video.int_motion (keyframe 0, §3). The decode receipt (§15.1) is a sink’s statement of what the payload reconstructs to; the encode receipt (§15.3) is an encoder’s statement of the payload it wrote, the source it wrote it from, its parameters and its error. Each binds a measured energy figure and its acquisition class (energy-measurement §2) beside the figures anyone can recompute, and keeps the two apart. The reference is wai-rs’s video_receipt (VideoReceipt, VideoEncodeReceipt; feature video_receipt).

In this section, integers in a signing payload are big-endian and those hashed into a frame or clip commitment little-endian, as the reference writes them; BLAKE3 is the hash of every commitment.

15.1 The decode receipt

fieldwhat it is
codestream_hashBLAKE3 over the payload’s bytes
h, wthe payload’s geometry
framesfor each decoded frame i, from 0: BLAKE3 over wai:video-rframe (ASCII), 0x01, i (u64) and the frame’s RGB8 bytes
clip_hashBLAKE3 over wai:video-clip, 0x01, h (u32), w (u32), the number of frames (u64), then each frame’s i (u64) and hash
merkle_rootthe jwp-receipts §2.2 Merkle root over the frame hashes, in order
work_pixel_opsframes × h × w × 3, saturating at 2⁶⁴ − 1: the decoded subpixels
joules_micro, the classthe energy the sink spent reconstructing, in microjoules, and its acquisition class; 0 and no class is the unmetered marker
parent_receipt_hashthe receipt hash of the receipt before it, or none
signer_pubkey, signer_id, sigthe signer’s Ed25519 key, its id, and the signature

The signing payload is wai:video-receipt and 0x01, then codestream_hash, clip_hash, h (u32), w (u32), merkle_root, the number of frames (u64), work_pixel_ops (u64), joules_micro (u64), the class, the parent (0x00, or 0x01 and the hash), each frame’s i (u64) and hash, signer_pubkey, and signer_id’s length (u32) and UTF-8 bytes. The class is one tag byte and its evidence: 0 the unmetered marker; 1 HwShunt (legacy), 2 ModelBased, 3 Estimator, with none; 4 OnChipCounter, then relative_ppm (u32), window_us (u64) and the filtering state (u8: 0 Unknown, 1 Off, 2 On); 5 CalibratedInstrument, then relative_ppm (u32), window_us (u64), and the certificate reference’s length (u32) and UTF-8 bytes. sig is Ed25519 over the payload, verified strictly (SPEC §7 Signatures). The receipt hash is BLAKE3 over wai:video-receipt-id, 0x01, the payload and sig.

A receipt is consistent when merkle_root, clip_hash and work_pixel_ops are what its frames give, its frames are numbered 0 upwards, its figure and class cohere (0 with no class, or a figure above 0 with a class whose evidence energy-measurement §2.3 requires and whose declared relative_ppm × joules_micro reaches the resolution floor of 288676), and its signature verifies. It is re-derived from a payload when the payload’s BLAKE3 is its codestream_hash, the payload decodes (§3), and the decode has its geometry and, frame by frame, its frame hashes; and then it is consistent. The figure is attested by the signature and by nothing else: another sink measures another figure for the same decode.

15.2 The JSON form

A decode receipt’s JSON form is one object of these members, in this order: profile ("wai.video.int_motion/receipt"), codestream_hash, clip_hash, h, w, frames (each frame’s hash), merkle_root, work_pixel_ops, joules_micro, uj_per_kilopixel_op, energy_provenance, the class’s evidence (for OnChipCounter, energy_uncertainty {"relative_ppm", "window_us"} and energy_counter_filtering; for CalibratedInstrument, energy_uncertainty and energy_calibration_ref; for the others, none), parent_receipt_hash, signer_pubkey, signer_id, sig and receipt_hash. Hashes, keys and signatures are lowercase hexadecimal; h, w and the figures are integers in plain decimal; energy_provenance is the class’s label, or null when unmetered, and parent_receipt_hash null when there is none. uj_per_kilopixel_op is derived: null when the receipt is unmetered or its work is 0, and otherwise joules_micro × 1000 / work_pixel_ops, computed in IEEE 754 binary64 (the product, then the quotient) and written with six digits after the point, correctly rounded.

A reader reads the text strictly (RFC 8259, no member named twice), requires exactly the members the receipt’s class writes, each of its form, reads the class and its evidence as energy-measurement §2.7 reads them, and refuses a uj_per_kilopixel_op that is not, spelled as above, the receipt’s own, and a receipt_hash that is not its receipt hash. The members’ order and the whitespace between them are not held. A receipt read so is then consistent or not, and re-derived from its payload or not, as §15.1 says.

15.3 The encode receipt

An encode receipt (wai.video.int_motion/encode-receipt) states the payload an encoder wrote and the source it wrote it from:

fieldwhat it is
codestream_hashBLAKE3 over the payload’s bytes
h, wthe source’s geometry
source_framesfor each source frame i, from 0: BLAKE3 over wai:video-sframe (ASCII), 0x01, i (u64) and the frame’s RGB8 bytes
source_hashBLAKE3 over wai:video-source, 0x01, h (u32), w (u32), the number of frames (u64), then each frame’s i (u64) and hash
block, search, mv_precision, qthe encoder’s parameters: the payload’s block, mv_precision and q (§3), and the motion search’s radius, which the payload does not carry
sseΣ (source − decode)² over every sample of every frame (u64): the error the payload decodes with
work_pixel_opsframes × h × w × 3, saturating at 2⁶⁴ − 1: the encoded subpixels
joules_micro, the classthe energy the encoder spent, in microjoules, and its acquisition class; 0 and no class is the unmetered marker
parent_receipt_hash, signer_pubkey, signer_id, sigas §15.1’s

Its signing payload is wai:video-encode-receipt and 0x01, then codestream_hash, source_hash, h (u32), w (u32), the number of frames (u64), block (u32), search (u32), mv_precision (u8), q (u32), sse (u64), work_pixel_ops (u64), joules_micro (u64), the class (§15.1’s encoding), the parent, each source frame’s i (u64) and hash, signer_pubkey, and signer_id’s length (u32) and bytes. Its receipt hash is BLAKE3 over wai:video-encode-receipt-id, 0x01, the payload and sig. It is a profile defined after the hardware class was refined (energy-measurement §2.6, New profiles): its class is never the legacy HwShunt.

An encode receipt is consistent as §15.1 says of a decode receipt, with source_hash for clip_hash and no Merkle root, and the new-profile rule for its class. It is re-derived from a payload and a source when the payload’s BLAKE3 is its codestream_hash; the payload reads as WIV1 keyframe 0 (§3) of its h, w and number of frames, with its block, mv_precision and q; each source frame hashes to its own; the payload decodes; and the decode against the source gives its sse; and then it is consistent. Re-deriving it proves the payload decodes to frames within sse of the stated source. It does not prove which encoder wrote it, or that search was the radius used.

Its JSON form is one object of profile ("wai.video.int_motion/encode-receipt"), codestream_hash, source_hash, h, w, source_frames, block, search, mv_precision, q, sse, work_pixel_ops, joules_micro, energy_provenance, the class’s evidence (as §15.2), parent_receipt_hash, signer_pubkey, signer_id, sig and receipt_hash, read as §15.2 reads a decode receipt’s.

16. WIF1 — one frame of integer inter-frame video

WIF1 is the frame form of WIV1 (§3): the payload of wai.video.int_motion.frame (keyframe 0) and wai.video.int_hyper.frame (keyframe 1), one frame an object. frame-stream says how frames make groups and streams, the manifest member that links them, and what a sink checks across them; this section pins one frame’s bytes and its decode. A frame is an intra frame, which opens a group, or an inter frame, decoded against the frame before it in its group.

offsizefieldrule
04magic WIF1
41version1
51kind0 intra, 1 inter
61keyframe0 or 1
71mv_precisioninter: 1 (whole-pel) or 2 (half-pel); intra: 0
84h≥ 1
124w≥ 1
164blockinter: ≥ 1; intra: 0
204qintra: 1..=65535; inter: 0
244offseti32; inter: 0
282cdf_lenintra: 3..=4097; inter: 0
302reserved0

An intra frame continues with:

sizefieldrule
cdf_len·4the residual table§2
4 + Lkeyframe 0: the frame’s residual stream§2
4 + Kkeyframe 1: u32 K, then a WIH1 payload (§4)K is exactly the bytes that remain

and an inter frame with:

sizefieldrule
nb·4motion vectorsi16 dy, i16 dx, nb = ⌈h/block⌉·⌈w/block⌉, the block grid row-major
4 + Lthe frame’s residual stream§2

Checks. §2’s order, with these rules: the header’s fields in the order kind, keyframe, mv_precision, h and w, block, q, offset, cdf_len (field_range), then the keyframe kind (keyframe_capability_mismatch: keyframe 0 is wai.video.int_motion.frame’s, 1 is wai.video.int_hyper.frame’s); then the caps of §7 on the frame h·w·3 and, for an inter frame, the motion-vector field 2·nb (too_large); then the sections in the order above, and nothing after them (trailing, or nested_length for a keyframe). A field one kind of frame does not use is fixed at 0, so each frame has one header. The header alone sizes every section, so a frame parses without the frames before it.

What a group’s intra frame fixes. h, w, the residual table, q and offset are the group’s: an inter frame reads its stream against its intra frame’s table at its offset, steps by its q, and has its intra frame’s h × w (geometry_mismatch, frame-stream §5). mv_precision and block are each inter frame’s own.

Decode. Each frame has a residual stream of its own, decoded by E1 rans64_decode (t4097, lenient, canonical) and ending canonically after the frame’s last symbol (§2): h·w·3 symbols, raster (y, x), then R, G, B.

The output is one RGB8 frame of h × w: the buffer of the frame’s frame-state-equivalence digest (SPEC §7 Digests). The decode is §3’s frame step, through the same stage operators (integer-operators §13.5), so a group whose intra frame carries a clip’s table, q, offset and frame 0 (or keyframe), and whose inter frames carry the clip’s vectors and each frame’s residual symbols, decodes to the clip’s frames, byte for byte (frame-stream §6). Since every frame’s stream is its own, a frame’s decode reads no byte of another object; what it reads of another frame is the decoded frame before it, which frame-stream §5 holds to be its predecessor’s.

Reference implementation. int_payload::wire::{parse_wif1, write_wif1}; int_video::decode_unit_typed (the frame step, through int_video::ops::decode_unit); frame_stream::IntFrames, the stateful decoder; frame_stream::split_wiv1, which cuts a WIV1 clip into one group of WIF1 frames. Conformance: int-payload-conformance/frame-stream/.

17. WIC1 and WCF1 — integer conditional inter-frame video

wai.video.int_cond codes each frame conditionally: not as a residual against a prediction of its pixels, but as a latent coded given a temporal context, the previous frame’s decoded feature, and decoded back into a feature and a frame with that context as a second input. Four integer networks carry the decode, each a WHM1 network (§9) lowered by whm1/1 (integer-operators §13.2), so every scale, every table, every feature and every frame is an integer function of the payload and the set, the encoder’s on every decoder. Its payload is WIC1, a clip of WCF1 frames; wai.video.int_cond.frame, its frame form (frame-stream), carries one WCF1 frame an object. Both decode against one parameter set of one file, model.wcb, a WCB1 bundle (§17.3). Their registry class is NeuralIntegerExact, at decode-equivalence (determinism-tiers), and their outputs are frames, each a frame-state-equivalence digest’s buffer (SPEC §7 Digests).

17.1 WCF1 — one frame

offsizefieldrule
04magic WCF1
41version1
51kind0 intra, 1 inter
62reserved0
84h≥ 1: the frame’s height
124w≥ 1: the frame’s width
164m≥ 1: latent channels
204sh≥ 1: the latent’s and the feature’s height
244sw≥ 1: their width
284cf≥ 1: the reference feature’s channels
324 + Lthe latent stream§2

Checks. §2’s order: the header’s fields kind, then h, w, m, sh, sw and cf (field_range); then the frame h·w·3, the scales and means 2·m·sh·sw and the reference feature cf·sh·sw, each at most 2^26 elements (too_large); then the stream, and nothing after it (trailing). The header alone sizes the frame, so it parses without the frames before it.

17.2 WIC1 — a clip

offsizefieldrule
04magic WIC1
41version1
53reserved0
84n_frames1..=2^20
12…n_frames frameseach a u32 F, then F bytes: one WCF1 frame

Checks. §2’s order: n_frames at least 1 (field_range) and at most 2^20 (too_large); then each frame in order: its length and its bytes present (stream_length), its bytes a WCF1 frame (§17.1’s codes), frame 0 intra and every later frame inter (field_range); after frame 0, the clip n_frames·h·w·3 at most 2^30 bytes (too_large); every later frame with frame 0’s header, h to cf (geometry_mismatch); and nothing after the last frame (trailing).

A clip is one group of the frame form (frame-stream §2): its split is its frames’ bytes, copied, frame 0 the group’s intra frame; its join writes a group’s frames back into a clip, which must then parse as one (a group of no frame, an intra frame after the first or another header is refused with the parse’s code). Each gives the other’s input back, byte for byte, and since every frame’s stream is its own, a group decodes to its clip’s frames, frame for frame.

17.3 The parameter set: WCB1

One file, read from a set as model.wcb: the magic WCB1, then five sections, each a u32 length and that many bytes, in this order:

sectionwhat it is
ctxthe temporal context network: the reference feature [cf, sh, sw] → the context [cc, sh, sw]; cc is its output channels
priorthe entropy context network: the context → [2·m, sh, sw], the scales σ (channels [0, m)) and the means μ ([m, 2·m))
decthe conditional decoder: concat(ŷ, context), [m + cc, sh, sw] → the frame’s feature [cf, sh, sw]
reconthe synthesis: the feature → [3, h, w]
the tablesa JSON object: afrac; scale_edges, N integers each fitting i64; cdfs, N arrays of integers each fitting i32; cdf_lengths, N non-negative integers; offsets, N integers each fitting i32. Other members are ignored

Its rules, each refused model_mismatch: each section inside the bundle, and nothing after the fifth; each network a WHM1 network (§9); the four sharing one fixed-point scale afrac in 1..=30, which is the tables’ afrac; the tables read as §9 reads entropy.json (a JSON text, read strictly, whose numbers are integers as spelled); N from 1 to 65536; and each table, over its first n_b entries, a table of §2: 3 ≤ n_b ≤ 4097, n_b no more than its array’s length, from 0 to 65536, strictly increasing. The networks’ arithmetic is integer-operators §13.2, and the bundle’s lowering is wcb1/1 (§13.4a there).

17.4 Decode

A decode checks in this order and refuses with the first that fails:

  1. the parse (§17.1, or §17.2 for a clip);
  2. the set’s member model.wcb, present (set_incomplete) and a WCB1 bundle that keeps §17.3’s rules (model_mismatch);
  3. the four networks planned on the frame’s header (§7, Planning the networks), each layer by §7’s three rules (model_mismatch, too_large), all four before any shape is compared: ctx on [cf, sh, sw]; prior on [cc, sh, sw]; dec on [m + cc, sh, sw]; recon on [cf, sh, sw]. Then ctx keeps sh × sw, prior gives [2·m, sh, sw] and dec [cf, sh, sw] (each geometry_mismatch), and recon gives three channels (model_mismatch) of h × w (geometry_mismatch);
  4. the four networks lowered and loaded under acc64 (model_mismatch).

Then each frame, in order. Its reference feature F is zero, [cf, sh, sw], for an intra frame, and the feature the frame before it in its group left for an inter frame (in a clip, the frame before it):

  1. C = ctx(F), then prior(C), whose channels [0, m) are σ and [m, 2·m) are μ;
  2. the latent ŷ [m, sh, sw], element by element in order (channel, then row, then column): its table is b, the ceil bucket of its σ over the scale edges (integer-operators E3, as §4 step 3 gives it); its symbol r is the next of the frame’s stream against table b at its offset (§2); and it is r·2^afrac + μ, which must fit i64 (field_range). A symbol that runs the stream out is refused before its value is formed (stream_exhausted), and after the last symbol the stream ends canonically (stream_noncanonical);
  3. F' = dec(concat(ŷ, C)), the latent’s channels first;
  4. recon(F'), then the RGB8 step (integer-operators §13.0): the frame, as RGB8 bytes, rows top to bottom, pixels left to right, R, G, B;
  5. F' is the next frame’s reference feature.

Each network runs through its lowering, whose check ranges refuse a value a layer’s arithmetic could take out of i64 (field_range, §7 Exactness). The stage operators are E1 rans64_decode (t4097, lenient, canonical, one stream a frame), E2 dequantize and E3 bucket (ceil); concat, the zero feature and the split of prior’s output are driver steps (integer-operators §13.5).

Memory and work. The header and the set declare the decode’s output and work, as §7 says: the frames, n_frames·h·w·3 bytes; m·sh·sw symbols a frame; the networks’ multiply-accumulates a frame, layer by layer, from their plan; and the reference feature, cf·sh·sw values of 8 bytes, kept from frame to frame.

17.5 The frame form

wai.video.int_cond.frame carries one WCF1 frame an object, in groups (frame-stream §2): a group’s intra frame decodes against the zero feature, and each inter frame against the feature the frame before it left, which frame-stream §5 holds to be its predecessor’s. An inter frame carries its group’s intra frame’s header, h to cf (geometry_mismatch); then it is decoded by §17.4 steps 2 to 9, through the set its pin selects. A sink keeps one feature for each stream: no pixel of an earlier frame is read.

17.6 Reference implementation and conformance

17.7 Weights from a pretrained model (informative)

The committed vector’s networks were trained for this topology. A pretrained conditional codec’s own weights are lowered onto it by post-training quantisation, as wai.neural.int_hyper’s are from the pretrained mbt2018_mean (tools/wai_sota_int_synth_export.py). DCVC-RT (wai.neural.dcvc_rt, SPEC §5) reads, as this topology does, the previous frame’s decoded feature as its temporal context, with no motion estimation or compensation, so lowering its published weights takes no warp or motion-compensation operator. It takes the following; the second column says what this revision’s forms hold of each.

what the lowering takesin this revision
The checkpoint, pinned by its full SHA-256 and read as tensors only, so that the set the lowering writes is a function of known bytesas for mbt2018_mean; the set is one file, pinned by its SHA-256 (SPEC §3.1)
Its modules in the four roles: the adaptor that turns the previous decoded feature into the temporal context is ctx; the path from that context to the latent’s scales and means is prior; the decoder from the latent and the context to the feature is dec; the reconstruction from the feature to the frame is recon. The analysis side is the encoder’sthe four sections of WCB1 (§17.3)
Its blocks: residual additions around a block, which a WHM1 chain cannot state; depthwise convolutions and sub-pixel resampling, which a chain states only as dense convolutions (a depthwise one with zero weights off its diagonal, its work multiplied by its channel count; a pixel shuffle as a transposed convolution with one weight of 1 an output)each is a graph operator of wai-ops/1: conv with groups (integer-operators §5.2), add (§5.5), pixel_shuffle and pixel_unshuffle (§5.10), so wai-ops/1 gains no operator for them. Networks built of them are WHG1 graphs (§10 there) with a bounds certificate (§8 there), while WCB1’s sections are WHM1 chains: such a set is a bundle of graphs
The entropy model’s other inputs: a hyperprior, a second latent with a stream and tables of its own, whose decoded output joins the temporal context before the scales and means are formed; and spatial context, the latent decoded in passes over a partition of its positions, the scales and means formed again before each pass from the elements already decodedeach has an integer form here: a hyper-latent stream and its tables in WIH1 (§4, §9), passes in the coding order of WIM1 (§14), a 2 × 2 partition of positions in subpos_split and subpos_merge (integer-operators §5.10). WCF1 carries one stream decoded in one pass (§17.4), and a registered payload format is fixed (SPEC §8): a payload that carries them is a registration of its own
Its rate: the per-channel scales a rate index selects, at the latent, the feature and the reconstructionone set a rate: a scale on a layer’s output folded into that layer’s requantisation multipliers, one on a layer’s input into its weights. WCF1 names no rate, so a payload that names one is, again, a registration of its own
The quantisation: per-output-channel weights, activations at 2^afrac calibrated on the model’s own activations, and every layer within §7 Exactness at acc64WHM1 holds 16-bit weights (this vector uses 12 bits), and the four networks share one afrac (§17.3). A device’s 32-bit subset (wai-ops/1-i32, integer-operators §9.1) runs a network at acc32 only where its certificate bounds every intermediate in 32 bits (§7.2, §8 there)
The tables: the model’s scale table as scale_edges at 2^afrac, and one table an edge from its Gaussian at that scalethe tables section, as entropy.json holds mbt2018_mean’s (§9): up to 65536 edges, each table of up to 4097 entries
The picture format: the samples the model codesthe decode ends in the RGB8 step, three channels at h × w (§17.4 step 8); a model that codes another sampling, such as 4:2:0, takes an output step of its own
The check: the integer decode held to the floating-point decode of the same streams, as tests/int_hyper_synth.rs holds int_hyper’s within 1 dBthe export records each frame’s PSNR against its source (expected.json)