Skip to main content

WAI Extension: Packaging

Mirrored from the canonical text at commit 498e6b88 ().

Status: Draft. A packager turns sources into WAI objects under a declared policy; this extension fixes what a verifier recomputes. This revision defines the policy, its digest and what it may ask a packager to emit (§3), how a source is cut into pieces (§4), the objects a packager writes, refinements and stills among them, a piece’s ladder, and the components a package carries beside its lead, audio, a score, vector graphics, timed text and a feed, each in the capability that suits it (§5), the stage index (§6), the floor, one or several, supplied as OBU streams or as an existing ladder’s CMAF files, and a sound track’s Opus floor (§7), the protected floor, those CMAF files under Common Encryption (cenc and cbcs) with W3C Clear Key and content protection systems signalled by their system IDs (§7.10), what each class is offered over HTTP and MoQ and how its delivery is simulated (§8), the package receipt and live windows (§9), the package, its verification and a re-pack from a parent (§11), and where each step of a pack or a verify may run (§11.6), and how a programme’s audio renditions are written for HLS and DASH, audio description among them (§8.10), a lead’s marks, its shots, chapters, break candidates and clips, derived from its stage index and written as WebVTT chapters, HLS date ranges and DASH event streams (§8.11), a package that holds them and names them in its receipt (§8.12), a package that carries a signed statement beside each of its playlists and MPDs, and a protected presentation beside each of its own and its index (§8.13), and a programme’s loudness, measured in integers, with the gain to a target in a signed statement (§12). Reference impl: wai-pack — policy, capcheck, source, scenecut, classify, plan, encode, png, floor, cmaf, cmaf_read, opus_floor, cenc, av1_tiles, protect, catalog, marks, statements, files, pack, multi, live, delivery, verify, fabric, fabric_meter, floor_decode, and wai-pack-gpu for the steps a GPU runs; in wai-rs, the encoder int_video_enc, the refinement encoder staged::wir1, the stage index stage_index, the receipt package, the delivery simulator stage_measure::sim, the marks pack_marks and a packager’s statements manifest_statement::package; the corpus pack-conformance/, whose multi/ holds packages of several components (§5.8), and beside it the composing sink’s reference package wai-web/compose/, five components written by wai-pack and held to the corpus check. Keywords MUST, MUST NOT, SHOULD, MAY are RFC 2119/8174.

1. Scope

A packager turns one or more sources into WAI objects, stage indexes, catalogs and a package receipt, under a declared policy. This extension fixes what a package contains and what a verifier recomputes. It fixes an encoder’s choices only where it names them.

2. Terms

3. Policy

3.1 A package is made under exactly one policy, a TOML 1.0 document of schema wai.pack.policy/1, in UTF-8 with no byte-order mark.

3.2 The effective policy is the policy with every default written in (§3.3) and every override the packager applies (such as a command line forcing royalty_free_only). An override only ever forces royalty_free_only on; it never turns off what the policy asks for. Its JSON form is §3.3’s members, with an optional member that has no default omitted when absent, and arrays of tables in document order. Its policy digest is

BLAKE3("wai:pack-policy\x01" ‖ JCS(effective policy as JSON))

where JCS is RFC 8785. Two spellings of one policy (member order, inline or standard tables, dotted keys, string or integer notation, comments, an omitted default or the default written out) therefore have one digest; the order of an array of tables is part of the policy.

A packager MUST refuse with policy-toml, before anything else, a policy whose bytes are not UTF-8, that begins with a byte-order mark, that is not TOML 1.0 (an integer outside TOML’s 64-bit range included), or that nests arrays and tables more than 32 deep: the document’s own table is at depth 1, and an array or a table directly inside one at depth k is at depth k + 1. Then, before it reads any key, it MUST refuse one holding anywhere a float (policy-float), a date or time (policy-datetime), a negative integer (policy-negative), or an integer above 2^53 − 1 (policy-integer-range), visiting members in byte order of their keys and arrays in order.

3.3 Schema. Within each table, a key this section does not define is refused first (policy-unknown-key, the first in byte order), then each member in the order listed: a required member that is absent (policy-missing-key), a member of another type (policy-type), a value outside its range or vocabulary (policy-value). An absent table reads as empty.

MemberTypeDefaultRange
schemastringrequiredwai.pack.policy/1 (else policy-schema)
asset_idstringrequired1 to 255 bytes
royalty_free_onlybooleanfalse
leadstringrequired1 to 255 bytes
componentarray of tablesrequired
component.idstringrequired1 to 255 bytes
component.mediastringrequiredvideo, image, audio, world, feed, avatar, film, graphics, text
component.sourcestringabsent
component.timebasetablerequiredmembers num, den, each 1 to 2^32 − 1, required
component.profilestringabsentfloor-only (§7.8); absent, the standard profile
component.langstringabsent1 to 63 ASCII letters, digits and hyphens, none first, last or doubled: a text component’s language, a BCP 47 tag (§5.8)
segmentation.target_msinteger2000
segmentation.min_msinteger1000
segmentation.max_msinteger4000
segmentation.scene_cutstringwai.pack.scenecut/1wai.pack.scenecut/1
segmentation.cut_abs_permilleinteger3001 to 1000
segmentation.cut_rel_permilleinteger30000 to 1000000
segmentation.cut_windowinteger83 to 64 (a window of fewer than 3 scores never applies cut_rel_permille, §4.3)
segmentation.lookahead_framesinteger60 to 64
segmentation.closedbooleantrue
ladderarray of tablesempty
ladder.idstringrequired1 to 255 bytes
ladder.componentstringrequired1 to 255 bytes
ladder.capabilitystringrequired
ladder.rolestringrequiredbase, refinement, floor
ladder.refinesstringabsent
ladder.paramstableemptyby capability, below
floor.supplied_listingstringabsent64 lowercase hexadecimal digits
floor.extra_keyframesbooleanfalse
floor.supplied_formstringabsentcmaf (§7.7); absent, the OBU form (§7.3)
protectiontableabsentabsent, the package has no protected floor; present, cpix_sha256 and key are required (§7.10)
protection.cpix_sha256stringrequired64 lowercase hexadecimal digits: the SHA-256 of the CPIX document the keys come in
protection.systemsarray of stringsemptysystem IDs of the DASH-IF registry in canonical lowercase UUID text, none twice, none all zero, neither 1077efec-c0b2-4d02-ace3-3c1e52e2fb4b nor e2719d58-a985-b3c9-781a-b030af78d30e
protection.licence_uristringabsent1 to 1024 of RFC 3986’s characters, no brace: the Clear Key licence URI
protection.key_uristringabsentas licence_uri, holding {kid} exactly once and no other brace: the HLS key URI of an identity key
protection.keyarray of tablesrequiredat least one
protection.key.floorstringrequired1 to 255 bytes: a floor name (§7.1)
protection.key.schemestringrequiredcenc, cbcs
protection.key.kidstringrequireda KID in canonical lowercase UUID text, not all zero
markstableabsentabsent, the package holds no marks; present, its lead’s (§8.12)
marks.program_date_time_msinteger00 to 253402300799999: the program date-time of media time 0, in milliseconds since 1970-01-01T00:00:00Z, none past 9999-12-31T23:59:59.999Z (§8.11.5)
marks.langstringabsenta language tag (live-events §4.10): the chapters’ language
marks.chapterarray of tablesemptyat most 1,024
marks.chapter.titlestringrequireda title (§8.11.1)
marks.chapter.from_msintegerrequiredthe instant the chapter begins at
statementstableabsentabsent, the package carries no statement; present, a statement beside each of its playlists, multivariant playlists and MPDs, and its protected presentation one beside each of its own and its index (§8.13). It has no member
dvb_dashtableabsentabsent, the package holds no DVB-DASH MPD; present, dash/dvb.mpd and the numbered copies of its lead’s floor segments it names (§8.14). It has no member
live.layer_lag_msarray of integersone 0 per layer§3.4
classarray of tablesempty
class.idstringrequired1 to 255 bytes
class.max_layerinteger00 to 255
class.max_kbpsintegerabsent1 to 2^32 − 1
class.floor_onlybooleanfalse
momentarray of tablesempty
moment.idstringrequired1 to 255 bytes
moment.from_ms, moment.to_msintegerrequiredto_ms above from_ms
moment.tagsarray of stringsempty
rulearray of tablesempty
rule.idstringrequired1 to 255 bytes
rule.when.piece_kindarray of stringsabsentnatural, static, synthetic, motion_high
rule.when.moment_tagarray of stringsabsent
rule.then.ladderarray of stringsabsent
rule.then.max_layerintegerabsent0 to 255
rule.then.deadline_msintegerabsent1 to 2^32 − 1
defaults.deadline_msinteger15001 to 2^32 − 1
classify.static_permilleinteger50 to 1000
classify.static_mad_milliinteger300
classify.synthetic_binsinteger120 to 64
classify.motion_high_milliinteger9000

A rung’s params are those of its capability; every other capability takes none.

CapabilityParameterDefaultRange
wai.video.int_motionblock164, 8, 16, 32
search70 to 32
mv_precision21, 2
q_candidates[8]a non-empty array, each 1 to 255
max_sse_px_milli0
wai.video.int_refineqrequired1 to 255
wai.video.av1, wai.video.av1.losslesssourcesuppliedsupplied, encode
wai.audio.int_refineto_rate, qrequired1 to 384000; 1 to 65535
wai.audio.flacsourceencodeencode, supplied
wai.audio.opussourcesuppliedsupplied
wai.text.zstdsourceencodeencode

wai.audio.int_layered is reserved for the layered integer audio base: it is not registered yet, and its parameters are defined when it is.

3.4 Constraints. A policy that passes §3.3 is checked in this order, and the first that fails is the refusal:

  1. 1 to 8 components (policy-too-many-components); at most 64 rungs (policy-too-many-rungs), 32 classes (policy-too-many-classes), 32 rules (policy-too-many-rules) and 32 moments (policy-too-many-moments).
  2. Ids unique among components, rungs, classes, moments and rules taken together, so a reference never names two things (policy-duplicate-id).
  3. lead names a component (policy-lead).
  4. segmentation.closed is true: every piece is closed, §4.1 (policy-open-gop).
  5. 1 ≤ min_ms ≤ target_ms ≤ max_ms ≤ 600000, and max_ms is at most 2^24 units on every component’s timebase (§4.4) (policy-piece-bounds).
  6. Every rung names a component (policy-ladder-component).
  7. A rung carries refines exactly when its role is refinement (policy-refines-role); refines names a rung (policy-dangling-refines); no chain of refines returns to a rung it passed (policy-refines-cycle); every chain stays in its component and ends at a base (policy-refines-base); no rung is refined twice (policy-refines-branch).
  8. Every component has a base rung (policy-no-base), but a floor-only one (§7.8), which has no base and no refinement rung, no source, and at least one floor rung (policy-floor-only).
  9. A rung’s layer is 0 for a base or a floor, and one more than the rung it refines for a refinement; the policy has one more layer than its highest rung layer. No class’s and no rule’s max_layer exceeds the highest (policy-class-layer).
  10. A rule’s then.ladder names known rungs, all of one component, each once: exactly one base (none, for a floor-only component), every refinement over a rung it names (so its layers are a chain from that base), and every floor rung of the component (every piece has a floor, §7.1) (policy-rule-ladder).
  11. live.layer_lag_ms has one entry per layer, the first 0, the entries non-decreasing (policy-layer-lag).
  12. The deadline values (defaults.deadline_ms and every rule’s then.deadline_ms) number at most 4 (policy-too-many-deadlines), and for every refinement layer k and every deadline value d, d − layer_lag_ms[k] ≥ 1 (policy-deadline-unreachable).
  13. With a protection table (§7.10): every key names a floor with CMAF files, the lead’s AV1 floors and each sound track’s Opus floor, by its floor name; and for each scheme the keys name, every such floor has exactly one key of it, the floors of one component the same KID (policy-protection).
  14. With a marks table (§8.12): marks.lang is given exactly when there is a chapter; the chapters’ from_ms strictly ascend; and every moment’s id and each of its tags is a token (route-record §2), a moment’s tags distinct, as the marks’ derivation needs of the policy (§8.11.2 step 2) (policy-marks).
  15. With a statements table (§8.13): every component id and floor name is at most 58 bytes, so that each track a statement names (<f>/floor, <c>/l<k>, <scheme>/<f>) is at most 64 bytes, and a protection table names at most 64 keys, one track of the index each (policy-statements).
  16. With a dvb_dash table (§8.14): the lead is a video component with 1 to 16 floor rungs, the representations one adaptation set of a DVB-DASH MPD holds (ETSI TS 103 285 V1.4.1 clause 4.5) (policy-dvb-dash).

Rules fold in document order: every matching rule applies, and a later then member overrides an earlier one.

3.5 A package receipt carries the policy digest. A verifier holding the effective policy recomputes it; it cannot check that the packager followed the policy, except by packing again.

3.6 What a packager may emit. A packager MUST refuse a policy with a rung it may not emit, reading SPEC §5’s registry, never the policy. For each rung in ladder order the first rule that holds is its refusal, and a policy’s refusal is its first rung’s:

  1. SPEC §5 does not register the capability: capability-unknown:<cap>. One capability is read before this rule with the classes stated here, and with no licence class: wai.text.timed, which SPEC §5 registers as a record, as the base of a text component, whose payload is a WebVTT segment (§5.8): standalone media of class StandardDefined. Rules 2 to 4 read those classes, and under royalty_free_only rule 5 refuses it license-unclassified:<cap>. (wai.graphics.vector is registered, as standalone media of class StandardDefined, royalty-free: graphics-vector §1.8.)
  2. SPEC §5 registers it as a record or a derivation operation, which names no payload format: capability-not-payload:<cap>.
  3. It is a candidate, whose payload format no revision has pinned (SPEC §5, Candidate capabilities): capability-candidate:<cap>.
  4. A base or a floor whose capability is not standalone media, or a refinement whose capability is not refinement-only: capability-role:<cap>.
  5. Under royalty_free_only, a capability the registry gives no licence class: license-unclassified:<cap>; one of any class other than royalty-free: license-refused:<cap>:<label>, with the class’s SPEC §5 label. The class is the registry’s, “licensed royalty-free by its standards body”; it is not an opinion on freedom to operate, and a package states nothing about patents.

4. Pieces

4.1 Every piece MUST be closed: its first frame, sample or tick is coded with no reference outside the piece, and no later object of the package references into it.

4.2 Piece boundaries are a pure function of the source frames and the effective policy; a floor-only component’s, which has no source, of its first floor rung (§7.8). Where the policy names wai.pack.scenecut/1, the boundaries of a component with a source MUST be those §4.3 computes. The list of piece starts, in timebase units, is the plan, and its digest is

BLAKE3("wai:pack-plan\x01" ‖ u64 LE n ‖ n × u64 LE start)

4.3 wai.pack.scenecut/1. All arithmetic is integer; >> is a shift and every division rounds down. For each RGB8 frame f of w × h pixels:

  1. Luma Y = (77R + 150G + 29B + 128) >> 8, and the histogram H_f of 64 bins, Y >> 2.
  2. D(a, b) = (Σ_i |H_a[i] − H_b[i]|) · 1000 / (2wh): the share of pixels, in thousandths, whose bin differs. The score of frame f ≥ 1 is s_f = D(f, f − 1).
  3. The current piece starts at frame p, and len = f − p. Its window W_f is the scores s_g for p < g < f, the last cut_window of them; m_f is their lower median (sorted ascending, the element at index (|W_f| − 1) / 2).
  4. Hard cut at f: len ≥ min, s_f ≥ cut_abs, and either |W_f| < 3 or s_f · 1000 ≥ cut_rel · m_f.
  5. Soft cut at f: len ≥ target and 2 · s_f ≥ cut_abs.
  6. Return clause. A cut at f is rejected when D(f + k, f − 1) < cut_abs for some k in 1..=min(L, last − f): the content came back within L frames, so f starts a flash.
  7. Revisit clause. A cut at f is rejected when D(f, f − 1 − j) < cut_abs for some j in 1..=min(L, f − 1): f returns to content seen within L frames, so it ends a flash. Frames before p count.
  8. A hard or soft cut that clauses 6 and 7 do not reject starts a piece at f. Otherwise, when len ≥ max, a piece starts at f.

Frame 0 starts the first piece. L is lookahead_frames, last the index of the source’s last frame, cut_abs and cut_rel the policy’s permille values, and min, target and max the policy’s milliseconds on the component’s timebase (§4.4). A decision about frame f reads no frame after f + L, so a live segmenter decides it once frame f + L has arrived, and a live and an on-demand segmenter with equal L produce equal boundaries. A piece that follows a cut is flagged a scene cut; one started at max, forced.

4.4 A piece’s duration is an integer number of timebase units, and lies between min and max except the last piece of a source, which may be shorter than min. A duration in milliseconds converts to timebase units as floor(ms · num / (1000 · den)); a result of 0 becomes 1, and one above 2^64 − 1 becomes 2^64 − 1.

A piece is at most ten minutes long (max_ms at most 600000) and at most 2^24 units on its component’s timebase (§3.4 item 5). A verifier takes a piece whole: it decodes every frame of a base, holds a still’s one frame for every frame of its piece and hashes each (§5.7, §6.5), and reads one floor unit per frame (§7.2). So the longest piece a policy allows is the most work a verifier does for one piece, and a package states it. Ten minutes is far past any piece a delivery cuts (the default max_ms is four seconds); 2^24 units bounds that work on any timebase, since a timebase may have up to 2^32 − 1 units a second.

4.5 Components. Every component of a package shares one presentation timeline that starts at 0. The policy names one lead component; every other component cuts its pieces at the lead’s boundary instants, each converted to its own timebase as floor(start_lead · num_c · den_lead / (den_c · num_lead)) (above 2^64 − 1, 2^64 − 1). A sink presents piece p of every component from that start. A component’s piece ends where its next one starts, and its last piece at the lead’s end converted the same way, so every component spans the lead’s whole timeline; a piece that would last no unit, or more than 2^32 − 1, is refused (pack-piece-length). Each piece of a component beside the lead has flag bit 3, aligned to the lead, and no other flag (§6.2).

4.6 Classification. Each video piece has a kind, which rules match (rule.when.piece_kind). Over the piece’s consecutive frame pairs, with Y the luma of §4.3:

The kind is the first that holds of: static (mad_milli ≤ static_mad_milli and changed_permille ≤ static_permille); synthetic (occupancy ≤ synthetic_bins); motion_high (mad_milli ≥ motion_high_milli); natural.

4.7 Sources.

5. Objects

5.1 One envelope. Each object is exactly one WAI1 or WAI2 envelope and nothing after it (http-delivery §5.1).

5.2 Position independence. An object’s bytes MUST NOT depend on its position in a package or on the package it belongs to: no member of its manifest names a piece, a time, an asset, a class or a component, so two pieces with identical content are one object. The reference packager writes a base’s manifest as SPEC §3 orders it, with media the component’s media, intent replicate, model_requirement the rung’s capability with no fallback member, conditioning.kind the capability’s last dot-separated segment (int_motion, av1), and target {"frames": n, "h": h, "w": w} (the piece’s frame count and dimensions, for a still too, whose one frame is held for the piece, §5.7; for a floor, its sequence header’s maximum frame height and width); a floor’s derived_from (§7.2) comes after conditioning.

5.3 Name. An object is named by its content hash (jwp-receipts §2.1), BLAKE3("moq-jwp:object\x01" ‖ envelope), as objects/<64 lowercase hex>.wai: the same value is its stage-index entry (§6), its receipt leaf, its MoQ content hash and its HTTP strong ETag (http-delivery §5.2).

5.4 Refinements. A refinement object carries refines (staged-delivery §3): base the content hash of its piece’s base, layer its layer, and as producers in prev only objects of the layers below it, the object its entry refines (§6.2) among them.

The reference packager writes, for each refinement rung of a piece’s ladder (§5.7), one object of wai.video.int_refine over the piece’s wai.video.int_motion base, layer by layer. Its manifest is the one wai stage refine writes, as SPEC §3 orders it: media the component’s, intent replicate, the capability with no fallback, conditioning.kind wir1, refines with base the base’s content hash, prev the content hash of the object of the layer below (the base for layer 1) as a string, and layer, and target null. Its payload is a WIR1 (staged-delivery Appendix A) of unit kind RGB8, one unit per frame of the piece, the identity predictor and one table, written by wai-rs’s refinement encoder (staged::wir1::encode) at the rung’s step q:

The stage the layer reconstructs, clamp(below + r·q, 0, 255), is what the next layer refines. The packer applies each layer to the decoded base with the reference sink (staged-delivery §4) before it keeps it, and the sink’s output must be the encoder’s.

5.5 Integer video. An object of wai.video.int_motion carries a WIV1 payload with keyframe 0, as SPEC §5 pins it (integer-payloads §3); this extension defines no other layout for it, and its decode is that section’s. The reference packager writes these with one encoder (wai-rs int_video_enc) at the step §5.6 chooses. The encoder’s choices make a payload a function of the source frames, the rung’s block, search and mv_precision (§3.3), and the step q:

The encoder also states the SSE of its reconstruction against the source: the sum of squared differences over every sample. It codes in two passes (the first finds the vectors, the reconstruction and the residuals’ counts; the second derives each residual again and codes it last first), so beyond its input it holds the reconstruction, the vectors, the stream and the residual table: at most n·h·w·3 + 24n + 12·v + 3·(2·n·h·w·3 + 16) + T bytes for n frames and v motion vectors (int_video_enc::working_bytes). T is the widest table’s, of S = 2·255 + 1 symbols: its fit, 16·(S + 1) + 2·8·S + 4·(S + 2) bytes (a frequency for each symbol and the escape, an index for each symbol and the sort’s scratch, the table), and the table’s three copies and the payload’s header, 3·4·(S + 2) + 64. For a clip past the format’s caps, which the encoder refuses before it allocates, the figure is 2^64 − 1.

5.6 Choosing the step. For each piece, the base encode of a wai.video.int_motion rung chooses its step from q_candidates, read as a set (a value listed twice is one candidate). It encodes the piece at each candidate, from the largest down, and chooses the first whose floor(SSE · 1000 / (3·w·h·n)) ≤ max_sse_px_milli, where SSE is the encoder’s (§5.5) and n the piece’s frames: the largest step that meets the bound, so candidates that meet it alike are decided by their step. Where no candidate meets it, it chooses the smallest. The function the reference packager’s base encode calls is wai_pack::encode::choose_q.

5.7 A piece’s ladder, and stills. A piece is coded in the rungs of its ladder: those the matching rules’ then.ladder names (§3.4 folds them: the last that sets it), where they are rungs of the piece’s component; otherwise the component’s default ladder, its first base rung in ladder order, the chain of refinement rungs over it, and its floor rungs. A rule’s ladder therefore chooses a piece’s representation: its base, how many layers refine it, and nothing else (§3.4 item 10). A piece has one entry per rung of its ladder: its base, each refinement in layer order, then each floor (§6.2), and a verifier holds an index to that (§11.3).

A base of wai.image.png is a still: the piece’s first frame, held for the piece. A sink presents that frame for each of the piece’s frames, so its prefix is the frame repeated and its digest the clip digest of the repeats (§6.5), its sse is over the repeats against the piece’s frames (§6.6), and its manifest’s target names the piece’s frames (§5.2). The reference packager writes the frame as a PNG every byte of which is a pure function of the frame, so an independent implementation writes the same file:

A verifier reads a still as a PNG of 8-bit RGB without interlace, by PNG’s chunk rules: every chunk’s type four letters and its CRC checked; IHDR first; an optional PLTE (a suggested palette) before the image data; the IDATs consecutive; ancillary chunks read past; any other critical chunk refused; IEND last, empty, and nothing after it. It inflates the zlib stream as the zlib library’s inflate does, so two verifiers agree on every stream: the header’s check bits, method 8, a window of at most 32 KiB and no preset dictionary; a stored block’s length and its complement agree; at most 286 literal/length and 30 distance codes, a complete code-length code, a repeat with a length before it and within the code lengths, and a code for the end of the block; no Huffman code over-subscribed, and none incomplete but a literal/length or distance code whose longest code is one bit (a lone one-bit code, or none); a code with no symbol, or a distance past the bytes written, refused where it is read; the Adler-32, and nothing after it. The rows are exactly 1 + 3·width bytes each, under the five filters. A still a verifier does not read is prefix-digest (§11.3).

5.8 Components beside the lead. A package carries up to 8 components on one presentation timeline (§3.4 item 1, §4.5). Its lead is a video component, coded as §5.1–§5.7 say; every other component is cut at the lead’s piece starts (§4.5) and each of its pieces is one base object of the component’s base rung, and, where its media class has one, one floor object (§7.1). A component beside the lead has one base rung and no refinement rung. Each base object is §5.2’s manifest for its capability with target as below, and states its piece’s decoded-output digest (§6.5) where its tier is decode-equivalence (§6.4). Its source is the file the packager is given for the component’s id.

mediabase capabilitysourcethe piece’s objecttargetdecoded-output digest (§6.5)
audiowai.audio.flaca WAV (§4.7) of one or two channels, at the component’s timebase rate/1the piece’s samples as FLAC, by the encoder below{"channels", "rate", "samples"}, samples the piece’s sample framesSHA-256 of the piece’s samples as interleaved s16le
audiowai.audio.scorea WSCR (score §2) of one or two channels at sample_rate/1a WSCR of the same contract with duration the piece’s samples, holding each note that sounds in the piece, its start made relative to the piece, in the source’s order, and in a voiced score (score §2.4) each one’s voiceas wai.audio.flac, channels the contract’sSHA-256 of the piece’s mixdown (score §3.4) as s16le, two channels interleaved
graphicswai.graphics.vectora WVG1 (graphics-vector §2) whose t0 is 0 and whose dur is the component’s units times P of its ratethe source with t0 and dur set to the piece’s ticks, piece start · P and piece units · P, and each SIZE.cost restated for that window (graphics-vector §3.4(b)); nothing else changes{"frames", "h", "w"}, the first listed sizethe clip digest (SPEC §7) of the piece’s frames g as RGBA8P at the first listed size and the component’s rate
textwai.text.timeda WebVTT document in the component’s lang (timed-text §5.1, read by §5.3), on the timebase 1000/1the WebVTT segment of the piece’s span (timed-text §5.2, mpegts0 0){"lang", "ms"}: the component’s lang, which a segment does not state, and the piece’s millisecondsSHA-256 of the segment §5.2 writes of the cues the payload reads to (§5.3), which is the payload
feedwai.feed.statea WWLD of the sim wai.world.sim.feed0 (state-feeds) with no assets, at tick_rate/1a WWLD of the source’s contract, its graph the state after every op of a tick before the piece, its ops those of the piece’s ticks as the source signs them, sealed with the package’s key at the piece’s last tick{"ticks"}none: its seal states its state hash

wai.text.timed is a record (SPEC §5), which a sink does not dispatch to a decoder: in a package its payload is the WebVTT segment its extension writes, which a sink reads and draws in its own text layer (browser-sink §14.3), and a packager reads it as standalone media of class StandardDefined, with no licence class (§3.6 rule 1): the reader of timed-text §5.3 reads exactly what §5.2 writes, so the cues a segment reads to are written back byte for byte. rate is the component’s timebase num, whose den is 1; P is graphics-vector §3.1’s ticks per frame. A component’s source spans the lead’s whole timeline: the WAV’s sample frames, the score’s duration and the document’s dur are the lead’s end converted (§4.5), every cue ends by it and every op’s tick is before it (pack-source-length). A source of another rate than the component’s timebase, or a graphics document that does not list it, is refused (pack-source-rate), and one that does not read as its format, pack-source-format (a WAV’s and a WebVTT document’s own reasons, §4.7 and timed-text §12, first).

The FLAC encoder (flac-i16/1, wai-rs codecs::audio::flac_encode_i16) writes 16-bit samples of one or two channels as RFC 9639 FLAC, every byte a function of the samples, the channel count and the rate: fLaC; one STREAMINFO block, the last metadata block, with minimum and maximum block size 4096, minimum and maximum frame size 0, the rate, the channels, 16 bits per sample, the inter-channel sample count and an MD5 of zeros; then per 4096 inter-channel samples (the last block fewer) one frame of fixed block size: sync 0xFFF8, block-size code 7, sample-rate code 0, independent channels, sample-size code 4, the frame number (RFC 9639 §9.1.5), the block size less one in 16 bits and the header’s CRC-8; per channel, the first of the shortest of the fixed predictors of orders 0 to 4 (below the block’s length; Rice coding, partition order 0, the parameter 0 to 14 whose code is shortest, the least on a tie) and then verbatim; no wasted bits; zero bits to a byte; the frame’s CRC-16.

6. Stage index

6.1 A package carries, per component, one stage index, the file index-<id>.wsix (WSIX), per window: one window for on-demand content, a chain of windows for live (§9.2). A window holds whole pieces. Integers are little-endian.

6.2 Layout. A 160-byte header, then n_pieces piece records of 32 bytes, n_entries entry records of 128 bytes, n_classes class records of 16 bytes, the class plan, and the string table. Nothing follows the string table. Reserved and padding bytes are zero.

OffsetSizeHeader field
04"WSIX"
42version, 1
62media_class: 0 video, 1 image, 2 audio, 3 world, 4 feed, 5 avatar, 6 film, 7 graphics, 8 text
84tb_num
124tb_den
164n_pieces
204n_entries
244n_strings
284n_classes
3232policy_digest (§3.2)
6432prev_index_digest: the previous window’s index digest; zero for on-demand content and a first window
968first_piece: the absolute number of the window’s first piece
1048first_entry: the absolute ordinal of the window’s first entry across the chain
11232plan_digest (§4.2) of the pieces’ starts
1442component: the component’s id, as a string index
14614reserved
OffsetSizePiece record field
08start, in timebase units
84duration, in timebase units
124first_entry, window-relative
162n_entries
181kind: 0 natural, 1 static, 2 synthetic, 3 motion_high (§4.6), 4 world segment, 5 audio (an audio component’s piece, §5.8), 6 none (a floor-only component’s piece, which no source classifies, §7.8, and a piece of a graphics, text or feed component, §5.8)
191flags: bit 0 scene cut, bit 1 forced at max (§4.3), bit 2 started by a cue, bit 3 aligned to the lead (§4.5)
204rules: bit i set when rule i matched the piece (§8.1)
244moments: bit i set when the piece lies in moment i (§8.1)
284reserved
OffsetSizeEntry record field
032content_hash (§5.3)
328byte_len: the whole envelope’s length
408piece: the absolute piece number
484refines: the window-relative index of the entry this one refines; 0xFFFFFFFF for none
522capability, as a string index
541role: 0 base, 1 refinement, 2 floor, 3 snapshot, 4 ops, 5 receipt, 6 asset
551layer
561prefix_tier: 0 none, 1 entropy-consistency, 2 decode-equivalence (§6.4)
571flags: bit 0 deferrable, bit 1 duplicate (an earlier entry of the index has the content hash), bit 2 the manifest carries derived_from
582reserved
604deadline_ms: 0 for none (§8.6)
6432prefix_digest (§6.5)
968sse (§6.6); 2^64 − 1 when not computed
1044manifest_len: the envelope’s man_len
1084reserved
1128payload_len: the envelope’s payload bytes (for a WAI2, every component’s)
1208reserved

byte_len, manifest_len and payload_len are the envelope’s own lengths; for an object whose payload format WAI checks they are its counted bytes (staged-measurement §3), so a verifier also requires each such payload to parse as its format, after the other object rules of §6.3 (object-lengths).

A class record is u16 name (a string index), u8 max_layer, u8 flags (bit 0 floor only), u32 max_kbps (0 for none) and 8 reserved bytes, in the policy’s class order. The class plan is n_classes rows of n_pieces bytes, each the highest layer the class lists for the piece or 0xFF for the floor only (§8.1), zero-padded to a multiple of 8. The string table is n_strings strings, each a u16 length and that many bytes of UTF-8, at least one, zero-padded to a multiple of 8. It is the component’s id, then the class ids in policy order, then each capability at its first use, and a package’s verifier holds an index to that (§11.3).

index_digest = BLAKE3("wai:pack-stage-index\x01" ‖ file)
leaf_i       = BLAKE3("wai:pack-stage\x01" ‖ entry record i)
stage_root   = the jwp-receipts §2.2 Merkle root over the leaves

A reader MUST refuse a file that is not this layout, the first that applies, in this order:

  1. its first four bytes are not WSIX, a file of fewer included (index-magic);
  2. a file of fewer than six bytes (index-truncated), or a version other than 1 (index-version);
  3. a file shorter than the header, or than the tables its counts give: the piece, entry and class records, the class plan with its padding, each string’s length and bytes, and the string table’s padding, every size taken from its count before anything is allocated (index-truncated);
  4. a reserved or padding byte that is not zero (index-reserved);
  5. a media class with no meaning (index-media-class); then, entry by entry, a role, then a tier, with none (index-role, index-tier);
  6. a string that is empty or not UTF-8, then a string index past the table (index-string);
  7. bytes after the string table’s padding (index-trailing).

6.3 Rules. An index lists every WAI object of its component, in piece order. Files that are not WAI envelopes (CMAF initialization segments and fragments, FLAC files) MUST NOT appear in it. A verifier MUST refuse an index that breaks one of these, checked in this order:

  1. A timebase of zero (index-timebase). An index with no previous window (prev_index_digest zero: on-demand content, or a chain’s first window) whose first_piece or first_entry is not 0, or whose first piece does not start at 0, the start of the presentation timeline (§4.5) (window-start). Then piece by piece: a piece that lasts no time or does not start where the one before it ended (piece-order); whose first_entry is not the number of entries before it, or that has none (piece-entries); of a kind or with a flag that has no meaning (piece-kind). Then pieces whose entries are not the index’s entries (piece-entries).
  2. Entry by entry, in order: an entry that does not name its piece’s absolute number (piece-entries); a flag with no meaning (entry-flags); within its piece, a role before one that comes earlier in the order base, refinement, floor, any other, or a refinement at a lower layer than the refinement before it (entry-order); an entry that is not a refinement above layer 0, a refinement at layer 0, or refines on an entry that is not a refinement or missing from one (layer-role); a refinement whose refines is outside the window (refines-outside-window), is not an earlier entry of its own piece (refines-mismatch) or is not one layer below it (layer-gap); its tier (§6.4: tier-entropy-passthrough, tier-weakest, then tier-mismatch); a non-zero prefix_digest with a tier other than decode-equivalence (digest-without-tier); a deadline on an entry that is not a refinement (deadline-role); a floor whose flag bit 2 is not set exactly when its piece has a base, so a floor presents its piece’s base and a piece of floors alone presents none (floor-derived-from); a content hash an earlier entry has at another length (dup-length); the duplicate flag other than set exactly when an earlier entry has the content hash (dup-flag).
  3. Class by class: a flag with no meaning (class-flags); a plan value other than 0xFF for a floor-only class, or other than 0xFF or at most both the class’s max_layer and the piece’s highest layer otherwise (class-plan).

With the objects in hand, a verifier MUST also refuse, entry by entry: bytes of another length (object-length) or that do not hash to the entry’s content hash (object-hash); bytes that are not one WAI1 or WAI2 envelope and nothing else (non-envelope-entry); a man_len or a payload length other than the entry’s (object-lengths); a capability other than the entry’s (object-capability; for a WAI2, its first entry’s); a refinement whose object is not a WAI1 whose refines names its piece’s base, its layer and producers of the layers below among which the entry it refines (§5.4, object-refines); and derived_from (SPEC §7.1) present otherwise than bit 2 says, or not well formed (object-derived-from). A member whose value is null is absent; for a WAI2, derived_from is present when any entry carries it, and well formed when every entry that carries it is. A manifest that does not parse, or names a member twice, is not an envelope (non-envelope-entry).

6.4 Tier. The tier a capability’s own decode holds is decode-equivalence for a capability whose registry class reconstructs exactly (StandardDefined, and NeuralIntegerExact, whose pin a sink verifies before it decodes), entropy-consistency for NeuralEntropyExact, and none otherwise (SPEC §5, §7). A capability whose registration is pending is read with the class its extension states (§3.6 rule 1): wai.graphics.vector holds decode-equivalence, and so does wai.text.timed as a text component’s base, whose segment reads to cues written back byte for byte (§5.8).

6.5 prefix_digest is the SHA-256 decoded-output digest (SPEC §7, Digests) of the prefix the entry completes when prefix_tier is decode-equivalence, and all zero otherwise: for video, the clip digest over RGB8 frames (for a still, of its frame held for the piece’s frames, §5.7); for an image, the RGB8 frame digest; for audio, the digest of the s16le interleaved samples (a score’s, of its mixdown); for graphics, the clip digest over the overlay’s RGBA8P frames at the payload’s first listed size and the component’s rate; for text, the SHA-256 of the segment the payload reads to and §5.8 writes again; for world, feed, avatar and film entries, zero (their state hashes live in their receipts, and a feed piece’s in its seal, §5.8). A verifier decodes the prefix and compares.

6.6 sse is the integer sum, over every output sample of the prefix, of the squared difference from the source sample the packager coded (RGB8 channels, or s16 samples). It is the producer’s statement: a verifier holding the source recomputes it, and one without the source cannot. A floor’s sse is 2^64 − 1.

7. Floor and derived_from

7.1 Every piece of a video, image or audio component MUST have a floor rendition in a capability of SPEC §5’s mandatory floor or recommended set; a packager refuses a video component with no floor rung (floor-missing). A film or world package SHOULD have one. A component may have several floor rungs: each piece then has one floor object per floor rung, in ladder order (§5.7), so one package carries a ladder of standard-codec renditions over one base, which is coded once. Each floor rung has a floor name (§2): the component’s id when the component has one floor rung, the rung’s id when it has several. A floor name names files (§7.3, §7.6, §8.3), so one that cannot be one path segment of a listing, or two floor names of a package equal once A–Z are mapped to a–z, are refused (pack-id, §11.1). This revision packs supplied AV1 floors (wai.video.av1 or wai.video.av1.lossless, source = "supplied"), one stream per floor rung, and, for a score beside the lead, a FLAC floor it encodes from the score’s mixdown (§5.8), and, for a graphics component, a text floor it encodes from each piece’s transcript (§5.8). An audio component whose base is wai.audio.flac needs no floor rung, its base being of the mandatory floor; it may have one, of wai.audio.opus (SPEC §5’s recommended set), supplied in the CMAF form (§7.9).

A video component’s first floor rung, in ladder order, is its floor rendition: the one SPEC §5’s video floor has every sink that presents video decode. It MUST be wai.video.av1 in the video-floor profile (SPEC §5, Video floor): AV1’s Main profile, every operating point at the Main tier and at level 4.1 or below, and its largest frame and its frame rate (the component’s timebase) within level 4.1’s limits. After that rung’s checks of §7.3 or §7.7, and before it writes anything, a packager refuses a first floor rung that is not: floor-av1-profile for another capability or profile, floor-av1-level for the High tier, a higher level, or a frame or a rate past the level’s limits. Its other floor rungs may be any rendition this section admits.

7.2 Floor objects. Each piece’s floor is one object of the floor rung’s capability (§5.2), whose manifest, except in a floor-only component (§7.8), carries derived_from (SPEC §7.1, Derived renditions) naming the piece’s base: motif the base’s content hash, motif_capability its capability, renderer the floor’s maker, and settings BLAKE3("wai:render-settings\x01" ‖ JCS(settings)) (§3.6). For a supplied floor, renderer is supplied@<the supplied listing's SHA-256> and the settings are {"source": "supplied", "supplied_listing": "<that SHA-256>"}, so neither names a position. This profile fixes these members: they are not the producer’s statement here (SPEC §7.1), and a verifier holds every floor object to them, to its framing header and to its target (§11.3). An AV1 floor object’s payload is the length-prefixed framing the reference sink decodes (wai-rs codecs::video): n, h, w, fps_num, fps_den as u32 little-endian (the piece’s temporal units, the sequence header’s maximum frame height and width, the component’s timebase num and den), then each temporal unit as a u32 little-endian length and its bytes as the stream holds them.

7.3 A supplied floor is a directory holding, per floor rung of a video component, <floor name>.obu (§7.1): an AV1 stream in the low-overhead OBU format (every OBU with its size field), one temporal unit per source frame. Its listing (SPEC §3.1) of every file is pinned by the policy’s floor.supplied_listing. A packager MUST refuse, before it writes anything, the directory and then each floor rung’s stream in ladder order:

Key-frame sequence headers (informative). A floor taken from an existing ladder without re-encoding (its CMAF samples demuxed to this form, each with its temporal delimiter put back) needs a sequence header in every unit that starts at a shown key frame. Where a ladder’s key-frame samples lack one, a producer that puts one back must put back the stream’s own first sequence header OBU, byte for byte, after each such unit’s temporal delimiter: §7.3 refuses any other bytes (floor-av1-malformed). The sequence header in the ladder’s sample entry (av1C) is not always the stream’s: an encoder may write it before it settles its coding tools, so that it differs from the in-band one in its tool flags, and a stream with that header put back is refused. The packager writes its own av1C from the in-band header (§7.6). A ladder whose key frames carry no in-band header cannot be taken in the CMAF form (§7.7), whose units are its samples as they are.

The plan a supplied floor is made against is the keyframe plan, the file plan-<id>.txt: each piece’s first frame in decimal, one per line, each line ended by a line feed. Its pieces’ starts are the plan of §4.2, and wai-pack plan writes it from the source and the policy before any floor is encoded.

7.4 The floor is presentation. Its output is never compared with the primary’s. A floor object’s manifest writes intent: "replicate", which promises only what its capability’s decoder conformance promises (SPEC §7: wai.video.av1 is registered standard-tolerance); no tier and no claim against a primary or a source covers it.

7.5 A floor produced by an encoder that is not a pure function of its input (§11) MUST name that encoder and its version in derived_from.renderer, and, where the encoder is hardware, the device and the driver that ran it. An AVC or HEVC rendition encoded over a package’s pieces names its encoder this way, and binds the device and the driver that ran it through its settings (platform-encode §9); it is beside the floor of §7.1, not in its place, and the package does not list it.

7.6 CMAF. Each floor rung’s floor is also written as CMAF files, which no stage index lists: floor/<floor name>/init.mp4, and one fragment per piece, floor/<floor name>/<SHA-256 of the fragment>.m4s. Box layouts are ISO/IEC 14496-12’s, with the AV1 binding’s sample entry; every integer is big-endian and every time 0:

The codecs string of the floor is the AV1 binding’s short form, av01.<profile>.<level, two digits><M or H>.<bit depth, two digits>.

7.7 A supplied CMAF floor. With floor.supplied_form = "cmaf", a supplied floor is an existing ladder’s CMAF files rather than OBU streams: per floor rung of floor name f (§7.1), a directory <f>/ holding init.mp4 and the floor’s media segments, which are every other file of <f>/, each named *.m4s and in no directory below it, in byte order of their names, at least one. The floor’s temporal units are the segments’ samples, in order, each a temporal unit without its temporal delimiter: a unit is the two bytes 0x12 0x00 (a temporal delimiter OBU with its size field and no extension) followed by the sample. Boxes are ISO/IEC 14496-12’s; a box’s 32-bit size is at least 8, a size of 1 is the 64-bit size that follows its type (at least 16), and a size of 0 is refused; a box’s children tile its body:

A packager MUST refuse, before it writes anything, after the directory’s listing (§7.3), floor rung by floor rung in ladder order and segment by segment in order:

  1. a floor rung with no <f>/init.mp4 (floor-stream-missing); another file of <f>/ that is not a media segment, or no media segment (floor-cmaf-malformed);
  2. an init.mp4 that is not the above (floor-cmaf-malformed), or whose timescale is 0 or whose edit list is another (floor-cmaf-timing);
  3. a media segment that is not the above, or that holds a sample of no bytes (floor-cmaf-malformed), or a sample with a composition offset other than 0 or a moof whose decode time is not where the one before it ended (floor-cmaf-timing), each where it is read; then one whose first sample is not a sync sample (sample_is_non_sync_sample set) (floor-cmaf-malformed); then a sample that does not last one frame (duration · num = T · den), or a segment whose first decode time is not the frame where the segment before it ended, the first at frame 0 (dts · num = frame · T · den) (floor-cmaf-timing); then its units, as §7.3 reads a stream’s (floor-av1-malformed, a unit’s own faults held to the end);
  4. then, as §7.3: a unit count other than the source’s frames (floor-frame-count), and the faults held back (floor-av1-malformed); then an init.mp4 whose av01 entry has no av1C, or whose av1C profile, level, tier, bit depth, monochrome and chroma fields, or whose sample entry’s width and height, are not the sequence header’s and its maximum frame size (floor-cmaf-malformed): a player configures its decoder from them, and §8.3’s playlists name the sequence header’s; then a sample that is a sync sample where its unit does not start at a shown key frame, or that is not one where its unit does (floor-cmaf-malformed): the ladder’s sample flags reach players unchanged, and players seek by them;
  5. other than one media segment per piece of the plan, each holding exactly its piece’s frames (floor-cmaf-pieces);
  6. a key frame §7.3 refuses (floor-keyframe-mismatch).

The floor’s CMAF files are then the supplied ones, byte for byte: floor/<f>/init.mp4 the supplied init.mp4, and per piece floor/<f>/<SHA-256>.m4s its media segment (§7.6’s writer is not used), which §8.3’s playlists and MPDs list. The package also carries the supplied floor’s listing as floor.listing: its SHA-256 is floor.supplied_listing, which the policy digest, and so the receipt, binds, and it names each of the ladder’s files by its SHA-256, so the receipt binds the ladder’s bytes. A server may hold those files once, for the ladder and the package alike (by a hard link, or by their content address): they are the same bytes. The floor objects carry the units as §7.2 frames them, as in the OBU form.

7.8 The floor-only profile. A component of profile = "floor-only" is its floor rungs alone: no base, no refinement and no source (§3.4 item 8). It is for a library that delivers its standard-codec renditions as WAI objects, each named by its content hash, listed in a signed stage index and bound to a signed receipt, without the integer lane.

7.9 An Opus floor. A sound track beside the lead (§5.8) whose base is wai.audio.flac may have one floor rung of wai.audio.opus, source = "supplied", only in the CMAF form (floor.supplied_form = "cmaf") and only on a timebase of 48000/1: anything else is refused pack-capability-unsupported:wai.audio.opus. Its floor is an existing ladder’s Opus in ISOBMFF, the directory <f>/ of the supplied floor (its floor name, §7.1) beside the lead’s floors, under the one listing the policy pins: <f>/init.mp4 and its media segments, the other files of <f>/, each *.m4s, in byte order of their names (§7.7). Boxes are read as §7.7 reads them.

A packager MUST refuse, before it writes anything, after the lead’s floors, in this order: init.mp4 (floor-stream-missing without it, then the rules above); segment by segment, its boxes, its samples, its first sample’s decode time where the segment before it ended, the first at 0 (floor-cmaf-timing), and each packet; then a sample that does not last its packet’s samples, but the floor’s last, which may last from 1 to them (the programme’s end) (floor-cmaf-timing); then packets that last, from decode time 0, fewer than the component’s samples N and P together, or that last them without their last (floor-frame-count); then media segments other than one per piece and at most one more, the tail, and a packet in another segment than this: piece k’s segment holds the packets whose decode time is at least the piece’s start s_k and below the next piece’s start, the last piece’s those from its start on; where there is a tail, it holds the packets whose decode time is N or later (floor-cmaf-pieces). A muxer that cuts its segments at the pieces’ starts on the decode timeline writes such a floor: the packet that carries a programme’s last P samples (the encoder’s delay, which the pre-skip removes) starts at N when the pieces end on a whole packet, and a muxer cutting at every piece start writes it into a segment of its own, the tail.

The floor objects. Each piece’s floor is one object of wai.audio.opus (§7.2: its manifest §5.2’s, target the piece’s {"channels", "rate", "samples"} as its base’s, derived_from naming the piece’s base, renderer supplied@<the supplied listing's SHA-256> and the settings {"source": "supplied", "supplied_listing": "<that SHA-256>"}). Its payload is the Ogg Opus stream (RFC 7845) that decodes alone to exactly the piece’s samples: the packets from the one holding decode time max(0, s_k + P − 3840) (80 ms of pre-roll, RFC 7845 §4.6, so a decoder that starts there has converged by the piece) to the one holding e_k + P − 1, e_k the piece’s end; its OpusHead (version 1, the dOps’s channels, input sample rate and output gain, mapping family 0) states the pre-skip s_k + P less the first packet’s decode time (a piece whose pre-skip would not fit 16 bits is refused floor-cmaf-malformed); its OpusTags holds the vendor string wai-pack and no comment; and its pages, of serial number 0, are OpusHead (beginning of stream), OpusTags and then one page per packet, sequence numbers from 0, each page’s granule position the samples of the object’s packets through it, the last page’s (end of stream) the pre-skip and the piece’s samples, each page’s CRC RFC 3533’s. Every byte is a function of the floor’s packets and the piece, so a verifier writes it again. Packets are shared between adjacent pieces’ objects (the pre-roll, and the packet a piece’s last samples sit in after the pre-skip); the CMAF files are the ladder’s own, each written once.

The files. The package carries the floor’s CMAF files as the supplied ones, byte for byte: floor/<f>/init.mp4, and floor/<f>/<SHA-256>.m4s for every media segment, piece k’s and the tail with the last piece, named in floor.listing (§7.7). This revision’s catalogs (§8) list the lead’s floors; a player reads an Opus floor’s files from floor.listing.

What it attests. The receipt signs the policy, so the listing that names the ladder’s bytes, and the index, so each floor object’s content hash; a verifier holds every packet to its segment, the segments to the pieces and each object to the stream above, so each object carries exactly its piece’s packets. A floor states no tier (§7.4): Opus is standard-tolerance (SPEC §7), and no digest says what a decoder makes of the packets. The FLAC base is what states each piece’s samples. A piece is closed for presentation (each object decodes alone), not in its packets: Opus carries decoder state from packet to packet, which the pre-roll restores.

7.10 The protected floor. A package whose policy has a protection table (§3.3) has a protected presentation: its floor CMAF files (the lead’s AV1 floors, §7.6 and §7.7, and each sound track’s Opus floor, §7.9) encrypted with Common Encryption (ISO/IEC 23001-7) under the cenc scheme, the cbcs scheme or both, with the keys the table names by KID, and HLS and DASH presentations that carry them. The package itself does not change: the table is in the policy, so the policy digest, and the receipt that signs it (§9.1), names every KID and the SHA-256 of the CPIX document they come in. The presentation is written beside the package by whoever holds the keys, and is a function of the package’s files and the keys: a verifier holding the CPIX document writes it again byte for byte. The reference is wai-pack’s protect (wai-pack protect, wai-pack protect-verify), with cenc and av1_tiles; the corpus is pack-conformance/protected/, whose second implementation is protected_verify.py.

Texts read for this section: the AV1 Codec ISO Media File Format Binding v1.3.0 (3 April 2024), §2.4 and §4; the AV1 Bitstream and Decoding Process Specification, §5.5, §5.9, §5.11.1, §7.8, §7.20 and §7.21; the W3C notes “ISO Common Encryption (‘cenc’) Protection Scheme for ISO Base Media File Format Stream Format” and “‘cenc’ Initialization Data Format”, §4 (the common system ID and its pssh); the W3C Encrypted Media Extensions §9.1 (Clear Key); DASH-IF IOP v5.1.0 Part 6 (December 2025), §6.2 to §8; the DASH-IF registry of content protection system IDs; draft-pantos-hls-rfc8216bis-22 §4.4.4.4, §4.4.6.5 and §5; “Encapsulation of Opus in ISO Base Media File Format” §4.3.5, which defines no subsample for Opus; and key-release §4.4 for the CPIX document. ISO/IEC 23001-7 and ISO/IEC 14496-12 were not available to this revision: the boxes and scheme rules below are written as the texts above restate them, and a browser’s own decryptor plays the corpus (pack-conformance/protected-browser/).

The keys. The CPIX document is read as key-release §4.4 steps 1 to 3 read one; a writer refuses, before it writes anything, a package that does not verify (§11.3; protect-package) or whose policy has no table (protection-absent), a document whose SHA-256 is not cpix_sha256 (protection-cpix-mismatch), one key-release refuses (protection-cpix), a key whose KID the document does not hold (protection-kid-unknown), and one whose KID the document gives another scheme than the key’s (protection-scheme-mismatch). A floor’s key under a scheme is the one key the table names for it (§3.4 item 13). DASH-IF IOP Part 6 §6.4 asks the representations of one content under two schemes to use two keys: a CPIX key has one scheme, so a table that uses both schemes names two KIDs per component.

The files, each scheme the keys name, cenc first, and each floor in ladder order:

&lt;scheme&gt;/floor/&lt;f&gt;/init.mp4                  the floor's init, protected
&lt;scheme&gt;/floor/&lt;f&gt;/&lt;SHA-256 of it&gt;.m4s       each media segment of the floor, protected, in order
&lt;scheme&gt;/hls/master.m3u8, &lt;scheme&gt;/hls/floor-&lt;f&gt;.m3u8     (when a key line can be written, below)
&lt;scheme&gt;/dash/manifest.mpd
protection.json                              the index
&lt;file&gt;.sig                                   beside each playlist, MPD and the index, under a
                                             policy with a statements table (§8.13)
LISTING                                      the listing of every other file (SPEC §3.1)

A floor’s segments are its classical playlist’s, in its order (§8.3), for an AV1 floor, and the media segments floor.listing names in the floor’s directory, in its order, for an Opus floor (§7.9).

An init is the clear one with three changes, and none other. Its one sample entry, av01 or Opus, is renamed encv or enca and gains a sinf as its last box: frma holding the clear entry’s type, schm (version 0, flags 0, the scheme’s four characters, scheme version 0x00010000), and schi holding tenc. Under cenc, tenc is version 0: two zero bytes, default_isProtected 1, default_Per_Sample_IV_Size 8 and the KID. Under cbcs it is version 1: a zero byte, the pattern (crypt_byte_block 1 and skip_byte_block 9 for video, the binding’s recommended 1:9; 0 and 0 for audio, no pattern: every whole block), default_isProtected 1, default_Per_Sample_IV_Size 0, the KID, default_constant_IV_size 16 and the constant IV. The last box of moov is one pssh of the common system ID (version 1, one KID, the track’s, and no data), which every EME key system reads (the W3C note’s §4); system-specific pssh data, whose formats the registry does not publish, is not written. Box sizes are rewritten, each header in the form it had.

A media segment is the clear one with these changes. Each traf gains, after its last box, saiz (version 0, flags 0: a default size where every sample’s auxiliary information has one length, else 0 and each length), saio (version 0, flags 0, one entry: the offset of the first sample’s entry in senc, from the moof’s first byte) and senc (version 0, flags 2 for video, whose samples carry subsamples, and 0 for audio; then each sample’s IV at the track’s IV size and, for video, its subsample count and subsamples); for a cbcs audio track, whose samples have no auxiliary information, senc alone (DASH-IF IOP Part 6 §6.2). Each trun’s data offset grows by what its moof grew, a sidx’s first offset and referenced sizes are the protected boxes’ (a reference to another sidx is refused, protected-box), and each sample’s protected bytes are encrypted in place. Version 0 of saiz holds a size up to 255, so a sample with more subsamples than fit is refused (the binding’s §4.1 note).

What a sample protects. A video sample by the binding’s §4.1, from where each tile’s bytes (its decode_tile structure with any trailing bits) lie: every OBU header, size field, sequence header, frame header (in a frame OBU too), redundant frame header, metadata, padding and tile group header stays clear; under cbcs each tile is one subsample whose protected bytes are the tile’s, a tile of under 16 bytes included; under cenc a tile of at least 16 bytes is one subsample whose protected bytes are its whole 16-byte blocks counted back from its last byte, and a shorter tile stays clear. Clear runs join the protected range after them, a trailing clear run is a subsample of no protected bytes, and a clear run past 65 535 bytes is split into subsamples of 65 535 clear bytes. The tiles are found by reading each frame header (av1_tiles), which reads the sequence header and what the reference frames before it left (their sizes, order hints and segmentation features), so a track’s samples are read in decode order from its av1C’s sequence header on; a scalable stream (an operating point other than 0), a tile list OBU and a frame header the AV1 text does not allow are refused (protect-av1). An audio sample is protected whole: Opus in ISOBMFF defines no subsample.

The ciphers, AES-128 with the key. cenc: counter mode over the sample’s protected bytes in order, joined across its subsamples, from the counter block IV ‖ 0 (an 8-byte IV, the low 64 bits counting blocks), the last partial block encrypted. cbcs: each subsample’s protected bytes (an audio sample’s, whole) a chain of its own from the constant IV, over the blocks the pattern encrypts (with 1:9, the first block of every ten; with none, every whole block), a last partial block clear.

The IVs. A cenc sample’s IV is the first 8 bytes of SHA-256("wai:cenc-iv\x01" ‖ key ‖ u32 len(f) ‖ f ‖ the clear segment's SHA-256 ‖ u32 i), f the floor name and i the sample’s place in its segment from 0; a track’s cbcs constant IV is the first 16 bytes of SHA-256("wai:cbcs-iv\x01" ‖ key ‖ u32 len(f) ‖ f). So the IVs are unpredictable without the key, and no counter block repeats under a key unless two segments are the same bytes, whose ciphertexts are then the same too.

The catalogs. Under each scheme, the DASH presentation is one Period of an adaptation set of the AV1 floors (each a Representation with its codecs, width and height) and one per Opus floor (codecs="opus", its AudioChannelConfiguration), each SegmentList as §8.3 writes one, in the track’s timescale, naming the protected files; each adaptation set carries the ContentProtection descriptors of DASH-IF IOP Part 6 §7 and §8: urn:mpeg:dash:mp4protection:2011 with the scheme and cenc:default_KID; the common system ID’s with its pssh as cenc:pssh; with licence_uri, DASH-IF’s Clear Key (e2719d58-a985-b3c9-781a-b030af78d30e, value="ClearKey1.0", dashif:Laurl licenseType="EME-1.0"); and an empty descriptor per system ID systems names. The HLS presentation is written when every track has a key line: with key_uri, an identity key (#EXT-X-KEY:METHOD=SAMPLE-AES-CTR under cenc, METHOD=SAMPLE-AES with the constant IV under cbcs, URI the key URI with {kid} the KID’s 32 lowercase hexadecimal digits, KEYFORMAT="identity", KEYFORMATVERSIONS="1"); and under cbcs, when systems names 94ce86fb-07ff-4f43-adb8-93d2fa968ca2, that system’s skd key line (URI="skd://<KID, hex>" and the KEYFORMAT it registers, com.apple.streamingkeydelivery). Each media playlist is §8.3’s floor playlist with its key lines before its EXT-X-MAP (an EXT-X-KEY covers the initialization sections declared after it), its segment durations the samples’; the master names the key lines once each as EXT-X-SESSION-KEY, the Opus floors as an AUDIO group, and each AV1 floor as a variant whose BANDWIDTH and AVERAGE-BANDWIDTH add the audio group’s highest to the floor’s (§8.3’s rates over the protected bytes).

The index, protection.json, is RFC 8785 JSON and a line feed: cpix_sha256, the package’s package_receipt (its receipt hash) and policy_digest, and tracks, one per scheme and floor in the order above, each {"floor", "init", "kid", "scheme", "segments"}, init and each segment the pair [clear SHA-256, protected SHA-256]. It is what pairs each protected file with the clear one the receipt binds (§7.6, §7.7). Under a policy with a statements table (§8.13) the writer, given the package’s key, signs a statement of each playlist, the MPD and the index, and refuses before it writes anything without that key or with another (protect-statement-key): the index is then signed by the key that signs the receipt, and a party that replaces a protected segment and the index together is found.

Checking. A presentation is held to its package, which must verify (§11.3), and, every failure reported with its file: LISTING (protected-listing); the statements, as §8.13 checks a package’s, protection.json among the files stated; the index names the package, the CPIX digest and one track per scheme and floor, each with the package’s clear init and segments (protected-index); each file it names is present (protected-missing) and each segment named by its SHA-256 (protected-name); each init reads back, with the three changes removed, to the clear init (protected-mismatch), its boxes as above (protected-box, protected-scheme), its IV size the scheme’s (protected-iv-size), its KID the table’s (protected-kid) and its pssh the one common one of that KID (protected-pssh); each segment’s boxes (protected-box), its senc at the init’s IV size with saiz and saio agreeing (protected-iv-size), each sample’s subsamples those the clear sample gives (protected-subsample), each cenc IV distinct in its track (protected-iv), and the segment read back to the clear one in every byte no protected range covers (protected-mismatch); the catalogs and the index written again from those files (protected-catalog, protected-index); and no other file (protected-unexpected). With the CPIX document, also each IV and constant IV the derived one (protected-iv), each segment decrypted to the clear one byte for byte, and every file written again byte for byte (protected-mismatch) but the statements, which the package’s key signs and the keys cannot write, and LISTING, which lists them.

What it attests. With the receipt, that the clear floor’s every sample is in the protected presentation, encrypted under the keys whose KIDs the policy names, by the binding’s rules; with the CPIX document, that it decrypts to that floor byte for byte. What it does not: a licence. The content protection systems are named by ID, and their licence exchanges and their own pssh data are theirs (key-release §9.2): none is tested here, and none is claimed. The WAI objects of the package, its bases and floor objects among them, are not encrypted by this section, so a protected presentation protects the floor’s delivery only where the package’s other files are not served beside it. Under a policy with no statements table the index is not signed. Clear Key is for test and development (DASH-IF IOP Part 6 §8): it hands the browser the key.

8. Classes, catalogs and delivery

8.1 What a class lists. The stages listed for a class are a pure function of the class, the piece and the effective policy, so every viewer of a class receives the same catalog bytes.

8.2 A catalog MUST NOT list a refinement without the stages below it.

8.3 HTTP. A package is offered in http-delivery’s Mode N (§11.1 there): the classical presentation is the floor alone, and WAI objects are offered in playlists of their own.

Every HLS file above and every MPD below is a rendering of http-delivery’s (Appendix G for a media playlist): the reference packager writes them with the binding’s one HLS and one DASH renderer.

DASH, the same as two MPDs (http-delivery §11.1), each <?xml version="1.0" encoding="UTF-8"?>, then <MPD xmlns="urn:mpeg:dash:schema:mpd:2011" profiles="urn:mpeg:dash:profile:full:2011" type="static" mediaPresentationDuration="PT<S.mmm>S" minBufferTime="PT<t>S"> over the component’s whole duration and its target duration, <Period id="p0" start="PT0S">, one AdaptationSet, </Period> and </MPD>, indented by two spaces a level:

The media type application/vnd.wai is not registered yet (http-delivery §3). The HLS, DASH and MoQ catalogs of this revision are the lead’s: each lists the lead component alone, as for a package of one component. Every component is listed in the stage views (§8.8), which name each of its objects by content hash, so a sink reaches a component beside the lead (§5.8) through them.

8.4 MoQ. moq/catalog.json is an MSF catalog (MSF §5) and a line feed: {"tracks": [...], "version": "draft-01", "initDataList": [...]}, its version the string MSF §5.1.1 gives draft-ietf-moq-msf-01. It is RFC 8785 canonical JSON but for initDataList, which MSF §5.1.7 places after tracks: it is written last, after version, each member’s value canonical. A component’s bases and refinements are a stage set of moq-streaming-format §4.1, one track per (layer, deadline), by layer, then by deadline ascending, each written as that section’s entry: namespace the policy’s asset_id; name <c>/l<k>, or <c>/l<k>@<deadline> for a refinement whose deadline is not defaults.deadline_ms; packaging "wai"; isLive false; role video; renderGroup 1; wai_staged with set the component’s id, layer k and lead_ms null (an on-demand package has no live schedule); wai_capabilities its entries’ capabilities by first appearance (a still among them on the base track, §5.7); wai_determinism the weakest tier of its entries, absent when that is none; and receipts declared: wai_receipts true, wai_receipt_alias the first eight bytes of BLAKE3("wai:pack-receipt-alias\x01" ‖ namespace ‖ 0x00 ‖ name) as a little-endian integer, less than 2^53 (so two packages published under one key keep apart chains), and wai_receipts_track <name>/receipts. A track depends on every track that carries an entry below one of its own entries, down the chain to the base, in track order: a track of each layer below it, none at or above it, and never one the catalog does not hold. A sink receives every track a track depends on (moq-streaming-format §4.1), so a dependency is all of them. A refinement track carries wai_delivery_timeout_ms (§8.6), a catalog field this extension defines under the wai_ prefix of moq-streaming-format §4. The base track has altGroup 1 when the component has a floor: each floor rung’s floor, in ladder order, is a track <f>/floor for its floor name f (§7.1), of packaging "cmaf" (draft-ietf-moq-cmsf-01), in the same namespace, with isLive false, altGroup 1, renderGroup 1, codec, width and height, and initRef f; and a sink that selects a floor subscribes to no layer track. Its initialization segment, floor/<f>/init.mp4 (§7.6, §7.7), is the entry of initDataList that initRef names, as CMSF §3.1 requires: {"data", "id": "<f>", "type": "inline"}, data the segment in base64 with padding (RFC 4648 §4); the entries follow the floor tracks’ order. The receipts tracks follow, one per media track in its order, each {"isLive": false, "name": "<name>/receipts", "namespace", "packaging": "wai-receipts"}. The catalog passes moq-streaming-format §4.1’s check, and a packager writes no other. A package holds no receipt: the publisher signs each track’s groups (a group a piece) on that track’s chain as it publishes them (jwp-receipts §2.4.1; §8.9 simulates it).

8.5 Priority. The base track’s publisher priority is 64 and layer k’s 64 + 32·k: a refinement track’s value is strictly greater than that of every track it depends on, so it is delivered after them (MOQT: a lower value is delivered first), and moq-streaming-format §4.1’s rule holds; a receipts track’s is its media track’s. The priority is MOQT’s (DEFAULT_PUBLISHER_PRIORITY), set by the publisher, not a catalog field.

8.6 Deadline. An entry’s deadline_ms is the longest a refinement may arrive after the base of its piece was published and still be presented; the rules’ or defaults.deadline_ms. A refinement track carries DELIVERY_TIMEOUT deadline_ms − live.layer_lag_ms[k] (§3.4 has refused a policy where that is below 1). Entries of one layer with different deadlines travel on different tracks. A relay may drop a late refinement (MOQT §5.2), so a refinement never delays the base.

8.7 Presentation. A sink presents piece p at the highest prefix whose objects all arrived and verified by p’s presentation instant, and MUST NOT delay the base to wait for a refinement. What the receipts of a dropped object say is jwp-receipts’ to define, not this extension’s.

8.8 Stage views. stages/<class>.json is RFC 8785 canonical JSON and a line feed: {"class": "<class>", "components": {"<c>": {"media_playlist": "../hls/<class>/wai-<c>.m3u8", "pieces": [...]}}, "policy_digest": "<hex>"}, with media_playlist only when the class’s HTTP view lists a WAI object (§8.3), and each piece {"dur", "p", "start", "stages": [...]} with "floor_only": true when the class takes it as floor only (and stages empty), and each listed stage {"bytes", "h", "layer"}, a refinement’s with "refines" (the content hash of the entry it refines) and "deadline_ms". A component beside the lead (§5.8) is a member of components too, without media_playlist: each piece with its base as its one stage where the class lists it, {"floor_only": true} and no stage where the class takes it as floor only (§8.1, from the component’s own entries and timebase).

8.9 Delivery, simulated. How a package’s objects reach a subscriber over a link is simulated by staged-measurement’s delivery simulator (staged-measurement §10, Appendix B); its figures are labelled simulated, with their plan, and are never a measurement of a network. The reference packager’s delivery states the model and the plan, and reads the outcome:

Under §8.5’s priorities a refinement delays a base by at most one chunk and one decode (staged-measurement §10.4). The publisher seals each track’s group with a group receipt on the track’s chain (wai_receipt_alias, §8.4); a subscriber accepts a delivered object by its Merkle proof, and holds a dropped one by inclusion alone, never as delivered (jwp-receipts §4.4).

8.10 Audio renditions. A programme’s audio is offered as renditions: its main audio in each language it has, dubs, alternates, commentaries and audio description. Each is described by an entry, and a set of entries is written as HLS EXT-X-MEDIA tags of type AUDIO (draft-pantos-hls-rfc8216bis-22 §4.4.6.1) and as DASH audio adaptation sets, with the roles and accessibility descriptors of ETSI TS 103 285 V1.4.1 §6.1.2. An audio description rendition is a broadcast mix: the programme audio with an access-services §5 description mixed in by its mixing instructions. Its entry names the description record by its record id (access-services §2), and the writer holds the entry to that record. The reference is the audio_renditions module of wai-rs (feature audio_renditions); its corpus is pack-conformance/renditions/, and its second implementation pack-conformance/renditions_verify.py.

8.10.1 An entry is a JSON object with exactly these members, each required:

membervalue
id1 to 64 of the ASCII letters, digits, ., _ and -: the DASH representation’s id
groupas id: the HLS group, GROUP-ID
name1 to 64 of the ASCII letters, digits, space, ., _, -, ( and ), neither first nor last a space: what a player shows, NAME
languagea language tag read as live-events §4.10 reads one
rolemain, alternate, dub, commentary or description
capabilitythe capability of its samples, from the table below
codecsits RFC 6381 codecs value, from that capability’s row
channelsan integer, 1 to 32: the most channels a segment of it holds
bandwidthan integer, 1 to 2^32 − 1: its peak rate in bits per second
uri1 to 1024 of the ASCII letters, digits, ., _, -, /, :, ~ and %: its media playlist
defaultyes or no
descriptionfor the role description, the description record’s id in 64 lowercase hexadecimal digits; null otherwise
capabilitycodecs
wai.audio.aacmp4a.40.2, mp4a.40.5, mp4a.40.29
wai.audio.opus.packetopus
wai.audio.flacfLaC
wai.audio.ac3ac-3
wai.audio.eac3ec-3

A rendition’s media segments hold its samples as the ISO base media file format carries its codec (for AC-3 and E-AC-3, ETSI TS 102 366 Annex F, which ATSC A/52 Annex F names); WAI decodes none of them.

8.10.2 Rules. The entries are held to these rules in order, the first that fails naming the refusal:

  1. every entry has §8.10.1’s shape, and there is at least one (rendition-malformed);
  2. each entry’s capability is in the table (rendition-capability) and its codecs value in its row (rendition-codecs);
  3. no two entries share an id, a uri, or a group and a name: the HLS text requires the names of a group to differ (rendition-duplicate);
  4. a default of yes is a main’s, and a group has at most one (rendition-default);
  5. an entry names a description exactly when its role is description, and the record it names is among those given (rendition-description-unbound); that record reads and verifies as access-services §2 reads a record, is of kind wai-description, and meets access-services §5.2’s rules (rendition-description-invalid); and its lang is the entry’s language, byte for byte (rendition-description-language);
  6. no two entries of a group that a player chooses unasked (§8.10.3) have one language and one set of characteristics (rendition-ambiguous): the HLS text asks such members to differ in them;
  7. every group has the same members, by name, each with the same language, role, default and description in every group (rendition-groups-differ): the HLS text requires the groups of one type to hold the same members with identical attributes but for their URI and their channels, so a group of stereo AAC and a group of 5.1 E-AC-3 offer the same choices.

8.10.3 HLS. Each entry, in order, is one tag:

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="G",NAME="N",LANGUAGE="L",DEFAULT=D,AUTOSELECT=A,CHANNELS="C",CHARACTERISTICS="X",X-WAI-DESCRIPTION="R",URI="U"

G, N, L, U and C are its group, name, language, URI and channels; D is YES for a default and NO otherwise; A is YES for the roles main, dub and description, the renditions a player may choose because they match its language or its listener’s accessibility setting, and NO for alternate and commentary, which a listener asks for. X is, for an audio description, public.accessibility.describes-video, then public.machine-generated when its record’s voicing is machine or machine-reviewed, as access-services §5.6 item 5 requires the synthesized voice to be labelled; CHARACTERISTICS is left out when X is empty. R is an audio description’s record id, and X-WAI-DESCRIPTION is left out for any other role: a client ignores an attribute it does not recognise (the HLS text, §6.3.1), and one that reads it fetches the record and its DAPT document (access-services §5.5).

8.10.4 DASH. Each entry, in order, is one adaptation set, its id 2 and up (1 is the video’s, §8.3), indented as §8.3’s:

&lt;AdaptationSet id="K" contentType="audio" mimeType="audio/mp4" lang="L" codecs="C" segmentAlignment="true"&gt;
  &lt;Role schemeIdUri="urn:mpeg:dash:role:2011" value="O"/&gt;
  &lt;Accessibility schemeIdUri="urn:tva:metadata:cs:AudioPurposeCS:2007" value="1"/&gt;
  &lt;SupplementalProperty schemeIdUri="tag:transaction.science,2026:wai/description" value="R"/&gt;
  &lt;AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="H"/&gt;
  &lt;Representation id="I" bandwidth="B"/&gt;
&lt;/AdaptationSet&gt;

O is the role, but alternate for an audio description, with the Accessibility descriptor of value 1 (for the visually impaired): ETSI TS 103 285 §6.1.2 Table 5’s broadcast-mix audio description. The Accessibility and SupplementalProperty lines are written for an audio description only. H is the channels, I the id, B the bandwidth and C the codecs. The segment addressing of each representation is §8.3’s.

8.10.5 What it does not do. The writer states renditions; it does not mix. The mix of an audio description is its producer’s, by the record’s mixing instructions, and a receiver-mixed description (a description stem a player mixes) is not a rendition here.

8.10.6 Reasons.

reasonraised at
rendition-malformed§8.10.2 rule 1
rendition-capability§8.10.2 rule 2
rendition-codecs§8.10.2 rule 2
rendition-duplicate§8.10.2 rule 3
rendition-default§8.10.2 rule 4
rendition-description-unbound§8.10.2 rule 5
rendition-description-invalid§8.10.2 rule 5
rendition-description-language§8.10.2 rule 5
rendition-ambiguous§8.10.2 rule 6
rendition-groups-differ§8.10.2 rule 7

8.11 Marks. A lead offers marks a sink navigates by: its shots, the pieces that follow a cut of §4.3 as its stage index flags them; its chapters, titled spans that each begin at a piece’s start; its break candidates, the piece starts at which a break may be inserted without cutting into a shot or a clip; and its clips, the run of whole pieces each moment of the policy covers (§8.1). The marks are a pure function of the lead’s stage index, the effective policy and the chapters a packager is given, so a verifier holding the index and the policy derives them again and compares. They are written as one JSON document, the record wai.delivery.marks (SPEC §5), and in three forms a player of each kind reads: a WebVTT chapters document, the date ranges of an HLS media playlist and the event streams of a DASH Period. Each form reads back to what it was written from. A sink that assembles a highlight reel of its own takes its clips from the document (sink-predicate §9). The reference is the pack_marks module of wai-rs (feature pack_marks); its corpus is pack-conformance/marks/, and its second implementation pack-conformance/marks_verify.py. The module writes the marks from a package’s index and policy; a package holds them, and its receipt names them, when its policy asks (§8.12).

Texts read for this section, each at its source: WebVTT, W3C Candidate Recommendation Draft of 2026-05-20, §1.6, §3.5, §4.1, §4.2.3, §4.5.1, §4.6.2 and §6.6; draft-pantos-hls-rfc8216bis-22 §4.2 and §4.4.5.1; and the MPD schema of ISO/IEC 23009-1:2026 at its revision of 2026-06-13 (its types PeriodType, EventStreamType and EventType, and its example MPD G9). The clause text of ISO/IEC 23009-1 was not read.

What the shots rest on. wai.pack.scenecut/1 cuts a source in integers, over its frames’ luma histograms, under thresholds the policy pins (cut_abs_permille, cut_rel_permille, cut_window, lookahead_frames, §4.3), and its cuts are the piece boundaries. The index flags each piece that follows a cut (§6.2, flag bit 0) and binds the policy by its digest. So a shot of the marks is a cut that is also a closed piece’s start, where a sink can seek and a break can be spliced; a cut within min_ms of a piece’s start starts no piece (§4.3 clause 4) and is no shot here. A verifier holding the source recomputes the cuts (§11.3); one holding the index and the policy holds the marks to them.

8.11.1 Inputs. The lead’s stage index (§6.2); the effective policy in its JSON form (§3.2); and the chapters a packager is given: a language, and 0 to 1,024 chapters, each a title and the instant it begins at, from_ms, in milliseconds on the presentation timeline.

8.11.2 Derivation. The steps run in order, and the first that fails names the refusal.

  1. The index. It reads (§6.2) and holds §6.3’s rules 1 to 3; it is on-demand content (prev_index_digest zero) of at least one piece; its plan_digest is §4.2’s over its pieces’ starts; every s(p) and e(p) is at most 2^53 − 1; the s(p) strictly ascend, so no two pieces begin in one millisecond; and the last piece’s e is above its s. Otherwise marks-index.
  2. The policy. It is a document (route-record §2) whose digest (§3.2) is the index’s policy_digest and whose lead is the index’s component. Its moment is an array of objects, each of exactly the members id (a string), from_ms and to_ms (integers, to_ms above from_ms) and tags (an array of strings). Otherwise marks-policy. Then there are at most 32 moments, their ids are distinct tokens, and each moment’s tags are distinct tokens; otherwise marks-moment.
  3. The moments and the clips. For every piece p and every bit k from 0 to 31 of its moments, the bit is set exactly when there is a moment k and p lies in it, s(p) below its to_ms and e(p) above its from_ms (§8.1); otherwise marks-moment. Each moment some piece lies in is a clip: the moment’s id and tags; its first piece, the first that lies in it; its number of pieces, to the last that lies in it; and its start_ms and end_ms, s of the first and e of the last. A moment no piece lies in has no clip.
  4. The chapters. There are at most 1,024; a language is given exactly when there is a chapter, and it reads (marks-chapter). Every title is one (marks-title). The instants strictly ascend, each below the lead’s end (marks-chapter). A chapter begins at the last piece whose s is at or before its instant, and no two chapters begin at one piece (marks-chapter). Its start_ms is s of that piece, and its end_ms the next chapter’s start_ms, or the lead’s end for the last.
  5. The shots. Every piece after the first whose flag bit 0 is set, with its s.
  6. The break candidates. Every piece after the first, with its s and the first of these that holds: its flag bit 2 is set, cue (a cue started it); it lies inside a clip, one of a clip’s pieces other than its first, and it is no candidate; a chapter begins at it, chapter; its flag bit 0 is set, cut. A piece of which none holds is no candidate.

A break candidate is an offer: a decider or a sink places a break there or not (live-ads, sink-predicate), and the marks place none. Piece 0 is no candidate: a break before the programme is a pre-roll, which a schedule places.

8.11.3 The document. The marks are a JSON object of exactly these members, written as its RFC 8785 canonical form and a line feed:

membervalue
kind"wai-pack-marks"
version1
componentthe lead’s id, 1 to 255 bytes
indexthe lead’s index digest (§6.2): 64 lowercase hexadecimal digits
duration_msthe lead’s end, at least 1
langthe chapters’ language, or null when there is no chapter
shotseach shot, {"piece", "at_ms"}, in piece order
chapterseach chapter, {"title", "from_ms", "piece", "start_ms", "end_ms"}, in order
breakseach break candidate, {"piece", "at_ms", "why"}, in piece order; why is "cue", "chapter" or "cut"
clipseach clip, {"id", "tags", "first_piece", "pieces", "start_ms", "end_ms"}, in the order of its moment in the policy

Its digest is BLAKE3("wai:pack-marks\x01" ‖ document), over the document’s bytes with its line feed.

A reader is given a text and refuses it marks-malformed unless it holds at most 16,777,216 bytes and ends with one line feed, before which is a document (route-record §2) that is its own canonical form, of exactly the members above, each of its form, kind and version as stated, and:

A verifier is given the lead’s index, the effective policy and a text. It reads the text, derives the marks (§8.11.2) from the index, the policy and the text’s own language and chapters (each a title and its from_ms), and refuses with the derivation’s reason, or marks-mismatch unless the document of what it derived is the text, byte for byte. So a document that leaves out a shot, offers a candidate inside a clip, widens a clip or begins a chapter anywhere but at its piece’s start is refused, and one whose titles another packager chose is not.

8.11.4 WebVTT. The chapters as a WebVTT file using chapter title text (WebVTT §4.6.2): the bytes WEBVTT and a line feed; then for each chapter k, from 1, a line feed; k in decimal and a line feed, the cue’s identifier; its start_ms and end_ms each as HH:MM:SS.mmm (hours of two digits or more, minutes and seconds of two, milliseconds of three), joined by -->, and a line feed; and its title, with &, < and > written &amp;, &lt; and &gt;, and a line feed. The cues are consecutive, so the file uses only nested cues (WebVTT §4.5.1), and each cue’s payload is chapter title text (§4.2.3 there) with no tag, which WebVTT §6.6 reads back to the title. The file states no language: a player takes the track’s, and a reader is given it. A reader of the form reads each chapter’s title and span, and refuses as marks-vtt a text that is not what this section writes of them: another header, a byte-order mark, a line end other than a line feed, a character reference other than the three, a < or a >, a time not as written, an identifier out of sequence, a title that is not one (§8.11.1), or chapters that last no time or do not each end where the next begins.

8.11.5 HLS. The lines a media playlist carries, given pdt0, the program date-time of media time 0 in milliseconds since 1970-01-01T00:00:00Z, each ended by a line feed:

D is pdt0 plus the mark’s start, written as pdt0 is; S is its end_ms less its start_ms as seconds with three digits after the point (5.600); T its title, W its why and I its id. A candidate is an instant, of duration 0 (§4.4.5.1 there). A packager writes the lines after a media playlist’s header tags and before its first segment, whose media time is 0, so that the program date-time applies to it. A mark whose date would fall after 9999-12-31T23:59:59.999Z is refused marks-hls. The three classes are WAI’s, and X-WAI-TITLE, X-WAI-WHY and X-WAI-TAGS are attributes of those classes; no CUE is written, since a candidate triggers nothing, and a player that knows none of the classes passes the tags by. A reader, given pdt0, reads each line’s attribute list (§4.2 there) and refuses as marks-hls lines that are not what this section writes of the marks they carry: a first line other than the program date-time, a date before pdt0 or not as written, a class this section does not write, attributes in another order, a value unquoted that is written quoted, a title, tag or reason that is not one, or chapters that do not meet.

8.11.6 DASH. The event streams of the Period that holds the lead, written before its first AdaptationSet (an EventStream precedes it in the schema’s PeriodType) and indented as §8.3’s MPD is, each line ended by a line feed. For each of the chapters, the break candidates and the clips, in that order, when there is one: the line <EventStream schemeIdUri="tag:transaction.science,2026:wai/marks" value="V" timescale="1000">, V being chapter, break or clip; for each mark k, from 1, the line <Event presentationTime="P" duration="L" id="k">C</Event>; and the line </EventStream>. P is the mark’s start in milliseconds (the Period starts at 0, §8.3), L its duration (0 for a candidate) and C its content, with &, < and > written as in §8.11.4: a chapter’s title, a candidate’s why, or a clip’s id and then its tags, each after one space. The content is the event’s own text, which the schema’s EventType holds (it is mixed, and messageData is prohibited there), as its example G9 carries a phone number. A reader refuses as marks-dash text that is not what this section writes.

8.11.7 Reasons.

reasonraised at
marks-index§8.11.2 step 1
marks-policy§8.11.2 step 2
marks-moment§8.11.2 steps 2 and 3
marks-chapter§8.11.2 step 4
marks-title§8.11.2 step 4
marks-malformed§8.11.3
marks-mismatch§8.11.3
marks-vtt§8.11.4
marks-hls§8.11.5
marks-dash§8.11.6

8.12 Marks in a package. A package whose policy has a marks table (§3.3) holds its lead’s marks; one whose policy has none holds nothing of them, so a policy written before the table was defined has the digest, and its package the bytes, it had. The table is part of the effective policy and so of the policy digest (§3.2): whether a package has marks, and which chapters, is signed with everything else the policy states. The reference is wai-pack’s marks module; its golden is pack-conformance/packs/video-marks-01/ and its planted faults pack-conformance/marks-mutations/.

What the packager writes. The marks are §8.11.2’s, derived from the lead’s stage index, the effective policy’s JSON form and the chapters the table declares (marks.lang, and each marks.chapter’s title and from_ms, in document order). A packager refuses with the derivation’s reason a policy whose marks do not derive (a chapter at or past the lead’s end, two chapters in one piece: marks-chapter; two pieces that begin in one millisecond: marks-index), and marks-hls one of whose dates falls past the year 9999. It then writes:

The MoQ catalog and the stage views carry nothing of the marks: a sink that reads those takes the document. The presentation wai-pack protect writes (§7.10) and Mode F’s tracks (http-delivery §11.7) are written from a package’s floors and objects and carry no date range or event stream in this revision. A live packer (§9.2) refuses a policy with a marks table (marks-index): a window’s index is not the lead’s whole index (§8.11.2 step 1).

What a verifier checks (§11.3 step 6, after the floors and before the catalogs), in this order, refusing at the first that fails:

  1. The policy has no marks table: the receipt has no marks, the package has no marks-<lead>.json and no marks-<lead>.vtt, no media playlist of the lead holds an #EXT-X-DATERANGE line, and no MPD holds the scheme tag:transaction.science,2026:wai/marks; otherwise marks-unexpected. Nothing more is checked.
  2. The policy has one: the receipt has marks and both files are present (marks-missing).
  3. The document reads (§8.11.3: marks-malformed), and its component is the lead and its index the digest of the lead’s index file (marks-package).
  4. The marks derive from the lead’s index, policy.json and the table’s chapters (the derivation’s reason, or marks-hls for a date past the year 9999), and their document is the file, byte for byte (marks-mismatch). The chapters are the policy’s, not the file’s own: a document with other titles, a chapter anywhere but at its piece’s start, a shot left out or a candidate with another reason is refused here.
  5. The receipt’s marks is the document’s digest (marks-receipt).
  6. marks-<lead>.vtt is §8.11.4’s file of the document’s chapters (marks-vtt).
  7. In each media playlist of the lead, the lines that begin #EXT-X-PROGRAM-DATE-TIME: or #EXT-X-DATERANGE:, in order, are §8.11.5’s (marks-hls); in each MPD, the lines that begin with four spaces and <EventStream or six and <Event , and its lines </EventStream>, in order, are §8.11.6’s (marks-dash). Where those lines sit is then the catalogs’ comparison (catalog-mismatch), each catalog rendered with them in the places above.
reasonraised at
policy-marks§3.4 item 14
marks-unexpectedcheck 1
marks-missingcheck 2
marks-packagecheck 3
marks-receiptcheck 5

8.13 Statements in a package. A package whose policy has a statements table (§3.3) carries beside each of its playlists, multivariant playlists and MPDs a packager’s statement of it (http-delivery §8.10), <file>.sig: the statement and one line feed. One whose policy has none carries none, so a policy written before the table was defined has the digest, and its package the bytes, it had. The table is part of the effective policy and so of the policy digest (§3.2): that a package carries statements is signed by its receipt, so a verifier or a player that holds the receipt requires every statement the table implies, and a path that strips them is found. The reference is wai-pack’s statements module over wai-rs’s manifest_statement::package; the golden is pack-conformance/packs/video-statements-01/, its planted faults pack-conformance/statement-mutations/, and its protected presentation pack-conformance/protected/video-statements-01/.

What a statement states, of the file at path p (under the package’s directory, or a protected presentation’s) holding the bytes b: name p; sha256 the SHA-256 of b; package the package receipt’s hash (§9.1); issued_ms, expires_ms, position, skip and delegation null; and it is signed by the package receipt’s signer. Its form is protection-index for a protected presentation’s protection.json, dash-mpd for an MPD, and for a playlist hls-multivariant when one of its lines begins #EXT-X-STREAM-INF:, else hls-media. Its tracks, each with alias null, in the order the file lists them, are:

A track’s first and last are null where the file lists no segment of it. So a multivariant playlist’s statement names its variants under the package’s key, and a client compares them with the package’s floors. §3.4 item 15 makes every such track one a statement can name.

What the packager writes. A statement beside each of the package’s HLS and DASH files (§8.3), signed with the package’s key; each is a file of the package and a line of LISTING. It is written after the receipt is sealed, since it names the receipt, as LISTING and a measurement claim are, so a metered figure (§9.4) does not cover it. The MoQ catalog and the stage views are not stated: http-delivery §8.10 has no form for them. A live packer’s windows hold no playlist or MPD, so the table asks nothing of them. wai-pack protect (§7.10), given the package’s key, writes a statement beside each of the presentation’s playlists and MPDs and beside protection.json.

What a verifier checks (§11.3 step 5, once the receipt verifies; and, in a protected presentation, after its LISTING). With the table, each file named *.m3u8 or *.mpd, and in a protected presentation protection.json, in byte order of its path, is held to the statement beside it, refusing at the first that fails: http-delivery §8.10’s five steps, the statement required (statement_not_served), read as a packager’s statement in its one canonical form and a line feed (statement_malformed), of this file (statement_other_resource), of these bytes (statement_mismatch), and signed by the package receipt’s signer (statement_signature); then

  1. its package is the package receipt’s hash (statement-package);
  2. its form and tracks are those this section states of the file’s path and bytes (statement-tracks).

Then, with the table or without it, a file whose name ends .sig that is not a statement those checks held is refused (statement-unexpected): a statement in a package whose policy has no table, or beside a file this section does not state. A statement that holds binds the bytes served to the package’s key and receipt. What those bytes list is then the catalogs’ comparison (§11.3 step 6), which a verifier holding the package makes; a player holds a playlist’s segments to floor.listing and a multivariant playlist’s variants to the package’s floors (player-integration §3).

reasonraised at
policy-statements§3.4 item 15
statement-packagecheck 1
statement-trackscheck 2
statement-unexpectedafter the checks
protect-statement-key§7.10, writing

8.14 DVB-DASH in a package. A package whose policy has a dvb_dash table (§3.3) holds dash/dvb.mpd, its classical MPD in the form of the DVB profile of MPEG-DASH (ETSI TS 103 285 V1.4.1), so that a player built to that profile, a broadcaster’s or an HbbTV terminal’s own, plays the floor from it. One whose policy has none holds neither it nor its copies, so a policy written before the table was defined has the digest, and its package the bytes, it had. The table is part of the effective policy and so of the policy digest (§3.2). What the form is and why is decision D56; the reference is wai-pack’s catalog::render_dash_dvb over wai-rs’s http_binding::dvb_dash; the golden is pack-conformance/packs/video-dvb-01/; the corpus of the form’s rules, with its planted faults and its verifier, is dvb-dash-conformance/.

The MPD is dash/classic.mpd (§8.3) but for these, in this order, indented the same way:

The copies. A SegmentTemplate names segments by number and cannot spell a file’s SHA-256, so the package holds floor/<f>/dvb/<k>.m4s for each floor rung f and k from 1: the floor’s k-th segment in piece order, byte for byte. A plain file server then serves the form; an origin MAY instead answer those names from the hash-named files by this rule, which is a function of the classical MPD alone. A copy is held to the listing by its position, as a client holds a hash-named segment to it by its name (player-integration §3).

The sidecars. The receipt, the policy, the indexes and floor.listing are where §11.1 puts them, at the package’s root, which the receipts descriptor names; a statement of the MPD (§8.13), dash/dvb.mpd.sig, is beside it and is not named inside it, since it covers the MPD’s bytes. A player that does not know a SupplementalProperty’s scheme ignores the descriptor, and one that does not know an EssentialProperty’s ignores its parent (TS 103 285 clause 10.10), which would hide the floor: so no descriptor of WAI’s in this form is an EssentialProperty, and nothing a DVB-DASH player reads names a statement.

What the packager refuses (pack-dvb-dash): floor rungs of two picture aspect ratios, which one adaptation set cannot hold; a piece but the last under 960 ms, or any over 15 s; and an MPD over 256 Kbytes (TS 103 285 clause 4.5). Its codec, AV1, is in no codec clause of TS 103 285: a player that does not decode it ignores the set (clause 10.1, NOTE 1), as for any codec it lacks.

What a verifier checks (§11.3 step 6): dash/dvb.mpd is the catalog this section gives, byte for byte (catalog-mismatch), and each copy is held and is its segment (dvb-dash-copy). The form’s rules are held by the corpus’s verifier, which shares no code with the reference.

reasonraised at
policy-dvb-dash§3.4 item 16
pack-dvb-dashwriting
dvb-dash-copy§11.3 step 6

9. Package receipt

9.1 A version-2 package receipt binds the asset id; the policy digest; its parent kind and parent receipt hash; per component its id, media class, index digest and stage root; the entry, distinct-object, referenced-byte and stored-byte counts, which a verifier recomputes from the indexes; the packager builtin:<crate>@<version>; the energy figure with its class, or the unmetered form; the grant; and the signer. Its signing payload, big-endian:

labelled_domain("wai:pack-receipt\x01", label)
u32 len ‖ asset_id ‖ policy_digest[32]
u8 parent_kind (0 none, 1 live-window, 2 re-pack) ‖ [parent_receipt_hash[32] if not 0]
u32 n ‖ n × ( u32 len ‖ id ‖ u8 media_class ‖ index_digest[32] ‖ stage_root[32] )
u64 total_entries ‖ u64 unique_objects ‖ u64 total_bytes ‖ u64 stored_bytes ‖ u64 work_units
u32 len ‖ packager
u64 joules_micro ‖ write_label(label)
grant_hash[32] ‖ u32 len ‖ capability ‖ u64 joule_ceiling_micro ‖ u8 0 | u8 1 ‖ u64 funds_ceiling
signer_pubkey[32] ‖ u32 len ‖ signer_id
[u8 1 ‖ marks[32] if the receipt names marks]

receipt_hash = BLAKE3("wai:video-package-receipt-id\x01" ‖ payload ‖ sig)

A receipt names marks when its package’s policy has a marks table (§8.12): marks is the digest of the lead’s marks document (§8.11.3), and the payload ends with one byte 1 and the digest. A receipt that names none ends at its signer’s id, so a receipt written before marks were defined keeps its bytes, its signature and its hash. The two payloads cannot be taken for one another: the signer’s id carries its length, and nothing follows it but this.

labelled_domain and write_label are energy-measurement §2.8’s: unmetered, the domain as written and no label; metered, its final byte \x02 and the class’s signed encoding. total_entries counts every entry; unique_objects the distinct content hashes over every component; total_bytes the entries’ byte_len; stored_bytes the byte_len of each content hash’s first entry; work_units equals total_entries. A receipt whose content hash comes at two lengths has no counts. The signature is Ed25519 by signer_pubkey, verified as SPEC §7 Signatures requires; a signature or a key that rule refuses is receipt-signature.

Its JSON form, package.receipt.json, is RFC 8785 canonical, hexadecimal lowercase, integers as numbers no greater than 2^53 − 1 (a packager refuses to seal anything else, receipt-json): asset_id; components, each {"id", "index": "index-<id>.wsix", "index_digest", "media_class", "stage_root"}; grant {"capability", "funds_ceiling", "grant_hash", "joule_ceiling_micro"} with null for no ceiling (a joule ceiling of 2^64 − 1); joules_micro; kind "package"; packager; parent_kind (none, live-window or re-pack); parent_receipt_hash (null for none); policy_digest; receipt_hash; sig; signer_id; signer_pubkey; stored_bytes; total_bytes; total_entries; unique_objects; version 2; work_units; energy_provenance, the class’s label (energy-measurement §2.7) or null when unmetered; and the class’s evidence members exactly as its class carries them. Every one of those members is present, with null where it has no value (parent_receipt_hash, energy_provenance, a grant’s ceilings) and nowhere else. One member more, marks, 64 lowercase hexadecimal digits, is present exactly when the receipt names marks, and is never null.

A reader refuses, before anything else, JSON that is not RFC 8259 or that names a member twice in one object, at any depth (receipt-json). It then reads "version": absent or 1 is a version-1 receipt (§9.3), 2 a version-2 one, anything else receipt-version. It then refuses (receipt-json) a member it does not define or one missing (marks may be absent), null where the member has a value, a value of another form (a number that is not an integer of its range, -0 among them; a string that is not Unicode scalar values), evidence other than exactly what the class carries, a class label energy-measurement §2.7 refuses, a parent kind and hash that disagree, an index other than index-<id>.wsix, and a receipt_hash that is not the receipt’s. A reader does not require the canonical form; a writer writes it, and a package’s package.receipt.json must be it (§11.3).

A verifier of a receipt against its indexes, in its components’ order, refuses, the first that applies: no component, or another number of indexes than components, a live window’s receipt included (receipt-index-count); an index that breaks §6 (its reason); one that does not hash to its index_digest (receipt-index-digest), has another stage root (receipt-stage-root), names another component or media class (receipt-component) or another policy digest (receipt-policy-digest); two components of one id (receipt-component); other counts, or a content hash at two lengths in any of the indexes (receipt-counts); a parent kind that does not match the indexes’ prev_index_digest, which is zero in every index for none and a re-pack and non-zero in every one for a live window (receipt-parent-kind); an energy figure and class that do not cohere, or a legacy label (receipt-energy); a signature that does not verify (receipt-signature).

9.2 Live windows chain by the parent link with parent kind live-window. A re-pack of an earlier package MAY name it with parent kind re-pack, and one that takes its bases and refinements from it MUST (§11.5).

The reference live packer (wai-pack live, wai_pack::live) takes a source a frame at a time, cuts it with the segmenter of §4.3, which decides a frame once lookahead_frames more have arrived, and codes each piece as it closes with the coder the on-demand packer uses (§5.4–§5.7). So a live and an on-demand pack of one source under one policy write the same plan and the same bases and refinements, byte for byte. It writes no floor: a supplied floor is made from the whole source (§7.3), so it is the on-demand package’s. Its pieces are gathered into windows: each a stage index of its pieces’ bases and refinements (first_piece and first_entry continuing the chain, prev_index_digest the previous window’s index digest, zero for the first) sealed by a version-2 receipt whose parent is the previous window’s (live-window; none for the first). A piece’s objects are published when the frame that decides its end has arrived: its base then, and layer k layer_lag_ms[k] after it (§8.6, §8.9).

It writes as it goes: each object when it is published, and each window’s index and then its receipt when the window is sealed, every file whole (written beside its place and renamed into it); the plan file and LISTING at the end of the source. It keeps no record per file of the session: each file’s listing line goes to a hidden file beside the package as the file is written, and is sorted into LISTING at the end (the one moment it holds a line per file); an object already written is known by its file; an object’s bytes are not held once written. What it holds is what no window has sealed yet: those pieces’ entry fields and the content hash of each object they published first. So with windows of n pieces that is at most n pieces’, and with one window over the whole source (the default) every piece’s until the end. Besides those it holds the frames of the piece being cut and each piece’s first frame, eight bytes a piece, for the plan file. A source that fails before its end (a short frame, a frame of another size) leaves every sealed window and the objects it lists, removes the objects only unsealed pieces published, writes no plan file or LISTING, and is an error that names what was kept.

9.3 A version-1 receipt (wai.video.package’s segment map) keeps its bytes and still verifies. Its JSON form is RFC 8785 canonical with version 1: asset_id, energy_provenance (null when unmetered) with its evidence, grant, joules_micro, kind "package", parent_receipt_hash, receipt_hash, segment_map_root, segments (each {"byte_len", "content_hash", "duration_ms", "index"}), sig, signer_id, signer_pubkey, stored_bytes, total_bytes, total_segments, unique_segments, work_segment_ops. A reader reads one without version as version 1, and, because version-1 receipts were issued before this form, keeps them verifying: energy_provenance may be absent (read as unmetered), an evidence member written null is absent, and an on-chip counter’s filtering, absent, reads as unknown. It refuses a member named twice as version 2’s reader does. Its signing payload, big-endian, is

"wai:video-package-receipt\x01"
u32 len ‖ asset_id ‖ segment_map_root[32]
u64 total_segments ‖ u64 unique_segments ‖ u64 total_bytes ‖ u64 stored_bytes ‖ u64 work_segment_ops
u64 joules_micro ‖ class (u8 0 when unmetered, else the class's signed encoding)
the grant as §9.1 writes it
u8 0 | u8 1 ‖ parent_receipt_hash[32]
u64 n ‖ n × BLAKE3("wai:video-package-seg\x01" ‖ u64 index ‖ content_hash[32] ‖ u64 duration_ms ‖ u64 byte_len)
signer_pubkey[32] ‖ u32 len ‖ signer_id

segment_map_root is the jwp-receipts §2.2 Merkle root over those segment leaves, the counts are the segments’ (the bytes of one copy per content hash for stored_bytes), and receipt_hash is version 2’s. Both versions share the identity domain, so the figure profile wai:video-package-receipt-id (energy-measurement §6.1) covers both; a version-2 receipt’s figure is read only with the indexes it verifies against.

9.4 Where the energy figure is not the unmetered form, the packager MUST publish a measurement claim about it (jwp-receipts §2.4.8, energy-measurement §6) as claims/<receipt_hash>.measurement.json: subject the receipt, figure profile wai:video-package-receipt-id, signed with the package key and checked against the receipt before it is written. A verifier refuses a metered package without it (measurement-claim-missing) or with one that does not verify and check against the receipt (measurement-claim-invalid). The figure brackets the whole pack: it starts once the policy’s own refusals (§3.4, §11.1) have passed, before the source’s first read, and covers both reads of the source, the supplied floor’s checks (§7.3, §7.7), a parent’s verification (§11.5), every piece’s coding and every file written, and stops before the receipt is sealed. A packager MUST NOT leave any of that work outside it. What names the sealed receipt is written after it and is not in the figure: the claim itself, LISTING, and the statements of §8.13.

10. Royalty-free packaging

A packager run with royalty_free_only = true MUST refuse to emit any object whose capability the registry does not classify as royalty-free, including one it does not classify at all (§3.6, rule 5). A package receipt says nothing about patents. royalty_free_only records the registry class the packager read, “licensed royalty-free by its standards body”; it is not a freedom-to-operate opinion.

11. Package, re-pack equivalence and verification

11.1 The package is a directory of these files and no other file or directory (every directory below it holds one of them):

policy.toml  policy.json  plan-&lt;c&gt;.txt  index-&lt;c&gt;.wsix  package.receipt.json
objects/&lt;content hash&gt;.wai …
floor/&lt;f&gt;/init.mp4  floor/&lt;f&gt;/&lt;sha256&gt;.m4s …   (per floor rung, of floor name f)
floor.listing                                 (a supplied CMAF floor's listing, §7.7)
hls/classic/…  hls/&lt;class&gt;/…  dash/…  moq/catalog.json  stages/&lt;class&gt;.json
claims/&lt;receipt_hash&gt;.measurement.json      (a metered package)
marks-&lt;lead&gt;.json  marks-&lt;lead&gt;.vtt          (a policy with a marks table, §8.12)
hls/…/&lt;file&gt;.sig  dash/&lt;file&gt;.sig            (a policy with a statements table, §8.13)
floor/&lt;f&gt;/dvb/&lt;k&gt;.m4s …                      (a policy with a dvb_dash table, §8.14)
LISTING

policy.toml is the policy as given; policy.json its effective canonical JSON (§3.2). LISTING is the listing (SPEC §3.1) of every other file, and its SHA-256 is the pack digest. A component or class id or a floor name (§7.1) that cannot be one path segment of a listing, two class ids or two floor names equal once A–Z are mapped to a–z, and a class named classic in any case (ids name paths, and paths fold case) are refused (pack-id).

11.2 Re-pack equivalence. Given the same source bytes, effective policy, supplied floor, signing key and packager identity, a conforming reference packager MUST produce byte-identical package files on every machine and whichever fabric runs each step (§11.6), except the energy figure and measurement claim of a metered package and floor objects §7.5 covers. A verifier MAY pack again and compare listings. A packager whose identity differs is not held to this. A re-pack from a parent (§11.5) is held to it with the parent among its inputs; and where the parent was packed by a conforming packager of the same identity, from the same source, under base and refinement rungs of the same capabilities and parameters, its files are those of a pack that codes every piece, but for its receipt’s parent and so LISTING.

11.3 Verification. A verifier of a package, without its sources, applies these checks in this order and refuses at the first that fails; a check that reads files a failed check names is not applied:

  1. Every file below the package’s directory, read strictly — a name beginning with . and a *.sha256 file included, as a listing (SPEC §3.1) would not skip them — is a regular file whose path a listing can carry; a symbolic link, a directory that cannot be read or holds nothing, or anything else is refused (unexpected-file).
  2. LISTING is present (listing-missing) and is the listing of every other file (listing-mismatch).
  3. package.receipt.json reads as a version-2 receipt (§9.1: receipt-json, receipt-version) and is that receipt’s canonical form, byte for byte (receipt-json).
  4. policy.json hashes to the receipt’s policy digest (policy-digest-mismatch); policy.toml is a policy (its own reason, §3.2–§3.4); policy.json is its effective canonical JSON, royalty_free_only forced on where policy.json says true (policy-json-mismatch; an override never turns it off, §3.2, so a policy.json that says false over a policy.toml that says true is refused here); its ids and floor names can name files (pack-id, §11.1); and under that effective policy it asks for nothing a packager may not emit (§3.6’s reason; under royalty_free_only, a patent-pool floor is license-refused).
  5. The receipt’s components are the policy’s, by id, in order, each of the policy’s media (policy-component-mismatch), so a package is never a receipt over fewer components than its policy has; each index is present (index-missing); the receipt verifies against them (§9.1, the index rules of §6 among its reasons); then the statements, by §8.13’s checks in their order (statement_not_served, statement_malformed, statement_other_resource, statement_mismatch, statement_signature, statement-package, statement-tracks, statement-unexpected).
  6. Per component, in order (a component beside the lead as the item after this list says):
    • the index against the policy (index-policy-mismatch): its timebase and media class are the component’s; its string table is the component’s id, the class ids in policy order, then each capability at its first use; its class records are the policy’s classes, in order, with their max_layer, floor-only flag and max_kbps; every piece’s kind is one a video piece has, none exactly in a floor-only component (§7.8); its rules and moments are §8.1’s from its kind and span; its entries are the rungs of its ladder (§5.7), by role, capability and layer: its base, each refinement in layer order, each floor (a floor-only component’s, its floors alone); each refinement’s deadline is its plan’s (§8.6); and its class plan is §8.1’s from its entries;
    • the plan file holds the index’s starts and its digest is the header’s (plan-mismatch);
    • every entry’s object is present (object-missing) and keeps the object rules of §6.3, its counted bytes included;
    • before any frame is decoded, each piece’s floor objects, where the policy has floor rungs and an object’s payload reads as §7.2’s framing, hold one unit per frame of their piece (floor-object-mismatch);
    • every base and refinement, in entry order: a base decodes by its capability (a still is a PNG §5.7 reads, its one frame held for the piece), or it is prefix-digest; its manifest is the one §5.2 states for its piece’s frames and its frame size (object-manifest); a refinement’s manifest is §5.4’s (object-manifest), and it applies in layer order, to every frame of the stage below it, by a staged-delivery §4 sink under the reference tools’ cost limit and chain budget, or it is prefix-digest; a base that does not decode, or a layer that does not apply, ends the check;
    • then every decode-equivalent prefix decodes to its prefix_digest (prefix-digest);
    • when all of that holds, the floors, floor rung by floor rung in ladder order: per piece, one floor object per floor rung, the rung’s the k-th floor entry for the k-th floor rung, of the rung’s capability, whose renderer, settings, framing header, unit count and target are §7.2’s (floor-object-mismatch) and which presents the piece’s base (floor-motif-mismatch), or, in a floor-only component, which carries no derived_from (floor-object-mismatch); its units each one temporal unit holding a frame, the first of the rung’s first piece carrying a sequence header, which, for the component’s first floor rung, is within the video floor as §7.1 holds the floor rendition (floor-av1-profile, floor-av1-level), checked before the header is read further; every sequence header the first’s bytes, every shown key frame carrying one (floor-av1-malformed); each piece opening on a shown key frame, and no other unit being one unless the policy allows extra key frames (floor-keyframe-mismatch); and the rung’s CMAF files are the ones its temporal units give (§7.6, floor-cmaf-mismatch); or, for a supplied floor in the CMAF form (§7.7), floor.listing is a listing that hashes to floor.supplied_listing, floor/<f>/init.mp4 is the file of its line <f>/init.mp4 and reads as §7.7’s initialization segment, its av1C and sample entry size the floor’s sequence header’s, and the listing’s media segments of <f>/, in its order, are one per piece, piece k’s floor/<f>/<its SHA-256>.m4s, reading as §7.7’s media segment, opening on a sync sample, each sample one frame and the first at the piece’s start, its samples the piece’s floor units without their temporal delimiters, each a sync sample exactly when its unit starts at a shown key frame (floor-cmaf-mismatch); and, with a dvb_dash table, each numbered copy floor/<f>/dvb/<k>.m4s is held and is the rung’s k-th segment, byte for byte (§8.14, dvb-dash-copy);
    • then the lead’s marks, by §8.12’s checks in their order (marks-unexpected, marks-missing, marks-malformed, marks-package, the derivation’s reason, marks-mismatch, marks-receipt, marks-vtt, marks-hls, marks-dash);
    • then the catalogs are the ones the index, the objects and the floor give, with the marks’ lines where §8.12 places them, and an index no catalog can be written for (a start past 2^53 − 1) is refused (catalog-mismatch). A component beside the lead (§5.8) is held, in this order: to the index against the policy as above, its pieces of kind audio for an audio component and none otherwise, with no length bound of its own, and to the lead’s pieces converted (§4.5), each flagged aligned to the lead and nothing else (index-policy-mismatch); to its plan file (plan-mismatch); to every entry’s object and the object rules of §6.3 (object-missing and §6.3’s reasons); then piece by piece, to its base, which decodes by its capability or is prefix-digest, is the envelope §5.8 writes of its own payload with its piece’s target (object-manifest), holds its piece’s span (a FLAC’s samples, a score’s duration, a document’s window as object-manifest, a segment’s X-TIMESTAMP-MAP time, a feed’s ticks and last tick), and decodes to its entry’s prefix_digest (prefix-digest): a segment’s, to the cues it reads to written again as the segment of its piece’s span, which is its payload byte for byte; a feed’s, to a replay of its sealed ops whose seal verifies under the receipt’s key and whose graph hashes to the state the piece before it seals; and a score’s floor, which is §5.8’s envelope for its piece’s base and decodes to that base’s mixdown, sample for sample, in its channels and at its rate (floor-object-mismatch); a graphics piece’s document digest, which is the pieces’ before it (graphics-document-mismatch, graphics-vector §6.2), and its text floor, which is the object graphics-vector §6.3 writes of its base’s transcript, byte for byte (floor-object-mismatch); and a sound track’s Opus floor (§7.9): once its first base holds, floor.listing hashes to floor.supplied_listing, the files of its <f>/ are the package’s floor/<f>/init.mp4 and floor/<f>/<SHA-256>.m4s, and they hold §7.9’s checks against the pieces (floor-cmaf-mismatch, naming §7.9’s reason); then piece by piece, each base’s samples are its entry’s digest, and its floor object names that base (floor-motif-mismatch), is §7.9’s envelope for it, and carries the Ogg Opus stream §7.9 writes of the floor’s packets for the piece (floor-object-mismatch). Its stage view members are then §8.8’s. Once every component holds, each graphics component’s caption intervals, from its pieces, are held to the cues of a text component of their language, from its segments (caption_without_track, graphics-vector §6.4).
  7. A metered package’s measurement claim (§9.4: measurement-claim-missing, measurement-claim-invalid, a fabric record naming a step this extension does not define, §11.6, among the latter); an object file no entry names (object-unlisted); a catalog file past the ones given (catalog-unexpected); any other file (unexpected-file).

11.4 What this revision packs. The reference packager packs a video component as its lead, and beside it up to seven components of the media and capabilities of §5.8, each from its source, one object a piece, a score’s encoded FLAC floor, and a sound track’s Opus floor supplied in the CMAF form (§7.9). The lead is packed as one video component: bases of wai.video.int_motion (§5.5, §5.6) and stills of wai.image.png (§5.7); refinements of wai.video.int_refine over an int_motion base (§5.4); each piece by its ladder, a rule’s ladder subset included (§5.7); and a supplied AV1 floor per floor rung, one or several (§7.1), in the OBU form (§7.3) or the CMAF form (§7.7); and floor-only components (§7.8). Its live packer packs the same pieces in windows, without a floor (§9.2). It runs base-search, sse and object-hash on a GPU that qualifies, where it is built to and asked to, and every other step on the CPU (§11.6). It refuses by name, before writing anything, what it does not pack yet: another medium (pack-media-unsupported), a floor to encode (pack-floor-encode-unsupported), and a rung of a capability it does not encode, or a refinement over a base it does not refine (pack-capability-unsupported:<cap>, every one naming the capability of the rung it does not pack: beside the lead, a refinement, a second base, or a floor its base does not take); a lead other than video, a medium beside the lead that §5.8 does not name, and a re-pack of several components (pack-media-unsupported); a source that does not span the lead’s timeline, is not of the component’s rate or does not read (pack-source-length, pack-source-rate, pack-source-format), a note across a piece start (pack-score-crossing), caption blocks in a graphics document that no text component carries (caption_without_track, graphics-vector §6.4), a text component with no lang (pack-text-lang), and a feed op that depends on another piece’s (pack-feed-crossing), §5.8; and a component of the standard profile with no source (pack-source-missing; a floor-only component has none, §7.8) or whose source’s frame rate is not its timebase (pack-source-rate). Where its own work fails it refuses by name too, and leaves nothing written (a file it wrote before the refusal is removed): an encode the encoder refuses, such as a piece past the samples it holds, or an object it wrote that does not count (pack-encode); a piece past 2^32 − 1 units (pack-piece-length); catalogs it cannot write, the case §11.3 refuses as catalog-mismatch (pack-catalog); a meter that was asked for and gives no figure, a span too short to reach a microjoule or a figure past 2^64 − 1 (pack-meter, never an unmetered package in its place); a measurement claim it cannot write (pack-measurement-claim); a listing it cannot write (pack-listing); and a source that does not read the same twice (pack-source-changed) or a supplied floor stream that does not (floor-listing-mismatch): it reads each a frame or a temporal unit at a time, once to cut and check it and once to code it, and holds neither whole. What it holds is not a function of the source’s length but for a record per piece and per entry, a byte per frame of each floor (whether it starts at a key frame) and a record per packet of an Opus floor (its segment, place, decode time and samples): the frames of the pieces it codes at once, which it chooses, each piece coded alone (§4.1), so the package is the same for any number; one piece of each floor; and, for the catalogs, one object at a time, read back from where it wrote it; except a re-pack, which holds its parent whole while it verifies it (§11.5), so its memory grows with the parent. Each file is written whole as it is made. It writes a package only into a directory that is new or holds nothing, and refuses any other before it packs (pack-out-not-empty): written among another package’s files, it would leave behind the ones it does not write.

11.5 Re-pack from a parent. A packager MAY take every piece’s base and refinements from an earlier package, its parent, instead of coding them, so a component’s integer base is coded once and taken by every package made from the same source after it. It reads the source as for a pack that codes (§4: its pieces and their kinds, §4.3, §4.6, are this source’s), and MUST refuse, checks 1 and 2 before it writes anything and check 3 as it reads each piece, leaving nothing written (§11.4):

  1. a parent that does not verify as §11.3 verifies a package (repack-parent-invalid, with the reason the parent fails for);
  2. a parent with no component of the component’s id, media and timebase, or whose pieces are not this source’s, piece by piece: another number of pieces, or a piece of another start, duration, flags or kind, or whose ladder (§5.7, under the parent’s policy) codes its base and refinements in rungs of other capabilities or other parameters (§3.3) than this policy’s ladder for the piece, in layer order, or with another deadline (§8.6) (repack-parent-mismatch);
  3. a piece whose prefixes, each the parent’s base decoded and its refinements applied in layer order by a staged-delivery §4 sink, do not reconstruct this source with the sse the parent’s entry states (§6.6) (repack-source-mismatch): a parent packed from another source.

It then writes, for each piece, the parent’s base and refinement objects and their entries’ fields (§6.2: content hash, lengths, capability, role, layer, tier, prefix digest, sse, deadline) in place of coding them, and the floors as it would; its receipt names the parent’s receipt with parent kind re-pack (§9.1). A verifier holding the parent MAY check a re-pack against it: the parent verifies (repack-parent-invalid), the re-pack’s receipt names the parent’s receipt with parent kind re-pack, and each component’s pieces are the parent’s, piece by piece, of the same start, duration, kind and flags, whose base and refinement entries are the parent’s entries, every field the same but the entry’s place in its index: the entry it refines counted from its piece’s first entry, and the duplicate flag, which an index’s own entries decide (repack-parent-mismatch). A re-pack holds no source of its parent’s bases; what binds a reused base to this source is the third check, and what binds it to the parent is the receipt.

11.6 Steps and fabrics. The steps of a pack and a verify are named by these terms:

StepWhat it computes
source-decodea source’s frames from a coded rendition, before a pack, by a producer’s tool (§4.7 reads the result)
source-reada source’s frames or samples (§4.7)
segmentthe pieces (§4.3) and their kinds (§4.6)
base-searchthe integer base’s motion search (§5.5)
base-codethe rest of the integer base (§5.5), its step (§5.6) and a still (§5.7)
refine-coderefinement layers, each applied by the reference sink (§5.4)
ssean entry’s sse (§6.6)
object-hashan object’s content hash (§5.3)
digestthe SHA-256 digests: decoded-output digests (§6.5), listings, file names
floor-reada supplied floor’s units, CMAF boxes and listing (§7.3, §7.7)
floor-decodea floor’s units decoded to pictures (item 4 below)
indexthe stage indexes, catalogs, receipt and claim (§6, §8, §9)
ioreading and writing files

Every step is a function of its inputs, defined in integers (and floor-decode by the AV1 decoding process), and none names a fabric:

  1. Packages. A packager MAY run a step on any fabric whose output is the step’s for every input. It MUST NOT run a step on a backend that has not, on each device it opens and before it is used, given the step’s output for every input of a qualifying set, nor at all on one that computes otherwise than the step’s definition, such as in floating point where the step is defined in integers, or with a learned approximation. A backend that cannot take a call (a size, a lost device) leaves it to the CPU, with the same output. So §11.2 holds across fabrics: the package files are the same bytes whichever fabric ran each step, and only a metered package’s measurement claim says where they ran. A qualifying set holds, at the least:
    • for the motion search (§5.5), frame pairs of noise, of a smooth field, of a flat field (every candidate ties in cost) and of alternating columns (two vectors tie in cost and in length), at sizes that are not a whole number of blocks, at three block sizes or more, at full-pel and half-pel precision, and with search ranges that reach the frame’s edge;
    • for SSE, inputs of at least three lengths, among them one byte and one past 4096 bytes;
    • for BLAKE3, messages of 0 and 1 bytes, of 1023, 1024 and 1025 bytes around a chunk, and longer than four chunks. (Informative: the reference packager’s set is wai_pack::fabric::qualifying_frames and the inputs of wai_pack::fabric::qualify. Its GPU kernels are held to integer types, with no subgroup operation and no extension, on the IR of the translator that compiles them, wai-pack-gpu/tests/integer_only.rs.)
  2. Verdicts. A verifier’s verdict and its reason are functions of the package (§11.3). A verifier MAY compute a check on another fabric where the package states the value the check compares with, such as an object’s content hash: a match shows that the fabric computed the check, and a mismatch MUST be computed again on the reference implementation before it is reported. Every other check runs on a backend held to the reference’s output as item 1 holds a packager’s. A verdict that changes with the fabric is the verifier’s fault, never the package’s.
  3. Hardware encoders. A hardware encoder’s output depends on the chip, its firmware and driver, and its settings, and no rule fixes it, so no step of this extension is computed by one. A floor a packager is given is an input, whatever made it (§7.3, §7.7); a floor a packager encodes on such hardware is one §7.5 covers, whose derived_from.renderer names what §7.5 requires, and §11.2 does not hold its bytes.
  4. Decoding a floor. The AV1 decoding process fixes every output sample of a stream whose sequence header has film_grain_params_present 0, and of every stream before film grain is applied (AV1 §7.2), so conformant decoders on any fabric give the same pictures of a floor’s units. A floor states no decoded-output digest (§7.4), so a decode of it has no stated value to compare with: a verifier that decodes floors MUST NOT refuse a package for what a decoder other than the reference gives, nor read a hardware decoder’s error or concealment as a verdict. A hardware decoder’s frames may be held in a wider format than the stream’s bit depth (a 10-bit sample shifted into the high bits of a 16-bit word); a reader of them recovers each sample exactly or refuses the frame, and never rounds. (Informative: wai-pack floor-decode reports each piece’s digest from either decoder, for comparing them.)
  5. The fabric record. A metered package’s measurement claim MAY carry a fabric record (jwp-receipts §2.4.8, energy-measurement §6.3): each step the metered span ran, with every fabric that ran it and its backend, and the figure’s share on each. Its steps are this section’s terms (§11.3 step 7). An unmetered package carries no claim, so no record: which fabric ran a step is not part of the package, and two packages that differ only in where their steps ran are the same files.

12. Loudness

12.1 A packager states the loudness of each audio component’s programme and the gain that brings it to a target, in a signed loudness statement. The measurement is ITU-R BS.1770’s integrated loudness (K-weighting, 400 ms blocks every 100 ms, an absolute gate at −70 LKFS and a relative gate 10 LU below), computed in integers by the procedure below, so that every implementation states the same figure to the thousandth of an LU and a verifier holding the programme recomputes it exactly. The figures are thousandths: an LU, a dB, LKFS. The statement says how far a sink scales its output; the packaged audio is the source’s, unchanged. The reference is the loudness module of wai-rs (feature loudness) and wai-pack loudness; the corpus is pack-conformance/loudness/, and its second implementation pack-conformance/loudness_verify.py.

12.2 Measurement. The programme is 16-bit samples, interleaved, at 48 000 Hz; another rate, or a layout other than one channel, two (L, R) or six (L, R, C, LFE, Ls, Rs, WAV’s order), is not measured in this revision (loudness-unsupported). The channels’ weights, in hundredths, are 100 for L, R, C and a single channel, 141 for Ls and Rs (BS.1770’s 1.41), and 0 for the LFE, which is excluded.

  1. Filter. Each channel’s samples, each multiplied by 2^8, pass through two second-order sections in turn, each computing, from its inputs x and its outputs y, yₙ = ⌊(b0·xₙ + b1·xₙ₋₁ + b2·xₙ₋₂ − a1·yₙ₋₁ − a2·yₙ₋₂ + 2^29) / 2^30⌋, from a state of zeros, with BS.1770’s coefficients at 48 kHz in units of 2^−30, each rounded to the nearest integer: the first section (b0, b1, b2, a1, a2) = (1 648 327 767, −2 890 186 776, 1 286 764 483, −1 815 331 593, 786 495 243), the second (1 073 741 824, −2 147 483 648, 1 073 741 824, −2 136 797 184, 1 063 081 984). Every product and sum is exact (they fit in 128 bits).
  2. Blocks. A programme of F frames has ⌊(F − 19 200) / 4 800⌋ + 1 blocks when F ≥ 19 200 and none otherwise; block j is frames 4 800·j to 4 800·j + 19 199. Its power Pⱼ is the sum over the channels of the weight times the sum of the squares of the filtered samples in it, an exact integer.
  3. Loudness. The loudness of a power P summed over n blocks is −691 + M(P, 100 × 19 200 × 2^46 × n), M of §12.3: BS.1770’s −0.691 + 10 log10 of the mean weighted square, with 2^46 undoing the 2^8 and the 16-bit full scale (2^15 squared). A block’s loudness is that of Pⱼ over one block; a block of power 0 has none.
  4. Gates. The blocks whose loudness is above −70 000 pass the absolute gate. With none, the programme has no integrated loudness (null). Otherwise the relative gate is the loudness of their powers’ sum over their count, less 10 000; the blocks of those whose loudness is above it are the gated blocks, and the integrated loudness is the loudness of their powers’ sum over their count.
  5. Peak. The peak is the greatest absolute sample, 0 to 32 768, over every channel; its figure is 2 × M(peak, 32 768), in thousandths of a dB below full scale, null for a peak of 0. It is the sample peak: BS.1770’s true peak, from oversampling, is not measured here.

12.3 The logarithm. For integers p, d ≥ 1, M(p, d) = ⌊10 000 × (L(p) − L(d)) × 1 292 913 986 / 2^64⌋, where 1 292 913 986 is log10(2) in units of 2^−32, rounded, and L(x) is log2(x) in units of 2^−32 by this procedure: n is the index of x’s highest set bit; m is x shifted to put that bit at bit 62 (left by 62 − n, or right by n − 62, dropping the bits shifted out); r = n × 2^32; then, for k from 31 down to 0, m becomes ⌊m² / 2^62⌋, and where m ≥ 2^63, m is halved (dropping its low bit) and 2^k added to r; L(x) is r. Every step is an exact integer operation (m² fits in 128 bits), and ⌊·⌋ rounds toward negative infinity. A 1 kHz full-scale tone on one channel measures −3 004, BS.1770’s calibration of −3.01 LKFS.

12.4 The gain. Given a target T (thousandths of an LKFS) and a peak ceiling C (thousandths of a dB of full scale), the gain is T minus the integrated loudness, unless that would put the peak above C: where the peak’s figure is not null and the gain exceeds C minus it, the gain is C minus the peak’s figure and the statement says the ceiling limited it. A programme with no integrated loudness has no gain. A sink that normalises scales its output by 10^(gain / 20 000).

12.5 The statement is RFC 8785 canonical JSON and a line feed, with exactly these members: kind "wai.pack.loudness"; version 1; method "bs1770-int/1"; component, the component’s id (1 to 255 of the ASCII letters, digits, ., _ and -); source {"channels", "frames", "rate", "sha256"}, the programme’s layout, frame count, rate and the SHA-256 of its samples as 16-bit little-endian, interleaved; blocks, gated and peak (integers); integrated_milli, peak_milli and gain_milli (signed integers, or null as §12.2 and §12.4 say); target_milli and ceiling_milli (signed integers); limited ("yes" or "no"); receipt_hash, the package receipt it belongs to (§9.1, lowercase hexadecimal), or null; signer_pubkey; and sig. A signed integer is a JSON string: 0, or an optional -, a digit 1 to 9 and any digits, less than 2^53 in magnitude; every other integer is a JSON number below 2^53. The signature is Ed25519 by signer_pubkey, the package’s key, over wai:pack-loudness\x01 followed by the statement’s canonical form without sig. A package with an audio component carries its statement as loudness/<component>.json; this revision’s packer packs audio components (§5.8) without one, and wai-pack loudness writes and verifies a component’s statement from its WAV source (§4.7).

12.6 Verification. A verifier refuses, in order, the first that applies:

  1. text that is not JSON, names a member twice, or has another member, a member missing or of another form (loudness-malformed);
  2. a signature that does not verify as SPEC §7 Signatures requires (loudness-signature);
  3. a method, rate or layout this revision does not measure (loudness-unsupported);
  4. figures that do not follow from each other: blocks other than §12.2’s count for frames; gated above blocks; gated 0 and an integrated loudness, or the reverse; a peak figure that is not peak’s; a gain or limited that is not §12.4’s from the integrated loudness, the peak figure, the target and the ceiling (loudness-gain). This needs no samples;
  5. with the receipt hash it expects, another (loudness-receipt-mismatch);
  6. with the programme’s samples: another SHA-256, rate or layout (loudness-source-mismatch), then any figure of §12.2 that its recomputation does not give (loudness-not-recomputed).

12.7 Reasons.

reasonraised at
loudness-malformed§12.6 step 1
loudness-signature§12.6 step 2
loudness-unsupported§12.2, §12.6 step 3
loudness-gain§12.6 step 4
loudness-receipt-mismatch§12.6 step 5
loudness-source-mismatch§12.6 step 6
loudness-not-recomputed§12.6 step 6

12.8 Conformance. pack-conformance/loudness/ holds vectors.json: programmes as formulas of integers (sine_1k, one period of a 1 kHz tone at 48 kHz, 32 767 × sin(2πk/48) rounded to an integer as the table below fixes it, scaled and floored; a linear congruential generator’s noise; silence) with their measurements, among them BS.1770’s calibration tone, a programme the relative gate cuts in half, one under the absolute gate, 5.1 with the surround weight and with the LFE alone, and one shorter than a block; §12.3’s L of chosen integers, from 1 to 2^128 − 1; and signed statements, each verified or refused with a reason of §12.7, every reason given. wai_loudness_vectors verify and wai-rs/tests/loudness.rs require every outcome and the file byte for byte, and §12.2’s coefficients and the table below as this text states them; the generator holds each programme to the loudness BS.1770’s calibration or its construction declares. loudness_verify.py, written from this text and sharing no code with wai-rs, does the same, mutates copies of the corpus and requires every copy to fail, and drops each of its own checks in turn and requires the corpus to catch the weakened verifier.

The tone’s period, which the corpus holds as sine_1k:

sine_1k = [0, 4277, 8481, 12539, 16383, 19947, 23170, 25996, 28377, 30273, 31650, 32487,
   32767, 32487, 31650, 30273, 28377, 25996, 23170, 19947, 16383, 12539, 8481, 4277, 0,
   -4277, -8481, -12539, -16383, -19947, -23170, -25996, -28377, -30273, -31650, -32487,
   -32767, -32487, -31650, -30273, -28377, -25996, -23170, -19947, -16384, -12539, -8481,
   -4277]

13. Reasons

ReasonWhere
policy-toml (not UTF-8, a byte-order mark, not TOML 1.0, nesting past 32), policy-float, policy-datetime, policy-negative, policy-integer-range§3.2
policy-unknown-key, policy-missing-key, policy-type, policy-value, policy-schema§3.3
policy-too-many-components, policy-too-many-rungs, policy-too-many-classes, policy-too-many-rules, policy-too-many-moments, policy-duplicate-id, policy-lead, policy-open-gop, policy-piece-bounds, policy-ladder-component, policy-refines-role, policy-dangling-refines, policy-refines-cycle, policy-refines-base, policy-refines-branch, policy-no-base, policy-floor-only, policy-class-layer, policy-rule-ladder, policy-layer-lag, policy-too-many-deadlines, policy-deadline-unreachable, policy-protection, policy-marks, policy-statements, policy-dvb-dash§3.4
capability-unknown, capability-not-payload, capability-candidate, capability-role, license-unclassified, license-refused, each naming the capability (and the licence label)§3.6
y4m-header, y4m-interlaced, y4m-colorspace, y4m-size-change, y4m-truncated, wav-format, wav-truncated§4.7
index-magic, index-version, index-truncated, index-reserved, index-media-class, index-role, index-tier, index-string, index-trailing§6.2
index-timebase, window-start, piece-order, piece-entries, piece-kind, entry-flags, entry-order, layer-role, refines-outside-window, refines-mismatch, layer-gap, tier-entropy-passthrough, tier-weakest, tier-mismatch, digest-without-tier, deadline-role, floor-derived-from, dup-length, dup-flag, class-plan, class-flags§6.3, §6.4
object-length, object-hash, non-envelope-entry, object-lengths, object-capability, object-refines, object-derived-from§6.3
floor-missing, floor-listing-mismatch, floor-stream-missing, floor-av1-malformed, floor-frame-count, floor-keyframe-mismatch, floor-av1-profile, floor-av1-level§7.1, §7.3
floor-stream-missing, floor-cmaf-malformed, floor-cmaf-timing, floor-cmaf-pieces§7.7
floor-stream-missing, floor-cmaf-malformed, floor-cmaf-timing, floor-opus-malformed, floor-frame-count, floor-cmaf-pieces, pack-capability-unsupported:wai.audio.opus§7.9
floor-keyframe-mismatch, floor-piece-length§7.8
protect-package, protection-absent, protection-cpix-mismatch, protection-cpix, protection-kid-unknown, protection-scheme-mismatch, protect-av1, protect-listing, protect-statement-key§7.10, writing
protected-listing, protected-index, protected-missing, protected-unexpected, protected-box, protected-scheme, protected-iv-size, protected-kid, protected-pssh, protected-subsample, protected-iv, protected-name, protected-mismatch, protected-catalog§7.10, checking
floor-object-mismatch, floor-motif-mismatch, floor-av1-profile, floor-av1-level, floor-av1-malformed, floor-keyframe-mismatch, floor-cmaf-mismatch, against a package§11.3
receipt-json, receipt-version, receipt-index-count, receipt-index-digest, receipt-stage-root, receipt-component, receipt-policy-digest, receipt-counts, receipt-parent-kind, receipt-energy, receipt-signature§9.1
measurement-claim-missing, measurement-claim-invalid§9.4
unexpected-file, listing-missing, listing-mismatch, policy-digest-mismatch, policy-json-mismatch, policy-component-mismatch, index-missing, index-policy-mismatch, plan-mismatch, object-missing, object-manifest, prefix-digest, catalog-mismatch, catalog-unexpected, object-unlisted§11.3
pack-id, pack-source-missing, pack-source-rate, pack-media-unsupported, pack-floor-encode-unsupported, pack-capability-unsupported (naming the capability)§11.1, §11.4
pack-encode, pack-piece-length, pack-catalog, pack-meter, pack-measurement-claim, pack-listing, pack-source-changed, pack-out-not-empty§11.4
pack-source-length, pack-source-format, pack-score-crossing, pack-text-lang, pack-feed-crossing§5.8, §11.4
caption_without_track, writing and against a package; graphics-document-mismatch, against a package§5.8, §11.3, graphics-vector §6
repack-parent-invalid, repack-parent-mismatch, repack-source-mismatch§11.5
loudness-malformed, loudness-signature, loudness-unsupported, loudness-gain, loudness-receipt-mismatch, loudness-source-mismatch, loudness-not-recomputed§12
rendition-malformed, rendition-capability, rendition-codecs, rendition-duplicate, rendition-default, rendition-description-unbound, rendition-description-invalid, rendition-description-language, rendition-ambiguous, rendition-groups-differ§8.10
marks-index, marks-policy, marks-moment, marks-chapter, marks-title, marks-malformed, marks-mismatch, marks-vtt, marks-hls, marks-dash§8.11
marks-unexpected, marks-missing, marks-package, marks-receipt, and §8.11’s reasons against a package§8.12, §11.3
statement_not_served, statement_malformed, statement_other_resource, statement_mismatch, statement_signature (http-delivery §8.10), statement-package, statement-tracks, statement-unexpected, against a package and a protected presentation§8.13, §11.3, §7.10
pack-dvb-dash, writing; dvb-dash-copy, against a package§8.14, §11.3