WAI Extension: Platform and Hardware Encoding
Mirrored from the canonical text at commit 260cbd09 ().
Status: Draft. Registers the record
wai.video.encode. A sink or a packager that needs an AVC (ISO/IEC 14496-10) or HEVC (ISO/IEC 23008-2) rendition encodes it with an encoder the device already has: the operating system’s media framework and its hardware media engine, a GPU’s encode engine through the video acceleration API or through the GPU driver’s own encode interface, or a software AVC library the user obtains separately and names at run time. WAI writes no AVC or HEVC encoder. This extension fixes the shape the encoder’s output is held to (§4), what names the encoder (§5), and the encode record, a signed statement of who made an elementary stream, from what source, with which settings and at what measured cost, and, where its signer decoded the stream, the quality of the pictures it decodes to (§6 to §8). A verifier holds the stream to its digest and its shape; it never re-encodes. Decoding the stream itself and holding the source, it recomputes the quality; pictures handed to it show only that the figures are theirs (§7). An encode is never a deterministic step: no tier and noreplicateclaim covers the encoded pictures against the source or against a base. An encode made over a package’s pieces is framed as one floor object per piece, each naming its encoder inderived_fromas packaging §7 requires in its paragraph 7.5 (§9). No new transport. The reference implementation is theplatform_encodemodule ofwai-rs(featureplatform_encode), thewai encode avc|hevc,wai encodersandwai encode-record verifycommands, andwai-pack encode-rungandwai-pack verify-rung; the corpus isencode-conformance/, and its second implementation isencode-conformance/encode_verify.py. Keywords MUST, MUST NOT, SHOULD, MAY are RFC 2119/8174.
1. Scope
In scope: what an encode is handed and how a ladder rung’s pictures are made (§4.1, §4.2); the shape of the elementary stream (§4.3); the encoder interfaces by role and the identity each reports (§3, §5); the encode record and its signature (§6); its verification (§7); the energy and the quality of an encode (§8); an encode over a package’s pieces (§9); conformance and measurement (§10).
Out of scope: choosing an encoder’s coding tools, which is the encoder’s; any claim about an encoder’s rate and distortion beyond the quality of one stream (§8.3), which a record states and never compares; any encoder written for WAI. The record states what the encoder made, and a verifier checks the bytes against it.
2. What WAI writes, and what it does not
2.1 WAI writes no AVC or HEVC encoder and ships none. The reference implementation links no encoder: it loads the operating system’s media framework, the video acceleration API’s library, the GPU driver’s encode library, or a software library at a path the user names, each at run time, and a build of it runs on a host without any of them and says which is missing.
2.2 Patent position, by mechanism. Both standards’ essential patents are licensed for
royalties (SPEC §5, Broadcast and streaming codecs: patent-pool). An encoder an operating
system or a GPU driver provides is supplied by the device’s maker with the device, under that
maker’s terms. A software library the user names is obtained by the user from its distributor,
separately from WAI, under the terms that distributor states, which may cover the royalties for
some uses and not for others. This extension states which encoder made a stream (§5); it states
nothing about whether a use of it is licensed. A deployer decides that from the encoder’s
supplier’s terms and its own use.
2.3 Header syntax under the video acceleration API. That interface leaves the parameter sets and the slice headers to the application, and the drivers reached for this revision emit none unless the application hands them over. The reference adapter writes them (ISO/IEC 14496-10 §7.3.2.1, §7.3.2.2, §7.3.3; ISO/IEC 23008-2 §7.3.2.1 to §7.3.2.3, §7.3.6.1) from the values it gives the driver in its parameter buffers, and chooses each picture’s QP, which it writes in both the parameter buffer and the slice header so the two agree. No picture sample passes through it: the driver’s engine codes every slice’s data.
2.4 Not a deterministic step. An encoder is not a pure function of its input
(packaging §11): a hardware engine’s output is fixed by no specification, and
the media framework’s hardware encoder was measured making other bytes from the same pictures
in each round (§10.3). So no step of a pack or a verify is an encode, no verdict depends on an
encoder’s output beyond the digest a record states, and an encoded rendition promises only its
capability’s decoder conformance (SPEC §7: wai.video.avc and wai.video.hevc are
StandardDefined, so every conforming decoder outputs the same pictures from the stream).
3. Encoder interfaces
An encoder is reached through one of four interfaces. Each record names the one it used in
encoder.api.
api | what it drives | where |
|---|---|---|
platform | the operating system’s media framework: its hardware media engine, or its software encoder where the engine is refused | the desktop, phone and TV operating systems that ship the framework |
va | the video acceleration API, whose driver drives a GPU’s encode engine | Linux, through a render node |
gpu-driver | the encode interface a GPU’s driver exports from its own library | Linux |
software | a software AVC encoder library loaded at run time from a path the user names | any Unix |
3.1 A va or gpu-driver encode is a hardware encode. A software encode is not, and
codes AVC only in this revision. A platform encode is either, and says which (§5).
3.2 The phone and TV operating system’s codec interface (informative). On the mobile and
TV operating system whose media codec interface exposes hardware and software encoders by
name, an adapter would: open an encoder for the codec’s MIME type by name, read the name, the
canonical name and whether the interface reports it hardware-accelerated for encoder.name and
encoder.hardware; configure the codec’s flexible 4:2:0 input, the target bit rate, no
B-pictures and no periodic key frame; request a key frame for exactly the pictures the plan
names; and read the device’s hardware and system-on-chip identifiers and the operating system’s
release for encoder.device and encoder.driver. Its output (the codec’s configuration buffer,
then one buffer per access unit) is an Annex B stream held to §4.3 like any other. This revision
builds no adapter for it.
4. The encode
4.1 The source
A source is a YUV4MPEG2 stream: 8-bit 4:2:0 (C420, C420jpeg, C420paldv, C420mpeg2, or no
C tag), progressive (Ip, I? or no I tag), each side even and at most 16384, its frame rate
F<num>:<den> both non-zero, and at least one frame. XCOLORRANGE=FULL makes it full range;
anything else is the video range. Each frame is its luma plane, then its Cb and Cr planes, each
row-major with no padding. The source’s digest is SHA-256 (FIPS 180-4) over every frame’s
three planes, one frame after another, with no frame header.
4.2 A ladder rung’s pictures
A rung asks for a size W × H (each even, from 16 to 8192), a frame rate (the source’s), a
target average bit rate in kilobits per second (1 to 1000000) and a range (the source’s). Where
the size is the source’s, the encoder is handed the source’s frames. Otherwise each plane is
resampled by wai.resample.two-tap, chroma at half the luma size: for an axis of n_in
samples resampled to n_out, output sample o reads the source at
p = ((2·o + 1) · n_in · 128) div n_out − 128
in 1/256ths of a sample, clamped to 0 ..= 256 · (n_in − 1); with x0 = p >> 8, f = p & 255
and x1 = min(x0 + 1, n_in − 1), the sample is (s[x0] · (256 − f) + s[x1] · f + 128) >> 8.
Rows are resampled first, then columns, each pass rounding to 8 bits. The resampler is a pure
function in integers: every host hands its encoder the same samples.
The IDR plan is the frame indices at which the stream has an IDR access unit: sorted, the first 0, each below the source’s frame count. A package’s plan is its pieces’ starts (§9).
4.3 The stream
An encoder’s output MUST be, and a producer MUST refuse to record one that is not:
- an Annex B byte stream (ISO/IEC 14496-10 Annex B; ISO/IEC 23008-2 Annex B): it begins with
zero bytes and a start code prefix; a NAL unit ends where the next three bytes are
00 00 00or00 00 01; no NAL unit is empty, holds00 00 02, or has its forbidden bit set; an HEVC NAL unit’snuh_temporal_id_plus1is not 0; every sequence parameter set reads, an AVC one through its VUI (Annex E.1.1) where it has one and an HEVC one through its sub-layer ordering information (sps_max_latency_increase_plus1); and the start of every slice header this section reads (below) reads (encode-stream-malformed); - exactly one access unit per source frame, in presentation order (
encode-frame-count). An access unit begins at the first of an access unit delimiter, a parameter set, a prefix SEI or the other non-VCL types the standard lists after the previous picture’s slices (ISO/IEC 14496-10 §7.4.1.2.3, its types 6 to 9 and 14 to 18; ISO/IEC 23008-2 §7.4.2.4.4, its types 32 to 35, 39, 41 to 44 and 48 to 55), or at a slice whosefirst_mb_in_sliceis 0 (AVC; the first syntax element of its slice header) or whosefirst_slice_segment_in_pic_flagis 1 (HEVC; the first bit). NAL units after the last picture’s slices that begin an access unit holding no slice areencode-stream-malformed, checked with item 1; - an access unit’s slices all IDR or none (
encode-stream-malformed); no random access point but an IDR (HEVCnal_unit_type16 to 23 other than 19 and 20); and an IDR access unit (AVCnal_unit_type5; HEVC 19 or 20) at exactly the plan’s frames (encode-keyframe-mismatch); - every IDR access unit carrying the sequence and picture parameter sets (and the video
parameter set for HEVC), the same bytes in the same order as the first IDR access unit’s; no
other access unit carrying parameter set bytes the first IDR’s do not; no access unit carrying
a sequence parameter set other than the stream’s first; and no access unit before the first
sequence parameter set (
encode-parameter-sets); - 8-bit 4:2:0 in one colour plane; progressive, which for AVC is
frame_mbs_only_flag1 (this revision reads no HEVC field signalling); a single layer: no AVC NAL unit of type 14, 15, 20 or 21, and HEVCnuh_layer_id0 throughout; and AVCpic_order_cnt_type0 or 2, the types this revision reads (encode-stream-unsupported); - with no reordering: HEVC
sps_max_num_reorder_picsof the highest sub-layer 0; AVC the VUI’smax_num_reorder_frames0 where the SPS has one, and, forpic_order_cnt_type0, each access unit’s top field order count (§8.2.1.1:PicOrderCntMsb + pic_order_cnt_lsb, from the first slice’spic_order_cnt_lsband the previous reference picture’s, both reset to 0 at each IDR) above the one before it since the last IDR (encode-stream-reordered). The slice header is read only as far aspic_order_cnt_lsb; this revision does not readdec_ref_pic_marking, so a stream whose pictures carrymemory_management_control_operation5 may be refused here; - of the rung’s size: the sequence parameter set’s cropped picture is
W × H(AVC frame cropping, HEVC conformance window), and where an AVC VUI signalsvideo_full_range_flagit is the rung’s range (encode-sps-mismatch).
Item 1 and item 2’s count are checked over the whole stream first; then items 3 to 6 access unit by access unit, in stream order, each access unit’s items in that order; then item 7. The first that fails is the reason. They hold the stream to what a piece of a package needs: each piece starts at an IDR access unit that carries its parameter sets, so it decodes with nothing outside it, and access unit i is the picture of frame i.
5. The encoder’s identity
5.1 An encode names its maker with six strings and one flag, each as the interface reports it at run time, read as the adapter documents:
| member | meaning |
|---|---|
api | the interface (§3) |
name | the encoder, as the interface names it: the framework’s encoder identifier and display name; the profile and entrypoint the driver offered, with the driver’s vendor string; the codec and preset the GPU driver’s interface names; or the library’s file name with the settings it was opened with |
version | the encoder’s version: the framework’s bundle version; the video acceleration API’s version; the encode interface’s version and the highest the driver supports; or the library’s own version |
hardware | yes when a fixed-function engine coded the pictures, no otherwise |
device | the device the encoder ran on: the processor’s name and model; the render node and its PCI identifiers; the GPU’s name, ordinal and PCI bus |
driver | the driver, or for a platform or software encoder the operating system’s release and build, that ran it |
library_sha256 | for software, the SHA-256 of the file loaded; otherwise absent (null) |
5.2 A producer MUST NOT state an identity it did not read from the interface. Where an interface reports nothing for a member, the adapter writes what it read instead and says so in the string; no member is empty.
5.3 A runtime-loaded library is refused at load unless it is of the interface version the adapter was written for: its structures may differ between versions. The record names the file loaded by its SHA-256, so a relying party can hold it to the distributor’s published digest.
6. The encode record
6.1 Members
The record is a JSON object (RFC 8259) read as contribution §2 reads a
record: strict JSON (no member named twice in one object, at most 32 levels), integers unsigned,
decimal, with no sign, fraction, exponent or leading zero and at most 2^53 − 1, no other number,
no boolean; byte strings in lowercase hexadecimal; a term 1 to 64 bytes of a–z, 0–9,
-, _ and .; a date-time RFC 3339 in UTC, YYYY-MM-DDTHH:MM:SS, at most nine fraction
digits, then Z, its day one its month has in the proleptic Gregorian calendar and its second
at most 60; a decimal a string of 1 to 39 decimal digits with no leading zero but 0
itself, an integer below 2^128; a signed decimal a decimal, or - and a decimal other than
0, an integer from −2^32 to 2^32. It has exactly these members, at every level, none missing
and none added, but quality: a record with no such member, as one written before the member
was, reads as a record whose quality is null does, and neither states any figure (§8.3.2). A
record that does not hold to this, or a member of another type or encoding (any quality that
is neither null nor an object of §8.3’s members among them), or outside the fixed words the
table gives kind, hardware, range and tier, is record_malformed. Every other set or
range of values is a §7 step’s, and refused with that step’s reason.
| member | type | meaning |
|---|---|---|
kind | string | "wai-encode" |
capability | string | wai.video.avc or wai.video.hevc |
encoder | object | §5.1’s members: api a term; name, version, device, driver strings of at most 1024 bytes; hardware "yes" or "no"; library_sha256 32 bytes or null |
settings | object | width, height, fps_num, fps_den, kbps integers; range "video" or "full"; resample a term, none or wai.resample.two-tap (§4.2) |
source | object | frames, width, height integers; sha256 32 bytes, the source’s digest (§4.1) |
idr | array | the IDR plan (§4.2), integers |
stream | object | sha256 32 bytes, the stream’s SHA-256; bytes, units integers; profile_idc, level_idc integers, the SPS’s (general_profile_idc and general_level_idc for HEVC); tier "main" or "high", HEVC’s general_tier_flag, and "main" for AVC, which has none (held to the stream at §7 step 8) |
energy | object or null | §8 |
wall_us | integer | the encode’s wall time, from the first picture handed over to the last byte back |
package | object or null | §9 |
quality | object or null | §8.3: the quality of the pictures the stream decodes to, null when the signer did not measure it |
issued_at | date-time | when the signer signed it, self-asserted |
signer_pubkey | 32 bytes | an Ed25519 key (RFC 8032) |
sig | 64 bytes | §6.2 |
6.2 Signature and digest
sig = Ed25519(signer_key, "wai:video-encode\x01" || JCS(record without "sig"))
digest = SHA-256("wai:video-encode-record\x01" || JCS(record))
JCS is RFC 8785. The signature is verified strictly (SPEC §7 Signatures): a key
that fails step 1 or 2 is bad_key, a signature that fails a later step bad_signature. The
record’s text as the reference writes it is its JCS form. The signer is whoever ran the encode;
the record is that signer’s statement of what the interface reported (§5.2).
7. Verification
A verifier checks, in order, and the first that fails is the reason:
- the record reads (§6.1) (
record_malformed); - its signature (§6.2) (
bad_key,bad_signature); capabilityiswai.video.avcorwai.video.hevc(encode-capability);- the identity:
name,version,deviceanddrivereach hold a character other than Unicode White_Space (encode-identity-incomplete);apiis one of §3’s;library_sha256is present exactly forsoftware; asoftwareencode ishardware"no"and ofwai.video.avc; avaorgpu-driverencode ishardware"yes"(encode-identity-inconsistent); - the settings and the source: the rung’s width and height even and from 16 to 8192, the
source’s even and from 2 to 16384; both rate members non-zero and at most 2^32 − 1;
kbpsfrom 1 to 1000000; at least one source frame; andresamplenoneexactly where the rung’s size is the source’s,wai.resample.two-tapotherwise (encode-settings-invalid); - the IDR plan: its first entry 0, each above the one before, the last below
source.frames; and, wherepackageis notnull, its pieces’ starts equal to the plan, each piece of at least one frame and starting where the one before it ended, their frames summing tosource.frames(encode-plan-invalid); - the energy, where it is not
null(§8.1) (encode-energy-invalid,encode-energy-coverage); - with the stream held: its SHA-256 and length are
stream.sha256andstream.bytes(encode-stream-mismatch); it holds to §4.3 forsource.framespictures and the plan, its reasons as there; its access units numberstream.units(encode-frame-count); its SPS’s cropped size is the settings’width × height, its profile, level and tierstream’s, and an AVC VUI’s signalled range the settings’ (encode-sps-mismatch); - with the source held: it reads as §4.1 says, its digest is
source.sha256, its width, height and frame count aresource’s, its frame rate’s numerator and denominator, as its header writes them, are the settings’fps_numandfps_den, and its range is the settings’ (encode-source-mismatch); - with the package and the floor objects held, §9.3 (
encode-package-mismatch); - with the source and pictures held, where
qualityis notnull: the pictures aresource.framespictures of the settings’width × height, their SHA-256 isquality.decoded_sha256, and each ofsse_y,sse_cb,sse_crandssim_y_q32is the figure §8.3 computes from them and the pictures the encoder was handed (encode-quality-mismatch).
Steps 8 to 11 are made only where the verifier holds what they read; a verdict names which it made. Nothing in them re-encodes: the source is held to its digest, the stream to its digest and its shape, and the pictures to their digest.
Step 11 reads pictures, and only a decode ties pictures to a stream: quality.decoded_sha256
is the signer’s statement of what the stream decodes to, and a signer can state any figures and
supply pictures that give them. So a verdict on a record stating figures also names what they
rest on:
| verdict | when | what it says of the figures |
|---|---|---|
decoded | step 11 was made over pictures the verifier decoded itself, with a conforming decoder, from the stream step 8 held | they are the stream’s |
handed | step 11 was made over any other pictures, whoever made them | they are those pictures’, and nothing of the stream |
stated | step 11 was not made | they are the signer’s statement |
A relying party MUST NOT read a figure as the stream’s on a verdict other than decoded. A
record stating no figures has no step 11 and no such word, whatever pictures are held.
8. Energy and quality
8.1 The figure
energy restates a measured figure with the members jwp-receipts §2.4.5
gives an energy claim, and three of this extension’s own:
| member | type | meaning |
|---|---|---|
joules_micro | integer | the figure, microjoules, for the whole encode the record names |
energy_provenance | string | an acquisition class of energy-measurement §2 |
energy_uncertainty | object or null | { "relative_ppm", "window_us" }, integers, as §2.3 there |
energy_counter_filtering | string or null | an OnChipCounter’s filtering state |
energy_calibration_ref | string or null | a CalibratedInstrument’s calibration reference |
coverage | string | Whole or Partial (energy-binding §4) |
counter | term | the counter read (§8.2) |
attribution | string | task, rail or bracket (§8.2) |
fabrics | array of strings | what the counter’s reading covers, each of cpu, gpu, media, npu, at least one, in that order, none twice |
It is held to jwp-receipts §2.4.5’s rules 1 to 3, read as written there: a non-zero figure; a
class of energy-measurement §2 other than HwShunt; the class’s evidence whole, including the
resolution floor joules_micro × relative_ppm ≥ 288 676, an OnChipCounter’s filtering state
of On, Off or Unknown, and a CalibratedInstrument’s non-blank calibration reference; and
to a coverage of Whole or Partial, an attribution of the three, and fabrics as the
table says (encode-energy-invalid). A hardware encode’s figure whose fabrics names
neither gpu nor media counts none of the engine that coded the pictures, so it MUST be
Partial (encode-energy-coverage). A relying party MUST NOT report a Partial figure as
the whole cost of the encode.
8.2 How the reference measures it
The figure is the median of an odd number of rounds (at least three) of the same encode, after
one unmetered round that loads and warms the encoder. A round’s figure is a counter’s advance
over the encode (task), or energy-measurement §3’s marginal: the advance over the encode less
the mean power over an idle window just before it, times the encode’s length (bracket); a
round whose marginal is not above zero, or whose counter fails a read, makes the whole figure
absent rather than estimated. The declared uncertainty is the rounds’ spread read as rectangular,
(max − min) / (2·√3) relative to the median, rounded up, or energy-measurement §2.3’s rounding
floor if that is larger; its window is the median round’s. The class is OnChipCounter with
filtering Unknown, and the coverage Partial: no counter the reference reads sees all of a
host’s energy.
counter | what it reads | attribution | fabrics |
|---|---|---|---|
task | the operating system’s account of the encoding process’s own energy, with the work other processes did for it billed to it | task | cpu |
rapl | the processor package zones’ energy counters, summed | bracket | cpu |
gpu-board | a discrete GPU driver’s total-energy counter for its device | bracket | gpu, media |
rapl-and-gpu-board | both, summed | bracket | cpu, gpu, media |
The table is the reference’s practice, not a rule a verifier applies: §8.1 is. No counter is
divided among its users by busy time. A task figure counts none of a hardware
engine’s draw, since the operating system gives an unprivileged process no account of it; a
bracket figure on a host other work shares is machine-wide, and its rounds’ spread says how
much that work moved it.
8.3 Quality
A record whose signer decoded the stream states the quality of the decoded pictures against
the pictures the encoder was handed (§4.2) in quality, an object of exactly:
| member | type | meaning |
|---|---|---|
decoded_sha256 | 32 bytes | the SHA-256 of the decoded pictures: each picture’s luma plane, then its Cb and Cr planes, row-major with no padding, one picture after another in output order, as §4.1 digests a source |
sse_y, sse_cb, sse_cr | decimal | Σ (aᵢ − bᵢ)² over that plane of every picture, a the samples the encoder was handed and b the decoded ones |
ssim_y_q32 | signed decimal | the structural similarity of the luma planes: staged-measurement §4’s, its windows and integers as its paragraph 4.2 gives them, with each plane’s samples read as its luma, pooled over every picture as its paragraph 4.5 pools units, one mean over every window of every picture, in Q32 (2^32 is 1) |
The decoded pictures are source.frames pictures of settings.width × settings.height, 8-bit
4:2:0. The handed pictures are the source’s frames where resample is none, and their
wai.resample.two-tap resampling to the rung’s size (§4.2) otherwise.
8.3.1 A figure of the stream. Every conforming decoder outputs the same pictures from the
stream (§2.4: StandardDefined), and the figures are integers computed from those pictures and
the handed ones alone. So the figures a stream gives are the same whichever conforming decoder
made the pictures, and a verifier that decodes the stream itself recomputes them exactly (§7, the
verdict decoded). A record signs them for that reason. Pictures handed to a verifier show only
that the record’s figures are theirs: nothing but a decode ties them to the stream (§7, the
verdict handed). The time and the energy of an encode are figures of the encoder, the host,
its load and its counter, and §8.1, §8.2 and §10.3 state them as such.
8.3.2 A record whose signer did not measure the stream’s pictures, because it did not decode
the stream or its decode gave no pictures to measure, states quality null: the encode stands,
and the record says nothing of its quality. The reference’s wai_encode_measure writes the figures
from the pictures the platform’s decoder makes, or from pictures another conforming decoder made,
which it is handed (§10.3). PSNR, 10·log10(255²·N / sse)
over a plane’s N samples, is derived from the sums for display only, as staged-measurement
§4 derives it in its paragraph 4.7, and a sum of 0 has no finite PSNR and shows as exact; a
record carries the integers. The figures state the distortion of one stream against the
pictures it was made from; this extension compares no figure with another.
8.3.3 A record given figures after it was signed is a new statement: its signer signs it
again, and its issued_at is when (§6.1). Another key does not add to a record: the record is its
signer’s (§6.2).
9. An encode over a package’s pieces
9.1 The rendition
An encoded rendition of a package (packaging §11) is one encode of a video
component’s source whose IDR plan is the component’s pieces’ starts, as its stage index lists
them (§6.2 there), and whose rung is named by a term, its rung id. Each piece is framed as
one floor object (packaging §5.2, §7.2): a WAI1 envelope (SPEC §2) whose manifest is the JSON
text, with no whitespace and its members in this order,
{"wai":"1.0","media":"video","intent":"replicate","model_requirement":{"capability":C},
"conditioning":{"kind":K},"derived_from":{"motif":M,"motif_capability":MC,
"renderer":R,"settings":D},"target":{"frames":F,"h":H,"w":W}}
with C the capability, K avc or hevc, F the piece’s frames, H and W the rung’s
height and width, each string written as JCS writes it (RFC 8785 §3.2.2.2), and the
derived_from members:
motif: the piece’s base object’s content hash, the stage index’s entry of role base and layer 0 for that piece, as 64 lowercase hexadecimal digits;motif_capability: its capability;renderer:<encoder.name>@<encoder.version>;settings:BLAKE3("wai:render-settings\x01" ‖ JCS(S)), withSthe object{"source": "encoded", "api", "hardware", "device", "driver", "library_sha256", "stream_sha256"}, each member butsourcethe record’sencodermember of that name, andstream_sha256the record’sstream.sha256; the digest written as 64 lowercase hexadecimal digits.
So each object names the encoder and its version in renderer (packaging §7.5) and binds its
device and driver through settings, which any verifier holding the record recomputes. Its
payload is its piece’s access units as NAL units each after the four bytes 00 00 00 01, in the
order the stream holds them: the stream’s own NAL units, re-framed with one start code length,
beginning with the piece’s IDR access unit and its parameter sets.
9.2 The record’s package
| member | type | meaning |
|---|---|---|
receipt_sha256 | 32 bytes | the SHA-256 of the package receipt’s file (packaging §9) |
rung | term | the rung id |
pieces | array | one object per piece, in order: start and frames integers, the piece’s first frame and frame count; motif 32 bytes, the base it presents; object 32 bytes, its floor object’s content hash (jwp-receipts §2.1) |
The objects are written objects/<object>.wai, the stream <rung id>.h264 or <rung id>.h265,
and the record <rung id>.encode.json, beside the package and not inside it: the package
receipt does not list them, and a package made before them is unchanged.
9.3 Verification against the package
With the package receipt’s file, the component’s stage index and the floor objects held, a
verifier checks, in order, every failure encode-package-mismatch: the receipt file’s SHA-256
is receipt_sha256; the stage index’s pieces, in order, have the record’s starts and frame
counts (a video component’s timebase unit is one frame, packaging §2), and each piece’s one
entry of role base and layer 0 has the record’s motif; and each piece’s object is present,
its content hash is object, it is a WAI1 envelope with exactly §9.1’s manifest for this
record and piece, and its payload is §9.1’s bytes from the held stream.
A sink that decodes an encoded rendition decodes each object’s payload as its capability’s stream; the record tells it who made the bytes, and nothing more.
10. Conformance and measurement
10.1 The corpus. encode-conformance/ holds small elementary
streams made by encoders through each interface (fixtures, pinned by their SHA-256 in the
corpus’s manifest and never regenerated, since no encoder is a pure function), the source they
were made from, the pictures a conforming decoder made from one of them (a fixture too, pinned
in decoded.json), and records generated reproducibly over them from a fixed-seed key with a
fixed issued_at: accepted cases, among them a record with quality over a resampled rung, one
stating null, and records with no such member, and negative controls each refused with its
reason, among them that record with each figure one away from the pictures’ and with another’s
pictures; and an encoded rendition of a packaging golden with its objects, accepted and mutated. A
case with pictures states how the verifier came by them, decoded or handed, and an accepted
record stating figures states §7’s verdict. The
pinned pictures stand in for a verifier’s own decode in a case of decoded, since §10.2 holds
them to what a conforming decoder makes of the stream, and a verifier with a decoder may decode
the stream instead. The corpus records what pictures handed over cannot show: a record stating
the perfect score of the encoder’s own input, which its stream does not give, is accepted as
handed with that input (handed/, generated) and refused as decoded. wai-rs
tests/platform_encode.rs requires the generator to reproduce the corpus byte for byte and every
case to get its outcome and its verdict; wai-pack tests/encode_rung.rs does the same for the
rendition. The second implementation, encode-conformance/encode_verify.py, is written from this
text: it shares no code with wai-rs or wai-pack, reads the Annex B stream, its access units
and its parameter sets itself, decodes nothing, resamples the source and recomputes every quality
figure itself, verifies signatures with WAI’s strict Ed25519 (claim-conformance/_ed25519.py) and
content hashes with BLAKE3 (claim-conformance/_blake3.py), and mutates copies of the corpus and
requires every copy to fail.
10.2 Decoder conformance. Each corpus stream, and each stream the reference encoders made
for §10.3, decodes in two independent conforming decoders, a hardware decoder and a software
one, to the same pictures byte for byte, as StandardDefined requires.
10.3 Measurement. wai_encode_measure encodes a source through one interface at a rung,
rounds as §8.2, and decodes the result with the platform’s decoder, or reads the pictures
another conforming decoder made, to write the record’s quality (§8.3), from which it prints
PSNR for display (luma, each chroma plane, and the 6:1:1 weighted mean of their squared
errors) and which pictures it read. With no pictures to measure it writes the record with
quality null and says why (§8.3.2). Its time and energy figures are informative: each is of
the encoder, the host, its load and its counter at the time it was taken, and none is compared
with a figure taken under another counter.
11. Relationship to other extensions
- packaging §7, in its paragraph 7.5, requires a floor made by an encoder that is not a pure function of its input to name the encoder and its version; §9 here is how an AVC or HEVC rendition over a package’s pieces does it. Packaging §7.1’s floor remains a capability of SPEC §5’s mandatory floor or recommended set: an encoded rendition is beside it, not in its place.
- energy-measurement gives the acquisition classes, the declared uncertainty and the bracketed marginal (§2, §3); jwp-receipts §2.4.5 gives the members and rules §8.1 restates.
- SPEC §5, Broadcast and streaming codecs, registers
wai.video.avcandwai.video.hevc, their payload (the Annex B byte stream) and their licence class.
12. Reasons
| reason | step | meaning |
|---|---|---|
record_malformed | §7 step 1 | the record does not read as §6.1 says |
bad_key | step 2 | the signer’s key fails SPEC §7’s strict rule |
bad_signature | step 2 | the signature fails it |
encode-capability | step 3 | not wai.video.avc or wai.video.hevc |
encode-identity-incomplete | step 4 | an identity member is blank |
encode-identity-inconsistent | step 4 | the interface, hardware, library_sha256 and capability disagree |
encode-settings-invalid | step 5 | a size, rate, bit rate, frame count or resample out of range or inconsistent |
encode-plan-invalid | step 6 | the IDR plan, or the package’s pieces, out of order or out of range |
encode-energy-invalid | step 7 | the figure’s members break §8.1 |
encode-energy-coverage | step 7 | a hardware encode’s figure that counts no engine is stated Whole |
encode-stream-mismatch | step 8 | the stream’s SHA-256 or length is not the record’s |
encode-stream-malformed | §4.3 items 1 to 3 | not an Annex B stream, NAL units after the last picture that hold no slice, or an access unit mixing IDR and other slices |
encode-frame-count | §4.3 item 2, step 8 | another number of access units |
encode-keyframe-mismatch | §4.3 item 3 | an IDR off the plan, a planned picture not an IDR, or another random access point |
encode-parameter-sets | §4.3 item 4 | parameter sets missing or changed |
encode-stream-unsupported | §4.3 item 5 | not 8-bit 4:2:0 progressive single-layer, or a picture order type this revision reads not |
encode-stream-reordered | §4.3 item 6 | the stream reorders pictures |
encode-sps-mismatch | §4.3 item 7, step 8 | the coded size, profile, level, tier or range is not the record’s |
encode-source-mismatch | step 9 | the source’s digest is not the record’s |
encode-package-mismatch | step 10 | the rendition does not match the package, the stream or the record |
encode-quality-mismatch | step 11 | the pictures held are not the record’s, or a quality figure is not the one they give |
encoder-unavailable | producer | no encoder of the interface for the codec here (not a record reason) |
encoder-refused, encoder-failed | producer | the interface refused the settings, or failed an encode (not record reasons) |
13. Security considerations
- A record is its signer’s statement. It names what the interface reported; a signer can name an encoder that did not run. The stream is checked, not the claim of who made it: a relying party that must know the maker trusts the signer, or runs the encode itself.
- A runtime-loaded library is code the user chose. The adapter refuses a library of another
interface version and states the digest of the file it loaded; it does not check the file
against any list. A deployer that must hold the library to its distributor’s published digest
compares
library_sha256with it. - Quality is the signer’s statement until a verifier decodes the stream. A verifier that
decodes the stream it holds with a conforming decoder recomputes every figure from what it
decoded and the source, and its verdict is
decoded(§7). Pictures handed to it, whose digest isdecoded_sha256and which give every figure, show only that the figures are theirs: a signer can state any figures and supply such pictures, so that verdict ishanded, and of the stream the figures stay their signer’s. - A stream is checked before it is recorded. §4.3 is applied by the producer before it signs, and again by every verifier holding the stream, so a record never names a stream whose pieces do not each start at a self-contained IDR access unit.