WAI Extension: Opus Deep Redundancy, Decoded Exactly
Mirrored from the canonical text at commit 260cbd09 ().
Status: Draft. Registers two companion-only capabilities for the deep redundancy (DRED) of draft-ietf-mlcodec-opus-dred-07: a compact neural latent of the recent speech, carried in an Opus packet’s padding, from which a receiver rebuilds the acoustic features of packets that were lost.
wai.audio.opus.dredis that decoding as the draft defines it, held to the draft’s own conformance test (its §4.1), which admits any decoder whose features lie within its thresholds of the reference.wai.audio.opus.dred.intis the same decoding through an integer profile,dred-int/1: the draft’s own decoder network and its own weights file, pinned by digest, lowered towai-ops/1graphs (integer-operators) whose output is the same integers on every machine and on every fabric whose operator implementation conforms. With it, a stream under loss has one reconstruction, named by a digest that a verifier recomputes from the packets, the loss and the weights, and a player can sign that digest in the log of synthesis §10.3. This extension also says where the working group’s other documents stand in WAI’s registry and in synthesis’s lattice (§2.3, §7.1), and measures the profile against the floating-point network (§6). No new transport. The reference implementation is thedred_exactmodule ofwai-rs(featuredred_exact); the corpus islive-conformance/dred/, and its second implementation islive-conformance/dred_verify.py. The integer profile as a proposal to the working group is drafted inproposals/dred-integer-profile.md, which is not filed. Keywords MUST, MUST NOT, SHOULD, MAY are RFC 2119/8174.Texts read for this revision, each pinned to the version named: draft-ietf-mlcodec-opus-dred-07 (4 August 2026), with the files its §7 references by SHA-256 ([DRED-Weights], [DRED-Test-Vectors], [DRED-Compare], [DRED-Ref-Impl]); draft-ietf-mlcodec-opus-extension-06; draft-ietf-mlcodec-opus-speech-coding-enhancement-04; draft-ietf-mlcodec-opus-scalable-quality-extension-02; draft-ietf-mlcodec-test-battery-00; RFC 6716 §3, §4.1 and §5.1; RFC 7587.
1. Scope
The draft’s decoder has two parts (its §4): a feature decoder, a recurrent network that turns the latents into 20 acoustic features per 10 ms, which the draft makes normative and tests within tolerances; and a vocoder that turns features into samples, which the draft leaves to the implementation (its §2.2.4, §4.2). This extension’s profile is of the first part.
Conformance targets.
- A sink that advertises
wai.audio.opus.dred.intfinds DRED in a packet (§3), decodes it throughdred-int/1(§4), and gives exactly the features §4 gives. - A verifier of a reconstruction recomputes a stream’s reconstruction from its packets, its loss pattern and the weights, and checks a record of it (§5) and a player’s statement of it (§7.2).
Out of scope.
- The vocoder. A sink renders features to samples with any vocoder the draft admits; those samples are its own, and §7.2 says what a player states of them.
- The decoding of an Opus packet’s own frames (RFC 6716 §4), and concealment of a frame no DRED rebuilds. §7.1 classes both.
- Encoding. The draft’s encoder is not normative (its §2.2.1); the corpus’s packets are written by a generator that quantizes as §9 describes.
2. Registration
2.1 Two strings, one payload
| string | kind | role | replicate class | parameter set |
|---|---|---|---|---|
wai.audio.opus.dred | companion | enhancement layer over wai.audio.opus.packet | StandardTolerance: features within the draft’s §4.1 thresholds of the reference | the draft’s, fixed by the draft |
wai.audio.opus.dred.int | companion | enhancement layer over wai.audio.opus.packet | NeuralIntegerExact: the features of §4, exactly | one file, the draft’s weights file (§4.1) |
Both carry the same payload. They differ in what a sink that advertises them promises, which is
what a sender chooses between: a sender that needs any rebuilt speech names the first; a sender
whose reconstruction is to be recomputed, signed or presented at decode-equivalence names the
second, with a pin on the weights file (SPEC §3.1) that declares tier decode-equivalence.
Two strings are needed because a capability’s class bounds the tier a pin may claim (SPEC §3.1,
Tier): a decode admitted within tolerances cannot back replicate, and the draft’s decoding is
admitted within tolerances.
2.2 Why a companion, not a refinement
A refinement-only capability is applied to the output of the stage below it, bound to that
output’s producer by content hash (staged-delivery): its payload corrects
what the base decoded. DRED corrects nothing the base decoded: it is a second, coarser encoding of
the same speech, sent ahead so that a receiver can rebuild what a loss took, and it is read only
where the base’s own frames are missing. Its payload has no meaning without the encode it was
made with: its offsets index that encode’s frames (draft §3), and its first rebuilt samples
continue that encode’s last decoded ones (draft §2.2.4). That is the model of an enhancement
layer (SPEC §5): a companion-only capability, decoded by its own standard over one base
encode, bound to exactly one selectable rendition of a base it lists, and the base of no second
layer. So both strings are registered as enhancement layers. Their one base is
wai.audio.opus.packet, one Opus packet per object, because there the envelope itself aligns the
DRED payload with the packet it belongs to; this revision pins no alignment of a separate DRED
stream with an Ogg Opus stream (wai.audio.opus), as SPEC §5 pins none for a separately carried
LCEVC stream.
DRED is carried one of two ways, as an enhancement layer is (SPEC §5, Enhancement layers):
- Embedded. The DRED extension rides in the base packet’s padding (draft §3), under extension
ID 32, or, until the IANA action the draft asks for, under the experimental ID 126 with the
two-byte prefix
'D', 12 (draft §3, §5). The payload is the base’s bytes, under the base’s capability; a sink without DRED discards the padding (draft-ietf-mlcodec-opus-extension-06 §2). No DRED component exists. - Separate. A
WAI2companion of the base rendition (SPEC §7.1),role"companion",companion_ofwai.audio.opus.packet. Its payload is one DRED extension payload as draft §3 defines it (the bytes after the extension’s ID byte and any length, with no experimental prefix), for frame 0 of the base packet in the same envelope (a frame offset of 0, §3.3).
2.3 The working group’s other documents
- draft-ietf-mlcodec-opus-extension-06 defines the padding’s extension format that §3.2 reads. It carries no payload of its own and registers no capability here.
- draft-ietf-mlcodec-opus-speech-coding-enhancement-04 sets requirements for a decoder-side
enhancement of the Silk layer’s output, and for a blind extension of its bandwidth above
8 kHz. It changes no byte of a packet, so it is no WAI capability: it is a property of the
sink’s Opus decoder, and a stream decoded with it is still
wai.audio.opusorwai.audio.opus.packet, decoded within RFC 6716 §6’s tolerances by a decoder that also passes that draft’s objective tests. §7.1 classes what it does to an object. - draft-ietf-mlcodec-opus-scalable-quality-extension-02 adds refinement bits for the CELT layer, and bands above 20 kHz, in the same padding. It is not a neural decoder, and the refinement rides embedded in the base packets, decoded by the base’s decoder; this revision registers no string for it.
- draft-ietf-mlcodec-test-battery-00 describes test material for these extensions. WAI reads it as informative.
2.4 Licence class
Both strings are classed patent-pool, the conservative class REGISTRY.md gives a capability
whose royalty status is not established. One IPR disclosure on the individual draft that became
draft-ietf-mlcodec-opus-dred, the IETF’s disclosure 6063, states an intent to license
royalty-free if the technology is included in a standard the IETF adopts; until that happens the
status is not established. A later revision MAY move both strings to royalty-free once it is.
3. Finding DRED and its integers
3.1 The packet
A sink reads an Opus packet’s framing as RFC 6716 §3.2 defines it, and holds it to §3.4’s rules
R1–R7. A packet that breaks one is not a valid Opus packet: nothing in it is read, and its DRED
is refused (packet_malformed). From a valid packet it takes the number of frames, each frame’s
duration in units of 2.5 ms (RFC 6716 §3.1: 4, 8, 16 or 24 for configurations 0–11 by
configuration modulo 4; 4 or 8 for 12–15 by configuration modulo 2; 1, 2, 4 or 8 for 16–31 by
configuration modulo 4), and the padding of a code-3 packet.
3.2 The extensions
The padding’s extensions are read as draft-ietf-mlcodec-opus-extension-06 §2 defines them, including the frame separator (ID 1) and the repeat mechanism (ID 2, §2.3), in bitstream order, each instance with its frame index. Iteration stops at the first extension whose length, or whose frame index, would read past the padding or the packet’s last frame (§2.7); the instances before it stand.
The packet’s DRED payload is the first instance, in that order, that is either:
- ID 32 with at least one byte of data: the data; or
- ID 126 whose data is longer than two bytes and starts with
'D'(0x44), 12: the data after those two bytes.
Its frame offset is its instance’s frame index times the frame duration, in units of 2.5 ms. A packet with no such instance carries no DRED. (The reference implementation the draft names in its Appendix B finds DRED this way, under ID 126.)
3.3 The payload
The payload is decoded with the range decoder of RFC 6716 §4.1, as draft §3 and §3.1 define it. This section fixes the details the draft’s prose leaves open, as the draft’s reference decoder does; where the two could be read apart, the reference decoder’s reading is this profile’s.
- The header:
q0 = uint(16),dQ = uint(8),X = uint(2); ifX = 1, an extended offsetE = uint(256), elseE = 0; an indicatorI = uint(32).uint(n)isec_dec_uintof RFC 6716 §4.1.5 forn ≤ 256, one symbol ofnequally likely values.encoded_offset = 32·E + I. qmax: 15, unlessq0 < 14anddQ ≠ 0. Then, withn = 14 − q0, a symbol is decoded over a total of2n(ec_decode): a values ≥ ngivesqmax = q0 + (s − n) + 1and is consumed as[s, s + 1); a values < ngives 15 and is consumed as[0, n).- The offset,
dred_offset = 16 − encoded_offset + frame_offset, in units of 2.5 ms: the newest feature vector the payload rebuilds ends this long after the start of the frame that carries the extension. - The initial state: 50 coefficients
k = 0 … 49, at quantizerq0, with the state tables (draft Tables 6, 8 and 9). Then latent vectorsl = 0, 1, …, newest first, each of 25 coefficients at quantizerq_l = min(qmax, q0 + ⌊(D[dQ]·l + 8) / 16⌋)withD = [0, 2, 3, 4, 6, 8, 12, 16], the slope table of draft §3 in sixteenths, with the latent tables (Tables 2, 4 and 5). Before each latent vector, if8·len − tell ≤ 7, wherelenis the payload’s length in bytes andtellisec_tell(RFC 6716 §4.1.6.1), no more are decoded. - A coefficient with decay
rand zero probabilityp0(the tables’ entries for its coefficient and quantizer): ifr = 0orp0 = 255, it is 0 and nothing is decoded. Otherwise, withP = 128·p0andS = 32768 − P, a symbol is decoded byec_dec_icdf16(withftb = 15) against the table[S, ⌊S/2⌋, 0]: 0 is a zero coefficient, 1 positive, 2 negative. A non-zero coefficient’s magnitude is then1 + v_1 + v_2 + …, eachv_jdecoded byec_dec_icdf16againsticdf[0] = max(7, 128·r),icdf[i] = max(7 − i, ⌊icdf[i − 1]·128·r / 2^15⌋)fori = 1 … 6,icdf[7] = 0, while the symbol just decoded is 7. When the magnitude being summed passes2^16, the payload is refused (dred_latent_range): the draft’s escape rule bounds nothing, and past a payload’s last byte the decoder reads zero bytes (RFC 6716 §4.1.2), so a crafted payload could otherwise keep a decoder reading.
The draft’s test bit files carry payloads with no header (each record gives q0, with no slope,
and a count of latents); §6 reads them so.
4. The integer profile dred-int/1
4.1 The parameter set
The parameter set of wai.audio.opus.dred.int is one file: the decoder weights file draft
§2.2.2.1 and §7.1 name ([DRED-Weights]), whose SHA-256 is
394f0ecf19926cebd678019dc94a54c88ae078340c16325069707f286cdf28a6. A sink MUST check the file
against that digest before it reads it, and MUST refuse any other file
(weights_digest_mismatch): the draft makes another set of weights non-conforming (its §4.1, “an
alternate DNN model is not allowed”), and §4.4 makes these weights’ integers a function of this
file’s bytes. A pin over the set names that digest.
The file is read as the draft’s Appendix A defines it. The profile reads, for each layer of the
table below, the arrays dec_<layer>_weights_float and dec_<layer>_bias, and for a block-sparse
layer dec_<layer>_weights_idx; it reads no other array. A dense matrix is stored input-major
(weights[in·outputs + out], A.3.1). A block-sparse one (A.3.2) is a run of groups of 8 outputs,
each a count of blocks and then each block’s first input (a multiple of 4), and its weights are
blocks of 8 outputs by 4 inputs stored input-major inside the block (w[in·8 + out]); every
position no block covers is 0.
| layer | inputs | outputs | sparse |
|---|---|---|---|
dense1 | 26 | 96 | no |
hidden_init | 50 | 128 | no |
gru_init | 128 | 320 | yes |
gru1_input … gru5_input | 96, 192, 288, 384, 480 | 192 | yes |
gru1_recurrent … gru5_recurrent | 64 | 192 | no |
glu1 … glu5 | 64 | 64 | no |
conv_dense1 … conv_dense5 | 160, 256, 352, 448, 544 | 32 | yes |
conv1 … conv5 | 64 | 32 | no |
output | 576 | 80 | yes |
The weights are not in this repository. The corpus names the file by digest, and its fetch script
(live-conformance/dred/fetch.sh) fetches it from the location the draft gives and checks the
digest.
4.2 The tables
The decoder reads draft Tables 2, 4, 5, 6, 8 and 9 (scale, decay and P(0), for the latents and
for the state). Serialized row by row (coefficient k from 0, the sixteen quantizers in order,
one byte each), in the order latent scale, latent decay, latent P(0), state scale, state decay,
state P(0), their SHA-256 is
03f7d6b22e2aca5f5a28525a1311e1113d94b917fbc74a5493b04bbed5c97ded. An implementation that
transcribes them checks its transcription against this digest.
4.3 The inputs
Every graph input is an integer at 2^16 (Q16). A coefficient x of the initial state, with
state scale s at q0, and a coefficient of latent l, with latent scale s at q_l, stand
for x·256 / max(1, s) (draft §3.1), which the input holds as
⌊(2·x·2^24 + s′) / (2·s′)⌋ with s′ = max(1, s): the quotient rounded to the nearest
integer, a tie upward. A latent’s 26th input is q_l/8 − 1 (draft §2.2.2), held as
(q_l − 8)·2^13.
4.4 Weights to integers
Every weight becomes an integer at 2^24, and every bias an integer at the scale of the
accumulator it joins: 2^40 for dense1 and hidden_init, whose inputs are at 2^16, and
2^39 for every other layer, whose inputs are activations at 2^15. Each is computed from the
float32 value’s bits alone, with no floating-point arithmetic:
- a float32 with sign
σ, exponent fieldeand fraction fieldmis(−1)^σ·M·2^E, withM = m,E = −149whene = 0, andM = m + 2^23,E = e − 150otherwise; - at scale
2^f, its integer is(−1)^σ·R_half_even(M, −(E + f))whenE + f < 0, and(−1)^σ·M·2^(E + f)otherwise, withR_half_evenof integer-operators §2 (0 for a shift of 63 or more).
A value that is not finite (e = 255), a weight whose integer does not fit i32, a missing array,
an array of the wrong length, a block that starts at an input that is not a multiple of 4 or past
the layer’s inputs, a group that names one block twice, or a record the file format refuses,
makes the set fail to lower (model_mismatch, integer-operators §10.5). The pinned file does
none of these.
4.5 Activations
Activations are integers at 2^15 (Q15), in [−32767, 32767]. tanh and the sigmoid are
lut16 tables (integer-operators §5.6), read at 2^12 and 2^11. Their entries are this
integer computation, not the functions’ exact values:
C_0 = 4609434768280048980(⌊e^(−1/2048)·2^62⌋), andC_(k+1) = ⌊C_k² / 2^62⌋;- for
0 ≤ u < 2^15,X(u)starts at2^62, and for each bitkofufrom bit 0 to bit 14 that is set, in that order,X ← ⌊X·C_k / 2^62⌋; tanh: entryv + 32768, forvin−32767 … 32767, issgn(v)·min(32767, ⌊(2a + b) / (2b)⌋)witha = 2^15·(2^62 − X(|v|))andb = 2^62 + X(|v|);- sigmoid: entry
v + 32768ismin(32767, ⌊(2a + b) / (2b)⌋)withb = 2^62 + X(|v|), anda = 2^77forv ≥ 0,a = 2^15·X(|v|)forv < 0; - entry 0 (
v = −32768) of each is 0, aslut16requires. No certified input reaches it.
Every intermediate fits 128 bits. Serialized as their 65,536 entries in order, each a 16-bit
two’s-complement little-endian integer, the tables’ SHA-256 are
aea887ef206a10ea37bd31ef310a75c2ff05ca17df7edbbc15916969bccb4c22 (tanh) and
1fe2d7c6b3255127f612576093f2f50e9e13d88da546ad4c04de264e2bbc8c36 (sigmoid).
4.6 The graphs
dred-int/1 lowers the set to two WHG1 graphs (integer-operators §10), init and step, in
which every tensor is [C, 1, 1]. Below, dense(x; L, s) is a conv with kh = kw = 1 and
groups = 1 over x, with layer L’s integer weights (w_bytes 4), and an RQ with no
multiplier, one shift s, L’s integer bias placed pre (b_bytes 8) and round half_up;
T_tanh and T_sig are lut16 with the tables of §4.5; C(·) marks an output tensor whose
policy is clamp [−32767, 32767] (every table input, and every GRU state); mul(a, b) is mul
with an RQ of shift 15, no bias, half_up; [a, b] is concat; every other tensor’s policy is
derived.
init. Input 0 is the initial state [50], policy clamp [−(2^31 − 1), 2^31 − 1].
a = T_tanh(C(dense(IS; hidden_init, 28)))[128];s = T_tanh(C(dense(a; gru_init, 27)))[320];- outputs: the five GRU states
h_k = split(s, 64·(k − 1), 64),k = 1 … 5.
step. Inputs: the latent Z [26], policy clamp [−(2^31 − 1), 2^31 − 1]; the GRU states
h_1 … h_5 [64]; the convolution memories c_1 … c_5 [32]; each of the last ten with policy
clamp [−32767, 32767]. Then:
t = T_tanh(C(dense(Z; dense1, 28)))[96].- For
k = 1 … 5, withx = t,h = h_k,I = gru<k>_inputandU = gru<k>_recurrent:- the gates:
g = T_sig(C(conv([x, h]))),[128], where theconvisdensewith shift 28 whose outputo < 128has the weights of rowoofIfollowed by those of rowoofU, and the biasb_I[o] + b_U[o](each converted by §4.4 on its own);z = split(g, 0, 64),r = split(g, 64, 64); - the candidate:
p = dense(x; rows 128–191 of I, 19) + mul(r, dense(h; rows 128–191 of U, 19))(add), at2^20; thenhc = T_tanh(C(rescale(p))), therescalewith one shift 8, no bias,half_up; - the state:
h′ = C(hc + mul(z, h + rescale⁻(hc))), whererescale⁻isrescalewith one multiplier −1 (m_bytes4), shift 0, no bias,floor, and each+isadd. This is draft §2.2.2’sz·h + (1 − z)·hc, written so that each product has one rounding; with0 ≤ z < 2^15its value lies betweenhcandh, so the clamp changes nothing; - the GLU:
u = mul(h′, T_sig(C(dense(h′; glu<k>, 28)))); t ← [t, u];- the causal convolution:
c = T_tanh(C(dense(t; conv_dense<k>, 27)))[32]andv = T_tanh(C(dense([c_k, c]; conv<k>, 27)))[32]; t ← [t, v]; the newh_kish′and the newc_kisc.
- the gates:
x = dense(t; output, 23)[80], at2^16, policyclamp [−2^31, 2^31 − 1].
Outputs, in order: x, the five new GRU states, the five new memories.
Tensors are numbered as integer-operators §13.0 numbers a lowered graph (inputs first, then each
operator’s output in operator order), every input declares max_h = max_w = 1, every exponent is
none, and the operators are written in the order above (for the GRU: concat, the gates’ conv,
lut16, the two splits, the candidate’s two convs, mul, add, rescale, lut16, then
rescale⁻, add, mul, add). For the pinned set, the graphs’ identities (the SHA-256 of their
WHG1 bytes) are 6527304e48bbd6c972fb41cb24ab5f387a3adee6f88347dfc8f10f47060e5c9d (init)
and 4c76f2e5f6b2657b985aa171f7872fc18ae48b9419447e69cb3e52a3225b34d2 (step).
4.7 The driver
A payload is decoded as a sequence of graph runs:
initon the state input (§4.3); its outputs are the GRU states. The memories are 0.- For latent
l = 0, 1, …, newest first:stepon the latent’s input (§4.3), the GRU states and the memories; its first outputxgives four feature vectors, and the others are the states and memories of the next latent.
The rows of the payload are those vectors, newest first: row 4l + j is
x[20j], …, x[20j + 19] of latent l (draft §2.2.2: the four vectors of a latent are in reverse
chronological order), and each row is 10 ms older than the row before it. Row 0 ends at the
payload’s offset (§3.3 step 3). Each value is the feature at 2^16, an i32.
Decoding the first n latents gives the first 4n rows, the same integers whatever follows:
the network runs newest first, so a row never depends on an older latent. A sink MAY stop after
the rows it needs.
4.8 Profile and fabrics
Every operator of both graphs is in wai-ops/1-i32 (integer-operators §9.1), and both graphs’
certificates meet acc64 and not acc32: a 16-bit activation times a 32-bit weight, summed over
up to 576 inputs, passes 32 bits. The registry’s ops member of wai.audio.opus.dred.int is
{"set": "wai-ops/1", "profile": "acc64", "subset": "full", "lowering": "dred-int/1"}; the
payload’s range decoder is RFC 6716’s and not a stage operator of integer-operators §5E, so the
member has no stage. On a GPU, the reference implementation’s WGSL kernels (wai-ops-gpu) run
both graphs at acc64, and wai-rs/tests/dred_exact_gpu.rs holds their outputs to the CPU’s
graph by graph.
5. A stream under loss
5.1 The loss pattern
A loss pattern is a seed of 32 bytes and two thresholds, enter and stay, each an integer in
0 … 2^16. Packet n (from 0) draws u_n, the first two bytes, big-endian, of
SHA-256("wai:dred-loss\x01" ‖ seed ‖ n as a big-endian u32). It is lost when u_n < enter and
the packet before it was received (or n = 0), and when u_n < stay and the packet before it was
lost.
5.2 Slots
A stream is a run of packets of one frame each, all of duration d (in units of 2.5 ms), so
packet n starts at n·d. Each lost packet’s frame is two 10 ms slots, half 0 (the earlier)
and half 1, and half h of frame m ends at E = m·d + 4·(h + 1). A maximal run of lost
packets is recovered by the packet after it, when that packet was received and carries DRED
whose payload is not refused (§3); then, with A = n·d + dred_offset the end of its row 0 (n
the recovering packet), half h of frame m takes row r = ⌊(A − E) / 4⌋ of it when A − E ≥ 0
and r is less than four times the payload’s latent count. Otherwise the slot is not
covered: no DRED rebuilds it.
5.3 The reconstruction buffer
The reconstruction of a stream is its slots in order (frame ascending, then half), as bytes: for each, the frame (u32), the half (u8) and 1 if it is covered or 0 if not (u8); for a covered one then the recovering packet (u32), the row (u32) and the row’s 20 features (i32 each); every integer little-endian. Its digest is SHA-256 over those bytes. The buffer of one run is the same over that run’s slots alone.
5.4 The record
A record of a reconstruction states:
weights_sha256: the weights file the decode used;lost: the indices of the packets the stream lost, ascending;slots: each slot as[frame, half, packet, row], or[frame, half, null, null]for one not covered;output: the reconstruction’s digest, hexadecimal.
5.5 Verification
A verifier is given the stream (each received packet as the object a live track carries it in:
a WAI1 envelope of wai.audio.opus.packet, with nothing where a packet was lost), its loss
pattern, the record, and the weights file. It checks, in order, and refuses at the first check
that fails:
- the record’s
weights_sha256is the digest of §4.1 (weights_digest_mismatch); - the packets the loss pattern loses are the record’s
lost, and are exactly the packets the stream lacks (loss_pattern_mismatch); - the slots §5.2 gives are the record’s
slots(reconstruction_mismatch); - the digest of the reconstruction §4 gives is the record’s
output(reconstruction_mismatch).
A reconstruction that passes is the one every conforming sink gives: the profile admits no other.
6. Measurement
What the profile changes is the arithmetic: the same network, the same weights, at fixed point.
wai-rs/tests/dred_exact_measure.rs measures how far that moves the features, on the draft’s own
test vectors ([DRED-Test-Vectors]): all eight DRED bit files, 20 records each, every record’s
latents decoded through the profile, and through the network in floating point (f64, from the
same float32 weights, with tanh and the sigmoid computed exactly to f64 precision); and every
packet of the eight packet files that carries DRED, its DRED decoded both ways. Each pair is
compared by the draft’s feature comparison (its §4.1 and its tool [DRED-Compare]): per feature,
the mean squared difference over the frames; total the square root of their sum, max of
their largest, the pitch feature left out of both; pitch the mean absolute pitch difference
over the frames the reference voices. The draft’s DRED decoding test passes at total ≤ 0.5,
max ≤ 0.15 and pitch ≤ 0.02. Two figures are added: the largest absolute difference of any
feature but pitch, and a band log-spectral distance. The 18 cepstral features are the orthonormal
DCT-II of the log10 band energies (draft §2.1.3), so the inverse DCT of their difference is the
difference of the log band energies, and ten times its root mean square over the bands is a
distance in decibels for each frame; its mean over the frames and its largest are given.
The reference features are the test vectors’ own. “8-bit emulation” is the floating-point network
computing each layer the weights file also holds 8-bit weights for as the reference
implementation does when it reads them: each input rounded to ⌊0.5 + 127·x⌋, the 8-bit weights,
each output scaled by its scale.
| compared | frames | total | max | pitch | largest difference | band LSD mean (dB) | band LSD max (dB) |
|---|---|---|---|---|---|---|---|
| integer, reference | 16000 | 0.1314 | 0.0767 | 0.00492 | 0.4139 | 0.2945 | 1.0034 |
| float, reference | 16000 | 0.1314 | 0.0767 | 0.00492 | 0.4151 | 0.2944 | 1.0060 |
| 8-bit emulation, reference | 16000 | 0.0831 | 0.0458 | 0.00255 | 0.2542 | 0.1844 | 0.6929 |
| integer, float | 16000 | 0.0021 | 0.0013 | 0.00011 | 0.0084 | 0.0048 | 0.0210 |
| integer, float (packets) | 80452 | 0.0022 | 0.0013 | 0.00004 | 0.0085 | 0.0049 | 0.0208 |
What the table shows:
- The profile passes the draft’s test against the draft’s reference features, on each of the
eight vectors and over all of them, with
total,maxandpitchwell inside the thresholds. - The profile is close to the network. Against the floating-point network on the same
latents, its features differ by
total0.0021, at most 0.0084 in any feature, 0.005 dB of band log-spectral distance on average: about one sixtieth of the distance between the network and the draft’s reference features. - The reference features are not the network’s. The floating-point network and the profile lie the same distance from them, and the 8-bit emulation lies closer, but not at them. So the draft’s test vectors carry the arithmetic of the build that wrote them.
The last point was also checked once, informatively, on one arm64 host, by building the draft’s
reference implementation ([DRED-Ref-Impl]) twice: in its default configuration, which reads the
8-bit weights, and with its float weights, which it computes in float32 with a rational
approximation of tanh. Over the eight vectors, the default build’s features lie at total
0.090–0.096 from the test vectors and the float build’s at 0.131–0.136; the profile and the
floating-point network each lie at 0.015–0.018 from the float build, and at 0.129–0.135 from the
default build. The two builds differ from each other by about 0.13 (on the three vectors compared
directly), and neither gives the test vectors’ features. Two builds of the draft’s own decoder on
one machine, and the build that wrote the test vectors, give three different sets of features
from the same latents; under the integer profile there is one.
7. Receipts and the synthesis lattice
7.1 Classes
In synthesis’s lattice (§3.2):
- A reconstruction from DRED is a
convert. The features are what the packets carried: a second encoding of the same speech, which a decoder with a pinned parameter set rebuilds (synthesis §3.2: “a decoder’s reconstruction (neural or not)”). The vocoder that renders them continues that reconstruction; it generates no content the packets did not describe. So an object rebuilt from DRED keeps its input’s class,reconstructedfrom a capture. - Concealment beyond DRED is a
generate. Where no DRED covers a slot (§5.2), a sink that fills it with speech predicted by a model (packet loss concealment, which draft §2 names as what DRED’s synthesis may continue from) produces content the packets never carried, by a trained model: that isgenerate, classsynthesized, with the model inmodels. It is notassistive: synthesis’sassistedits an input without substantially altering it, and a concealed slot has no input to edit. - Speech coding enhancement is an
assist. A decoder-side enhancement of decoded speech (draft-ietf-mlcodec-opus-speech-coding-enhancement-04) edits one input without substantially altering it, and that draft’s tests bound how far it may move the decoder’s own output (its §3 and §4); a blind bandwidth extension adds content above 8 kHz the packets did not carry, as an upscale adds detail, which synthesis also classesassist(§3.4). Neither is a pinned derivation of synthesis §4, so their evidence statestierattest.
7.2 The presentation statement
A player that presents a reconstruction under the profile signs it in its log (synthesis §10.3):
for each run of lost packets it presents, a present entry whose
subjectis the content hash (jwp-receipts §2.1) of the recovering packet’s object;tierisdecode-equivalence,decoderiswai.audio.opus.dred.intandvariantisdred-int/1;outputis the digest of that run’s reconstruction buffer (§5.3),originandclassas synthesis §10.3 has them (nullwhere the player applies no claim).
That variant is the lowering wai.audio.opus.dred.int’s registration names. A
reconstruct-session claim of the same decode may name its decoder entry’s path by it, by
exact or by null: each is an exact path of a NeuralIntegerExact capability
(jwp-receipts §2.4.12, variant). Any other term is not, and the entry is refused
(session_variant_not_exact).
A verifier checks the log as synthesis §10.3 does (record_malformed, bad_key, bad_signature,
player_chain_broken), then each present entry: its decoder, variant and tier are those above,
its subject is the content hash of the object that recovers a run of the stream, and its output
is the digest of that run’s buffer that §4 gives. Otherwise the entry states an output the object
does not decode to (present_output_irreproducible, synthesis §9.2). The features are the
statement’s subject because they are what the profile fixes; a player that also states the digest
of its vocoder’s samples states its own output, which no other sink reproduces unless it runs the
same vocoder, exactly.
8. Refusals
The codes this extension adds, each with where it is raised:
| reason | raised at |
|---|---|
packet_malformed | §3.1 |
dred_latent_range | §3.3 step 5 |
weights_digest_mismatch | §4.1, §5.5 step 1 |
loss_pattern_mismatch | §5.5 step 2 |
reconstruction_mismatch | §5.5 steps 3 and 4 |
A set that fails to lower or load is refused model_mismatch (integer-operators §10.5). A
verifier of a presentation statement also raises synthesis’s codes for a log (§7.2).
9. Conformance
live-conformance/dred/ holds the corpus, written from fixed seeds
by wai_dred_vectors gen with the weights file of §4.1, which it names and does not hold:
vectors.json, its README, and fetch.sh. It has:
- The profile’s identities: the tables’ digest (§4.2), the constant and the two tables’ digests (§4.5), and the two graphs’ identities (§4.6).
- Payload vectors, each a packet and what a decoder states of it: for a packet with DRED, its
header, frame offset, offset, latent count, the digest of its integers, the digest of every row
and row 0 itself; for one without, that it has none; for one refused, the code. The digest of a
payload’s integers is SHA-256 over
q0,dQ,X,E,Iandqmax, one byte each; the latent count (u32); the state’s 50 indices (i32 each); then, for each latent, its quantizer (one byte) and its 25 indices (i32 each); little-endian. The digest of the rows is SHA-256 over every row in order, each feature an i32, little-endian. They cover the experimental ID with its prefix and ID 32; a slope with aqmaxbelow 15 and an extended offset; a DRED extension on a packet’s second frame after a frame separator; an experimental extension of another version, a truncated extension and plain padding, which carry none; two malformed packets; and a payload whose magnitude passes the cap. - Streams, each a run of packets as
WAI1objects with the lost ones absent, and a loss pattern, and cases over them, each a record, a player’s log, and the outcome: the reconstruction as the profile gives it, with a log of its recoveries, for two streams (one with a slope and an offset that leaves the newest frames of every run uncovered); and one refused case for each check of §5.5 and §7.2: a record naming another weights file; a record listing a received packet as lost; a record naming a row one older; a digest with one bit changed; the digest of a floating-point decode of the same latents, rounded to2^16; a log presenting that decode atdecode-equivalence; and a log entry whose signature is broken.
Every packet of the corpus is written by its generator: its DRED indices are drawn from the
draft’s own distributions, by decoding a seeded byte stream with §3.3’s coefficient rule at q0,
and requantized to each chunk’s quantizer (x·s_q / s_q0, rounded half up, where q codes the
coefficient). The primary frames are empty (length 0, RFC 6716 §3.2.1). No recorded speech is in
the corpus.
Feature equivalence. For each payload vector and each case, a conforming implementation reaches the outcome the vector states, and for an accepted one, the same digests, rows and statements, byte for byte.
On every fabric. wai-rs/tests/dred_exact_gpu.rs decodes the corpus’s payloads with every
graph on each GPU adapter found and holds the output to the CPU’s.
10. Security and privacy considerations
- Bounded work. A payload’s work is bounded by its length and by the magnitude cap (§3.3): without the cap, the escape rule could read without end past the payload’s last byte. A sink SHOULD also bound the latents it decodes to the span it needs (§4.7), as the draft lets it (its §3: “decoders MAY ignore any redundancy data beyond a certain amount”).
- No floating point in the decode. The profile’s decode forms no float, so the denormal slow-downs draft §6 warns of do not arise in it.
- Speech before joining. DRED can carry speech from before a receiver joined (draft §6). A
reconstruction record names the packets and slots it rebuilt; a verifier given a room’s
history learns nothing from a record that its packets did not already carry, but a relay that
forwards DRED forwards that speech, and the draft’s
ext32-dred-durationlimits apply. - What a statement proves. A presentation statement over a reconstruction’s digest proves which features the player’s decode fixed, from packets the verifier holds. It does not prove that the packets are authentic or that the samples a listener heard are those features rendered; the object’s own receipts (jwp-receipts) and the vocoder’s exactness are separate matters.