WAI Extension: World Voices (each voice heard from where its speaker stands)
Mirrored from the canonical text at commit 498e6b88 ().
Status: Draft. Registers the record
wai.live.voices: the room authority’s signed voice statement,WLV1, that a room carrying a shared world (live-worlds) renders each participant’s sound as an audio object at that participant’s place in the world, relative to each listener’s own place. The statement fixes the place rule (the world’s extent, the length of a move, how a listener faces) and the voice law (distance attenuation, a cutoff, a cap on the voices mixed, and an integer panning-and-delay law for two ears). A participant’s place is a pure function of the world’s admitted operations up to the tick of an instant (§4); the gain, the level difference and the time difference of each voice for a listener are integer functions of the two places (§5); and the stereo samples a listener hears are an integer mix of the voices’ decoded sound under those values (§6), so two sinks holding the same world, the same decoded sound and the same statement mix the same samples: spatial-audio’s mixdown-equivalence (§4 there), applied live, on live-compose §6’s mixer. No new track, no new receipt: the world tracks are live-worlds’, the sound tracks are the room’s. The reference implementation is theworld_voicemodule ofwai-rs(featureworld_voice); the corpus islive-conformance/voices/, and its second implementation islive-conformance/world_voices_verify.py. Keywords MUST, MUST NOT, SHOULD, MAY are RFC 2119/8174.Texts read for this revision: live-worlds, interactive-worlds (its operation, §5.1, and canonical order, §5.2), live-compose (its audio mix, §6), spatial-audio (its render contract and mixdown-equivalence, §3 and §4), live-rooms and live-events as merged; RFC 8032 and FIPS 180-4. No outside text or data is pinned: the voice law is stated here in full (§5, decision D125). The field’s measured numbers it was chosen against: a listener’s own head-related transfer functions showed no global benefit over another’s in the published latency-detection studies; consumer platforms keep personal transfer functions closed, and the one open exchange format (AES69, SOFA) carries measured sets, not a rendering rule; and a shared world is co-present only while its sound keeps up, at a few tens of milliseconds.
1. Scope
In scope: the voice statement, its signature and its rules (§3); places: the place body an actor’s action carries, ticks on the room clock, and an actor’s place at a tick (§4); the voice law: the listener’s frame, distance and gain, level and time difference, which voices are mixed, and each voice’s coefficients (§5); rendering a block of the mix, with its ramps and its history (§6); why this law (§7); where it runs (§8); verification (§9).
Out of scope: a transport; how a page captures, encodes and decodes sound, which live-rooms §8 profiles bound; how a world’s model renders what it shows (interactive-worlds §6, SPEC §5); a venue’s acoustics, elevation and spectral cues (§7); and how a participant chooses where to stand.
Relationship to the core spec: wai.live.voices is a record (SPEC §5): a
signed record a WAI tool emits and verifies. It names no payload format and is not
dispatchable.
Relationship to live-worlds: nothing in it changes. A room has at most one voice statement, and only beside its world statement. The world’s operations, tracks, receipts, verification and session hash are live-worlds’; this extension reads the ops a participant has admitted (live-worlds §9.2) and derives places from them. A participant that does not implement this extension holds the same world and hears the room as live-compose mixes it.
Relationship to live-compose: a voice’s gain and its balance are §6’s gain and pan, and its output sample is §6’s sum and shift: with no time difference and no change between blocks, one voice renders exactly as §6 mixes it (§6.3 here). This extension adds the far ear’s delay, a ramp from one block’s values to the next, and the rule that sets gain and pan from places instead of from the layout.
Relationship to spatial-audio: a voice is an audio object whose trajectory is its speaker’s place, sampled at each block’s instant, rendered to a declared layout, two ears under the law of §5; conformance is mixdown-equivalence over the decoded sound (§9.3), not a perceptual similarity. spatial-audio’s own scene, objects and floor are unchanged.
2. Terms
- Room, room authority, room clock, principal, grant: as in live-rooms §2.
- World, op, actor, admitted op: as in live-worlds §2. An action is an op of
wai.world.modelas live-worlds §9.2 step 5 reads it: mutation kind 1, no dependency, and a body that reads (dxi32,dyi32,buttonsu32,payload_lenu32 and the payload, model-generation §3.2). Its move is its body’sdxanddy, and its payload the body’s free bytes. The place rule (§4.3) is this extension’s reading of a move, whatever a world’s model makes of it. - Place: a point of the world’s floor, two integers in millimetres, and a heading, one of four directions (§4.1).
- Listener: the participant a mix is for, at a listening place: its own place when it is an actor, or a place it names when it is not (§5.1).
- Voice: a principal’s sound, decoded, as one mono source (§6.1).
- Block: N consecutive frames of the mix, N the statement’s
block_frames, and its instant, the room-clock instant its values are taken at (§6.2). - Q12: an integer in units of 1/4096; 4096 is one.
3. The voice statement
3.1 Layout
Integers are little-endian.
| field | size | meaning |
|---|---|---|
magic | 4 | WLV1 (ASCII) |
version | 1 | 1 in this revision |
room | 32 | the room id (live-rooms §3.2) |
law | 1 | the voice law: 1 itd_ild, the law of §5, the only one in this revision |
orientation | 1 | how a listener faces (§5.1): 0 map, the world’s own up; 1 heading, its own heading |
extent_x | 4 | u32: the floor’s width in millimetres, places from 0 to extent_x, 1 to 16 777 215 |
extent_y | 4 | u32: its depth, likewise |
step_mm | 4 | u32: the millimetres one unit of a move takes a place (§4.3), 1 to 16 777 215 |
sample_rate | 4 | u32: the frames a second of every voice and of the mix, 8 000 to 192 000 |
block_frames | 2 | u16: N, the frames of a block (§6.2), 16 to 9 600 |
ref_mm | 4 | u32: the radius within which a voice is at full gain (§5.2), at least 1 |
cutoff_mm | 4 | u32: the distance at and beyond which a voice is not mixed (§5.2), above ref_mm and at most 33 554 432 |
rear_q12 | 2 | u16: Q12 gain of a voice behind the listener (§5.2), 0 to 4096 |
ild_q12 | 2 | u16: Q12 depth of the level difference at a voice wholly to one side (§5.3), 0 to 4096 |
itd_max | 2 | u16: frames of the time difference at a voice wholly to one side (§5.3), 0 to 255 |
max_sources | 1 | the most voices mixed for one listener at once (§5.4), 0 to 64 |
sig | 64 | §3.2 |
A statement is 136 bytes.
3.2 Digest and signature
The digest of a statement is SHA-256 (FIPS 180-4) over its record from magic through
max_sources. The signature is Ed25519 (RFC 8032) by the room descriptor’s authority
over the 16 bytes wai:live-voices (ASCII) and 0x01, followed by the bytes the digest covers.
It is verified strictly, as SPEC §7 Signatures requires: a signature that fails
is bad_signature.
3.3 Reading and rules
A reader reads exactly the bytes it is given as one statement, in the order of §3.1, and refuses at the first of these that holds:
- Fewer than 4 bytes:
truncated. A magic other thanWLV1:malformed. No byte after the magic:truncated. A version other than 1:unknown_version. - The bytes end before a field the layout requires:
truncated. Bytes followsig:malformed. lawis not 1:voice_law_unknown.- A field breaks its rule in §3.1’s table (
orientation0 or 1, every range as stated):voice_statement_invalid. roomis not the room id of the descriptor the statement is read beside:wrong_room.- The signature does not verify under the descriptor’s
authority(§3.2):bad_signature.
A room has at most one voice statement. A voice statement is read only beside the room’s world
statement (live-worlds §3): a verifier given one with no world statement refuses it as
voices_without_world.
4. Places
4.1 The place body
An actor states where it stands by an action whose payload begins with the 13 bytes of a place body:
| field | size | meaning |
|---|---|---|
magic | 4 | WVP1 (ASCII) |
x | 4 | u32: millimetres from the floor’s left edge |
y | 4 | u32: millimetres from its top edge |
heading | 1 | 0 up (toward smaller y), 1 right (larger x), 2 down (larger y), 3 left (smaller x) |
A payload that is shorter than 13 bytes, whose first four bytes are not WVP1, or whose
heading is above 3, holds no place body. The bytes after the 13 are not read here: another
extension may define them. The
floor’s axes are the world’s own: x grows to the right and y downward, as a map is drawn.
4.2 Ticks on the room clock
This extension reads a wai.world.model world’s ticks as its frames, on the room clock: with
the world statement’s frame rate r (live-worlds §3.2), the tick of room-clock instant t
(microseconds) is ⌊t × r / 10⁶⌋, computed without overflow (for example as
⌊t / 10⁶⌋ × r + ⌊(t mod 10⁶) × r / 10⁶⌋). An actor that wants an op to apply from
an instant gives it that instant’s tick as its effect_tick (interactive-worlds §5.1).
4.3 An actor’s place at a tick
An actor’s place at tick k is what the following gives over the actor’s admitted actions
whose effect_tick is at most k, taken in canonical order (interactive-worlds §5.2), from no
place:
- An action whose payload holds a place body sets the place to (min(
x,extent_x), min(y,extent_y)) and itsheading. Its move is not applied. - Any other action, when the actor has a place, moves it:
xbecomesx+dx×step_mmandybecomesy+dy×step_mm, each then held to 0 …extent_xand 0 …extent_y(computed in integers wide enough not to overflow: at least 64 bits). When the move is not (0, 0) the heading becomes the move’s: right when |dx| ≥ |dy| anddx> 0, left when |dx| ≥ |dy| anddx< 0, otherwise down whendy> 0 and up whendy< 0. - Any other action, when the actor has no place, changes nothing.
An actor whose actions up to k hold no place body has no place at k, and is unplaced. A place is a function of the admitted ops alone: two participants holding the same ops hold the same places at every tick, whatever order the ops arrived in.
5. The voice law itd_ild
5.1 The listener and its frame
A listener is either an actor, whose listening place is its own place at the tick (§4.3) and whose own voice is never mixed for it, or a participant that is not an actor (a viewer), which names a listening place of its own choosing: a point, two u32 in millimetres taken as given and not held to the floor, and a heading, 0 to 3. An actor that is unplaced at a tick hears no voice at that tick (§6.4).
For a voice placed at (xₛ, yₛ) and a listener at (xₗ, yₗ), with Δx = xₛ − xₗ and Δy = yₛ − yₗ, the voice’s position in the listener’s frame is r, toward the listener’s right, and f, toward its front:
| the listener faces | r | f |
|---|---|---|
| up (0) | Δx | −Δy |
| right (1) | Δy | Δx |
| down (2) | −Δx | Δy |
| left (3) | −Δy | −Δx |
Under orientation 0 (map) every listener faces up, whatever its heading; under 1
(heading) it faces its own heading. The distance d is ⌊√(r² + f²)⌋, the integer
square root.
5.2 Distance and gain
A voice at a distance at or beyond cutoff_mm is out of range and is not mixed. Within
range, its gain g, in Q12, is:
- gₐ = 4096 when d ≤
ref_mm, and ⌊4096 ×ref_mm/ d⌋ beyond it: the gain halves as the distance doubles; - g = ⌊gₐ ×
rear_q12/ 4096⌋ when f < 0, the voice behind the listener, and gₐ otherwise.
The rear gain is the law’s one cue of front and back (§7).
5.3 Level difference and time difference
The voice’s lateral sine s, in Q12 from −4096 (wholly to the left) to 4096 (wholly to the right), is 0 when d is 0, and otherwise sign(r) × ⌊(4096 × |r| + ⌊d / 2⌋) / d⌋. Since |r| ≤ d, |s| ≤ 4096.
- Its pan p is sign(s) × ⌊(
ild_q12× |s| + 2048) / 4096⌋, from −4096 to 4096: live-compose §6’s pan, a balance that leaves the near ear at the voice’s gain and attenuates the far ear, byild_q12/ 4096 at a voice wholly to one side. - Its time difference e is ⌊(
itd_max× |s| + 2048) / 4096⌋ frames: the far ear (the left when s > 0, the right when s < 0) hears the voice e frames later than the near ear, whose delay is 0. When s is 0 both delays are 0.
5.4 Which voices are mixed
At a block’s tick, the candidates for a listener are the principals that are placed and in
range, the listener itself excluded, whether or not their sound is held. They are ordered by
distance, and principals at one distance by their keys, ascending bytewise; the first
max_sources are mixed. A voice’s place, not its loudness, decides: the choice is a
function of the world and the listener alone.
5.5 The coefficients
A mixed voice’s coefficient for each ear is its gain times that ear’s balance, live-compose §6’s product: with aL = 4096 − max(p, 0) and aR = 4096 + min(p, 0), cL = g × aL and cR = g × aR, each from 0 to 2²⁴. Its delay for each ear is e for the far ear and 0 for the near one, as §5.3 has it. The four values (cL, cR, eL, eR) are the voice’s values at the block.
6. Rendering
6.1 The voices
Each voice is one mono source at sample_rate. A decoded block of two channels is mixed down
first: each frame’s sample is ⌊(xL + xR) / 2⌋ (an arithmetic shift right by 1 of the sum).
A voice’s stream is its blocks in the order the renderer renders them, each N frames; a
block for which the renderer holds none of a voice’s sound is N zero samples of that voice’s
stream. A frame before the stream’s first block is 0.
6.2 One block
A renderer renders blocks in order, each at an instant; the instants increase, and a renderer
given an instant at or below the previous one refuses it as voice_instant_regress. For block
b the renderer takes the values of §5 at the tick of b’s instant (§4.2), and remembers those
of the block before (none for the first). The voices of b are those mixed at b or at the
block before. For each, with N the block’s frames:
- a voice mixed at both has its previous values and its values at b;
- a voice mixed only at b has previous coefficients of 0 and previous delays equal to its delays at b: it fades in;
- a voice mixed only at the block before has coefficients at b of 0 and delays at b equal to its previous delays: it fades out.
At frame n of the block, 0 ≤ n < N, each ear’s coefficient and delay run from the previous value to the value at b: c(n) = ⌊(cprev × (N − n) + c × n) / N⌋ and e(n) = ⌊(eprev × (N − n) + e × n) / N⌋, so that a voice that moves does not click and one whose values do not change keeps them. The ear’s term is x × c(n), where x is the voice’s stream sample e(n) frames before frame n of the block, read across the block’s start into the blocks before when e(n) > n.
Each output sample, left and then right, interleaved, is the sum s of its ear’s terms over the block’s voices, in signed 64-bit integers, then ⌊(s + 2²³) / 2²⁴⌋ (the arithmetic shift right by 24 bits of s + 2²³), saturated once to −32 768 … 32 767: live-compose §6’s output rule. A term is at most 2³⁹ in magnitude and a block has at most 128 voices, so the sum is exact and its order changes nothing.
6.3 One voice, held still
A voice mixed at a block and at the block before with the same values, whose time difference is 0, renders each frame’s sample as live-compose §6 mixes a mono voice of gain g and pan p into two channels: the same products, the same shift, the same saturation.
6.4 What a renderer reports
For each block a renderer states: the mixed voices in the order §5.4 mixes them, each with its principal, its distance, g, p, e and the far ear (by the sign of s, §5.3, whatever e is; none when s is 0); the voices fading out; the principals other than the listener whose sound it holds for the block that are unplaced; the mixed voices whose sound it does not hold for the block, mixed as silence (silent), in mix order; whether the listener is unplaced; and the block’s samples. The voices fading out and the unplaced principals are each listed in ascending bytewise order of their keys. An unplaced principal’s sound is not mixed, and a silent voice keeps its place in the mix: neither is an error, and neither is a voice at a gain of 0.
A renderer refuses as live-compose §6’s mix_input_malformed a block of a voice that is not
N frames of 1 or 2 channels, and a block that gives one principal’s sound twice.
7. Why this law
A binaural mix needs a rule for each ear. Two kinds were weighed (decision D125):
- A pinned set of head-related transfer functions, a measured SOFA set (AES69) under an open licence, convolved per voice per ear. Bit-exact convolution needs the set’s impulse responses in integers and an integer rule between its measured directions; at 64 voices and responses of 256 taps it is about 1.6 × 10⁹ multiply-adds a second, beyond the audio thread of a page on a phone. A measured set is still another person’s head: the published studies found no global benefit of a listener’s own set over another’s for detecting latency, and the field offers no exchange of personal sets.
- A stated integer law of level and time difference, which this revision registers as
itd_ild: the time difference by a sine law (§5.3) whose maximum the statement states (31 frames at 48 kHz is the spherical-head maximum, (a / c)(π/2 + 1) ≈ 0.656 ms for a head radius a of 87.5 mm and c = 343 m/s), the level difference as live-compose §6’s balance at a stated depth, inverse-distance gain from a stated radius with a hard cutoff, and a stated rear gain. It costs two multiply-adds per voice per frame, needs no data from outside, is byte-exact across machines, and its every value fits 53 bits, so a page computes it in its worklet with ordinary numbers.
The law gives up elevation, the spectral cues of the outer ear and most of front and back (only
the rear gain tells them apart), and its sine law gives a voice 30° to one side 0.33 ms where a
spherical head gives 0.26 ms. A later law, for example a pinned set under integer convolution,
takes another law value and changes nothing of §4.
8. Where it runs
A participant’s page that holds the room’s world (live-worlds §9) and its voice statement computes each block’s values at the block’s instant from the places of the ops it has admitted, and renders the block in its audio worklet. A room service gives the statement beside the world statement. A page that has admitted fewer ops than another holds other places for a while, and mixes other samples while it does; once the two hold the same ops, they mix the same samples for the same blocks. A recording of a world room renders again, block for block, from the world’s ops and the decoded sound.
9. Verification
9.1 Inputs
A verifier is given: the room’s descriptor; its world statement; the voice statement; the world’s admitted ops (live-worlds §9.2), in any order, an op given twice byte for byte being one op; the listener, an actor’s key or a listening place; and the blocks, in order, each with its instant and, for each principal whose sound it holds, that block of decoded sound with its channel count.
9.2 Steps
The verifier reads the descriptor (live-rooms §3.4), refusing as it does and naming the
descriptor; refuses a voice statement given with no world statement as voices_without_world,
naming the statement; reads the world statement beside the descriptor (live-worlds §3.4),
refusing as it does and naming the world statement; reads the voice statement (§3.3), refusing at
its first failure and naming the statement; reads each op in the order given
(interactive-worlds §5.1), refusing one that is not exactly one op as op_malformed, one that
is not an action (§2) as op_not_action, and one with the op id of an op given before it and
other bytes as equivocation (live-worlds §9.2 steps 4 and 5), naming the op; and then renders
each block in order (§6), refusing at the first block that breaks §6.2 or §6.4, its instant
checked before its sound, and naming it.
9.3 Outcomes
A verification that refuses states the reason and where. One that passes states, for each block, what §6.4 lists. Two verifications of the same inputs state the same; the samples are mixdown-equivalent (spatial-audio §4) over the decoded sound: they are byte-exact given the same decoded blocks, and as portable as the voices’ decoding is (live-compose §6).
10. Reasons
The codes this extension adds, each with where it is raised:
| reason | raised at |
|---|---|
voice_law_unknown | §3.3 step 3 |
voice_statement_invalid | §3.3 step 4 |
voices_without_world | §3.3 |
voice_instant_regress | §6.2 |
Of live-events §9’s codes, a verifier also raises truncated, malformed and
unknown_version when a statement does not read, bad_signature for its signature, and
wrong_room for a statement of another room (§3.3); of live-worlds §11’s, op_malformed,
op_not_action and equivocation (§9.2); and of live-compose §10’s, mix_input_malformed
(§6.4).
11. Conformance
live-conformance/voices/ holds the corpus, in one file,
vectors.json, written from fixed seeds by wai_world_voice_vectors gen. Each vector gives the
descriptor, the world statement, the voice statement, the ops, the listener and the blocks, and
either the outcome of §9.3, every block’s report and samples, or the refusal, its reason and
where:
- Accepted: a listener among four placed actors, one to each side, one ahead and one
behind, under each orientation; actors that move and turn across blocks, ramping; a move held
at the floor’s edge and a place body beyond it held to it; ops whose ticks come after a block,
which the block does not yet apply; a voice beyond the cutoff and one at the reference radius;
more candidates than
max_sources, ties broken by key; a viewer at a listening place of its own; stereo voices mixed down; a voice held still at the centre, which renders as live-compose §6; a block of real length, 960 frames at 48 kHz; the place rule’s edges (ops given out of canonical order and one given twice, a move before any place, a place body carrying a move and one followed by more bytes, payloads that hold no place body, a diagonal move, instants between ticks, a listener facing left, one whose headingmapignores); a voice to one side with no time difference; and two voices fading out and two unplaced in one block. - Gap: a voice with sound and no place (unplaced), a mixed voice with no sound for a block (silent), and a listener that is unplaced.
- Refused: each code §10 adds that a verifier raises, the codes of §3.3 a statement can
reach and every bound of §3.1’s table at both ends,
op_malformed,op_not_action,equivocationandmix_input_malformed, each vector a control of one defect.
A conforming verifier gives every vector its outcome, every block’s report and samples, or the
reason and the place. wai_world_voice_vectors verify and the test tests/world_voice.rs
check that the file regenerates byte for byte, that wai-rs gives every vector its outcome, that
every reason of §10 a verifier raises appears, and that §10 lists exactly the reference’s codes.
live-conformance/world_voices_verify.py is the second implementation, written from this text:
it shares no code with wai-rs; it checks every signature with WAI’s strict rule
(claim-conformance/_ed25519.py); and it mutates copies of the corpus and requires every copy to
fail, and drops each of its own checks in turn and requires the corpus to catch the weakened
verifier. The reference room client’s web/world/voices.js renders each block from the corpus
in Node and in a page’s audio worklet, and its tests require the corpus’s samples.
12. Security and privacy considerations
- What a place proves. A place is its actor’s own claim, signed in its ops; nothing in this extension checks that an actor is where it says, and a world’s model may give a place no meaning. It decides only how loud and from where a voice is heard.
- Crowding a listener. An actor that stands close to a listener takes a place in its mix;
max_sourcesbounds how many can, and the room authority bounds who acts by the grants it issues. A listener that wants a voice out of its mix mutes it locally: muting is the sink’s choice, outside the mix this text holds sinks to. - Loudness. A voice’s gain never exceeds its decoded level (g ≤ 4096), so standing close makes a voice no louder than it was sent; many voices together saturate as live-compose §6 does.
- Privacy. A public world’s places, and so who stood near whom and when, are visible to every participant and every relay, as the ops are (live-worlds §13). In a private world they are its members’ alone.