Skip to main content

WAI Extension: World Voices (each voice heard from where its speaker stands)

Mirrored from the canonical text at commit 498e6b88 ().

Status: Draft. Registers the record wai.live.voices: the room authority’s signed voice statement, WLV1, that a room carrying a shared world (live-worlds) renders each participant’s sound as an audio object at that participant’s place in the world, relative to each listener’s own place. The statement fixes the place rule (the world’s extent, the length of a move, how a listener faces) and the voice law (distance attenuation, a cutoff, a cap on the voices mixed, and an integer panning-and-delay law for two ears). A participant’s place is a pure function of the world’s admitted operations up to the tick of an instant (§4); the gain, the level difference and the time difference of each voice for a listener are integer functions of the two places (§5); and the stereo samples a listener hears are an integer mix of the voices’ decoded sound under those values (§6), so two sinks holding the same world, the same decoded sound and the same statement mix the same samples: spatial-audio’s mixdown-equivalence (§4 there), applied live, on live-compose §6’s mixer. No new track, no new receipt: the world tracks are live-worlds’, the sound tracks are the room’s. The reference implementation is the world_voice module of wai-rs (feature world_voice); the corpus is live-conformance/voices/, and its second implementation is live-conformance/world_voices_verify.py. Keywords MUST, MUST NOT, SHOULD, MAY are RFC 2119/8174.

Texts read for this revision: live-worlds, interactive-worlds (its operation, §5.1, and canonical order, §5.2), live-compose (its audio mix, §6), spatial-audio (its render contract and mixdown-equivalence, §3 and §4), live-rooms and live-events as merged; RFC 8032 and FIPS 180-4. No outside text or data is pinned: the voice law is stated here in full (§5, decision D125). The field’s measured numbers it was chosen against: a listener’s own head-related transfer functions showed no global benefit over another’s in the published latency-detection studies; consumer platforms keep personal transfer functions closed, and the one open exchange format (AES69, SOFA) carries measured sets, not a rendering rule; and a shared world is co-present only while its sound keeps up, at a few tens of milliseconds.

1. Scope

In scope: the voice statement, its signature and its rules (§3); places: the place body an actor’s action carries, ticks on the room clock, and an actor’s place at a tick (§4); the voice law: the listener’s frame, distance and gain, level and time difference, which voices are mixed, and each voice’s coefficients (§5); rendering a block of the mix, with its ramps and its history (§6); why this law (§7); where it runs (§8); verification (§9).

Out of scope: a transport; how a page captures, encodes and decodes sound, which live-rooms §8 profiles bound; how a world’s model renders what it shows (interactive-worlds §6, SPEC §5); a venue’s acoustics, elevation and spectral cues (§7); and how a participant chooses where to stand.

Relationship to the core spec: wai.live.voices is a record (SPEC §5): a signed record a WAI tool emits and verifies. It names no payload format and is not dispatchable.

Relationship to live-worlds: nothing in it changes. A room has at most one voice statement, and only beside its world statement. The world’s operations, tracks, receipts, verification and session hash are live-worlds’; this extension reads the ops a participant has admitted (live-worlds §9.2) and derives places from them. A participant that does not implement this extension holds the same world and hears the room as live-compose mixes it.

Relationship to live-compose: a voice’s gain and its balance are §6’s gain and pan, and its output sample is §6’s sum and shift: with no time difference and no change between blocks, one voice renders exactly as §6 mixes it (§6.3 here). This extension adds the far ear’s delay, a ramp from one block’s values to the next, and the rule that sets gain and pan from places instead of from the layout.

Relationship to spatial-audio: a voice is an audio object whose trajectory is its speaker’s place, sampled at each block’s instant, rendered to a declared layout, two ears under the law of §5; conformance is mixdown-equivalence over the decoded sound (§9.3), not a perceptual similarity. spatial-audio’s own scene, objects and floor are unchanged.

2. Terms

3. The voice statement

3.1 Layout

Integers are little-endian.

fieldsizemeaning
magic4WLV1 (ASCII)
version11 in this revision
room32the room id (live-rooms §3.2)
law1the voice law: 1 itd_ild, the law of §5, the only one in this revision
orientation1how a listener faces (§5.1): 0 map, the world’s own up; 1 heading, its own heading
extent_x4u32: the floor’s width in millimetres, places from 0 to extent_x, 1 to 16 777 215
extent_y4u32: its depth, likewise
step_mm4u32: the millimetres one unit of a move takes a place (§4.3), 1 to 16 777 215
sample_rate4u32: the frames a second of every voice and of the mix, 8 000 to 192 000
block_frames2u16: N, the frames of a block (§6.2), 16 to 9 600
ref_mm4u32: the radius within which a voice is at full gain (§5.2), at least 1
cutoff_mm4u32: the distance at and beyond which a voice is not mixed (§5.2), above ref_mm and at most 33 554 432
rear_q122u16: Q12 gain of a voice behind the listener (§5.2), 0 to 4096
ild_q122u16: Q12 depth of the level difference at a voice wholly to one side (§5.3), 0 to 4096
itd_max2u16: frames of the time difference at a voice wholly to one side (§5.3), 0 to 255
max_sources1the most voices mixed for one listener at once (§5.4), 0 to 64
sig64§3.2

A statement is 136 bytes.

3.2 Digest and signature

The digest of a statement is SHA-256 (FIPS 180-4) over its record from magic through max_sources. The signature is Ed25519 (RFC 8032) by the room descriptor’s authority over the 16 bytes wai:live-voices (ASCII) and 0x01, followed by the bytes the digest covers. It is verified strictly, as SPEC §7 Signatures requires: a signature that fails is bad_signature.

3.3 Reading and rules

A reader reads exactly the bytes it is given as one statement, in the order of §3.1, and refuses at the first of these that holds:

  1. Fewer than 4 bytes: truncated. A magic other than WLV1: malformed. No byte after the magic: truncated. A version other than 1: unknown_version.
  2. The bytes end before a field the layout requires: truncated. Bytes follow sig: malformed.
  3. law is not 1: voice_law_unknown.
  4. A field breaks its rule in §3.1’s table (orientation 0 or 1, every range as stated): voice_statement_invalid.
  5. room is not the room id of the descriptor the statement is read beside: wrong_room.
  6. The signature does not verify under the descriptor’s authority (§3.2): bad_signature.

A room has at most one voice statement. A voice statement is read only beside the room’s world statement (live-worlds §3): a verifier given one with no world statement refuses it as voices_without_world.

4. Places

4.1 The place body

An actor states where it stands by an action whose payload begins with the 13 bytes of a place body:

fieldsizemeaning
magic4WVP1 (ASCII)
x4u32: millimetres from the floor’s left edge
y4u32: millimetres from its top edge
heading10 up (toward smaller y), 1 right (larger x), 2 down (larger y), 3 left (smaller x)

A payload that is shorter than 13 bytes, whose first four bytes are not WVP1, or whose heading is above 3, holds no place body. The bytes after the 13 are not read here: another extension may define them. The floor’s axes are the world’s own: x grows to the right and y downward, as a map is drawn.

4.2 Ticks on the room clock

This extension reads a wai.world.model world’s ticks as its frames, on the room clock: with the world statement’s frame rate r (live-worlds §3.2), the tick of room-clock instant t (microseconds) is ⌊t × r / 10⁶⌋, computed without overflow (for example as ⌊t / 10⁶⌋ × r + ⌊(t mod 10⁶) × r / 10⁶⌋). An actor that wants an op to apply from an instant gives it that instant’s tick as its effect_tick (interactive-worlds §5.1).

4.3 An actor’s place at a tick

An actor’s place at tick k is what the following gives over the actor’s admitted actions whose effect_tick is at most k, taken in canonical order (interactive-worlds §5.2), from no place:

An actor whose actions up to k hold no place body has no place at k, and is unplaced. A place is a function of the admitted ops alone: two participants holding the same ops hold the same places at every tick, whatever order the ops arrived in.

5. The voice law itd_ild

5.1 The listener and its frame

A listener is either an actor, whose listening place is its own place at the tick (§4.3) and whose own voice is never mixed for it, or a participant that is not an actor (a viewer), which names a listening place of its own choosing: a point, two u32 in millimetres taken as given and not held to the floor, and a heading, 0 to 3. An actor that is unplaced at a tick hears no voice at that tick (§6.4).

For a voice placed at (xₛ, yₛ) and a listener at (xₗ, yₗ), with Δx = xₛ − xₗ and Δy = yₛ − yₗ, the voice’s position in the listener’s frame is r, toward the listener’s right, and f, toward its front:

the listener facesrf
up (0)Δx−Δy
right (1)ΔyΔx
down (2)−ΔxΔy
left (3)−Δy−Δx

Under orientation 0 (map) every listener faces up, whatever its heading; under 1 (heading) it faces its own heading. The distance d is ⌊√(r² + f²)⌋, the integer square root.

5.2 Distance and gain

A voice at a distance at or beyond cutoff_mm is out of range and is not mixed. Within range, its gain g, in Q12, is:

The rear gain is the law’s one cue of front and back (§7).

5.3 Level difference and time difference

The voice’s lateral sine s, in Q12 from −4096 (wholly to the left) to 4096 (wholly to the right), is 0 when d is 0, and otherwise sign(r) × ⌊(4096 × |r| + ⌊d / 2⌋) / d⌋. Since |r| ≤ d, |s| ≤ 4096.

5.4 Which voices are mixed

At a block’s tick, the candidates for a listener are the principals that are placed and in range, the listener itself excluded, whether or not their sound is held. They are ordered by distance, and principals at one distance by their keys, ascending bytewise; the first max_sources are mixed. A voice’s place, not its loudness, decides: the choice is a function of the world and the listener alone.

5.5 The coefficients

A mixed voice’s coefficient for each ear is its gain times that ear’s balance, live-compose §6’s product: with aL = 4096 − max(p, 0) and aR = 4096 + min(p, 0), cL = g × aL and cR = g × aR, each from 0 to 2²⁴. Its delay for each ear is e for the far ear and 0 for the near one, as §5.3 has it. The four values (cL, cR, eL, eR) are the voice’s values at the block.

6. Rendering

6.1 The voices

Each voice is one mono source at sample_rate. A decoded block of two channels is mixed down first: each frame’s sample is ⌊(xL + xR) / 2⌋ (an arithmetic shift right by 1 of the sum). A voice’s stream is its blocks in the order the renderer renders them, each N frames; a block for which the renderer holds none of a voice’s sound is N zero samples of that voice’s stream. A frame before the stream’s first block is 0.

6.2 One block

A renderer renders blocks in order, each at an instant; the instants increase, and a renderer given an instant at or below the previous one refuses it as voice_instant_regress. For block b the renderer takes the values of §5 at the tick of b’s instant (§4.2), and remembers those of the block before (none for the first). The voices of b are those mixed at b or at the block before. For each, with N the block’s frames:

At frame n of the block, 0 ≤ n < N, each ear’s coefficient and delay run from the previous value to the value at b: c(n) = ⌊(cprev × (N − n) + c × n) / N⌋ and e(n) = ⌊(eprev × (N − n) + e × n) / N⌋, so that a voice that moves does not click and one whose values do not change keeps them. The ear’s term is x × c(n), where x is the voice’s stream sample e(n) frames before frame n of the block, read across the block’s start into the blocks before when e(n) > n.

Each output sample, left and then right, interleaved, is the sum s of its ear’s terms over the block’s voices, in signed 64-bit integers, then ⌊(s + 2²³) / 2²⁴⌋ (the arithmetic shift right by 24 bits of s + 2²³), saturated once to −32 768 … 32 767: live-compose §6’s output rule. A term is at most 2³⁹ in magnitude and a block has at most 128 voices, so the sum is exact and its order changes nothing.

6.3 One voice, held still

A voice mixed at a block and at the block before with the same values, whose time difference is 0, renders each frame’s sample as live-compose §6 mixes a mono voice of gain g and pan p into two channels: the same products, the same shift, the same saturation.

6.4 What a renderer reports

For each block a renderer states: the mixed voices in the order §5.4 mixes them, each with its principal, its distance, g, p, e and the far ear (by the sign of s, §5.3, whatever e is; none when s is 0); the voices fading out; the principals other than the listener whose sound it holds for the block that are unplaced; the mixed voices whose sound it does not hold for the block, mixed as silence (silent), in mix order; whether the listener is unplaced; and the block’s samples. The voices fading out and the unplaced principals are each listed in ascending bytewise order of their keys. An unplaced principal’s sound is not mixed, and a silent voice keeps its place in the mix: neither is an error, and neither is a voice at a gain of 0.

A renderer refuses as live-compose §6’s mix_input_malformed a block of a voice that is not N frames of 1 or 2 channels, and a block that gives one principal’s sound twice.

7. Why this law

A binaural mix needs a rule for each ear. Two kinds were weighed (decision D125):

The law gives up elevation, the spectral cues of the outer ear and most of front and back (only the rear gain tells them apart), and its sine law gives a voice 30° to one side 0.33 ms where a spherical head gives 0.26 ms. A later law, for example a pinned set under integer convolution, takes another law value and changes nothing of §4.

8. Where it runs

A participant’s page that holds the room’s world (live-worlds §9) and its voice statement computes each block’s values at the block’s instant from the places of the ops it has admitted, and renders the block in its audio worklet. A room service gives the statement beside the world statement. A page that has admitted fewer ops than another holds other places for a while, and mixes other samples while it does; once the two hold the same ops, they mix the same samples for the same blocks. A recording of a world room renders again, block for block, from the world’s ops and the decoded sound.

9. Verification

9.1 Inputs

A verifier is given: the room’s descriptor; its world statement; the voice statement; the world’s admitted ops (live-worlds §9.2), in any order, an op given twice byte for byte being one op; the listener, an actor’s key or a listening place; and the blocks, in order, each with its instant and, for each principal whose sound it holds, that block of decoded sound with its channel count.

9.2 Steps

The verifier reads the descriptor (live-rooms §3.4), refusing as it does and naming the descriptor; refuses a voice statement given with no world statement as voices_without_world, naming the statement; reads the world statement beside the descriptor (live-worlds §3.4), refusing as it does and naming the world statement; reads the voice statement (§3.3), refusing at its first failure and naming the statement; reads each op in the order given (interactive-worlds §5.1), refusing one that is not exactly one op as op_malformed, one that is not an action (§2) as op_not_action, and one with the op id of an op given before it and other bytes as equivocation (live-worlds §9.2 steps 4 and 5), naming the op; and then renders each block in order (§6), refusing at the first block that breaks §6.2 or §6.4, its instant checked before its sound, and naming it.

9.3 Outcomes

A verification that refuses states the reason and where. One that passes states, for each block, what §6.4 lists. Two verifications of the same inputs state the same; the samples are mixdown-equivalent (spatial-audio §4) over the decoded sound: they are byte-exact given the same decoded blocks, and as portable as the voices’ decoding is (live-compose §6).

10. Reasons

The codes this extension adds, each with where it is raised:

reasonraised at
voice_law_unknown§3.3 step 3
voice_statement_invalid§3.3 step 4
voices_without_world§3.3
voice_instant_regress§6.2

Of live-events §9’s codes, a verifier also raises truncated, malformed and unknown_version when a statement does not read, bad_signature for its signature, and wrong_room for a statement of another room (§3.3); of live-worlds §11’s, op_malformed, op_not_action and equivocation (§9.2); and of live-compose §10’s, mix_input_malformed (§6.4).

11. Conformance

live-conformance/voices/ holds the corpus, in one file, vectors.json, written from fixed seeds by wai_world_voice_vectors gen. Each vector gives the descriptor, the world statement, the voice statement, the ops, the listener and the blocks, and either the outcome of §9.3, every block’s report and samples, or the refusal, its reason and where:

A conforming verifier gives every vector its outcome, every block’s report and samples, or the reason and the place. wai_world_voice_vectors verify and the test tests/world_voice.rs check that the file regenerates byte for byte, that wai-rs gives every vector its outcome, that every reason of §10 a verifier raises appears, and that §10 lists exactly the reference’s codes. live-conformance/world_voices_verify.py is the second implementation, written from this text: it shares no code with wai-rs; it checks every signature with WAI’s strict rule (claim-conformance/_ed25519.py); and it mutates copies of the corpus and requires every copy to fail, and drops each of its own checks in turn and requires the corpus to catch the weakened verifier. The reference room client’s web/world/voices.js renders each block from the corpus in Node and in a page’s audio worklet, and its tests require the corpus’s samples.

12. Security and privacy considerations