WAI Extension: Scene Carriage

Mirrored from the canonical text at commit 260cbd09 ().

Status: Draft. Maps WAI’s scene content — a world (WWLD, interactive-worlds), an audio scene (WSCN, spatial-audio), the assets a world names, and timed media — onto two scene-description formats, in both directions: ISO/IEC 23090-14 scene description, a set of glTF 2.0 extensions (§4, §5), and a USD layer in the text format of the USD Core Specification (§6, §7). A document WAI writes carries the WAI bytes exactly, beside the presentation a player of that format reads; reading it back gives the same bytes, and is refused unless writing them again gives the document. A document WAI did not write becomes new WAI bytes under stated rules, and is refused wherever a value would not carry exactly. No new capability and no transport. Reference implementation: the scene_map module of wai-rs (feature scene_map); corpus live-conformance/scene/, second implementation live-conformance/scene_verify.py. Keywords MUST, MUST NOT, SHOULD, MAY are RFC 2119/8174.

Texts read for this revision: glTF 2.0 as ISO/IEC 12113:2022; the descriptions and JSON schemas of the extensions of ISO/IEC 23090-14 that the glTF extension registry publishes (MPEG_media, MPEG_buffer_circular, MPEG_accessor_timed, MPEG_audio_spatial, MPEG_texture_video, MPEG_animation_timing, MPEG_scene_dynamic, MPEG_mesh_linking, MPEG_viewport_recommended; each states that it is based on ISO/IEC FDIS 23090-14, and the registry marks their licence as not yet determined, so none is copied here); and the USD Core Specification v1.0.1 (b7bc21a, 2025-12-12), published under CC-BY-ND-4.0. ISO/IEC 23090-14’s own text, including its Annex D (the attenuation functions) and the timed accessor information header of its clause 5, was not read.

1. Scope

In scope: the bundle a mapping reads and writes (§2); the numbers that cross it (§3); the document each direction writes or reads, for glTF (§4, §5) and for USD (§6, §7); the refusals (§8); the corpus (§9).

Out of scope:

2. The bundle

2.1 Worlds and audio scenes

A bundle holds at most one world (a WWLD payload of a wai.world.* capability), at most one audio scene (a WSCN payload of wai.audio.scene) and a list of timed media items (§2.2). The mapping reads each container’s section table (interactive-worlds §4: the magic, a little-endian u16 count, nine-byte entries of a kind, an offset from the end of the table and a length; every section within the payload, no kind twice) and:

Other sections are carried and not read. The checks run world, then audio scene, then media items in order.

2.2 Timed media items

A timed media item is a WAI object named by its content hash — the hash of interactive-worlds §4.1, over a complete envelope — with a name (a string), its capability, and for pictures a width and a height. Its capability is wai.video., wai.audio. or wai.image. followed by the capability grammar of http-delivery §4.1 (segments of a to z and 0 to 9, then segments that may also hold _); a picture’s (wai.video.*, wai.image.*) width and height are each 1 to 65535, and an audio item has neither. An item that breaks this is media_malformed. A document names a WAI object by the URI urn:wai:asset: followed by its hash in 64 lowercase hexadecimal digits.

2.3 What a document carries

Every document this mapping writes names its profile, wai-scene/1, and carries the bundle exactly: the world’s and the audio scene’s payloads in lowercase hexadecimal, and the media list as a JSON text — an array of objects, each of exactly name, asset (64 lowercase hexadecimal digits) and capability, then for a picture width and height, in that order, written as §4.1 writes JSON. A reader takes the WAI bytes from what is carried and nothing else; the rest of the document is presentation, a function of those bytes.

3. Numbers

3.1 Conversions

A position in WAI is three Q32.32 values (wai.det.fixed64: a signed 64-bit integer r standing for r / 2^32). Writing one, the mapping takes the binary32 (for glTF) or binary64 (for USD) nearest r / 2^32, ties to even; every non-zero result is a normal number. A time is a sample index s of an audio scene at rate R; writing it, the mapping takes the binary32 nearest s / R, ties to even. Reading, a binary32 or binary64 value v is a Q32.32 value only when v · 2^32 is an integer from −2^63 to 2^63 − 1 (position_not_exact otherwise), and a binary32 time t ≥ 0 is a sample index only when t · R is an integer of at most 2^64 − 1 (time_not_exact).

3.2 JSON numbers

glTF’s JSON numbers are written as RFC 8785 §3.2.2.3 writes a binary64 (the shortest digits that read back to it, spelt as ECMAScript’s Number.prototype.toString spells them), the binary64 being the binary32 value of §3.1 exactly. Integers are plain decimal. A reader takes a JSON number for a glTF float as glTF does: the binary32 nearest its decimal value, ties to even; one past the binary32 range is document_malformed.

3.3 USDA numbers

USDA numbers are written as the USD Core Specification (Double-Precision Floating Point Representation when Writing USDA content) writes a binary64: the shortest digits that read back to it; scientific notation, one digit before the point and a signed exponent with no +, when the decimal exponent is below −6 or at least 15; decimal otherwise; no trailing point, no trailing zero in a fraction, no leading zero but before the point of a value below 1. Integers are plain decimal. A reader takes a USDA number as the binary64 nearest its decimal value.

3.4 Positions by sim

A world node has a position when its sim’s state begins with one: wai.world.sim.kinematic0 and wai.world.sim.ballistic0, the first 24 bytes (three little-endian Q32.32 values); wai.world.sim.truss0, a particle (state byte 0 is 0), the 24 bytes after that byte. Other nodes and sims have none. A node with no position is presented with no translation.

4. Writing a glTF document

4.1 The document

The document is one JSON text in UTF-8, followed by a line feed. Every object is written with the members this section lists, in the order it lists them, and none other; no whitespace; strings escaped as RFC 8785 §3.2.2.2 escapes them; numbers as §3.2 writes them. A member a step does not produce is not written. The document’s members are, in order: asset ({"version":"2.0","generator":"wai-scene/1"}); extensionsUsed (the names of the extensions the document uses, in ascending byte order); extensions, holding MPEG_media when there are media items (§4.4); scene (0); scenes (one scene whose nodes lists every node, [{}] when there is none); nodes; cameras; buffers; bufferViews; accessors; animations; textures; extras (§4.5). First the bundle is checked (§2).

4.2 The world

Each node of the world’s tick-0 graph, in ascending id order, is a node {"name":"node-<32 lowercase hexadecimal digits of its id>","translation":[x,y,z]}, its translation the binary32 of §3.1 of its position (§3.4), absent when it has none.

4.3 The audio scene

An audio scene adds a camera, {"type":"perspective","perspective":{"yfov":1,"znear":0.25}}, and after the world’s nodes a listener node, {"name":"listener","camera":0,"extensions":{"MPEG_audio_spatial":{"listener":{"id":0}}}}. Then each object i, in table order, is a node {"name":"object-<i>","translation":[…],"extensions":{"MPEG_audio_spatial":{"sources":[{"id":i,"type":"Object","attenuation":"custom","accessors":[a]}]}}}, its translation its first key’s position (absent with no key), a its source’s timed accessor (§4.4). Its keys MUST be in strictly ascending sample order, and their times (§3.1) strictly ascending, or the bundle is not_presentable. The attenuation is custom, which ISO/IEC 23090-14 leaves outside its scope: WAI’s distance law and gain are in the carried bytes, and this mapping states no equivalence with a function of that standard’s Annex D.

An object with two or more keys is animated. The document’s first buffer holds, object by object, its times (binary32, little-endian), then its positions (three binary32 each), and is {"byteLength":n,"uri":"data:application/octet-stream;base64,<RFC 4648 base64 with padding>"}. For each animated object, in order: two buffer views, {"buffer":0,"byteOffset":o,"byteLength":4k} and {"buffer":0,"byteOffset":o',"byteLength":12k}; two accessors, {"bufferView":v,"componentType":5126,"count":k,"type":"SCALAR","min":[t0],"max":[tlast]} and {"bufferView":v+1,"componentType":5126,"count":k,"type":"VEC3"}; a sampler {"input":a,"interpolation":"LINEAR","output":a+1}; and a channel {"sampler":s,"target":{"node":n,"path":"translation"}}. One animation holds them all: {"name":"audio","channels":[…],"samplers":[…]}. glTF’s linear interpolation of positions between keys is the presentation; WAI’s own interpolation of keys is spatial-audio §2.2’s, in the carried bytes.

4.4 Media

MPEG_media’s media lists first the audio scene’s sources, each once, in object order, then the bundle’s other items in list order (an item whose hash is already listed is not listed again). Each is {"name":…,"autoplay":true,"alternatives":[{"uri":"urn:wai:asset:<hex>","mimeType":…}]}: for a source, the name of the first bundle item with its hash, else source- and the first 16 digits of its hash; the mimeType is application/wai (http-delivery §3) followed, when the item’s capability is known, by ; codecs="wai1.<the capability without its leading wai.>", the codecs grammar of http-delivery §4.1.

Each source, and each other item of a picture capability, gets in media order a circular buffer, {"byteLength":B,"extensions":{"MPEG_buffer_circular":{"media":m}}}, a view {"buffer":b,"byteLength":B}, and a timed accessor: for a source, {"bufferView":v,"componentType":5126,"count":1024,"type":"SCALAR","extensions":{"MPEG_accessor_timed":{"immutable":true}}} with B = 4096 (one frame of 1024 mono binary32 samples); for a picture of width w and height h, {"bufferView":v,"componentType":5121,"normalized":true,"count":w·h,"type":"VEC3","extensions":{"MPEG_accessor_timed":{"immutable":true}}} with B = 3·w·h, and a texture {"extensions":{"MPEG_texture_video":{"accessor":a,"width":w,"height":h,"format":"RGB"}}}. Buffers, views and accessors follow the animation’s. A circular buffer’s byteLength is one frame; how many frames a pipeline keeps is MPEG_buffer_circular’s count, left at its default. An accessor with no header view is immutable, as MPEG_accessor_timed’s schema requires.

The extensions used are MPEG_audio_spatial with an audio scene, MPEG_media with media items, MPEG_buffer_circular and MPEG_accessor_timed with a circular buffer, and MPEG_texture_video with a texture.

4.5 What is carried

The document’s extras is {"wai":{"profile":"wai-scene/1","world":"<hex>","audio":"<hex>","media":"<JSON text>"}} (§2.3), world, audio and media present when the bundle has them.

5. Reading a glTF document

5.1 Which reading

A document that is not a JSON text (RFC 8259, UTF-8, no member named twice, at most 64 levels) whose value is an object with asset.version "2.0" is document_malformed. One whose extras has a member wai is read as §5.2; any other, as §5.3.

5.2 A document WAI wrote

extras.wai is an object of the members of §4.5 and no other, profile being wai-scene/1, world and audio lowercase hexadecimal of even length, and media a JSON text of §2.3’s form; otherwise the document is carried_malformed. The bundle is what it carries. Writing that bundle (§4) MUST give the document’s bytes: a bundle §4 refuses is carried_malformed, and other bytes are presentation_mismatch. So a player’s view and the WAI bytes cannot drift apart unnoticed.

5.3 A document WAI did not write

Reading one needs the key that mints its node ids (an actor, 32 bytes) and the contract of the world and of the audio scene it yields. The steps run in order, and the first refusal is the one reported; each index a step follows MUST be in range, and a member a step reads MUST have the type it names, or the document is document_malformed.

  1. The nodes. The scene is scenes[scene] (scene defaulting to 0; no scenes, no nodes). Its nodes are visited depth first in preorder: its nodes in order, each node’s children in order. A node visited twice is document_malformed.
  2. Positions, in visiting order. A node with a matrix, a rotation other than (0, 0, 0, 1) or a scale other than (1, 1, 1), each read as binary32 (§3.2), is transform_not_carried: a node of the world has a position and nothing more. Its translation (zero when absent) is read as binary32 and then as Q32.32 (§3.1, position_not_exact), and its position is its parent’s plus its translation (position_not_exact past the Q32.32 range).
  3. Kinds, in visiting order: a node whose extensions.MPEG_audio_spatial has sources is an audio node; one with a camera, or with MPEG_audio_spatial and no sources, is skipped; every other node is a world node, and a world node with a mesh is reported as presentation not carried (node <i>: mesh).
  4. Animations, in document order, each animation’s channels in order: a channel’s sampler is followed first, then its target; a target with no node is ignored. A channel whose node is not an audio node, whose path is not translation, whose sampler’s interpolation (LINEAR by default) is not LINEAR, or that is a node’s second channel, is animation_not_carried.
  5. Audio objects, audio nodes in visiting order, each node’s sources in order. A source of type other than Object is audio_not_object (an HOA source has no position to carry); a pregain other than 0 is gain_not_exact (a gain in decibels has no exact linear value). Its first accessor’s buffer view’s buffer is a circular buffer whose media names an MPEG_media item; the source is the first of that item’s alternatives whose uri is urn:wai:asset:<hex> (media_not_wai if none). Its keys: with no channel, one key at sample 0 at the node’s position; with one, the audio contract is due now (contract_absent if absent, not JSON, or without a sample_rate of 1 to 2^32 − 1), and the sampler’s input (FLOAT, SCALAR) and output (FLOAT, VEC3) accessors, neither sparse, read from views with no byteStride of a buffer whose uri is data:application/octet-stream;base64, and canonical base64, each within its view and buffer, the output three values to each input value, give its keys: each time a sample (§3.1, time_not_exact), each position the parent’s position plus the output as Q32.32 (position_not_exact). Its gain is 1.
  6. Other media, the MPEG_media items no source uses, in order: its first urn:wai:asset alternative (media_not_wai if none), whose mimeType names a capability as §4.4 writes it (media_not_wai otherwise); its name (empty when absent); for a picture capability, the width and height of the first MPEG_texture_video texture whose accessor’s circular buffer is this item. An item that then breaks §2.2 is media_not_wai.
  7. The bytes. World nodes need the world contract (contract_absent if absent or not JSON), whose sim MUST be wai.world.sim.kinematic0 (sim_not_importable): the i-th world node, from 1 in visiting order, gets the id interactive-worlds §5.4 mints for the actor and lamport i, and a kinematic0 state at its position, at rest, with scalar 0. The world is the contract given, those nodes in ascending id order, no asset table, and an empty op log, laid out as interactive-worlds §4 lays a container out. Audio objects need the audio contract (as in step 5); the audio scene is the contract given, the objects in order (gain 1), and an asset table of each distinct source in object order, role 0, named by its item’s name. The media list is step 6’s.

6. Writing a USD layer

6.1 The layer

The layer is UTF-8 text in USD’s text format, every line ended by a line feed, indented four spaces a level:

#usda 1.0
(
    customLayerData = {
        string "wai:audio" = "<hex>"
        string "wai:media" = <string>
        string "wai:profile" = "wai-scene/1"
        string "wai:world" = "<hex>"
    }
    defaultPrim = "World"
    endTimeCode = <duration>
    startTimeCode = 0
    timeCodesPerSecond = <sample rate>
)

def Xform "World"
{
    …
}

wai:audio, wai:media and wai:world are written when the bundle has them, and the three time fields with an audio scene: time codes are its sample indices. The prims of §6.2 are World’s children, with one empty line before each but the first. First the bundle is checked (§2).

6.2 Prims

Prims inside a scope are separated by one empty line. A prim with metadata is written def "<name>" (, the metadata line, ), then its body { … } on the following lines; an empty body is { and }.

6.3 Strings

A string is written in double quotes, with \", \\, \n, \r and \t, and with \x and two uppercase hexadecimal digits for any other character below U+0020 and for U+007F. A character outside the grammar’s Utf8Character ranges has no spelling, and a bundle that needs one is not_presentable.

7. Reading a USD layer

7.1 The subset read

A layer that is not UTF-8, or does not begin #usda, is document_malformed; a version on that line other than 1.0 is usd_not_read. The rest is read as the core specification’s grammar reads it, for this subset:

Anything else is document_malformed.

7.2 A layer WAI wrote

A layer whose customLayerData has a string key beginning wai: is read as WAI wrote it: its wai: items MUST be the strings of §6.1, wai:profile being wai-scene/1 (carried_malformed otherwise), and writing the bundle they carry (§6) MUST give the layer’s bytes (carried_malformed for a bundle §6 refuses, presentation_mismatch for other bytes).

7.3 A layer WAI did not write

The root is the prim defaultPrim names (document_malformed if no root prim has that name), else the first root prim (document_malformed if none). Prims are visited depth first in preorder from the root, the root included, children in written order. For each, in visiting order:

  1. Its offset, from its properties in written order: xformOpOrder MUST be a list of the one string xformOp:translate with no time samples; xformOp:translate MUST have a default value and no time samples; any other xformOp: property is not carried; each breach is transform_not_carried. The translate’s default is a tuple of three numbers (document_malformed), each read as binary64 (§3.3) and then as Q32.32 (position_not_exact). With no translate the offset is zero. Its position is its parent’s plus its offset (position_not_exact past the range).
  2. Its kind. A prim with references is a set of assets: each asset path MUST be urn:wai:asset:<hex> (asset_not_wai), and gives an asset entry of role 0 named by the prim. Otherwise an Xform prim with an xformOp:translate is a world node at its position. Otherwise a prim of a type other than none, Xform and Scope is reported as presentation not carried (<path>: <type>).

Nodes or assets need the world contract (contract_absent if absent or not JSON), whose sim MUST be wai.world.sim.kinematic0 (sim_not_importable). The world is built as §5.3 step 7 builds one, with an asset table of the entries in visiting order. A layer gives no audio scene and no media: the core specification defines no audio prims, so the audio and media of a layer are read only from the bytes a WAI-written layer carries (§7.2).

8. Refusals

codewhere
world_malformed§2.1: the world does not read
audio_malformed§2.1: the audio scene does not read
media_malformed§2.2: a media item’s capability, width or height
not_presentable§4.3, §6.2, §6.3: keys out of order, two keys at one binary32 time, a time code past 2^53, a character with no USDA spelling
document_malformed§5.1, §5.3, §7.1, §7.3: the document does not read as its format, or a value a step needs is not there
carried_malformed§5.2, §7.2: what a WAI-written document carries does not read, or is a bundle §4 or §6 refuses
presentation_mismatch§5.2, §7.2: the document is not the export of the bundle it carries
usd_not_read§7.1: a construct outside the subset
transform_not_carried§5.3 step 2, §7.3 step 1: a rotation, scale, matrix or other transform operation
position_not_exact§3.1, §5.3, §7.3: a value that is not a Q32.32 value
animation_not_carried§5.3 step 4: an animation channel the mapping does not carry
time_not_exact§3.1, §5.3 step 5: a time that is not a whole sample
gain_not_exact§5.3 step 5: a pregain other than 0
audio_not_object§5.3 step 5: a source that is not an object
media_not_wai§5.3 steps 5 and 6: media that names no WAI object or capability
asset_not_wai§7.3 step 2: a reference to something other than a WAI object
contract_absent§5.3, §7.3: nodes or objects and no contract to give them
sim_not_importable§5.3 step 7, §7.3: a world contract whose sim is not wai.world.sim.kinematic0

9. Conformance corpus

live-conformance/scene/ holds bundles — worlds built in code (kinematic0, ballistic0 and truss0 nodes, an asset table) and the worlds and audio scenes other corpora commit, each pinned by SHA-256 — each written as a glTF document and as a USD layer and read back; bundles refused on export; documents WAI wrote, altered and refused; and documents WAI did not write, built in code, read to new WAI bytes or refused with each code of §8. A vector is an export (a bundle to a document’s bytes, or a code) or an import (a document, with the actor and contracts §5.3 needs, to a bundle and its report of presentation not carried, or a code). A verifier conforms when it reaches every vector’s outcome and bytes. wai_scene_vectors gen|verify writes and checks it, tests/scene_map.rs regenerates it, and live-conformance/scene_verify.py, written from this text and sharing no code with wai-rs, is its second implementation.

10. Reference implementation

The scene_map module of wai-rs (feature scene_map, on world and audio_scene): Bundle, MediaItem, ImportParams and Imported; gltf::export and gltf::import; usda::export and usda::import; num for §3; corpus for §9. Its unit tests check the conversions and spellings of §3 against fixed values, and that the containers §5.3 and §7.3 build are the bytes world::WorldContainer and audio_scene::SpatialScene write.