WAI Extension: Media Identification

Mirrored from the canonical text at commit 260cbd09 (2026-11-08).

Status: Draft. What a file is, from its bytes alone: the formats they fit, each with the bytes that say so and how far they were followed; what a registered probe reads of it; signatures found inside it; and measures that hold when no format is recognised. It decodes nothing, and what it does not establish it leaves out. The reference implementation is the identify module of wai-rs (feature identify), which wai identify runs (operation media.inspect); the corpus is identify-conformance/, and its second implementation is identify-conformance/identify_verify.py. Keywords MUST, MUST NOT, SHOULD, MAY are RFC 2119/8174.

1. Scope

§1.1 wai inspect reads what an envelope holds. Most files a toolchain meets are not envelopes, and the first question about one is what it is. Identification answers that for any file, and it is the first step of working out how a piece of media was made.

§1.2 A result states only what the bytes establish. A claim comes with the offsets of the bytes that make it, and a walk says where it stopped and why. A format is never guessed from a file name, and no confidence figure is given: a result has a strength (§4), which says what was checked. Where the bytes fit no format, the verdict is unknown and the measures (§8) are what is known.

§1.3 What a container states is not what its samples hold. A track row (§5.1) reports what the file says of a track, and no sample is read to confirm it. No payload is decrypted, and a protected track is reported as protected and no further.

§1.4 This extension adds no capability and changes no dispatch (SPEC §4). A capability in a result is the registered capability whose payload a format or track is: it says what wai wrap would carry it under, not that a decoder here accepts it.

2. The result

§2.1 A result is a JSON object:

§2.2 A candidate is a JSON object:

3. Common rules

§3.1 Integers are read as each section says. u8, be16, be24, be32 and be64 are big-endian, le16 and le32 little-endian. All arithmetic of a result is integer arithmetic, but for the sample rate of §5.4; two implementations give the same result for the same bytes on any machine.

§3.2 What is read. The head is the first 1 MiB of the file (1,048,576 bytes), or all of it if shorter. The walks of §5.5, §5.9, §5.10, §5.13, §5.17, §5.18, §5.19 and §5.20, the table of §6 and the probes read the head alone. The other walks read the file where their lengths lead. The scan is the first 16 MiB of the file (16,777,216 bytes), or all of it if shorter: §7 and §8 read it.

§3.3 Walks. A walk reads a format’s structures in order and is bounded: at most 4,096 structures (§5.6: 256), and 64 tracks. A walk that stops before the file’s end says why, in stopped:

walked_to and a stop’s at are never past the end of the file: an offset past it is taken as the file’s length. A walk that reads the head alone and reaches its end while the file goes on has stopped there, limit.

§3.4 Strings. A string read from the file enters a result only through this filter: its first 64 bytes, each of 0x20 to 0x7E but \ as itself, and any other byte as \xNN in lowercase hexadecimal. A file is untrusted input: a reader MUST treat these strings as data, and a tool that gives a result to an agent MUST mark it as holding them.

§3.5 Probes. The probes of the table of format-probes §2 that can name a format from the bytes (wai.image.jxs, wai.video.jxs, wai.video.hevc, wai.video.mvhevc, wai.video.vvc, wai.video.lcevc, wai.audio.mpegh, wai.video.avs3, wai.audio.avs3, in that order) each read the head. The other two read a payload declared to be theirs, have no signature, and say nothing of a file they are merely given, so they are not run. For each probe, in order:

• Where it accepts, the capability joins probes_accepted. The first candidate of kind elementary that maps to that capability (or, for wai.video.mvhevc, to wai.video.hevc) has its strength raised to parameters, the capability added to its capabilities, and the probe’s result set in its facts.probe under the capability. With no such candidate, a candidate of kind elementary is added whose id is the capability, of strength parameters, with evidence {at: 0, len: <head's length>, is: "probe"}, that capability and the same facts.probe.
• Where it refuses with no_signature, nothing is recorded.
• Where it refuses otherwise, and a candidate of kind elementary already maps to that capability, its facts.probe_refused holds the code under the capability.

4. Strength and verdict

§4.1 Strength says how far a candidate’s bytes were followed:

• signature: a format’s fixed bytes are where the format puts them. Nothing else was checked, and short ones occur by chance: evidence shows how many bytes it was.
• structure: what follows the signature is what the format’s section requires of it.
• parameters: a probe of format-probes read the format’s headers and accepted them.

§4.2 The verdict comes from the candidates at the highest strength present: unknown if there are no candidates; ambiguous if more than one; signature_only if one, of strength signature; else identified. Two formats that both fit are both reported, and no ranking between them is invented.

§4.3 Order. Candidates are in order of strength, strongest first. Within a strength they keep the order of the sections that define them: §5.2 to §5.20, then §6’s table in its order, then those a probe alone added, in the order of §3.5.

5. Walks

§5.1 Tracks. A row of tracks holds track (the identifier the container gives, or null), media (video, audio, text or other), stated (what the container calls it, through §3.4), codec (the name of the table below, or null where this extension has no name for it), and width, height, channels, sample_rate and protected, each null where not stated. protected, on a track the container says is protected, is {scheme, original}, each null where not stated.

A row’s codec adds to its candidate’s capabilities the capability of this table, where it has one:

The names mp3, vorbis, theora, speex, ac3, eac3, webvtt and ttml map to none.

§5.2 WAI envelopes (wai, kind wai). The head begins WAI1 or WAI2: evidence magic:WAI1 or magic:WAI2, 4 bytes at 0, and facts.version. le32 at 4 is the manifest’s length. If it is absent the walk stops at 0 (overrun). If it exceeds 1 MiB the walk stops at 8 (limit); if the file holds less the walk stops at 8 (overrun); if the manifest is not JSON, at 8 (invalid). Evidence manifest. For WAI1: facts.capability is model_requirement.capability, and le32 after the manifest is the payload’s length, with evidence payload_length; where it or the payload does not fit the walk stops there (overrun) with walked_to there too, else walked_to is past the payload. For WAI2: le16 after the manifest is facts.renditions, evidence rendition_count, and walked_to is past it. A walk that reaches this is structure. wai inspect reads the rest.

§5.3 ISO base media (isobmff, kind container). The four bytes at 4 are one of ftyp, styp, moov, mdat, moof, free, skip, wide, sidx or pnot. Boxes are walked from 0 to the end of the file. A box: be32 size and four bytes of type. Every type byte is 0x20 to 0x7E or 0xA9, else invalid. Size 0 runs to the end of the file; size 1 means be64 at 8 is the size, which must be at least 16; any other size below 8 is invalid. Fewer bytes than a header needs is overrun. Each box whose header reads gives evidence box:<type>, 4 bytes at its offset plus 4, and its type joins facts.boxes (distinct, first 16); a box that does not end within the file stops the walk there (overrun). moof boxes are counted in facts.fragments, present when there are any.

ftyp that fits: fewer than 8 content bytes is invalid; else facts.major_brand is its first four, and facts.compatible_brands the first 16 of the four-byte brands from its ninth. A major or compatible brand avif or avis (among its first 4096 bytes) adds wai.image.avif.

moov that fits and is at most 64 MiB is read for trak boxes (above that, facts.moov_not_read is true). Inside a box, children are read as the top level is, stopping at the first that does not fit. A track needs mdia, and in it hdlr (the handler type is at content offset 8) and minf/stbl/stsd with a first entry (the entries start at content offset 8), else it has no row. track is from tkhd: be32 at content offset 12 if its first byte is 0, else at 20. media from the handler: vide, auxv, pict is video; soun is audio; text, subt, sbtl, clcp is text; else other. stated is the entry’s type. For video: width be16 at entry content 24, height at 26, child boxes from 78. For audio: channels be16 at 16, sample_rate be32 at 24 shifted right 16, and, where be16 at 8 is 0, child boxes from 28.

An entry of type encv, enca, enct or encs is protected: protected.scheme is from sinf/schm (content bytes 4 to 8) and protected.original from sinf/frma (its first four), and the codec is named from original where it is present. mp4a is named from its esds: aac for object type 0x40, 0x66, 0x67 or 0x68, mp3 for 0x69 or 0x6B, else null. The object type is reached as MPEG-4 Systems lays it out: after four version bytes, tag 3 and its size (one to four bytes of 7 bits, high bit set on all but the last); two bytes of identifier and one of flags, then 2 bytes if flag 0x80, a length byte and that many if 0x40, and 2 bytes if 0x20; tag 4 and its size; the next byte. Other entry types:

structure: at least two box headers read.

§5.4 Matroska and WebM (matroska, kind container). The head begins 1A 45 DF A3: evidence ebml, 4 bytes at 0. An element is an identifier of 1 to 4 bytes (its length in the leading zeros of its first byte, the marker kept) and a size of 1 to 8 bytes (the same, the marker removed; all ones is unknown). The header must read in the head, with a known size: else the walk stops at 0 (invalid for an element that does not parse, overrun for one that does not fit). facts.doc_type is its 0x4282, through §3.4, and facts.doc_type_version its 0x4287. The next element must be the Segment, 0x18538067: evidence segment; else the walk stops there, overrun if fewer than 5 bytes remain and invalid otherwise. A Segment with a known size that passes the file’s end sets facts.segment_overruns_by, and the walk ends at the file’s end.

Its children are walked to its end. A child that does not parse stops the walk (overrun if fewer than 12 bytes remain, else invalid); an unknown size, unknown_size; a child that passes the file’s end, overrun. Clusters (0x1F43B675) are counted in facts.clusters, present when there are any, one that passes the file’s end included. Tracks (0x1654AE6B) of at most 1 MiB gives evidence tracks and a row for each 0xAE: track 0xD7; media from 0x83: 1 video, 2 audio, 17 text, else other; stated and codec from 0x86; width and height 0xB0 and 0xBA in 0xE0; channels 0x9F and sample_rate 0xB5 in 0xE1, an IEEE float of 4 or 8 bytes taken toward zero; protected where 0x6D80 holds 0x6240 holding 0x5035. Unsigned values are big-endian, of 1 to 8 bytes. The codec is aac for an identifier beginning A_AAC, else:

structure: at least one child read.

§5.5 MPEG transport streams (mpeg-ts, kind container). For each layout in order, packet size and sync offset (188, 0), (192, 4), (204, 0), and for each start from 0 below the packet size: 0x47 at the sync offset of five packets from there. The first that holds is the layout: evidence sync, 1 byte at its first sync, facts.packet_size. Packets are read while 188 bytes from a sync offset lie in the head. A packet whose sync byte is not 0x47 stops the walk (invalid), at the offset of that byte. PID is 13 bits from byte 1; facts.scrambled_packets counts those whose byte 3 has either of its top two bits set. With an adaptation field (byte 3 bit 0x20), the payload is past its length byte and its length. With a payload (bit 0x10), PID 0 and each PID the program table names are gathered: a packet with the start bit (byte 1 bit 0x40) begins a section after its pointer field, and a section’s length is 3 plus the 12 bits from its second byte. A section is used only if at least 12 bytes long and its MPEG-2 checksum (polynomial 0x04C11DB7, from all ones, not reflected, over the section and its checksum) is 0.

The first section of table 0 on PID 0 is the program table: evidence pat, each entry of 4 bytes from offset 8 to 4 from its end (the first 64), with program number from the first two bytes and PID from the next 13 bits, program 0 not being counted. facts.programs is the number of programs. The first section of table 2 on each such PID: evidence pmt. Program information of the 12 bits from offset 10 starts at 12; streams follow, each a type byte, 13 bits of PID at the next two bytes and 12 bits of descriptor length at the next two, then that many bytes of descriptors. A row for each, after the last packet: track the PID, stated stream_type:0x<two lowercase hexadecimal digits>, protected where a descriptor with tag 9 is among the program’s or the stream’s (then {scheme: null, original: null}). Codec and media:

Any other type has codec null and media other. facts.packets_read counts the packets. If no sync byte stopped it, and the walk’s end is before the file’s: limit if the head is shorter than the file, else overrun. Always structure.

§5.6 Ogg (ogg, kind container). The head begins OggS and a zero byte. Pages from 0: a 27-byte header beginning the same, a table of as many bytes as its byte 26, and a body of as many bytes as that table sums to. A page that does not fit is overrun; a header that does not begin so, invalid; a checksum (le32 at 22) that does not hold over the page with its checksum bytes zero, under the polynomial 0x04C11DB7 from zero, not reflected, crc. At most 256 pages. Each gives evidence page, 1 page long, at its offset. Byte 5 bit 0x02 starts a stream: a row with track le32 at 14, read from the body: OpusHead (stated OpusHead, opus, audio, channels byte 9, sample_rate le32 at 12); \x01vorbis (stated vorbis, vorbis, audio, channels byte 11, sample_rate le32 at 12); \x7fFLAC (stated FLAC, flac, audio); \x80theora (stated theora, theora, video); Speex and three spaces (stated Speex, speex, audio); else stated is its first 8 bytes, media other. facts.pages_read. structure: two pages, or one that ends at the file’s end.

§5.7 ID3 and FLAC (flac, kind audio). An ID3v2 tag at 0, for this and §5.9: ID3, ten header bytes, bytes 3 and 4 not 0xFF, and bytes 6 to 9 each below 0x80. Its length is 10 plus those four as 7 bits each, plus 10 where byte 5 has bit 0x10. A head that begins ID3 with a header that is not one means neither walk applies. fLaC at 0, or after the tag: evidence fLaC, and wai.audio.flac. Blocks follow: a byte (bit 0x80: last; low 7 bits: type), then be24 length. The first must be type 0 of length 34, else invalid: evidence streaminfo, 34 bytes at its body, and facts.sample_rate (20 bits from body byte 10), facts.channels (3 bits, plus 1), facts.bits_per_sample (5 bits, plus 1) and facts.samples (36 bits). A block past the file’s end is overrun. After the last block, 2 bytes 0xFF and 0xF8 or 0xF9 are a frame’s sync: evidence frame_sync, and structure. Other bytes there stop the walk (invalid), and no bytes (overrun). walked_to is the offset after the last block read.

§5.8 RIFF (kind per form). The head begins RIFF, with at least 12 bytes. The form at 8: WAVE is wave, audio; AVI is avi, container; WEBP is webp, image; any other is riff, container. Evidence form:<form>, 4 bytes at 8, and facts.form. The declared end is le32 at 4 plus 8, in facts.declared_size where it is not the file’s size; the walk ends at the lesser of it and the file’s size. Chunks from 12: four bytes of type and le32 length, padded to an even length. A header past the end stops the walk (overrun) unless the walk is exactly at the end or beyond it; each header read gives evidence chunk:<type>, and its type joins facts.chunks (distinct, first 16). A chunk’s content past the end stops the walk there (overrun).

For wave: fmt gives, from at most its first 64 bytes, facts.format_tag le16 at 0 (where that is 0xFFFE, le16 at 24 instead), facts.channels le16 at 2, facts.sample_rate le32 at 4 and facts.bits_per_sample le16 at 14, each null where absent; data gives facts.data_bytes. For webp: VP8 with bytes 3 to 5 9D 01 2A gives facts.coding lossy, facts.width and facts.height from le16 at 6 and 8 in their low 14 bits; VP8L with first byte 0x2F and 5 bytes gives lossless, and, in le32 at 1, width its low 14 bits plus 1 and height the next 14 plus 1; VP8X with 10 bytes gives extended, and width and height the 24-bit little-endian values at 4 and 7, each plus 1. walked_to is the walk’s end, at most the end above. structure: one chunk read.

§5.9 MPEG audio (mpeg-audio, kind audio). From 0 or after an ID3v2 tag (§5.7). A frame: 0xFF; 3 high bits of the next byte set; version 2 bits (0: 2.5, 2: 2, 3: 1), layer 2 bits (1: III, 2: II, 3: I), the bitrate index in byte 2’s high 4 bits (not 0 or 15), rate index in the next 2 (not 3), padding the next bit; byte 3’s high 2 bits are the channel mode. The version value 1 and layer value 0 are not a frame. Rate is 44100, 48000 or 32000 by index, halved for version 2 and quartered for 2.5. The bitrate in kbit/s, by index from 1:

Frame length: layer I, (12000 × kbit/s ÷ rate + padding) × 4; layer III of version 2 or 2.5, 72000 × kbit/s ÷ rate + padding; otherwise 144000 × kbit/s ÷ rate + padding; division is integer division. Nothing is a candidate unless the first bytes are a frame and at least one whole frame is walked. Evidence id3v2, 3 bytes at 0, where a tag is there; frame_header, 4 bytes at the first frame. facts.version (1, 2 or 2.5), facts.layer, facts.bitrate_kbps, facts.sample_rate and facts.channels (1 for channel mode 3, else 2), of the first frame. Frames walk through the head. A frame must keep the first’s version, layer and rate, else the walk stops there (overrun if fewer than 4 bytes remain, else invalid). Exactly 128 bytes left beginning TAG: evidence id3v1, 3 bytes, and the walk ends after them. A frame past the file’s end is overrun, past the head’s alone limit. facts.frames_read. structure: three frames.

§5.10 ADTS (adts, kind audio). A frame: 0xFF; byte 1 with byte & 0xF6 == 0xF0; byte 2’s high 2 bits plus 1 the audio object type, its next 4 the rate index (below 13: 96000, 88200, 64000, 48000, 44100, 32000, 24000, 22050, 16000, 12000, 11025, 8000, 7350), and channel configuration from its low bit and byte 3’s high 2; frame length 13 bits from byte 3’s low 2, at least 7. Nothing is a candidate unless the head begins with a frame and at least one whole frame is walked. Evidence frame_header, 7 bytes at 0, wai.audio.aac, and facts.audio_object_type, facts.sample_rate, facts.channel_configuration of the first. Frames walk as §5.9’s do, keeping type and rate (overrun under 7 bytes), and with the same limits. facts.frames_read. structure: three frames.

§5.11 PNG (png, kind image). The head begins 89 50 4E 47 0D 0A 1A 0A: evidence signature, and wai.image.png. Chunks from 8: be32 length, four ASCII letters of type (else invalid), data, 4-byte checksum. The first must be IHDR of length 13, else invalid; its checksum (polynomial 0xEDB88320 reflected, from all ones, inverted at the end; over type and data) must hold, else crc. Then evidence ihdr, 25 bytes, facts.width, facts.height, facts.bit_depth, facts.colour_type, facts.interlace, and structure. acTL sets facts.animated; cICP with 4 bytes readable sets facts.cicp to them as a list. A chunk past the file’s end, or the end of the file before IEND, is overrun. IEND gives evidence iend, 12 bytes at its offset, and the walk ends after it. facts.chunks_read. Only IHDR’s checksum is computed.

§5.12 JPEG (jpeg, kind image). The head begins FF D8 FF: evidence soi, 2 bytes at 0, and wai.image.jpeg. Markers from 2: 0xFF and a code, a code of 0xFF being a fill byte that advances one. Fewer than 2 bytes is overrun; a first byte not 0xFF is invalid. A code of 0xDA gives evidence sos, 2 bytes, and ends the walk there. Others: be16 length, at least 2 (else invalid), overrun if it does not read or the segment passes the file’s end. 0xE0 to 0xEF name an application segment from its first 32 bytes: JFIF\0 jfif, Exif\0\0 exif, ICC_PROFILE\0 icc, MPF\0 mpf, the XMP namespace (http://ns.adobe.com/xap/1.0/\0) xmp, Adobe adobe, else other; facts.applications holds them, distinct, first 8. Codes 0xC0 to 0xC3, 0xC5 to 0xC7, 0xC9 to 0xCB and 0xCD to 0xCF are a frame header, needing 6 bytes (else overrun): evidence sof, the segment’s bytes, facts.process (0xC0 baseline, 0xC1 extended, 0xC2 progressive, 0xC3 lossless, 0xC9 to 0xCB arithmetic, else hierarchical), facts.precision, facts.height, facts.width, facts.components, and structure.

§5.13 GIF (gif, kind image). The head begins GIF87a or GIF89a: evidence signature:<it>, 6 bytes. Fewer than 13 bytes stops the walk at 6. facts.width and facts.height from le16 at 6 and 8; a global table of 3 × 2^(n+1) bytes follows byte 10 flag 0x80 (n its low 3 bits). Blocks from there: 0x3B the trailer (evidence trailer, 1 byte, and the walk ends after it); 0x21 an extension, sub-blocks from 2 on; 0x2C an image, 10 bytes (a local table likewise from its byte 9), one more byte, then sub-blocks, evidence image, 10 bytes, for the first; any other byte is invalid. Sub-blocks: a count byte and that many, ending after a count of 0. Bytes not in the head stop the walk: limit where the head is shorter than the file, else overrun. facts.frames_read. structure: one image, or the trailer.

§5.14 IVF (ivf, kind container). The head begins DKIF: evidence dkif, 4 bytes. Under 12 bytes stops at 0 (overrun); a version (le16 at 4) not 0 or a header length (le16 at 6) under 32 stops at 0 (invalid). A row: media video, stated the four bytes at 8, codec av1 for AV01, vp9 for VP90, vp8 for VP80, width and height le16 at 12 and 14. Frames from the header length: le32 size, 8 bytes of time, then that many. overrun where one does not fit. facts.frames_read. structure: one frame.

§5.15 YUV4MPEG2 (y4m, kind elementary). The head begins YUV4MPEG2 : evidence signature, 10 bytes. A line feed must end the header within 256 bytes: else stop at 0 (overrun if the file ends sooner, else invalid). Space-separated tokens: W width, H height, C colour (default 420), F frame rate; facts.width, facts.height, facts.colour and facts.frame_rate. A colour is a base, and optionally p and a decimal depth, 2 bytes a sample above 8. Bases: 420, 420jpeg, 420mpeg2, 420paldv (w·h + 2·⌈w/2⌉·⌈h/2⌉ samples a frame); 422 (w·h + 2·⌈w/2⌉·h); 444 (3·w·h); mono (w·h). Any other stops the walk (invalid), as do absent dimensions. Frames: a line beginning FRAME, within 256 bytes, then a frame’s bytes. facts.frames_read. structure: one frame.

§5.16 FLV (flv, kind container). The head begins FLV and 0x01: evidence signature, 4 bytes. be32 at 5, the data offset, must be at least 9, else stop at 0 (overrun under 9 bytes, else invalid). Tags from that offset plus 4: 11 header bytes (type in byte 0’s low 5 bits, one of 8, 9 or 18, else invalid; be24 length at 1), then the body, then a be32 that must be 11 plus the length (else invalid; absent, overrun). The first audio tag (type 8, with a body) gives a row: media audio, stated sound_format:<n> for its first body byte’s high 4 bits, codec mp3 for 2 and aac for 10. The first video tag (type 9, with a body) gives a row: where its first body byte has bit 0x80, stated is the next four bytes, with codec avc for avc1, hevc for hvc1, av1 for av01, vp9 for vp09; otherwise stated is codec_id:<n> for that byte’s low 4 bits, with avc for 7. The video row comes first. facts.tags_read. structure: one tag.

§5.17 AV1 OBU streams (av1-obu, kind elementary). An OBU: a header byte with bits 0x81 clear and 0x02 set; type its bits 3 to 6, one of 1 to 8 or 15; an extension byte where bit 0x04 is set; a LEB128 size of at most 8 bytes; that many bytes. The first must be type 1 or 2. They are walked through the head. A header that does not parse stops the walk at it (limit if the head is shorter than the file and fewer than 8 bytes remain, else invalid); one past the file’s end, overrun; one past the head’s alone, limit. The first type 1 gives evidence sequence_header, and is read as AV1 §5.5: facts.profile, facts.level (the first operating point’s seq_level_idx), facts.width and facts.height (max_frame_width_minus_1 and max_frame_height_minus_1, plus 1). facts.units_read, and wai.video.av1. A candidate only if a sequence header reads and at least two OBUs were walked, and then structure: two bytes that merely fit are no claim of AV1.

§5.18 HLS (hls, kind playlist). After an optional UTF-8 byte order mark, #EXTM3U and a line end or the end of the head: evidence extm3u, 7 bytes at it. Lines (ended by a line feed, less a final carriage return) are read: any beginning #EXT-X- is a tag; #EXT-X-STREAM-INF: counts facts.variants, and the first with CODECS=" gives facts.codecs; #EXTINF: counts facts.segments; #EXT-X-MAP: counts facts.maps; #EXT-X-ENDLIST sets facts.ended; #EXT-X-VERSION: gives facts.version where it is a decimal integer; and the first #EXT-X-KEY: or #EXT-X-SESSION-KEY: whose METHOD= (to the next comma) is not NONE gives facts.key_method. facts.playlist is multivariant with variants, else media with segments, else other. Every one of these facts is present; structure: at least one tag.

§5.19 DASH (dash, kind playlist). After an optional byte order mark and any white space, <?...?> and <!--...-->, then <MPD and white space or >: evidence mpd, 4 bytes at it. urn:mpeg:dash:schema:mpd:2011 anywhere in the head gives evidence namespace and structure. facts.type is the value of type=" in the root tag up to its first >. facts.periods, facts.adaptation_sets, facts.representations and facts.content_protection count <Period, <AdaptationSet, <Representation and <ContentProtection; facts.codecs holds the distinct values of codecs=", the first 16. It is no XML parser: these are substrings.

§5.20 Annex B streams (kind elementary). The head holds at least two zero bytes, then 0x01, and nothing before them. Units are split as format-probes §2 (its Annex B byte streams) does, the first 4,096 (trailing zero bytes removed). If the byte after the first prefix is 0xBA or 0xB3, §6 has the format, and no reading applies. Each reading below needs every unit’s header to hold and its parameter sets in order, each before the first coded picture of that reading; a stream two readings fit is reported as both, and no reading is preferred. A reading is structure, with facts.units_read. Where the head is shorter than the file, or 4,096 units were read, it stops at the last unit (limit). walked_to is null.

• avc-annexb: one header byte, bit 0x80 clear, type its low 5 bits and 1 to 21; types 5, 7 and 8 have nonzero bits 0x60. Order 7, 8; coded pictures are types 1 to 5. Its first type 7 is read as ITU-T H.264 §7.3.2.1.1, with emulation prevention removed: a profile_idc other than 66, 77, 88, 100, 110, 122, 244, 44, 83, 86, 118, 128, 138, 139, 134 or 135 or a failed reading rules the reading out. Evidence nal:sps; facts profile_idc, constraint_flags, level_idc, chroma_format_idc, width and height less cropping; wai.video.avc.
• hevc-annexb: two header bytes, bit 0x80 of the first clear, the low 3 bits of the second not zero, type the first’s bits 1 to 6 and one of 0 to 9, 16 to 21 or 32 to 40. Order 32, 33, 34; coded pictures are types 0 to 9 and 16 to 21. Evidence nal:sps; wai.video.hevc.
• vvc-annexb: two header bytes, bits 0xC0 of the first clear, the low 3 bits of the second not zero, type the second’s high 5 bits and at most 25. Order 15, 16; coded pictures are types 0 to 10. Evidence nal:sps; wai.video.vvc.

6. Formats known by their signature

§6.1 A format of this table is a candidate where every one of its fixed bytes is at its offset in the head: strength signature, evidence signature for each, and the capabilities it names. Bytes are in hexadecimal, each group at the offset before its colon. A format with two rows is either.

7. Signatures inside a file

§7.1 A reverse engineer’s first question about a blob is what it holds. The scan is searched for each of these fixed bytes, none shorter than 8 bytes, so that one in 16 MiB of noise is improbable; each hit past the file’s beginning gives {at, id}, at being where its structure begins, back bytes before them. A hit whose structure would begin at or before the file’s beginning is not reported. Hits are in the order their bytes are found, and within one offset in this table’s order. At most 32 are reported, and embedded_capped says whether another was found. A hit says only that the bytes are there.

8. Measures

§8.1 measures holds, over the scan: scanned, its length; distinct_bytes, how many of the 256 values occur; zero_bytes, how many are zero; chi_square, the floor of 256 · Σc² / n less n, where c are the counts of the values and n the length, and 0 for an empty scan; text (§8.2); windows and uniform_windows (§8.3); and periods (§8.4).

§8.2 text is true where the scan is not empty, is valid UTF-8 (a sequence the scan’s end cuts short allowed), and holds no byte below 0x20 but 0x09, 0x0A, 0x0C and 0x0D, and no 0x7F.

§8.3 A window is each whole 4096 bytes of the scan from its start. One is uniform when the sum of its counts squared is at most 70,496: its chi-square against a uniform spread is then at most 310, about the 99th percentile for 255 degrees of freedom. A compressed payload and an encrypted one both tend to be uniform; this measure cannot tell them apart, and says only that the bytes show no structure in their values.

§8.4 Repeats. Over the first 256 KiB of the scan (n bytes), for each lag l from 1 to L + 1, where L is the lesser of 4096 and ⌊n/4⌋: m = n − l; matches counts the i below m with byte i equal to byte i + l; expected is the floor of m · Σc² / n², at least 1, over this sample’s counts; the excess e is matches − expected; and z is, where e is positive, the floor of the square root of the floor of e² / expected, otherwise 0. With L below 2 there are no periods.

The floor is 8 times the median of |e| over those lags, the element at index ⌊k/2⌋ of the k of them sorted: data that repeats exactly is not independent from lag to lag, and its noise is many times what z assumes. A lag from 2 to L, in increasing order, is reported as {lag, z} where all of these hold, at most 8: z is at least 8; e is at least the floor; e exceeds twice the greatest of its neighbours’ excess and 0; if 2l ≤ L + 1, z at 2l is at least 8; and it is no multiple of a lag already reported. A repeat is a record size, a sample width or a table’s stride, found with no knowledge of the format.

9. Security

§9.1 Identification reads files nobody vouches for. A conforming implementation MUST NOT panic, read outside the file, allocate more than the file’s bounded windows and a bounded number of structures, or follow a URI in a playlist. The bounds of §3 are what make the cost of a hostile file bounded.

§9.2 A result can hold strings a file’s author chose (§3.4). The operation’s agent door marks its results untrusted_content.

§9.3 An implementation MUST NOT decrypt, or try to recover the keys of, a protected track. It reports the scheme the file states and stops.

10. Conformance

§10.1 The corpus identify-conformance/ holds files written by real muxers and encoders from synthetic inputs (ext/), generated cases and negative controls (gen/), and vectors.json, each file’s digest and whole result. prober.json is what an independent media prober reads of ext/, the reference checked against it wherever both state a value. A conforming implementation gives each file its result exactly.