WAI Extension: Timed Text and Live Captions

Mirrored from the canonical text at commit f871af55 (2026-10-20).

Status: Draft. Registers the record wai.text.timed: timed text, and live captions as events. A cue is a span of media time and its text, in a small, closed set of styling (italic runs, line breaks, and one of six anchors) that WebVTT and the IMSC1 Text Profile both express. A cue therefore maps onto each of them exactly, and onto the two MoQ caption timelines, CTA-608 and HLS. Where a target cannot hold a cue, the mapping refuses it and names the reason; it never drops meaning silently. A live caption is a caption event of live-events (§4.10 there), unchanged. A machine caption is revised in place until it is final: each revision is a new caption event, followed by a meta event whose payload is this extension’s annotation, WTX1. The annotation names the revision it supersedes, says whether the text is final and how much of it is stable, and carries the anchor and the italic runs. The event’s origin (human, machine or machine-reviewed), with the digest of the model that wrote it, is the disclosure label, and every mapping carries it or hands it to a sidecar. No new transport: field semantics and rules over them only. The reference implementation is the timed_text module of wai-rs (feature timed_text); the corpus is live-conformance/captions/, and its second implementation is live-conformance/timed_text_verify.py. Keywords MUST, MUST NOT, SHOULD, MAY are RFC 2119/8174.

Texts read for this revision, each pinned to the version named: WebVTT, W3C Candidate Recommendation Draft of 2026-05-20 (CRD-webvtt1-20260520); IMSC Text Profile 1.3, W3C Recommendation of 2026-05-21 (REC-ttml-imsc1.3-20260521); TTML Profiles for Internet Media Subtitles and Captions 1.0.1, W3C Recommendation of 2018-04-24 (REC-ttml-imsc1.0.1-20180424), for its profile signalling; IMSC Hypothetical Render Model, W3C Recommendation of 2024-04-25 (REC-imsc-hrm-20240425); draft-wilaw-moq-webvtt-msf-00, draft-law-moq-imsc1-msf-00 and draft-ietf-moq-msf-01; draft-pantos-hls-rfc8216bis-22; the C2PA Technical Specification, version 2.4; the IPTC Digital Source Type vocabulary as released on 2024-10-23; EN 301 549 V4.1.1 (2026-09); and 47 CFR 79.101 as in force on 2021-06-01 and 47 CFR 79.102, which reproduce the 608 and 708 decoder rules. A section number of one of those texts in this extension is that version’s. CTA-608-E and CTA-708-E themselves could not be read for this revision: their publisher distributes them through an account-gated store. §8.1 and §9 say what that leaves unchecked.

1. Scope

In scope: the cue model and the text subset WebVTT and the IMSC1 Text Profile both hold exactly (§3); live captions as caption events, the annotation, how revisions are ordered and checked, what a sink shows, the cues a stream resolves to, and what a subscriber joining mid-stream receives (§4); WebVTT documents and segments, and a reader of what this extension writes (§5); IMSC1 Text Profile documents and a check of the hypothetical render model (§6); the MoQ caption timelines and their catalog entries (§7); CTA-608 on field 1, channel CC1, pop-on and roll-up (§8); a CTA-708 mapping, specified only (§9); the disclosure label in every target (§10); and what a WAI player owes its viewers (§11).

Out of scope: a transport; speech recognition and translation, which produce the text; the fragmented MP4 packaging of an IMSC1 rendition (HLS draft-pantos-hls-rfc8216bis-22 §3.1.5 and ISO/IEC 14496-30 carry the document §6 writes); the paint-on style of 608, colours, underline and flash; audio description; and a sink’s own rendering of text, beyond §4.4 and §11.

Relationship to the core spec: wai.text.timed is a record (SPEC §5). Its unit is the signed caption revision, and the documents this extension writes are mappings of it, as live-events §8 maps an event onto a playlist tag. It names no payload format and is not dispatchable. A timed-text component that the graphics-vector §5.7 package rule asks for is a WebVTT or IMSC1 document written by §5 or §6.

Relationship to live-events: this extension changes nothing in it. Its caption payload has no field for a revision, so the annotation rides in the meta kind, which live-events leaves to applications (§4.2 there), and is bound to its caption by the event chain (§4.3 here). A later revision of live-events MAY fold the annotation’s fields into caption; this extension would then read both forms.

2. Terms

• Cue: one entry of the cue model (§3.1).
• Revision: one caption event, with what its annotation says (§4.3).
• Chain of a cue: its revisions, the first superseding none and each later one superseding the one before.
• Partial: a revision that is not final. Final: a revision that is.
• Stable prefix: the leading code points of a partial’s text that no later revision of its cue changes.
• Document order: cues by start, and in the order given among cues of equal start.
• Media time: milliseconds on the timeline of the media the cues accompany. The media time of the room’s t0 is pt0, as live-events §8.4 has it.
• Slot: one position on the 608 field-1 pair clock (§8.2).

3. The cue model

3.1 Fields

origin 3, machine-reviewed, is text a model produced and a person reviewed, or reviewed and corrected, before it was published. This is the reading C2PA 2.4 §18.28.3 gives the human oversight level human_validated (§10.3).

3.2 Text: the common subset

A cue’s text and italic ranges are in the subset when these rules hold. Each rule exists because one of the targets would otherwise change what the viewer reads.

1. The text has at least one code point. Every line has at least one code point: no U+000A at the start or the end, and no two in a row. (An empty line ends a WebVTT cue, WebVTT §4.1.)
2. Every code point other than U+000A is in U+0020–U+007E, U+00A0–U+D7FF, U+E000–U+FFFD or U+10000–U+10FFFF, and is not U+2028, U+2029 or U+FEFF. (These are the characters XML 1.0 holds that are neither controls nor line or paragraph separators; a tab is refused, since IMSC1’s white-space handling turns it into a space.)
3. No line begins or ends with U+0020, and no line holds two U+0020 in a row. (The default white-space handling of TTML collapses runs of spaces and removes spaces at a line’s edges, so an IMSC1 rendering would differ from the WebVTT one.)
4. The italic ranges are canonical: each has at least one code point and lies within the text; they ascend, and each ends before the next begins; no range holds a U+000A; no range begins or ends with U+0020; and between two ranges there is at least one code point other than U+0020. (So one rendering has one spelling: two italic words with a space between them are one range.)

3.3 Anchors

The caption area is the inner 80% of the picture, 10% to 90% across and down, which is the safe caption area of 47 CFR 79.101(n)(12). An anchor puts a cue at the bottom or the top of it, with its lines centred, at its left edge or at its right edge.

In WebVTT the cue box spans 10% to 90% of the width, its end edge at 90% of the height for the bottom and its start edge at 10% for the top; in IMSC1 the regions are the same bands of the root container (§6.1). The two place a cue in the same band with the same alignment. They are not the same pixels, and this extension does not claim that they are.

3.4 Checking cues

A cue is checked in this order, and the first failure names the refusal:

1. start < end ≤ 359 999 999: otherwise time_out_of_range.
2. lang reads, and origin is 1, 2 or 3: otherwise cue_malformed.
3. A model when origin is 1: cue_malformed. No model when origin is 2 or 3: model_missing.
4. §3.2 rules 1 to 3: otherwise text_not_in_subset.
5. §3.2 rule 4: otherwise cue_malformed.

A document is written for one language, lang, a parameter of the mapping. A mapping whose lang does not read is parameter_malformed. It checks each cue in the order given, then that every cue’s lang is the document’s: otherwise language_mismatch. A document is one language because an HLS rendition and an MSF track each state one (LANGUAGE, draft-pantos-hls-rfc8216bis-22 §4.4.6.1; lang, draft-ietf-moq-msf-01 §5.2.32).

3.5 What each target drops

4. Live captions

4.1 A caption event is a revision

A live caption is a caption event (live-events §4.10): duration_ms, origin, model, lang and text. Its t is the room-clock instant from which this revision is shown, and duration_ms is how long it is shown from t, or 0 when not known (§4.4). A cue revised in place is a chain of caption events, each followed by its annotation. A caption event with no annotation is a final revision that supersedes nothing, at anchor bottom-center, with no italic range. A human captioner whose captions need neither an anchor nor italics sends caption events alone.

Live-events verifies every event first (§6 there): signature, chain, rates and payload. A caption payload cannot express a machine caption without a model digest: live-events reads the 32 bytes after origin 2 or 3 as the digest, and refuses a payload too short for them, malformed_payload.

4.2 The annotation

An annotation is the payload of a meta event whose first four bytes are WTX1. Integers are little-endian.

An annotation is 42 + 4 × n bytes. It is refused annotation_malformed when: it is shorter than 42 bytes; its version is not 1; state is not 0 or 1; anchor is above 5; its length is not 42 + 4 × n; stable_cp exceeds the code points of its caption’s text; its state is final and stable_cp is not that count; or its ranges are not canonical over its caption’s text (§3.2 rule 4).

4.3 Revisions, their order and their checks

A verifier of captions is offered the records of one room in stream order. It offers each to a live-events verifier (live-events §6), which refuses it with that extension’s reason, ignores it as a duplicate, or accepts it. For each event accepted, in stream order, of principal P at seq s:

1. The caption awaiting its annotation. The verifier keeps, for each principal, the last caption event it accepted whose annotation is not yet settled. If there is one, at seq c, and this event is an annotation at c + 1, it is that caption’s annotation (step 4). Otherwise the caption has no annotation: it is final, its stable_cp is its text’s code points, its anchor bottom-center, with no italic range; and if s is above c + 1, its annotation is unknown, since the verifier does not hold the event that would carry it.
2. A caption. Its text MUST meet §3.2 rules 1 to 3, otherwise text_not_in_subset. It becomes the caption awaiting its annotation.
3. An annotation not at c + 1 of a caption awaiting it: when the verifier holds P’s event at s − 1, it is annotation_misplaced. When it does not, the annotation is an orphan of a delivery gap, and is read no further.
4. An annotation of the caption at s − 1, in this order:
   1. it reads (§4.2, its layout): otherwise annotation_malformed;
   2. its t equals its caption’s: otherwise annotation_misplaced;
   3. its stable_cp, its state and its ranges hold against its caption’s text (§4.2): otherwise annotation_malformed;
   4. when supersedes is not zero:
      • it names a caption event of P the verifier holds: otherwise, when the verifier holds every seq of P from 0 to s, supersede_unknown; when it does not, the reference is unresolved, which is not a refusal;
      • the revision it names is a partial: otherwise supersede_final;
      • no other revision supersedes it: otherwise supersede_forked;
      • it has the same lang and the same anchor as this one: otherwise supersede_mismatch;
      • this revision’s text begins with the first stable_cp code points of its text: otherwise stability_broken.
5. Any other event changes nothing here.

At the end of the stream, a caption still awaiting its annotation has none. The outcome is the one live-events §6.4 gives, with a refusal of this section in its place: refuse, with the reason and the index of the record; gap; or accept.

Revisions are ordered by their chain: a revision supersedes exactly one partial, which no other revision supersedes, of the same principal, so a cue’s revisions form one line from its first to its last. Since live-events holds a principal’s t non-decreasing, a later revision is shown no earlier than the one it supersedes. A principal MAY interleave the revisions of several cues.

4.4 What a sink shows

At room-clock instant x, a sink shows each revision r it holds for which all of these hold:

• t(r) ≤ x;
• no revision that supersedes r has t ≤ x;
• when r’s duration_ms is above 0, x < t(r) + duration_ms × 1000; when it is 0, no later caption event of r’s principal has t ≤ x.

A sink:

• MUST replace a revision by the one that supersedes it in the same place, the anchor and language being the same (§4.3 step 4.4);
• SHOULD NOT move or re-flow the stable prefix of a revision when its successor arrives, and MAY set the rest of a partial’s text apart, for example in a lighter weight;
• MUST show a revision whose annotation is unknown as a caption with no annotation is shown;
• MUST show the label of every machine and machine-reviewed revision (§11).

4.5 The cues a stream resolves to

A stream that a verifier accepts with no gap resolves to cues, one per chain. For each chain, from its first revision r₀ to its last revision rₖ:

• rₖ MUST be final: otherwise not_final, at the index of rₖ’s caption event;
• rₖ’s duration_ms MUST be above 0: otherwise duration_unknown, at that index;
• the cue: start = pt0 + ⌊t(r₀) / 1000⌋; end = pt0 + ⌊t(rₖ) / 1000⌋ + duration_ms(rₖ), which MUST NOT exceed 359 999 999 (otherwise time_out_of_range); lang, origin, model and text are rₖ’s caption’s; anchor and italic are rₖ’s;
• the cue was published at pt0 + ⌊t(rₖ) / 1000⌋ (§7.3).

The cues are in document order, cues of equal start in the order of their first revisions in the stream. The first chain that fails, in that order, names the refusal. A cue starts when its first revision was shown, so its final text is timed from when the viewer first saw the cue. The partials are not in any document, timeline or 608 stream: none of them can replace a cue already published (§3.5).

4.6 Joining mid-stream

A publisher of captions MUST add to each principal’s join set (live-events §8.2), at the start of group g:

• the latest revision of each cue that is shown at the instant (§4.4) and has no stated duration: a partial whose duration_ms is 0, which no revision before the instant supersedes, and which is its principal’s last caption event before the instant;
• right after each caption event of the join set, its annotation, when it has one.

Live-events already requires every caption whose interval holds the instant. With these, a subscriber that joins at group g verifies the join set and what follows, finds a gap before it (§6.2 there) and refuses nothing (an annotation naming a revision it lacks is unresolved, §4.3), and shows at every instant from the group’s start what a subscriber holding the whole stream shows.

5. WebVTT

5.1 Documents

A WebVTT document of cues in lang is checked (§3.4), then written as UTF-8 with U+000A line ends, no byte order mark, as:

• WEBVTT, U+000A, U+000A;
• then each cue in document order, the cues separated by one U+000A, each cue:
  • its timing line: start, -->, end, a space, and the anchor’s settings (§3.3), each time written HH:MM:SS.mmm (hours always two digits, the milliseconds three);
  • its payload: the lines of its text, separated by U+000A, then U+000A. In each line, each run of italic code points (§3.2) is written between <i> and </i>, and in all text &, < and > are written &, < and >. A machine or machine-reviewed cue’s payload begins with the label’s start tag (§10.4) and ends, after its last line, with </c>.

No cue identifier, region, style block or comment is written. The > escape also keeps the substring -->, which WebVTT §4.1 forbids in a payload, out of every payload.

5.2 Segments

The segments of cues in lang, at bounds b₀ < b₁ < … < bₙ in media milliseconds (at least two bounds, the last at most 359 999 999), with mpegts0 the 90 kHz MPEG-TS time of media time 0 (below 2³³), are refused parameter_malformed when the bounds or mpegts0 do not hold. Segment k, for media time bₖ to bₖ₊₁, is:

• WEBVTT, U+000A;
• X-TIMESTAMP-MAP=LOCAL:, bₖ as HH:MM:SS.mmm, ,MPEGTS:, (mpegts0 + 90 × bₖ) mod 2³³ in decimal, U+000A, U+000A;
• every cue whose start < bₖ₊₁ and whose end > bₖ, in document order and with its whole interval, written and separated as in §5.1.

This follows draft-pantos-hls-rfc8216bis-22 §3.1.4: a segment holds every cue shown during its period, with each cue’s total display time, and its X-TIMESTAMP-MAP maps the segment’s first cue time to the 90 kHz media time, re-anchored in every segment so that a wrap of the 33-bit time stays exact. A live publisher writes a segment from the cues final when it cuts it; a cue that becomes final later is in the later segments it overlaps, and a publisher SHOULD delay cutting a segment by the time its captions take to become final.

5.3 Reading the subset

A reader of WebVTT reads only what §5.1 and §5.2 write, given the document’s lang. It refuses as vtt_malformed anything else: another header; an X-TIMESTAMP-MAP not as §5.2 writes it; a timing line whose times are not HH:MM:SS.mmm (minutes and seconds below 60) or whose settings are not one anchor’s; an empty payload line; a blank line other than one between cues; a character reference other than &, < and >; a tag other than the label (§10.4), <i> and </i>; an <i> span that is empty, nested or holds a line end; two runs of one style next to each other; and starts that go back. A label with no model class is model_missing. Each cue it reads is then checked (§3.4). A WebVTT document or segment this extension writes reads back to its cues in document order.

6. IMSC1 Text Profile

6.1 Documents

An IMSC1 document of cues in lang is checked (§3.4) and checked against the hypothetical render model (§6.2), then written as UTF-8 with U+000A line ends:

<?xml version="1.0" encoding="UTF-8"?>
<tt xmlns="http://www.w3.org/ns/ttml" xmlns:ttp="http://www.w3.org/ns/ttml#parameter" xmlns:tts="http://www.w3.org/ns/ttml#styling" xmlns:ebuttm="urn:ebu:tt:metadata" xmlns:wai="tag:transaction.science,2026:wai/timed-text" xml:lang="LANG" ttp:profile="http://www.w3.org/ns/ttml/profile/imsc1/text">
<head>
<metadata>
<ebuttm:documentMetadata>
<ebuttm:conformsToStandard>http://www.w3.org/ns/ttml/profile/imsc1/text</ebuttm:conformsToStandard>
<ebuttm:conformsToStandard>http://www.w3.org/ns/ttml/profile/imsc1.3/text</ebuttm:conformsToStandard>
</ebuttm:documentMetadata>
</metadata>
<layout>
<region xml:id="top" tts:origin="10% 10%" tts:extent="80% 35%" tts:displayAlign="before"/>
<region xml:id="bottom" tts:origin="10% 55%" tts:extent="80% 35%" tts:displayAlign="after"/>
</layout>
</head>
<body>
<div>
P
</div>
</body>
</tt>

with LANG the document’s lang, and P one line per cue in document order (no line when there is no cue):

• <p begin="START" end="END" region="R" tts:textAlign="A", the times as HH:MM:SS.mmm, R and A the anchor’s (§3.3);
• for a machine or machine-reviewed cue, wai:origin="O" wai:model="M" (§10.4);
• >, the lines of the text separated by <br/>, each italic run written between <span tts:fontStyle="italic"> and </span>, &, < and > written &, < and >, then </p>.

The document uses features the Text Profile of every IMSC edition from 1.0.1 to 1.3 permits: media timing in clock time, two regions given by tts:origin and tts:extent in percent (IMSC 1.3 §8.1.19), tts:displayAlign, tts:textAlign, tts:fontStyle, span and br. Its regions lie within the root container and do not overlap (IMSC 1.3 §8.13.2), and at most two are presented at once (§8.13.3).

It signals its profile in two ways. The ttp:profile attribute on tt, equal to the IMSC 1.0 Text Profile designator, is what IMSC 1.0.1 §6.9 says SHOULD be present in a document that is not also an EBU-TT-D or SDP-US document; that edition is the IMSC1 that draft-pantos-hls-rfc8216bis-22 §3.1.5 names, and IMSC 1.3’s feature table permits #profile. The two ebuttm:conformsToStandard elements, the IMSC 1.0 and the IMSC 1.3 Text Profile designators, are the form IMSC 1.3 §8.10.2 specifies, which its profile resolution reads to set the override content profile (§6.3.2). Neither names an Image Profile, since IMSC 1.0.1 §6.9 forbids signalling both the Image and the Text Profile in one document; no ttp:profile element is written. ttp:contentProfiles is not written: IMSC 1.3 §8.10.1 does not recommend it in a document meant for profiles based on TTML1, such as IMSC 1.0.1. Attributes in this extension’s namespace are foreign attributes, which IMSC 1.3 §8.6 permits and a processor prunes. Default cell resolution (32 by 15) and font size (one cell) apply.

6.2 The hypothetical render model

IMSC 1.3 §8.2 says a document SHOULD conform to the IMSC-HRM. A writer of this extension MUST check that it does, as follows, and refuses hrm_exceeded when the check fails. The check is sound: a document that passes conforms to the model. It may refuse a document that conforms, since it takes the least favourable factor of each of the model’s tables.

• The intermediate synchronic documents are the spans between consecutive distinct times among every cue’s start and end, each holding the cues with start ≤ its beginning < end. One with no cue is empty.
• No region has a background, so painting a non-empty document clears the root container and paints no region: 1 / BDraw = 1/12 s (IMSC-HRM §7, §8).
• Every glyph’s normalised rendered area is (1/15)², the default font size of one cell of fifteen (IMSC-HRM §9). A glyph is a code point with its italic flag; br is not one. Rendering a glyph takes (1/15)² / 0.6 = 1/135 s and copying one (1/15)² / 3 = 1/675 s: the slowest Ren and GCpy the model’s tables give, whatever the script.
• For each non-empty document in order, the code points of the top region’s cues, then of the bottom region’s, each region’s cues in document order: a glyph in the glyph cache is copied and flagged; any other is rendered, added to the cache and flagged. More than 225 flagged glyphs at any point (a normalised size above 1) fails the check.
• The time it has is the time since the previous non-empty document’s presentation, when that is less than one second, and one second otherwise (the initial painting delay). The check fails when 1/12 s plus the time of every glyph exceeds it. In units of 1/2700 s: 225 + 20 per glyph rendered + 4 per glyph copied, against 2.7 per millisecond.
• At the document’s presentation, glyphs not flagged leave the cache, and the rest lose their flag. An empty document empties the cache, which can only lengthen a later painting.

6.3 The structure a verifier checks

An IMSC1 document written here is well-formed XML 1.0 in UTF-8, and parsed it has: a tt root in the TTML namespace with xml:lang and ttp:profile equal to the IMSC 1.0 Text Profile designator, no ttp:profile element, and no Image Profile designator anywhere; a head whose metadata names both designators of §6.1; a layout with exactly the two regions of §6.1, each with an xml:id, tts:origin and tts:extent in percent, no tts:position, within the root container and not overlapping; a body with one div of p elements, each with begin < end in clock time and a region that names one of them; in a p, only text, span with tts:fontStyle="italic", and br; no attribute in a TT namespace but those §6.1 writes; and the wai: attributes of §10.4 exactly on the p of a machine or machine-reviewed cue.

7. The MoQ caption timelines

The two caption timelines are MSF event timelines (draft-ietf-moq-msf-01 §8): each MOQT object is a JSON array of records, and each record an index and a data object. This extension writes JSON with no whitespace, its members in the order given, strings with " and \ escaped, U+000A as \n, any other code point below U+0020 as \u00xx, and every other code point as itself. draft-wilaw-moq-webvtt-msf-00 cites MSF section numbers that are not those of draft-ietf-moq-msf-01 (its event timeline is MSF §8, not §11); this extension cites MSF -01’s own.

7.1 The WebVTT timeline

A cue’s record (draft-wilaw-moq-webvtt-msf-00 §3, §3.1):

{"m":START,"data":{"start":START,"end":END,"text":TEXT,"settings":{"line":LINE,"position":"50%,center","size":"80%","align":ALIGN}}}

START and END are the cue’s media times, TEXT its WebVTT payload (§5.1) without the final U+000A, LINE "90%,end" for a bottom anchor and "10%,start" for a top one, and ALIGN the anchor’s alignment. No initialization record is written, since no region or style is used (§4 there).

7.2 The IMSC1 timeline, cue mode

The initialization record (draft-law-moq-imsc1-msf-00 §4.1):

{"m":0,"data":{"init":{"profile":"http://www.w3.org/ns/ttml/profile/imsc1/text","styles":{"center":{"textAlign":"center"},"left":{"textAlign":"left"},"right":{"textAlign":"right"}},"regions":{"bottom":{"origin":"10% 55%","extent":"80% 35%","displayAlign":"after"},"top":{"origin":"10% 10%","extent":"80% 35%","displayAlign":"before"}}}}}

A cue’s record (§4.2 there):

{"m":START,"data":{"start":START,"end":END,"region":R,"style":A,"content":[...]}}

R is the anchor’s region and A its alignment, as JSON strings. content holds, line by line, each run as {"text":T} or, italic, {"text":T,"fontStyle":"italic"}, and {"br":true} between lines. A machine or machine-reviewed cue’s data ends with "wai_origin":O,"wai_model":M (§10.4). The draft defines no such member; a consumer that ignores members it does not know loses the label there, and §10.6 applies.

7.3 Groups and the repeat at a group’s start

Each cue is placed by the media time it was published at: its start for cues given as a list, and for a stream the time of §4.5. Group g of a timeline covers media time g × group_ms to (g + 1) × group_ms, group_ms a parameter above 0. The mapping of groups g₁ to g₂ (g₁ ≤ g₂; otherwise, or when a cue is published before its start, parameter_malformed) writes, for each group:

• object 0: the IMSC1 initialization record (IMSC1 timeline only), then the record of every cue published before the group’s start whose end is after it;
• then one object per cue published within the group, of its one record.

Cues are taken in the order they were published, by media time, in the order given among equal times. This is the first object of each group repeating “all accumulated cue records up to that point” (draft-wilaw-moq-webvtt-msf-00 §5) and the independence of draft-ietf-moq-msf-01 §8.3, read as live-events §8.1 reads it: the records accessible at a group’s start are those of cues still to be shown, and earlier ones stay reachable by FETCH of earlier groups. A subscriber joining at any group holds the record of every cue that has not ended.

7.4 Catalog entries

A caption timeline’s MSF catalog track object:

{"name":NAME,"packaging":"eventtimeline","eventType":"urn:msf:captions:webvtt","mimeType":"application/json","depends":[...],"role":"caption","lang":LANG,"wai_machine_generated":B}
{"name":NAME,"packaging":"eventtimeline","eventType":"urn:msf:captions:imsc1","mimeType":"application/json","depends":[...],"role":"caption","lang":LANG,"imsc1Mode":"cue","wai_machine_generated":B}

B is true when any cue of the track is machine or machine-reviewed, and false otherwise. wai_machine_generated is a custom field, which draft-ietf-moq-msf-01 §5 lets a producer add and requires a parser to ignore when it does not understand it.

8. CTA-608

8.1 Sources

The basic character set, the special characters, the mid-row codes, the miscellaneous control codes, the preamble address codes, parity and the redundant sending of control codes are as 47 CFR 79.101 reproduces CTA-608’s decoder rules (paragraphs (e) to (j)), read as in force on 2021-06-01. The extended characters (first byte 0x12 or 0x13) are not in that regulation. This revision’s table of them (§8.4) is the one the 608 standard’s extended character set is commonly given as, and it has not been checked against CTA-608-E: a reviewer with the standard SHOULD check it. The one code whose character this revision could not settle, 0x12 0x29, is left unassigned: the encoder never sends it, and the decoder refuses it.

8.2 The slot clock and the frame rate

The encoder writes field 1, data channel 1 (CC1) only. It writes on a slot clock: one byte pair per slot, at num / den slots a second, a parameter. Slot k is at media time k × 1000 × den / num ms, and the slot of media time t ms is the first at or after it, ⌈t × num / (1000 × den)⌉. The rate MUST be a positive num and den of at most 1 000 000 each, otherwise parameter_malformed.

The stream depends on the rate in its timing and its capacity: every slot carries one pair, so a faster clock carries more bytes a second, and each cue’s display starts at a slot of that clock. On line 21, and in 608 data carried with video of 30000/1001 frames a second, the rate is 30000/1001. A carrier that gives field-1 pairs at another rate states it, and the same cues then give the same pairs in other slots, or are refused too_dense where the pairs no longer fit. How a carrier places slots in video frames (cc_data of CTA-708, for example) is the carrier’s and is not specified here.

8.3 Bytes

• Every byte has odd parity: bit 7 is set exactly when the other seven bits have an even number of ones.
• A code is either one byte of the basic set or a two-byte code whose first byte is 0x10 to 0x1F. Standard bytes are packed two to a pair, in order. A lone standard byte before a two-byte code, or at the end of a cue’s codes, is paired with 0x00.
• Every two-byte code, control code or character alike, is sent twice, in consecutive slots: a decoder ignores a pair that repeats, in the next slot, a two-byte pair it acted on (47 CFR 79.101(i)(4)). So two equal two-byte characters in a row take four pairs.
• A slot that carries nothing carries 0x80 0x80. The hexadecimal form of a stream lists only the slots that carry a pair, one per line in ascending order: the slot in decimal (no leading zero), a space, the pair as four lowercase hexadecimal digits, then U+000A.

8.4 Characters

Each code point of a cue’s text, after substitution, is sent as the first that applies:

• Substitution. U+2019 is sent as U+0027; U+2013, U+2010 and U+2011 as U+002D.

• The basic set, one byte: 0x20 to 0x7A as the ASCII character of that value, except 0x2A á, 0x5C é, 0x5E í, 0x5F ó and 0x60 ú; and 0x7B ç, 0x7C ÷, 0x7D Ñ, 0x7E ñ. So ASCII *, \, ^, _, `, {, |, } and ~ are not basic. 0x7F (the solid block) is not sent.

• A special character, 0x11 then:

  0x30	0x31	0x32	0x33	0x34	0x35	0x36	0x37	0x38	0x39	0x3A	0x3B	0x3C	0x3D	0x3E	0x3F
  ®	°	½	¿	™	¢	£	♪	à	U+00A0	è	â	ê	î	ô	û

  0x39 is the transparent space, written U+00A0 in a cue.

• An extended character, 0x12 or 0x13 then the code below, sent after a standard fallback character, which the extended code backspaces over:

  second byte	0x20	0x21	0x22	0x23	0x24	0x25	0x26	0x27	0x28	0x29	0x2A	0x2B	0x2C	0x2D	0x2E	0x2F
  0x12	Á	É	Ó	Ú	Ü	ü	‘	¡	*	(unassigned)	—	©	℠	•	“	”
  0x13	Ã	ã	Í	Ì	ì	Ò	ò	Õ	õ	{	}	\	^	_	|	~

  second byte	0x30	0x31	0x32	0x33	0x34	0x35	0x36	0x37	0x38	0x39	0x3A	0x3B	0x3C	0x3D	0x3E	0x3F
  0x12	À	Â	Ç	È	Ê	Ë	ë	Î	Ï	ï	Ô	Ù	ù	Û	«	»
  0x13	Ä	ä	Ö	ö	ß	¥	¤	¦	Å	å	Ø	ø	┌	┐	└	┘

  The fallback of an accented capital is its unaccented capital (Á À Â Ã Ä Å → A; É È Ê Ë → E; Í Ì Î Ï → I; Ó Ò Ô Õ Ö Ø → O; Ú Ù Û Ü → U; Ç → C), of an accented small letter its unaccented small letter (ã ä å → a; ë → e; ì ï → i; ò õ ö ø → o; ù ü → u; ß → s); of “ ” « » it is ", of ‘ it is ', of ¡ it is !, and of every other extended character a space.

A code point none of these sends is char_not_in_608. A cue’s text after substitution is what a decoder shows.

8.5 Rows, columns and italics

A cue’s lines are laid out, in this order of checks:

1. A pop-on cue has at most 4 lines, a roll-up cue at most as many as its window has rows: otherwise too_many_rows.
2. Every code point is sent (§8.4): otherwise char_not_in_608.
3. Each line is at most 32 cells, one cell per code point: otherwise row_too_long.
4. Every italic run of a line begins at the line’s start or right after a U+0020, and ends at the line’s end or right before a U+0020: otherwise style_not_expressible. A mid-row code is shown as a space (47 CFR 79.101(h)(1)(i)), so italics can change only where a space already stands.

The rows and columns are §3.3’s; in roll-up every line is written on the base row (§8.7). A row is written as:

• Position. When the line begins italic at column 0: the italic preamble address code of its row (second byte 0x4E or 0x6E). When it begins italic at column c ≥ 1: the cursor is placed at c − 1, then the italics mid-row code (0x11 0x2E), which takes that cell as a space. Otherwise the cursor is placed at c. The cursor is placed at a column x by the preamble address code of the row with indent 4 × ⌊x / 4⌋ (second byte 0x50 + 2 × ⌊x / 4⌋ above the row’s base), then, when x mod 4 is not 0, tab offset x mod 4 (0x17 0x21 to 0x23).
• Characters, in order. A U+0020 just before an italic run is sent as the italics mid-row code; a U+0020 just after one, as the white mid-row code (0x11 0x20), which ends italics. Every other code point is sent as §8.4 says.

The first byte of a preamble address code for rows 1 to 15 is 0x11, 0x11, 0x12, 0x12, 0x15, 0x15, 0x16, 0x16, 0x17, 0x17, 0x10, 0x13, 0x13, 0x14, 0x14; the base of its second byte is 0x60 for rows 2, 4, 6, 8, 10, 13 and 15, and 0x40 for the others.

8.6 Pop-on

The cues are checked (§3.4, §8.5) in document order, then: two cues whose intervals overlap (a cue starting before the one before it ends) are cues_overlap, since pop-on shows one caption at a time. With sᵢ and eᵢ the slots of cue i’s start and end:

• eᵢ MUST be at least sᵢ + 2: otherwise too_dense.
• End of Caption (0x14 0x2F), sent twice, takes slots sᵢ and sᵢ + 1: the cue is shown from the first.
• Erase Displayed Memory (0x14 0x2C), sent twice, takes slots eᵢ and eᵢ + 1 when cue i is the last or sᵢ₊₁ ≥ eᵢ + 2. Otherwise the next caption replaces this one at sᵢ₊₁.
• The load of cue i: Resume Caption Loading (0x14 0x20), Erase Non-displayed Memory (0x14 0x2E), then its rows top to bottom (§8.5), packed (§8.3). Its units, a pair or a doubled code (two consecutive slots), are placed last first, each in the latest free slots before the unit after it (before sᵢ for the last), and none before slot sᵢ₋₁ + 2 (slot 0 for the first cue). Slots taken by an erase are not free. A load that does not fit is too_dense.

8.7 Roll-up

With a window of D rows (2, 3 or 4), the cues are checked as for pop-on, and cues whose anchors are not all bottom or all top are style_not_expressible: the window would move with the lines in it. The base row is 15 for the bottom and D for the top. Each cue’s codes are: the roll-up command of D (0x14 0x25 to 0x27), for the first cue only; then for each line, a Carriage Return (0x14 0x2D) when the base row holds a line (any line after the first since the start or since an erase), then the row (§8.5). The codes, packed, take consecutive slots from sᵢ, which MUST NOT be before the slot after the previous cue’s last (its last pair, or its erase); they MUST end before eᵢ when the cue is erased, and before sᵢ₊₁ when it is not. Otherwise too_dense. The erase is sent as in pop-on, at eᵢ, when cue i is the last or sᵢ₊₁ ≥ eᵢ + 2. Text is shown as it arrives, and a cue followed directly by the next stays on screen above it until it rolls off.

8.8 Decoding

The decoder reads the hexadecimal form (§8.3) and refuses as cc_malformed: a line not in that form, slots that do not ascend, a byte with even parity, a two-byte code the encoder does not send, a standard byte outside the basic set, an extended code left unassigned (§8.4), and a character before any mode is set. It keeps the displayed and the non-displayed memory of 15 rows of 32 cells, the cursor, the italic attribute and the base row, and acts on each pair as 47 CFR 79.101(f) to (i) say, ignoring a two-byte pair that repeats in the next slot the one it acted on. It reports:

• at each End of Caption, show with its slot and the displayed rows;
• in roll-up, line with the slot of the line’s preamble address code and the base row, when a Carriage Return, an erase or the end of the stream commits it;
• at each Erase Displayed Memory, erase with its slot.

A row is read from its first to its last cell holding a character that is not a mid-row code; cells of mid-row codes within it read as spaces. A code point is italic when its cell is, and a space is in an italic range when the cells on both sides are italic characters. What the decoder reports of an encoded stream is, for pop-on, each cue shown at sᵢ with its rows and erased at eᵢ as §8.6 says; and for roll-up, each line of each cue in order, from a slot of its cue, with the erases of §8.7.

9. CTA-708 (specified only)

This revision specifies the mapping of cues onto CTA-708 caption service blocks and does not implement it. CTA-708-E could not be read for it, so this section gives the semantics of each step and no byte values; an implementation MUST take the command codes, the window and pen attribute layouts and the caption channel packet rules from CTA-708-E itself, and its corpus would be the check of this section.

• Service. Captions in the document’s language are caption service 1, the primary service (47 CFR 79.102(c)); a second language is service 2. In HLS the rendition is a CLOSED-CAPTIONS tag with INSTREAM-ID="SERVICE1" (draft-pantos-hls-rfc8216bis-22 §4.4.6.1), and in an MSF catalog an accessibility descriptor of scheme urn:scte:dash:cc:cea-708:2015 (draft-ietf-moq-msf-01 §5.2.44).
• Windows. One window per anchor in use, its window id the anchor’s code (§3.3), defined with relative positioning: anchor point bottom-left, bottom-centre or bottom-right at 90% of the height for a bottom anchor, top-left, top-centre or top-right at 10% for a top one, at 10%, 50% or 90% of the width for left, centre and right; rows and columns locked; at most 4 rows (47 CFR 79.102(f)(1)) and 32 columns, so that a 4:3 receiver shows it (79.102(e)(3)).
• Pop-on. A cue’s window is defined hidden and cleared, its lines written (a carriage return between lines), and then displayed at the cue’s start and hidden or deleted at its end, the display and hide commands taking the place of 608’s End of Caption and erase.
• Roll-up. One visible window of D rows at the base anchor, lines written with a carriage return before each, as in §8.7.
• Italics. Set by the pen attributes before a run and cleared after it, at any position: 708’s pen attributes take no cell, so §8.5 rule 4 does not apply.
• Characters. The G0, G1 and G2 sets as 47 CFR 79.102(d) requires decoders to support them, with its substitutions where a decoder lacks a G2 character; a code point in none of them is refused, as §8.4 refuses one that 608 cannot send.
• The label. 708 has no field for it: §10.6 applies, as for 608.

10. The disclosure label

10.1 Its source

The label of a cue is its origin and, for origins 2 and 3, its model digest, as the caption event carries them (live-events §4.10). A cue is machine-generated when its origin is 2 or 3. Every mapping of §5 to §8 carries the label, in band where the target can (§10.2 to §10.4) and in the sidecar where it cannot (§10.5, §10.6). A machine caption with no model digest is refused wherever it is met: malformed_payload in an event (§4.1), model_missing in a cue (§3.4) and in a WebVTT label (§5.3).

10.2 HLS

draft-pantos-hls-rfc8216bis-22 §4.4.6.1 defines, in CHARACTERISTICS: “Any Rendition MAY include the “public.machine-generated” characteristic to indicate the Rendition was authored or translated programmatically.” This extension uses that characteristic as the draft defines it; it is in the IETF draft itself, so no other document is needed for it. The EXT-X-MEDIA tag of a caption rendition is written, attributes in this order:

• a WebVTT or IMSC1 rendition: #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="G",NAME="N",LANGUAGE="L",CHARACTERISTICS="C",URI="U";
• 608 on CC1: #EXT-X-MEDIA:TYPE=CLOSED-CAPTIONS,GROUP-ID="G",NAME="N",LANGUAGE="L",INSTREAM-ID="CC1",CHARACTERISTICS="C".

C is the accessibility characteristics the producer states (for captions, public.accessibility.transcribes-spoken-dialog and public.accessibility.describes-music-and-sound), in its order, then public.machine-generated when any cue of the rendition is machine-generated, joined by commas; CHARACTERISTICS is left out when C is empty. A rendition with a machine-generated cue MUST carry the characteristic and one without MUST NOT. G, N, U and each stated characteristic are one or more of the ASCII letters, digits, space, ., _, -, / and :; a stated characteristic holds no comma and is not public.machine-generated itself; otherwise parameter_malformed. A CLOSED-CAPTIONS tag names a channel within the video renditions rather than a rendition of its own (§4.4.6.1 there), and this extension carries the characteristic there too.

10.3 C2PA and IPTC

For a document or rendition, its cues give:

• an IPTC digital source type (the vocabulary http://cv.iptc.org/newscodes/digitalsourcetype/): digitalCreation when every cue is human (or there is none), trainedAlgorithmicMedia when every cue is machine, compositeWithTrainedAlgorithmicMedia when every cue is machine-reviewed, and compositeSynthetic when the cues have more than one origin. It is the digitalSourceType of the c2pa.created action of the rendition’s C2PA manifest (C2PA 2.4 §18.15.4.5).
• one c2pa.ai-disclosure assertion (C2PA 2.4 §18.28) per model, in the order models first appear in document order, labelled c2pa.ai-disclosure, c2pa.ai-disclosure__1, … (C2PA 2.4 §6.4), each with modelType c2pa.types.model (the generic model type of Table 12 there), modelIdentifier tag:transaction.science,2026:wai/model/ and the digest in lowercase hexadecimal, and contentProfile.humanOversightLevel fully_autonomous when any cue of that model is machine, and human_validated otherwise. This pairing of digital source type and oversight level is C2PA 2.4 §18.28.3’s.

The statement is written as the JSON {"digitalSourceType":DST,"assertions":[{"label":L,"data":{"modelType":"c2pa.types.model","modelIdentifier":ID,"contentProfile":{"humanOversightLevel":H}}},…]}. Embedding it in a manifest, and signing that, is C2PA’s and provenance’s.

10.4 In WebVTT and IMSC1

• WebVTT. A machine or machine-reviewed cue’s payload is one cue class span (WebVTT §4.2.2) whose classes are wai-machine or wai-machine-reviewed, then wai-model- and the digest in 64 lowercase hexadecimal digits: <c.wai-machine.wai-model-…>…</c>. A WebVTT parser keeps the classes on the cue’s nodes, where a style sheet (::cue(.wai-machine)) or a script reads them, and draft-wilaw-moq-webvtt-msf-00 §3 keeps the tags in its text.
• IMSC1. The cue’s p has wai:origin (machine or machine-reviewed) and wai:model (the digest in lowercase hexadecimal), in the namespace tag:transaction.science,2026:wai/timed-text.
• IMSC1 timeline, cue mode. wai_origin and wai_model in the cue entry’s data (§7.2).

A human cue carries none of these.

10.5 The sidecar

The sidecar of a document or rendition lists every cue, in document order, with its label:

{"wai_timed_text":1,"lang":LANG,"cues":[{"start":S,"end":E,"origin":O},{"start":S,"end":E,"origin":O,"model":M},…]}

O is human, machine or machine-reviewed, and model is present for the last two. In HLS the sidecar is published with the multivariant playlist by #EXT-X-SESSION-DATA:DATA-ID="science.transaction.wai.timed-text",URI="U",LANGUAGE="L" (draft-pantos-hls-rfc8216bis-22 §4.4.6.4; JSON is the default format), one per caption rendition; in a package it is a component beside the rendition it describes.

10.6 A target that cannot carry it

A target that cannot carry the label in band (608, 708, and an IMSC1 cue-mode consumer that ignores unknown members) MUST be published with the sidecar, and with the rendition-level marks it can carry: the HLS characteristic (§10.2), the catalog field (§7.4) and the C2PA statement (§10.3). A mapping MUST NOT publish machine-generated captions in a target that carries no label while publishing no sidecar for them. A converter that reads a WAI caption target and writes another MUST carry the label into the new one.

11. Obligations of a WAI player

These are requirements on a WAI player that shows captions. Each cites the provision it answers by identifier; they are written as engineering requirements, and meeting them is not a statement about any law.

1. Captions can be shown. A player of video with synchronised audio MUST have a mode that shows the captions available, and MUST let the viewer choose to show closed captions (EN 301 549 V4.1.1 clause 7.1.1; WCAG 2.2 Success Criteria 1.2.2 and 1.2.4, which ask content to provide captions, prerecorded and live).
2. In time. A player MUST show a cue within 100 ms of its time (EN 301 549 V4.1.1 clause 7.1.2), and SHOULD within 20 ms.
3. Preserved. A player or converter that transmits, converts or records captions MUST keep their position and style (EN 301 549 V4.1.1 clause 7.1.3): this extension’s mappings keep the anchor and italics, or refuse (§3.5, §8.5).
4. Under the viewer’s control. A player that controls how captions look MUST let the viewer change their characteristics: at least size, colours and background (EN 301 549 V4.1.1 clause 7.1.4). A viewer’s settings override the anchors and italics of this extension where they conflict, as graphics-vector §5.7 has them override drawn captions.
5. Discoverable. A player whose purpose is video MUST offer turning captions on and off as a single operation, at the same level as the volume control (EN 301 549 V4.1.1 clause 7.3).
6. Persistent. A player MUST keep the viewer’s choice, captions on or off, their language and their characteristics, from one item and one session to the next, until the viewer changes it (EN 301 549 V4.1.1 clause 7.3, NOTE 3).
7. Labelled. A player MUST show, with every machine or machine-reviewed caption, that it is machine-generated, in a way the viewer can perceive without turning on another feature, and SHOULD let the viewer see the model that produced it. A player MUST NOT remove the label when the viewer’s style settings apply.

12. Reasons

A record live-events refuses keeps that extension’s reason (§9 there).

13. Conformance

live-conformance/captions/ holds the corpus, written from fixed seeds by wai_timed_text_vectors gen: vectors.json and every expected output, each a file of its own (.vtt, .ttml, .608.hex). It has:

• event vectors: WEV1 records in stream order, with the outcome §4.3 gives them, the revisions, what a sink shows at stated instants (§4.4), the cues the stream resolves to or the refusal (§4.5), and every target’s output. They are a human-captioned lesson, a machine-captioned lesson whose partials are revised to final, and a mid-stream join, both on the events track (the joiner’s records, its gap, and what it shows) and on the timelines; and one vector for each refusal of §4.
• cue vectors: cues given as a list, with each target’s output or refusal: every anchor and italic placement, the disclosure label through every target, 608’s special and extended characters and substitutions, roll-up with windows of 2, 3 and 4 rows and on a second slot clock, and one vector for each refusal of §3, §5, §6 and §8.
• read cases: WebVTT the reader refuses, each with its reason; and decode cases: 608 streams the decoder refuses.

Every 608 code path §8 uses is in a committed file: the preamble address codes of indent and of italics, each tab offset, both mid-row codes, special and extended characters, Resume Caption Loading, Erase Non-displayed Memory, End of Caption, Erase Displayed Memory, each roll-up command and Carriage Return. Every reason of §12 refuses a case.

A conforming implementation gives every vector and case the outcome, the reason, the index, the revisions, what is shown, the cues and every byte of every output the corpus states. wai_timed_text_vectors verify and the test tests/timed_text.rs check that every file regenerates byte for byte, that no other file is there, and that wai-rs gives every case its outcome. live-conformance/timed_text_verify.py is the second implementation, written from this text: it shares no code with wai-rs, verifies events with live_events_verify.py, writes WebVTT, IMSC1, the timelines, 608 and the disclosure from §5 to §10, checks every IMSC1 document’s structure (§6.3) with an XML parser and the hypothetical render model (§6.2), and requires every mutated copy of the corpus to fail.

14. Security and privacy considerations

• Every revision is a signed event, and its annotation another: a relay can drop either and cannot alter one. A dropped annotation makes its caption’s annotation unknown, which a sink shows as a plain caption (§4.4); a dropped revision is a delivery gap (live-events §6.2).
• Only a revision’s own principal can supersede it (§4.3), so one participant cannot rewrite another’s captions.
• A partial’s stable prefix is a promise its producer’s later revisions are checked against (stability_broken); it is not a check that the speech was transcribed correctly.
• The label is the producer’s statement. A model digest names a model; it does not prove that model wrote the text. A producer that labels machine text human misleads its viewers, and the signature on the event makes that statement attributable to it.
• Text is untrusted input. Writers escape it (§5.1, §6.1, §7) and the reader accepts only the subset (§5.3), so cue text cannot inject markup, a cue setting or a timestamp into a target.
• The model digest can identify the vendor or version of a recognition service. A producer that does not want to publish it can only use origin 1, which would mislabel machine text; this extension does not offer an anonymous machine origin.