Skip to main content

WAI Extension: Cache Metadata

Mirrored from the canonical text at commit c5e7133e ().

Status: Draft. Registers no capability and no record. It is the form in which the cache rules of http-delivery §9 are handed to a cache another party runs, an access network’s edge cache among them, in the metadata that delegation already uses: a CDNI metadata configuration (RFC 8006) for the paths WAI serves, in a form of RFC 8006’s objects alone and in a form that adds the cache-control and compression objects of the open-caching configuration interface (§4), and the capability advertisement (RFC 8008) by which such a cache says what it can enforce and where (§6). A writer renders the configuration; a reader holds one to WAI’s rules exactly and refuses it, with one reason, where it states anything else: a lifetime that overrides the origin’s, a range rule dropped, compression, a cache key that is not the path, a path of WAI’s that an earlier pattern takes (§5). A configuration is an instruction and evidence of nothing. A cache is not trusted for content: a sink holds every byte it is served to the publisher’s signed receipts, whatever the cache was told (jwp-receipts §3; cache-authority §6), and what a cache served is on the signed records other texts define (§7). The reference implementation is the http_binding::cdni module of wai-rs (feature http); the corpus is cache-conformance/cdni/, and its second implementation is cache-conformance/cdni_verify.py. Keywords MUST, MUST NOT, SHOULD, MAY are RFC 2119/8174.

Texts read for this revision: the WAI texts this one cites; RFC 8006 §3.3, §4.1, §4.2.1, §4.2.2.2, §4.2.6, §4.3.1 to §4.3.3, §4.3.8, §6.5 to §6.7 and §7.3; RFC 8008 §5.1 to §5.4, §5.7 and Appendix B; RFC 9388 §1 and §2; draft-ietf-cdni-cache-control-metadata-06 §3; draft-ietf-cdni-edge-control-metadata-13 §4. The open-caching configuration interface is published by the Streaming Video Technology Alliance as a set of documents listed on its site; their text was not retrievable without an account when this revision was written and was not read. The two drafts above are the form in which that interface’s cache-control and compression objects were brought to the IETF, and this text follows the drafts. A clause number of one of them in this text is that text’s.

1. Scope

In scope: the paths a configuration covers and the rule each takes (§4.1); the configuration document and the objects each path carries, in two forms (§4.2 to §4.5); how a reader holds a configuration to those rules (§5); a cache’s capability advertisement, its footprint, and whether it supports a configuration in a region (§6); and which of a cache’s roles and records this text relies on (§7).

Out of scope: the metadata interface itself (how a configuration is retrieved, RFC 8006 §6); request routing; access control metadata (location, time and protocol lists, authorisation), which this revision’s form does not carry; a configuration for paths that are not WAI’s; purge and pre-positioning; and whether a cache does what it was told, which cache-conformance/ tests against a running cache and a sink never relies on.

Relationship to the core spec: this text defines no capability, no payload format and no record (SPEC §5). It signs nothing. It restates no rule: the rules are http-delivery §9’s, and this is one way of writing them down for a cache.

2. Terms

3. What a configuration is

4. The configuration

4.1 The classes

A configuration covers ten classes of path. Each is one PathMatch (RFC 8006 §4.1.4) whose pattern is matched case-sensitively, and they are written in this order:

classpatternwhat is served therefreshness
index*.wai.receiptsa receipt index (http-delivery §8.2)revalidate
head-proof*.wai.hpa head proof (http-delivery §6.2)origin
object*.waian object under its name (http-delivery §5.2)origin
manifest-store*.c2paa C2PA manifest store (http-delivery §8.9)origin
receipt*/rcpt/*.jsona receipt file (http-delivery §8.1)origin
playlist-receipt*/mr/*.jsona playlist receipt (http-delivery §8.6)origin
set/.well-known/wai-sets/*a parameter set under the origin’s well-known root (http-delivery §8.7)origin
key-log/.well-known/wai-keysthe key log (http-delivery §8.4)origin
key-status/.well-known/wai-key-statuskey statuses (http-delivery §8.4)origin
grants/.well-known/wai-grants/*.jsona module’s grants (http-delivery §8.4)origin

A pattern is RFC 8006 §4.1.5’s: * matches any run of characters, / among them, ? one character that is not /, and $ before $, * or ? makes it a literal. A path’s class is that of the first pattern, in the table’s order, that matches it; a path none matches is not WAI’s and this text says nothing of it.

Freshness is the one rule that differs by class. origin: the cache stores the response and keeps it fresh for as long as the origin’s Cache-Control says, neither longer nor shorter, and does not revalidate it while it is fresh (http-delivery §9). revalidate: the cache revalidates with its source on every use, as §9 rule 6 requires of a receipt index.

An object served under an alias (http-delivery §5.2), a parameter set under a root other than the origin’s well-known one, and a playlist are the deployment’s own paths: an upstream that wants a cache to hold them lists them after the ten classes, and this text’s reader does not read them.

4.2 The document

A configuration is a HostIndex with a HostMatch for the host WAI’s paths are served under. The writer writes one host, in JCS (RFC 8785):

{"hosts":[{"host":"media.example","host-metadata":{
  "metadata":[ <MI.SourceMetadata> ],
  "paths":[
    {"path-metadata":{"metadata":[ <the class's objects> ]},
     "path-pattern":{"case-sensitive":true,"pattern":"*.wai.receipts"}},
    … the nine other classes, in §4.1's order … ]}}]}

The host’s metadata is inherited by every path that does not define the same type (RFC 8006 §3.3), which is why it holds the sources and nothing else: an object placed there would apply to every class without appearing in any.

4.3 A class’s objects

A class carries, in the order written:

formobjects
cdniMI.Cache, WAI.ByteStore
open-cachingMI.Cache, MI.CachePolicy, MI.StaleContentCachePolicy, MI.AllowCompress, WAI.ByteStore

Every class of one configuration is in the same form.

MI.Cache (RFC 8006 §4.2.6) is {"include-query-strings":[]}, with no exclude-path-pattern: the cache key is the whole path and nothing of the query. A name is then one stored object, whatever a request appends to it.

4.4 WAI.ByteStore

RFC 8006 has no object for range handling, for transformation, for request-header variation or for a lifetime, and the open-caching objects cover the last of these only. WAI.ByteStore is a GenericMetadata type of WAI’s own that states §9’s rules in one place. Its name identifies whose it is, as RFC 8006 §6.5 recommends for a type outside the registry; it is not registered in the CDNI Payload Types registry. Its value has exactly these members:

membervaluethe rule of http-delivery §9 it states
version1
store"exact"rule 1: stored and served byte for byte, never transformed, no content coding applied
range"single"rule 2: a single byte range answered 206 with the exact bytes from a stored complete object, 416 with Content-Range: bytes */<len> where it cannot be satisfied
headers"origin"rule 3: ETag, Content-Type, Cache-Control and, on a 200, Content-Length as the origin sent them
vary"none"rule 4: a stored object varied by no request header
if-range"strong"rule 5: If-Range evaluated with a strong comparison
freshness"origin" or "revalidate"rule 6, and the class’s freshness (§4.1)
otherwise"relay"rule 7: a request the cache cannot serve under these rules is passed through uncached and the origin’s bytes relayed unchanged

A cache that lists WAI.ByteStore in its advertisement (§6) says it enforces all of them. One value has one meaning: a later revision that adds a value or a member changes version.

4.5 The open-caching objects

In the open-caching form a class also states, in that interface’s own objects, the part of the rules they can express. They say nothing WAI.ByteStore does not; they are there so that a cache which applies those objects applies WAI’s values and not its defaults or a parent’s.

A class carries no MI.ComputedCacheKey (draft-ietf-cdni-cache-control-metadata-06 §3.5), which would key a stored object by something other than its path, and no other type.

5. Reading a configuration

Inputs. The configuration’s text and the host. Output. The form, the host as written and the sources, or a refusal with one reason. A configuration that is accepted, written again from that output, is the writer’s bytes for it.

The reader refuses at the first of these steps that fails. Wherever a step reads an object of RFC 8006’s or of §4, a value of another JSON type, a member the object’s definition does not give it, or a required member that is absent is cdni_form, unless the step names another reason.

  1. The document. The text is JSON in which no object has a member name twice (jwp-receipts §3.1 step 1 reads JSON so), an object with exactly hosts, an array: cdni_form.

  2. The host. The entries of hosts are read in order to the first whose host, in lowercase, is the host asked for, in lowercase. An entry met on the way that has a member href is cdni_link: a HostMatch given by reference could be the host’s, and hosts are matched first to last. No such entry: cdni_host_missing. Entries after it are not read.

  3. Embedded. Any object within that HostMatch that has a member named href is cdni_link. Then the HostMatch has exactly host and host-metadata, and the HostMetadata exactly metadata and paths, both arrays.

  4. The host’s metadata. Each entry of metadata, in order, is an object with exactly generic-metadata-type, a string, generic-metadata-value, an object, and mandatory-to-enforce, a boolean; mandatory-to-enforce of false is cdni_not_mandatory; and a type an earlier entry of the array has is cdni_metadata_duplicate. (RFC 8006 §3.3 has a receiver keep the first of two; a reader of WAI’s configuration refuses both.) Then the first entry whose type is not MI.SourceMetadata is cdni_computed_cache_key where it is MI.ComputedCacheKey and cdni_metadata_unknown otherwise. No MI.SourceMetadata: cdni_metadata_missing. Its value is as §4.2 says.

  5. The paths. Each entry of paths is an object with exactly path-pattern, itself exactly pattern, a string, and case-sensitive, a boolean, and path-metadata, itself exactly metadata, an array. Then, in this order:

    • a pattern of §4.1 that no entry has: cdni_path_missing;
    • a pattern of §4.1 that two entries have, or the entries with §4.1’s patterns in another order than the table’s: cdni_path_order;
    • an entry with another pattern before the last entry with one of §4.1’s: cdni_path_shadowed. Paths are matched first to last and only the first match is used (RFC 8006 §4.1.3), so such an entry may take a path of WAI’s, and the reader does not work out whether it does;
    • an entry with one of §4.1’s patterns whose case-sensitive is false: cdni_pattern_insensitive.

    Entries after the last class are not read.

  6. A class’s objects, for each class in §4.1’s order, step 7 for a class before step 6 for the next. Each entry of its metadata is read as step 4 reads one, with its reasons. Then the first entry whose type is none of the five of §4.3 is cdni_computed_cache_key where it is MI.ComputedCacheKey and cdni_metadata_unknown otherwise. Then the class’s types are exactly one form’s, in any order, and the same form as the classes before it: otherwise cdni_metadata_missing. The form of index, the first class, is the configuration’s.

  7. The class’s values, in this order:

    • MI.Cache: anything but exactly include-query-strings, an empty array, is cdni_cache_key;
    • MI.CachePolicy, in the open-caching form. internal and external are each a count of seconds or one of "as-is", "no-cache" and "no-store", and the two force- members booleans; an absent member has the draft’s default, "as-is" and false. For index, internal other than "no-cache", or force-internal not true: cdni_index_not_revalidated. For any other class, internal other than "as-is": cdni_ttl_override, forced or not, since an unforced value still applies wherever a response states no policy. Then external other than "as-is": cdni_ttl_override. An "as-is" policy says the same forced or unforced;
    • MI.StaleContentCachePolicy: stale-while-revalidating of true, a stale-if-error that lists anything, or a failed-revalidation-delta-seconds above 0: cdni_stale_serving. Absent members have the draft’s defaults, which serve nothing stale;
    • MI.AllowCompress: allow-compress of true: cdni_compression_allowed. Absent, it is false;
    • WAI.ByteStore: a member §4.4 does not list, or a version other than 1, is cdni_form. Then each member in this order, absent or of another value than §4.4’s: store, cdni_transform_allowed; range, cdni_range_rule_dropped; headers, cdni_headers_not_kept; vary, cdni_vary_allowed; if-range, cdni_if_range_weak; freshness, cdni_index_not_revalidated for index and cdni_ttl_override for any other class; otherwise, cdni_no_relay.

An accepted configuration states every rule of http-delivery §9 for every class, marks each mandatory, and lets no other entry take a path of WAI’s.

6. A cache’s advertisement

6.1 What a cache advertises

A cache that takes WAI’s configuration advertises three capabilities (RFC 8008 §5.1), each with the same footprint, in JCS:

{"capabilities":[
 {"capability-type":"FCI.DeliveryProtocol","capability-value":{"delivery-protocols":["https/1.1"]},"footprints":[…]},
 {"capability-type":"FCI.AcquisitionProtocol","capability-value":{"acquisition-protocols":["https/1.1"]},"footprints":[…]},
 {"capability-type":"FCI.Metadata","capability-value":{"metadata":["MI.SourceMetadata","MI.Cache","WAI.ByteStore"]},"footprints":[…]}]}

FCI.Metadata (RFC 8008 §5.7) lists MI.SourceMetadata and then the types of the form it supports, in §4.3’s order. RFC 8008 §5.7 reads the list as the only types the cache supports, so a cache that lists the open-caching form’s types supports the cdni form too.

6.2 Footprints

A footprint is written from regions in strictly ascending order of their bytes, each mapped to CDNI’s lowercase as rung-price §2 maps it:

A footprint object holds a region r: a countrycode one when it lists r’s country; a subdivisioncode one when it lists r; a footprintunion when one of its members holds r. A subdivision’s footprint never holds its country, as rung-price §2’s holds never does. A capability covers r when it has no footprints or every footprint object of its footprints holds r: several footprint objects on one capability narrow it (RFC 9388 §2.2).

6.3 Whether an advertisement supports a configuration

Inputs. The advertisement’s text; a configuration’s form and the protocols of its sources; the protocol the cache is to deliver by; and a region. The check is refused at the first of these that fails:

  1. The text is JSON as §5 step 1 reads it, an object with exactly capabilities, an array. Each entry is an object with exactly capability-type, a string, and capability-value, an object, and optionally footprints. The value of an FCI.DeliveryProtocol, an FCI.AcquisitionProtocol or an FCI.Metadata entry has exactly its one member (delivery-protocols, acquisition-protocols, metadata), an array of strings. An entry of another type is read no further than its footprints. Otherwise cdni_advert_form.
  2. Every entry’s footprints are read, entry by entry and in order, whatever the entry’s type. footprints is an array of objects with exactly footprint-type, a string, and footprint-value, an array: otherwise cdni_advert_form. A footprintunion whose value holds an entry that is not an object, or is itself a footprintunion, is cdni_advert_form (RFC 9388 §2.2). A footprint of any type other than the three of §6.2, or a value of a countrycode or subdivisioncode footprint that is not CDNI’s lowercase writing of a region code of that kind, is cdni_footprint_unmapped: an address block or a network number is not a region, and the check does not guess whether it lies in one.
  3. No FCI.DeliveryProtocol entry that covers the region lists the delivery protocol: cdni_delivery_unsupported.
  4. A source’s protocol that no FCI.AcquisitionProtocol entry covering the region lists: cdni_acquisition_unsupported.
  5. No single FCI.Metadata entry that covers the region lists MI.SourceMetadata and every type of the form (§4.3), compared without regard to case: cdni_capability_missing. The cache could not enforce an object the configuration marks mandatory, and RFC 8006 §6.6 would have it serve nothing.

An upstream SHOULD NOT delegate WAI’s paths to a cache whose advertisement fails this check for the region it would serve. An advertisement is the cache’s own unsigned word and places no key (§7).

7. Roles and records

This text adds no authority and no record. A cache that takes a configuration is, in the other texts’ terms:

8. Reasons

The codes this text adds, each with where it is raised:

reasonraised at
cdni_form§5
cdni_link§5 steps 2 and 3
cdni_host_missing§5 step 2
cdni_not_mandatory§5 steps 4 and 6
cdni_metadata_duplicate§5 steps 4 and 6
cdni_computed_cache_key§5 steps 4 and 6
cdni_metadata_unknown§5 steps 4 and 6
cdni_metadata_missing§5 steps 4 and 6
cdni_path_missing§5 step 5
cdni_path_order§5 step 5
cdni_path_shadowed§5 step 5
cdni_pattern_insensitive§5 step 5
cdni_cache_key§5 step 7
cdni_ttl_override§5 step 7
cdni_index_not_revalidated§5 step 7
cdni_stale_serving§5 step 7
cdni_compression_allowed§5 step 7
cdni_transform_allowed§5 step 7
cdni_range_rule_dropped§5 step 7
cdni_headers_not_kept§5 step 7
cdni_vary_allowed§5 step 7
cdni_if_range_weak§5 step 7
cdni_no_relay§5 step 7
cdni_advert_form§6.3 steps 1 and 2
cdni_footprint_unmapped§6.3 step 2
cdni_delivery_unsupported§6.3 step 3
cdni_acquisition_unsupported§6.3 step 4
cdni_capability_missing§6.3 step 5

A writer refuses, as cdni_form, a host that is empty or not in lowercase and sources §4.2 does not admit, and, as cdni_advert_form, regions that are not region codes in strictly ascending order and a protocol RFC 8006 §7.3 does not register; a check asked for a region that is not a region code is refused as cdni_advert_form too.

9. Conformance

cache-conformance/cdni/ holds the corpus, written by wai_cdni_vectors gen as vectors.json. Nothing in it is signed. It has:

A conforming implementation gives every case the outcome the corpus states. wai_cdni_vectors verify and the test tests/cdni.rs check that the file regenerates byte for byte, that wai-rs gives every case its outcome, that every reason of §8 appears, and that §8 and §4.1 list exactly the reference’s codes and classes. cache-conformance/cdni_verify.py is the second implementation, written from this text: it shares no code with wai-rs, uses the standard library alone, writes every configuration and advertisement itself and compares the bytes, reads every case, reads §4.1 and §8 from this text, mutates copies of the corpus and requires every copy to fail, and drops each of its own checks in turn and requires the corpus to catch the weakened verifier.

10. Security considerations