Skip to main content

WAI Extension: Peer Delivery

Mirrored from the canonical text at commit c5e7133e ().

Status: Draft. Registers no capability and no record. It is the exchange by which a sink fetches objects from the neighbours that hold them before it spends the backhaul, over the records other texts define: a neighbour’s holdings record says where to ask (holdings §5.4), the publisher’s receipts say what the answer must be (jwp-receipts §3), and the asker’s settlement acknowledgement says what was served (settlement §3). A sink joins by its own choice: asking a peer and serving one are each off unless the operator’s policy and the viewer’s setting both allow it (§3). The exchange is eight frames over a link that keeps message boundaries, a WebRTC data channel among them, and three requests over HTTP for a neighbour reached that way (§4). An asker asks only for an object whose object receipt it already holds, takes nothing a peer sent before the bytes verify under that receipt, sets aside a holder that served other bytes and a record that stated what its holder did not hold, and goes to the origin for what is left (§5). It signs an acknowledgement for what it verified, by holder and by window, so what each peer served recomputes from signed records (§6). §9 says what a peer learns of a viewer. WAI defines no transport. The reference implementation is the peer_delivery module of wai-rs (feature peer_delivery) and, in a page, the peer bundle of wai-web; the corpus is live-conformance/peer-delivery/, and its second implementation is live-conformance/peer_delivery_verify.py. Keywords MUST, MUST NOT, SHOULD, MAY are RFC 2119/8174.

Texts read for this revision: the WAI texts this one cites; RFC 8831 (§6.1, §6.6: a data channel’s reliability and its message size) and RFC 8832 (§5.1: the channel’s type, label and protocol); RFC 8828 (§3, §5.1, §5.2: which addresses a WebRTC endpoint gathers, and its four modes); RFC 8826 §4.2.4; draft-ietf-mmusic-mdns-ice-candidates-03 §1 and §3.1.1; RFC 7574 §3.2 and §5; RFC 9110 §15.5.5, §15.5.21 and §15.6.4; the W3C WebRTC: Real-Time Communication in Browsers Recommendation (a data channel’s label, protocol, order and largest message, and the transport policy relay of a connection’s configuration); the W3C Service Workers text, §5.4 (Cache); and the Web Incubator Community Group’s draft Local Network Access (§1, §2.1 to §2.3). A clause number of one of them in this text is that text’s.

1. Scope

In scope: who may ask and who may serve (§3); the exchange between an asker and a holder, over a link that keeps message boundaries and over HTTP (§4); what an asker does with its neighbours’ records and with each answer (§5); the acknowledgement of what a peer served, its object class and what a holder checks of it (§6); what a page uses of its browser (§7); and what each party learns (§9).

Out of scope: how two sinks are introduced to each other and how a link between them is set up, which is the deployment’s; how holdings records reach an asker before it has a link (holdings §1); how a node chooses what to keep; a price; and a record of service beside settlement’s acknowledgement, which this text does not add.

Relationship to the core spec: this text defines no capability, no payload format and no record (SPEC §5). Every signed byte it relies on is another text’s.

2. Terms

3. Joining

3.1 The operator’s policy

The operator of the service a sink takes delivery from states, in the configuration it gives its sinks, four values:

valuemeaningwhere the operator states nothing
askits sinks may fetch from peersno
serveits sinks may serve peers from what they holdno
max_object_bytesthe most bytes of one object a sink takes from a peer4,194,304
relay_onlyits sinks reach peers only through a relay (§9)no

How the configuration reaches a sink is the deployment’s. A sink with no configuration has an operator that states nothing.

3.2 The viewer’s setting

A sink keeps two settings of its own, ask and serve, each off until the person using the sink turns it on, and each theirs to turn off again. A sink MUST NOT turn either on by itself, and MUST show, where it offers them, what §9 says a peer learns.

3.3 The rule

A sink asks only while the policy’s ask and its own ask are both on, and serves only while the policy’s serve and its own serve are both on. Asking is not serving: either may be on without the other.

4. The exchange

The exchange is version 1. An asker opens it with record-want, which states the version; a holder that does not speak that version closes the link. Each frame is one message of the link. On a link, an asker has at most one request outstanding: it sends record-want, want or ack and waits for the whole reply before it sends another.

4.2 Frames

A frame is one byte of type, then its body:

typenamefrombody
1record-wantasker1 byte: the version, 1
2record-headholderlen, 8 bytes: the bytes of the holder’s record, 1 to 1,048,576, which follow in data frames
3wantaskergroup, 32 bytes, and index, 8 bytes: object index of the group with that link hash
4headholdergroup, index, and len, 8 bytes, at least 1: the object’s bytes, which follow in data frames
5dataholder1 to 16,384 bytes
6missholdergroup, index, and code, 1 byte: 1 the holder does not hold the object now, 2 its record does not list it, 3 it will not serve now
7ackaskerrecord_len, 4 bytes, at least 1; an acknowledgement of that many bytes (§6.1); then its item list, 1 to 256 items of 48 bytes
8ack-resultholderdigest, 32 bytes, the acknowledgement’s (settlement §3.5), and 1 byte: 0 taken, 1 not taken

A reader reads exactly the bytes of one message as one frame and refuses it peer_frame_malformed when it is empty, its type is not 1 to 8, its body is not of its type’s length, or a field is outside the range the table gives. A record-want of another version is peer_frame_malformed in this revision. An object’s bytes travel in frames of at most 16,384 bytes because RFC 8831 §6.6 asks a sender that cannot interleave messages to keep each within 16 KB.

4.3 The holder

A holder answers record-want with record-head and the bytes of the one record of its own that stands (holdings §5.3), and want with the first of these that holds:

  1. miss, code 3, when it may not serve (§3.3) or will not serve now;
  2. miss, code 2, when that record does not hold the object (holdings §5.4): a holder serves over this exchange only what it has told its neighbours it holds;
  3. miss, code 1, when it does not hold the object’s bytes now;
  4. head with the object’s length, then its bytes.

It answers ack as §6.3 says. A holder MUST NOT send bytes it has not itself verified under the object’s receipt, and learns nothing from doing so: its asker verifies them again (§5.3). A frame a holder is sent that is not record-want, want or ack, or that arrives before its reply to the one before is whole, is peer_frame_unexpected, and it closes the link.

4.4 A reply

The bytes a record-head or a head announces follow in data frames, each of 16,384 bytes but the last, which holds what remains. An asker holds a reply to what it asked:

After any of the three the asker closes the link and asks that holder nothing more over it. A reply is whole when its bytes are exactly len; an asker never holds more of one than its head stated, and it stores nothing of a reply that is not whole.

4.5 Over HTTP

A holder reached by HTTP answers, under its base URL:

requestanswer
GET holdings200 with the record’s bytes
GET object/<group>/<index>200 with the object’s bytes, whose length the response states; 404 for miss code 1 or 2; 503 for code 3
POST ack, its body an ack frame’s body200 with the acknowledgement’s digest when it is taken; 422 when it is not

<group> is the link hash as 64 lowercase hexadecimal digits and <index> the position in decimal with no leading zero. The rules of §4.3 and §4.4 hold with a response for a reply: a stated length above max_object_bytes, or a body that reaches it with no length stated, is peer_object_too_large, and the asker reads no further; any other status, a body shorter or longer than its stated length, and a request that fails are peer_link_failed. A 404 is read as code 1. An asker’s requests carry no credentials, no cookie and no referrer (§7).

5. The asker

5.1 Its records

An asker holds its neighbours’ records in the order it prefers them (holdings §5.4) and judges them at an instant as holdings §5.1 to §5.3 do, against the group receipts it holds: a record that is refused, superseded or set aside there takes no part. Of those that stand, it sets aside the ones §5.4 names. Its own record, where it has one, is in no route: an asker does not ask itself.

5.2 The route

For a wanted object, the asker’s route is, at an instant:

  1. the origin alone, peer_not_permitted, when it may not ask (§3.3);
  2. the origin alone, peer_receipt_not_held, when it does not hold the object’s object receipt and the group receipt it is included under, both verified (jwp-receipts §3 steps 1, 2 and 5): it could hold nothing a peer sent for it to a signed digest, so it asks no peer for it;
  3. otherwise the holders whose record stands (§5.1) and holds the object, in the asker’s order (holdings §5.4), then the origin.

An asker MUST NOT send a holder a want for an object that holder’s standing record does not hold. So a holder learns of an asker only that it wants objects the holder has itself told its neighbours it holds (§9).

5.3 The answer

The asker asks the holders of the route in order, one at a time, and stops at the first that serves bytes it takes:

When no holder is left it fetches the object from the origin. What comes from the origin is verified as every object is; this text adds nothing for it.

5.4 What an asker sets aside

Nothing here scores a holder beyond the session, and nothing is told to anyone: what an asker sets aside is its own state (holdings §9).

6. The acknowledgement

6.1 What the asker signs

For each holder it took objects from, an asker signs, at the end of a window, one acknowledgement (settlement §3) of exactly the objects that holder served and the asker verified in the window:

An acknowledgement sent over a link names at most 256 objects; an asker with more closes the window sooner. Where it took objects from several holders it signs one acknowledgement for each. It sends each to its holder in an ack frame with the entry’s item list, or as §4.5’s POST, and keeps it for the party it settles with.

An object is acknowledged only after it was taken (§5.3). Bytes that failed, a reply that was not whole and an object fetched from the origin are in no acknowledgement to a holder.

6.2 The class, and whose word

An object a holder serves in answer to a want is served from what it holds in answer to a request, so its entry is of class 2, fetch (settlement §3.3), whatever the object is: a retrieval class. This revision’s exchange carries no proof of possession of a grant (live-rooms §10.5), so its entries name no grant: each is a retrieval no grant was presented for, acknowledged, counted in a statement’s lines and classes, and free under a statement’s tariff (settlement §6.2, §7.2). A deployment that pays its peers pays them on the acknowledged bytes, outside the statement.

A payer honours an acknowledgement only from a key it can place (settlement §13). Outside a room no grant names the asker, so a payer takes an asker’s word when its key is in a population the payer stated (cache-authority §8.1), and reports any other acknowledgement apart, paid as no one’s. The holder’s key is placed as cache-authority §4.4 places any server key.

6.3 What a holder checks

A holder takes an acknowledgement sent to it when, in this order:

  1. the record reads and verifies (settlement §3.6), with that section’s reasons;
  2. it is of version 1, mode 1 and role 2, with one entry, of class 2, whose grant and proof are zero: otherwise peer_ack_invalid;
  3. its upstream is the holder’s key: otherwise ack_wrong_party;
  4. the item list reads and recomputes against the entry (settlement §3.6): items_invalid, items_mismatch;
  5. each item is an object the holder served over that link and holds no acknowledgement of, whole: otherwise ack_not_served.

It answers ack-result with the record’s digest and 0 when it takes it, 1 when it does not. An asker may acknowledge fewer objects than it was served: what it did not verify is not its to sign for, and what a holder served that nobody acknowledged it may state in a declaration (settlement §4).

7. In a page

A sink that is a page of a browser uses what the browser gives it:

8. Reasons

The codes this text adds, each with where it is raised:

reasonraised at
peer_not_permitted§3.3, §5.2
peer_frame_malformed§4.2
peer_frame_unexpected§4.3, §4.4
peer_object_too_large§4.4, §4.5
peer_link_failed§4.4, §4.5
peer_declined§5.4
peer_receipt_not_held§5.2
holdings_overstated§5.4
peer_bytes_refused§5.4
peer_ack_invalid§6.3
ack_not_served§6.3

An implementation also raises every code holdings §7 lists, for a record; jwp-receipts §3’s reasons, for a receipt and for bytes that fail one; and, for an acknowledgement, settlement §11’s ack_invalid, self_acknowledged, items_invalid, items_mismatch and ack_wrong_party, with truncated, malformed, unknown_version, bad_key and bad_signature.

9. Security and privacy considerations

Peers are not trusted. Nothing a peer says is evidence of an object’s bytes. An asker asks only for objects whose receipts it holds, takes nothing before it verifies, and a holder that sends other bytes gave it one wasted exchange and is asked nothing more. A holder’s record that overstates costs one exchange and is set aside. A holder cannot make an asker hold more than max_object_bytes of one reply, nor a byte more than a head stated.

What a holder learns of an asker, and what it does not:

What an asker learns of a holder: its address, as above; its record, which says what it holds (holdings §9); and whether it served.

What the service learns: whatever introduces two sinks learns that it did; the origin sees no request for an object a peer served, and learns of it from the acknowledgements, where the asker hands them on.

Whose choice. Joining is the sink’s (§3): a viewer who has not turned ask on has its address shown to no peer and names no object to one, and a viewer who has not turned serve on tells no neighbour what it holds. Turning a setting off stops the next exchange; what a peer already learned it keeps. Where a regime asks the operator to state the lawful basis on which it processes what an acknowledgement reveals, the operator states it as settlement §13 says.

A holder’s key on the link. A link is not authenticated by this exchange. A party that answers on a link in a holder’s place can serve only bytes that verify, and the acknowledgement names the record’s key, so what it gains is to have served for another’s credit. One that sends other bytes in a holder’s place has that holder set aside by that asker for its session: it costs the asker a route and the holder that asker’s requests, and nothing else.

Bounded work. One request outstanding per link; a reply of at most max_object_bytes; a record of at most 1,048,576 bytes; an acknowledgement of at most 256 items; a verification of one hash and one inclusion proof per object.

10. Conformance

live-conformance/peer-delivery/ holds the corpus, written from fixed seeds by wai_peer_delivery_vectors gen as vectors.json. It has:

A conforming implementation gives every case the outcome the corpus states. wai_peer_delivery_vectors verify and the test tests/peer_delivery.rs check that the file regenerates byte for byte, that wai-rs gives every case its outcome, that every reason of §8 appears, and that §8 lists exactly the reference’s codes. live-conformance/peer_delivery_verify.py is the second implementation, written from this text: it shares no code with wai-rs; it takes records, receipts and acknowledgements from the second implementations of the texts that define them (holdings_verify.py, jwp-conformance/jwp_verify.py, settlement_verify.py), writes the frames, the gate, the route, the session and the holder’s checks itself, rebuilds each acknowledgement’s bytes before its signature and verifies the signature, mutates copies of the corpus and requires every copy to fail, and drops each of its own checks in turn and requires the corpus to catch the weakened verifier.

In a page, wai-web/peer/ holds the client and two runs of it over the corpus’s world. verify_peer.mjs runs an asker and its holders in one process over links in memory and over a local HTTP link, with the faults above planted. run.mjs opens pages in a browser: holders that serve over data channels, one of them altering bytes and one listing what it does not hold, a local HTTP holder, an asker with both switches on and one with its setting off; it requires every object the asker shows to have verified, the altering peer to be named by no acknowledgement, every acknowledgement to be taken by its holder, the origin to have served only what no peer did, and the page with its setting off to have opened no connection.