Skip to main content

WAI Extension: Task Fidelity

Mirrored from the canonical text at commit 498e6b88 ().

Status: Draft. Names the contract for media whose correctness is a downstream model’s behaviour, not a byte comparison — the class determinism-tiers cannot express. Sibling to that file, not a tier inside it: the tiers there are reproducibility guarantees a sink can check from the bytes alone, and a task claim is not one. No new wire format; a claim shape + an evidence grading over §7, modelled on energy-measurement’s acquisition class. §5 registers one receipt class, split-computation (jwp-receipts §2.4.4): what each side of a network cut in two ran, and the energy it spent. Its reference is the split module of wai-rs’s claim_receipt and the machine_coding module (feature machine_coding); its corpus is live-conformance/machine-coding/, and its second implementation live-conformance/machine_coding_verify.py. Keywords MUST, MUST NOT, SHOULD, MAY are RFC 2119/8174.

1. Why a second axis

determinism-tiers.md answers one question: does the sink reproduce the same bytes? A growing class of coded media is not answering that question at all. Three independent arrivals, none of them WAI’s:

WAI registers the two parts as candidates, wai.video.vcm and wai.tensor.fcm (SPEC §5; neither is emitted or dispatched, and neither has a probe: format-probes §19), but a registration cannot state what conformance means for them. ReplicateClass::Behavioral is a disclaimer, not a criterion. Task-equivalent but not bit-equal is unstateable, so an implementation either overclaims a reproducibility tier it does not meet or says nothing at all.

The distinction that makes this a separate file. A reproducibility tier is checkable by the sink from the bytes it received. A task-fidelity claim is not: checking it requires a model, a corpus, and a test run. Listing the two together would blur the one property determinism-tiers.md currently keeps clean — that its tiers are verifiable without leaving the object.

2. The claim

A task-fidelity claim is a tuple, and it is meaningless with any element missing:

elementmeaning
modelcontent address of the network the claim is about — the identity under which δ was observed
testcontent address of the task, corpus and metric the claim was measured under
toleranceτ — the behavioural change the claimant asserts is acceptable, in the metric test names
observedδ — the behavioural change actually measured under test
evidencethe class in §3

A claim asserts δ ≤ τ under test, for model. It asserts nothing about any other model, any other corpus, or any metric test does not name.

3. Evidence class (REQUIRED)

What separates the rungs is provenance and scope, not accuracy — the same principle as energy-measurement.md §2, and for the same reason: a confidently formatted δ says nothing about whether anyone ran the test.

classmeaningrequires
Measuredδ was observed by running test against model on the stated corpus.retained artifacts sufficient to re-run
Transferredδ was observed under a different model or corpus and is argued to carry over.the source claim, and the argument, both named
Assertedδ is claimed without an observation.—

4. What the claim is, and is not

5. The split-computation receipt

Feature coding for machines makes the work partition between an origin and a sink a parameter of the wire: a network is cut at a layer, one party runs the layers before the cut, a tensor crosses, and another party runs the rest. That breaks an assumption of jwp-receipts, where joules_micro is one figure for producing an object by one producer. A receipt of a split computation needs three things: the energy spent before the cut, the energy spent after it, and an identity of the network under which the two halves provably belong together.

This section gives them as a pair of claims of one class, split-computation (a claim of jwp-receipts §2.4.3, registered here as its §2.4.4 provides). Each party signs its own half: a claim carries the authority of its signer and no more (jwp-receipts §2.4.4), and neither party read the other’s meter. The side that runs the layers before the cut is the head; the side that runs the rest is the tail. Both claims are about the tensor object, the object that carried the feature tensor across the cut, by its content hash (jwp-receipts §2.1).

A task-fidelity claim over a split computation MUST state the cut point, and MUST NOT report a single joule figure as if one producer had done all the work.

5.1 The body

A split-computation claim’s subject_type is object and its subject the tensor object’s content hash. Its body has exactly these members, read at jwp-receipts §3.1 step 2 in this order:

membertypemeaning
sidestringhead or tail.
networkhashThe network’s identity (below).
head_sethashThe digest of the head’s parameter set: the SHA-256 a pin of that set carries (SPEC §3.1).
tail_sethashThe digest of the tail’s parameter set, likewise.
cutobjectThe cut: { "layer", "dtype", "shape" }. layer is the number of layers the head runs, an integer of at least 1; dtype the element type of the tensor that crosses, one of u8, i8, u16, i16, i32, f16, f32, f64; shape its shape, an array of 1 to 8 integers, each at least 1.
consumedhashThe content hash of what this half took in: for a head, the source object; for a tail, the tensor object.
producedhashThe content hash of what this half gave: for a head, the tensor object; for a tail, its result’s bytes.
ran_onstringThe fabric this half ran on: cpu, gpu, media or npu (energy-measurement §6.3).
energyobject or nullThe energy this half spent, written as jwp-receipts §2.4.12 writes a session’s energy: { "joules_micro", "energy_provenance", "energy_uncertainty", "energy_counter_filtering", "energy_calibration_ref" }. null when this half was not metered, never a figure of 0 (energy-measurement §2).
attestsstringMUST be "computed". Any string passes step 2; another value is refused at step 4.

side, dtype and ran_on are read against their enumerations at step 2 (field_type), and so are layer and shape against their bounds. A hash is 32 bytes in lowercase hexadecimal. The body has no member for a total, and one that adds any is refused (unknown_member).

The network identity is

network = SHA-256("wai:split-network\x01" ‖ JCS({"cut": cut, "head_set": head_set, "tail_set": tail_set}))

over the RFC 8785 canonical form of the object of exactly those three members, cut as the body writes it and each set in lowercase hexadecimal. It names the two parameter sets in their order and the interface between them, so a head split for one tail does not name the network of another, and exchanging the sets names a different network.

5.2 Class rules

At jwp-receipts §3.1 step 4, in this order:

  1. attests is computed (split_not_computed).
  2. network is §5.1’s digest of the body’s own cut, head_set and tail_set (split_network_mismatch).
  3. The subject is the tensor object: a head’s produced, a tail’s consumed (split_subject_mismatch).
  4. consumed and produced differ: a half that gives back what it took ran nothing (split_no_work).
  5. Where energy is not null, its joules_micro is not 0 (energy_zero_figure), and its class is admitted as jwp-receipts §2.4.5 rule 2 admits one (energy_unknown_acquisition, energy_legacy_acquisition, energy_acquisition_evidence).

The energy figure of a half that is metered is an item of its own for signing authority (jwp-receipts §7.5): its signer needs split-computation and the energy authority of the figure’s class.

5.3 The pair

At step 6 a verifier checks one half against the other, each already through steps 1 to 5, and against the parameter sets it holds. In order:

  1. The other claim verifies through steps 1 to 4 and is of this class (context_invalid).
  2. One is a head and the other a tail (split_side_mismatch).
  3. Their subjects are equal: the tail consumed the tensor the head produced (split_tensor_mismatch).
  4. Their cuts are equal, member for member (split_cut_mismatch).
  5. Their network, head_set and tail_set are equal (split_halves_do_not_compose).
  6. Where the verifier holds the head’s parameter set or the tail’s, the digest it computes over the set it holds is the one the claims name (split_set_mismatch).

A pair that passes composes: both parties stated one network, cut at one place, and one tensor passed between them. A verifier that holds one half alone has no pair, and reports the half as verified through step 5 and nothing about the other side. Nothing here shows that either half computed what its network computes: that needs the parameter sets, the tensor and a run, and a claim about the task’s result is §2’s.

5.4 The account

A pair that composes gives an account, and the account is all a verifier reports of the pair’s energy:

A reader MUST NOT present a partial account’s figures as the energy of the whole computation, and MUST NOT add rows of different classes into one figure. A sum of a class’s figures past 2^53 − 1 has no account (context_invalid).

5.5 Reasons

reasonraised at
split_not_computed§5.2 rule 1
split_network_mismatch§5.2 rule 2
split_subject_mismatch§5.2 rule 3
split_no_work§5.2 rule 4
split_side_mismatch§5.3 step 2
split_tensor_mismatch§5.3 step 3
split_cut_mismatch§5.3 step 4
split_halves_do_not_compose§5.3 step 5
split_set_mismatch§5.3 step 6

5.6 What it does not say, and conformance

The class is independent of how the tensor is coded. wai.tensor.fcm, registered for ISO/IEC 23888-4, is a candidate with no pinned payload and no probe (format-probes §19), so no object of it is emitted; the claims name the tensor object by its content hash and say nothing of its bytes. A pair states what each party says it ran and spent. It is not a measurement of the network’s accuracy, and it does not make either half’s figure the other’s.

The corpus live-conformance/machine-coding/ holds network digests, single claims verified or refused at their step, and pairs with the account or the refusal each was written to have; wai_machine_coding_vectors verify and wai-rs/tests/machine_coding.rs replay it, and machine_coding_verify.py, written from this text and sharing no code with wai-rs, replays it with claim_verify.py for steps 1 to 4, requires every mutated copy to fail, and drops each of its own checks in turn.

6. Framing

Byte-comparability is becoming one fidelity contract among several rather than the only one. WAI’s position is unchanged by that — it names contracts and grades the evidence behind them, and does not adjudicate which contract a domain should want. What would be wrong is silence: a ratified class of media arriving with a conformance notion the container cannot state, described instead in the vocabulary of a guarantee it does not make.