# Conformance runs: `wai conform`

`wai conform <kind> --suite <suite> --impl <command or URL>` runs one of WAI's conformance
corpora against an implementation, yours or anyone's, and writes the outcome as a signed record.
Every case's expected answer is the corpus's own: a verdict, a digest or a reason the corpus
states, set from the specification's text.

```bash
wai conform --list                                                    # every kind and suite
wai conform verifier --suite claim --impl "my-verifier verdicts {corpus}"
wai conform decoder --suite score --impl "my-renderer {in} {out}"
wai conform origin --suite http --impl http://127.0.0.1:8380/
wai verify claim.wai-conform.json                                     # the signed result, checked
```

It runs in a checkout of the standard, which holds the corpora; `--root <checkout>/wai` names one
elsewhere.

## Kinds and suites

| Kind | Suite | Corpus | A case is | It passes when |
|---|---|---|---|---|
| verifier | `claim` | `claim-conformance/` | One vector | The verdict is the one declared: `accept link_hash=<hex>`, `accept ancestor=<hex>` or `reject step=<n> reason=<label>` (jwp-receipts §3.1) |
| verifier | `jwp` | `jwp-conformance/` | One vector | The verdict is the one declared (jwp-receipts §2–§6) |
| verifier | `signer` | `signer-conformance/` | One vector | The verdict is the one declared (jwp-receipts §7) |
| verifier | `conform-result` | `conform-result-conformance/` | One record | The outcome is the one declared: `accept conformant=<true or false>` or `reject step=<n> reason=<code>` (conformance-results §4) |
| decoder | `score` | `score-conformance/` | A `WSCR` score | The rendered `i16le` PCM has the declared SHA-256 |
| packager | `policy` | `pack-conformance/` | A packaging policy | It is admitted with the declared digest (`ok <digest>`), or refused with the declared reason (packaging §3, §12) |
| origin | `http` | `http-conformance/objects/` | An object | It is served as http-delivery §5.3 and §6.2 require: its bytes, `ETag`, cache and cross-origin headers, a byte range, `HEAD`, its head proof and the preflight |

Each verifier corpus's README states its verdict lines.

## Adapters

**A command.** `--impl` is a command line, run by the system shell, with these placeholders
replaced by paths:

| Placeholder | Replaced by |
|---|---|
| `{corpus}` | The corpus file whole: the command answers every case at once, one `name<TAB>verdict` line each (the verifier suites) |
| `{case}` | A file holding one case, as the corpus writes it |
| `{in}` | The case's input file |
| `{out}` | A file to write the output to |

Without `{corpus}`, `{case}` or `{in}`, the case or input is given on standard input; without
`{out}`, a decoder's output is read from standard output. The first line a command prints is its
verdict; a decoder that refuses exits non-zero. A command is stopped after `--timeout` seconds
(60 unless given).

**HTTP.** `--impl` is an `http://` URL. Each case is a `POST` to it: a verifier's case as JSON,
a decoder's or packager's input as bytes. The answer is the response's first line, or the member
`verdict` of a JSON response; a decoder answers `200` with the output, or another status with its
refusal. For an origin, the URL is where it serves the corpus's objects: `<url>/<cid>.wai` and
`<url>/<cid>.wai.hp`.

## The result record

Each run writes a record, `<suite>.wai-conform.json`; `--out` names another file, and a file that
exists is replaced only with `--yes`. The record, `conform-result/1`, is specified in
[conformance-results](https://git.transaction.science/Transaction-Science/open-standards/src/branch/main/wai/extensions/conformance-results.md):
its members and corpus digest (§2), its signature (§3), what a verifier checks and in what
order (§4), and the codes it refuses with (§5). Its corpus,
[`conform-result-conformance/`](https://git.transaction.science/Transaction-Science/open-standards/src/branch/main/wai/conform-result-conformance),
holds accepted and refused records with their outcomes, and `wai conform verifier --suite
conform-result` runs a verifier of these records against it.

```json
{ "wai": "conform-result/1", "kind": "verifier", "suite": "claim",
  "standard": "v0.4.0", "toolchain": { "name": "wai", "version": "0.3.0" },
  "corpus": { "dir": "claim-conformance", "files": 1, "digest": "sha256:…" },
  "implementation": { "adapter": "command", "impl": "my-verifier verdicts {corpus}",
                      "name": null, "version": null, "files": [{ "path": "…", "sha256": "…" }] },
  "started_at": "2026-10-27T12:00:00Z",
  "cases": [{ "name": "…", "pass": true, "expected": "…", "got": "…" }],
  "summary": { "cases": 1, "passed": 1, "failed": 0 }, "conformant": true,
  "key": "ephemeral", "signer_pubkey": "…", "sig": "…" }
```

- **`corpus.digest`**: the SHA-256 of the lines `<SHA-256 of the file>  <path under wai/>`, one for
  every corpus file the run read, sorted.
- **`implementation`**: the command and the SHA-256 of every file it names (its program found on
  `PATH`, and each argument that is a file), or the URL; `--name` and `--version` add what the
  implementer calls it.
- **`sig`**: Ed25519 (RFC 8032) by `signer_pubkey`, over the bytes `wai:conform-result` and `0x01`
  followed by the record without `sig` in RFC 8785 form. A key made for the run (`"key":
  "ephemeral"`) shows the record is as the run wrote it; a key given with `--key <file>` (a 32-byte
  seed in hexadecimal) also says who ran it.

`wai verify <path>` checks every record under a path, in the order of conformance-results §4:
that it reads, its version, its members, its signature, and that its summary and `conformant`
are what its cases say. A record changed after it was signed is refused.

## Exit status and JSON

| Exit | Meaning |
|---|---|
| 0 | Every case passed |
| 1 | A case failed |
| 2 | A usage error |
| 4 | The record would replace a file, and `--yes` was not given |
| 5 | The kind, suite, corpus or program is not found |
| 6 | The implementation's URL does not answer; safe to retry |
| 7 | An internal error |

`--dry-run` lists the cases and what would run, and writes nothing. With `--json` it prints one JSON
envelope (`"op": "conformance.run"`), its `data` the counts, the failures and the record's path;
without `--json` the output is text, on a terminal or not.

WAI's continuous integration runs the corpora's independent verifiers, second implementations
written from the specification's text and sharing no code with the reference, through `wai
conform`, as it would run any other implementation.