Gunnar

nordisk/nornir

.nornir/cli-command-contract.md

<!-- nornir:class: design -->
# CLI Command Contract — the autopilot's CLI ⟺ UI correspondence

**Status:** design + step 1 shipped (2026-06-23). Goal (Rickard): *"every UI element AND
every CLI command must correspond, and test data must be sane on BOTH. CLI commands don't
have a UI trait — fix that."* Autopilot testing is the goal.

## The asymmetry we're closing

The test matrix discovers **UI** surface mechanically and asserts **sane data** on each:

| UI element | discovery | sannr check (truth-verdict) |
|---|---|---|
| facett component | `impl Facet for *` ∩ `registry()` | `state_json()` non-empty / no error atom |
| viz tab/pane | `viz::surfaces::viz_surfaces()` | AccessKit atom-walk + `state_json[slice]` |
| wgpu 3D view | `*View` + render check | geometry + offscreen image |

**CLI** had only half of this:
- discovery was **shallow** — `Cli::command().get_subcommands().map(get_name)` saw the ~25
  top-level roots, NOT nested leaves (`bench history`, `workspace ls`, `mimir deps-of`);
- there is **no uniform sannr check (truth-verdict)** — each command prints ad-hoc; `--json` exists on
  *some* commands, inconsistently. There is no `state_json` equivalent the matrix can assert.

So a CLI command could rot (wrong/empty output) and the green matrix wouldn't notice — the
exact blind spot the autonom gate exists to forbid.

## The contract (CLI twin of `Facet`)

### 1. Discovery — recursive leaf enumeration  ✅ SHIPPED
`clap_command_paths()` (src/bin/nornir.rs) recurses the clap tree and returns every **leaf**
command path, space-joined (`"bench history"`, `"mimir deps-of"`, …). It feeds all three
autonom gate sites (`gate_coverage_for_repo` / `nornir_surface_inputs`), so
`Gap = Surface − Covered` now spans EVERY command, not just the roots. This is the CLI twin
of `viz_surfaces()` — mechanical, never a hand list (kills the drift in `tests/cli_parity.rs`).

### 2. Sane-data check — `CommandOutcome`  ⬜ NEXT
Every command handler returns a uniform, machine-readable outcome — the CLI's `state_json`:

```rust
pub struct CommandOutcome {
    pub command: String,           // canonical leaf path, e.g. "bench history"
    pub ok: bool,                  // semantic success (not just process exit 0)
    pub data: serde_json::Value,   // the READABLE DATA the command produced (the check target)
    pub human: String,             // the pretty/table rendering for a terminal
}
```

A global `--json` prints `data`; otherwise `human`. The dispatch in `main()` prints the
outcome centrally instead of each arm printing ad-hoc. The matrix then runs each enumerated
command against the **same fixture workspace** the viz atom-walk uses and asserts the
outcome is sane: `ok == true`, `data` non-empty / expected shape, no error sentinel
(`ERROR_MARKERS`). This is identical in spirit to the `deck_no_error_atoms` + `state_json`
check — "sane data on every CLI command."

Roll-out is per-command (whole-codebase), but mechanical and gate-driven: an un-migrated
command shows as an uncovered/`data: null` surface, so the gate names exactly what's left.

### 2a. `CommandOutcome` is presentation, NOT a transport  ⚠️ READ THIS
The data path is **unchanged** and stays canonical:

- **fat mode** → direct **nornir lib function calls** (Rust types) — e.g. `bench history` calls
  `nornir::bench::query_bench_runs(...)`.
- **thin mode** → **gRPC** to `nornir-server` (protobuf) — the server does everything via RPC.

`CommandOutcome` wraps the **already-fetched** result purely for OUTPUT: `human` to the
terminal, `data` (JSON) only for `--json`/tooling and the autopilot check. The JSON lives at
the **edge** — the same edge the UI's `state_json` lives at, and where MCP tools already speak
JSON. It is never the wire. So `data: serde_json::Value` is a *comparison/introspection*
shape (it lines up with the UI `state_json` slice for parity), not a serialization layer that
competes with gRPC or the lib calls.

### 3. CLI ⟺ UI parity — same data, two faces  ⬜ NEXT
Many commands are the CLI face of a viz tab (both read the same warehouse table):

| CLI leaf | viz surface | shared data |
|---|---|---|
| `bench history` | `BenchView` / Bench tab | `bench_runs` |
| `test all` / `test matrix` | 🧪 Test tab | `test_results` ⟕ `test_inventory` |
| `workspace ls --status` | Nornir tab roster | `clone_events` roll-up |
| `mimir deps-of` | Dep-graph pane | `call_edges` / dep graph |
| `arch` | Arch tab | wiring graph |

The deepest parity is asserted **at the lib / gRPC level, not by scraping JSON stdout**:
- **fat:** the CLI command and the viz tab call the **same lib fn** (`query_bench_runs`,
  `load_clone_events`, …). The parity test calls that fn directly and asserts both faces
  consume it — `cli_outcome.data` and `viz_state_json[slice]` are then equal *by construction*.
- **thin:** both go through the **same gRPC handler** (`Bench`, `Viz.CloneEvents`, …); the
  test asserts the handler is the shared source.

The `cli_outcome.data ≈ viz_state_json[slice]` comparison is the *edge* check on top of that
shared-source guarantee. Together they enforce the standing **CLI-parity LAW**
([[coolest-modern-CLI-parity]]) mechanically: same data path, two faces — if a tab shows data,
its CLI twin shows the same data, and vice-versa.

## Why this is the autopilot

Green matrix ⟺ (every UI atom emits sane data) **AND** (every CLI leaf emits sane data) AND
(every twin agrees). Both surfaces are discovered from facts (clap tree + Facet registry),
both are asserted on real injected data, and the completeness gate fails on any gap — no hand
lists, no silent rot. That is "autopilot testing."

## Build plan
1. ✅ Recursive `clap_command_paths()` + wire 3 gate feeders + unit test (nested leaves found).
2. ⬜ `CommandOutcome` type + global `--json` + central print in `main()`.
3. ⬜ Migrate handlers to return `CommandOutcome` (per-command; gate names the remainder).
4. ⬜ Generic CLI sannr check (truth-verdict) test (run every leaf vs fixture workspace, assert `data`).
5. ⬜ CLI⟺UI parity tests for the twin table above.
6. ⬜ Retire the hand lists in `tests/cli_parity.rs` (subsumed by 1+4).