# Bench vs test — for idiots
Short version. The long version is [`bencher.md`](bencher.md); the per-repo
work-list is [`bench-test-classification-audit.md`](bench-test-classification-audit.md).
## They are two different things
- **Test** = does the code give the right answer? Runs **automatically** on every
`nornir release`, every viz "Run full matrix", every gate. Output: pass/fail.
- **Bench** = how *fast* is it? Runs **only when you ask** (`nornir bench run`).
Output: a number (MB/s, rows/s, ms). Historized; gates on regression.
A plain `nornir test` runs **no benchmarks**. (The `bench-smoke` column just does
`cargo bench --no-run` = "does the bench code compile", nothing more.)
## The trap
Some **tests secretly do bench work** — download 26 JARs, build a graph over 50
crates, scan a 15 GB table. That's what made `nornir test` take a minute and spin
the fans. Fix it per test:
1. **Does it produce a meaningful speed number** (MB/s, rows/s, ms-for-N)?
- **Yes, and a bench already measures it** → just **shrink the test** (fetch 1
JAR, not 26). The bench keeps the full workload.
- **Yes, but no bench exists** → **make a bench** from it (full scale) **and**
keep a **shrunk** test for correctness.
- **No** (the slow part is a download or a server boot) → leave it a heavy test.
## Heavy = opt-in (the new rule)
Heavy / `#[ignore]`'d tests (big corpora, network, model downloads, scale
matrices) now run **only if you pass `--heavy`**. Default OFF for `release`,
`nornir test`, and viz. `--skip-heavy` does nothing now (off is already default).
```
nornir test # fast: no heavy arms
nornir test --heavy # includes the heavy/network/scale tests
nornir release # fast: synthetic tier gates; heavy OFF
nornir release --heavy # includes heavy arms
```
**Benches are NOT affected by `--heavy`.** You ran `nornir bench run` on purpose —
you want the heavy measurement. A bench that needs network/a giant fixture skips
itself via `nornir.toml`'s `network_required` + a fixture check, not `--heavy`.
## Don't chain tests by order (the idiot trap)
Tempting: "test #1 unpacks the OSM file, test #2 reads it." **Don't.** Tests run
in parallel and in any order, and if #1 is heavy it's skipped by default — then #2
blows up for no good reason.
- A test must **make its own inputs** (a `OnceCell` / `ensure_xxx()` helper that
downloads-or-reuses a cache). Then order doesn't matter and nothing breaks when
a heavy arm is off. (This is how znippy already does it — copy that.)
- If a step is genuinely expensive and shared, it's a **bench stage** with an
explicit order (`register_bench_ordered!`) that checks its prerequisite and
skips gracefully — not a sibling `#[test]`.
Coming soon: nornir will draw the **chain graph** (in the Test tab + warehouse)
and refuse a light test that depends on a heavy one. See
[`test-bench-chains.md`](test-bench-chains.md).