# znippy — full release runbook
How to cut a complete znippy release from this control plane: `nornir` runs every
bencher, gates on regression, renders docs, commits + tags, and publishes to
crates.io; you do the one `git push` it deliberately leaves to the operator. The
declarative gate contract is in [`SPEC.md`](SPEC.md); the manifest (publish order,
gates, bench requirements) is in [`nornir.toml`](nornir.toml).
(This workspace-wide release lives here in `workspace_znippy`, the control plane —
not in `znippy/.nornir`, which is the znippy repo's own docs.)
## TL;DR
Two commands, from the workspace root (where the sibling repos live):
```bash
cd /run/media/rickard/T9/git
NORNIR_CONFIG=workspace_znippy/release/nornir.toml nornir release run
git -C znippy push origin master --follow-tags # the step nornir leaves to you
```
There is no wrapper script — this is an all-Rust toolchain; the single non-Rust
action, `git push`, is a deliberate manual step (see *The one manual step* below).
## What `nornir release run` does (per repo, in order)
1. `cargo test --workspace` (serialized) — abort on failure.
2. **Bench** — runs `xtask/examples/nornir-bench.rs` (every registered `Bencher`:
the codec/stream/archive/uring/maven benches **and the stree size-sweep**),
parses the `BenchRun` JSON, persists it to the Iceberg `bench_runs` warehouse.
3. **Render docs** — rewrites the `<!-- nornir:gen -->` regions of `README.md`
from the fresh bench row (don't hand-edit inside those markers).
4. **Gates** — regression ≤ 10 % vs history, docs-fresh, no path-patches, metadata
(see SPEC.md's 5 gates).
5. **Capture** (skip with `--skip-snapshot`) — knowledge map / search index /
semantic vectors / DWARF facts → Iceberg.
6. **Commit + tag** — pure-Rust `gix` commit of the rendered docs + a `vX.Y.Z` tag,
*locally*.
7. **`cargo publish`** — walks `publish_order` (linflate → lgz/ljar/lzip/lbzip2 →
znippy-zoomies → osm-katana → znippy-common → compress/decompress → plugins →
iceberg → znippy → znippy-cli). "Already uploaded" is an idempotent skip.
8. **Lineage** — appends a `release_lineage` row to Iceberg.
## The one manual step nornir leaves: `git push`
By design, `nornir` does **not** push to the remote: gix 0.84 has no send-pack,
and nornir forbids `git` subprocesses + libgit2 (see `release/publish.rs::push`).
It creates the commit + tag locally and prints the exact `git push` command —
you run it (the TL;DR's second line). That keeps the toolchain pure-Rust; the
crates.io publish is automated, the remote push is the one deliberate human step.
## Prerequisites
- **`NORNIR_CONFIG`** must point at *this* config — nornir's auto-discovery is
hardwired to `workspace_holger/release/nornir.toml`. Use the env var or
`nornir --config workspace_znippy/release/nornir.toml …`.
- **crates.io token** — cargo's own auth (`CARGO_REGISTRY_TOKEN` or
`~/.cargo/credentials`); nornir shells out to `cargo publish`.
- **Iceberg warehouse** — now pinned by `[storage] local_path =
"workspace_znippy/.nornir"` in nornir.toml, so bench/release write
`workspace_znippy/.nornir/warehouse` (previously fell back to holger's). The
warehouse is local redb-backed and is created on first use.
- **A current `nornir` binary** — `cargo install --path nornir --bin nornir
--force` if yours is stale. The `bench_runs` Arrow schema evolved (now 7
fields); an older installed binary writes a mismatched batch and the local
append fails (`columns(N) must match fields(M)`). Reinstall to match source.
- **`NORNIR_CONFIG`** must point at *this* config — auto-discovery is hardwired to
`workspace_holger/release/nornir.toml`. Use the env var or `--config …`.
- **crates.io token** — cargo's own auth (`CARGO_REGISTRY_TOKEN` or
`~/.cargo/credentials`); nornir shells out to `cargo publish`.
- **git author** configured (`user.name` / `user.email`) — the tag step bails
without it.
- **Quiescent machine** — the regression gate compares same-machine numbers; close
other heavy jobs (and don't run two benches at once — concurrent benching
corrupts numbers).
- **Push auth** — codeberg over SSH for the push step.
- **`NORNIR_SERVER`** — if set (e.g. `http://127.0.0.1:7878`), the *standalone*
`nornir bench run` ships results to that server (which must serve the `znippy`
workspace) instead of the local warehouse. `release run` always uses the local
warehouse, so it is unaffected — but unset `NORNIR_SERVER` for ad-hoc local
`bench run`s, or register the workspace on the server
(`nornir workspace register znippy --descriptor workspace_znippy/nornir-workspace.toml`).
- **Heavy/planet bench fixtures** — a real `release run` runs the *full* bench
suite (no skips): the 500 MB/2 GB streams, planet `.osm.bz2`, and
`europe-latest.osm.pbf` must be present, or those benchers skip and the gate's
`required_results` may fail. For a quick path check use
`NORNIR_BENCH_SKIP_HEAVY=1 NORNIR_BENCH_SKIP_PLANET=1 nornir bench run znippy`.
## Status — what was blocking, now cleared (2026-06-11)
- ✅ **nornir compiles** (the earlier `viz/model.rs` gix error is gone).
- ✅ **znippy-common 0.9.2 published** — it had diverged from 0.9.1 without a bump
(the `MetaSinkFactory` iceberg-sink API), which would have failed the
`znippy-compress` publish step mid-release. Bumped + published + every
dependent's version requirement raised to 0.9.2.
- ✅ The decompressor crates were already perf-released this round (lbzip2 0.5.2,
lgz 0.3.3, ljar 0.2.1, lzip-parallel 0.2.2, znippy-compress 0.9.2) — a full
`release run` will idempotently skip those already-uploaded versions and pick
up whatever else changed.
- ✅ **Bench → persist path validated** (`NORNIR_BENCH_SKIP_HEAVY/PLANET=1`):
`nornir bench run znippy` persisted **21 results / 29 tests** into a fresh
`workspace_znippy/.nornir/warehouse` `bench_runs` (incl. `stree_8mb/_128mb`,
`lbzip2_scala`, `lgz_scala`). Fixes that got it there: reinstall the nornir
binary (schema skew), the `[storage]` pin above, and a one-time warehouse wipe
(the old table predated the 7-field schema).
So a full `release run` is runnable today. Dry-run tip: run `nornir bench run`
first (bench + persist, no gates/publish) to confirm the bench + warehouse path
on a quiescent box before the real release.