# znippy-zoomies — design (f0, the whole)
> **Class:** `design` · **Function axis:** `f0` (the toolbox) + `gatling` · `stree` ·
> `decompressors` · `chunk_revolver` · `vann` · scanners · `psort`.
> Strict: this file states *only the shape we ship*. Every *why*, every rejected
> alternative, lives in [`reasoning.md`](reasoning.md) and the per-function
> `*-reasoning.md` files.
## f0 — how the parts wire into one thing
`znippy-zoomies` is a single library crate (`src/lib.rs`) plus a workspace of
decompressor member crates (`Cargo.toml:17`). Everything shares **one substrate and
one law**: the no-barrier `gatling` engine, and **no rayon in the shipped tree**
(guarded by [`tests/rayon_free_law.rs`](../tests/rayon_free_law.rs)).
The dependency spine, top to bottom:
| layer | modules | what it gives the layer above |
|---|---|---|
| **engine** | `gatling`, `gatling_forkjoin`, `gatling::io`, `gatling::ordered` | 1 reader → N self-dispatching workers → ordered sink, `std::thread::scope` + one `AtomicUsize` |
| **buffers** | `chunk_revolver` | zero-allocation slot pool the engine fans over; workers borrow `&[u8]`, never copy |
| **consumers** | `decompressors` (`lgz`/`lbzip2`/`linflate`/`ljar`/`lzip-parallel`), scanners (`vtd`/`xml`/`json`), `stree`/`stree32`, `psort`, `vann` | each is a payload driven across the engine or a standalone cache-aware primitive |
The **reuse graph** (`uses`): `decompressors.uses = [gatling, chunk_revolver]`;
`scanners.uses = [gatling_forkjoin]`; `psort`, `stree`, `vann` are leaf primitives
that the engine *feeds* (a sort/search/score inside a worker) but do not depend on it.
`vann.uses = [psort]` conceptually (top-k and any future k-means both want the
parallel sort). This graph is why they live in one crate: the exact-key tree, the
fuzzy-vector index, the no-barrier driver, and the parallel sort are **one toolbox**,
reusable by `skade`, `nornir`, and `znippy` without pulling a whole application repo.
The crate also exposes a single introspection seam, `functional_status(component,
check, ok, detail)` (`lib.rs:35`) — a compiled-out no-op unless the `testmatrix`
feature is on, when each engine run appends a nornir test-matrix row.
## gatling — the no-barrier engine
`src/gatling.rs` (streaming), `src/gatling_forkjoin.rs` (fixed `0..n`),
`src/gatling_io.rs` (async I/O), `src/gatling_ordered.rs` (item-oriented ordered sink).
One idea in several bodies: a worker claims its next unit with a single atomic
`fetch_add`, runs it to completion without waiting on any sibling, and writes into a
**disjoint** output slot. The reader only reads; the collector only stitches in order;
all heavy work is in the middle. Full architecture in
[`gatling-design.md`](gatling-design.md).
## stree / stree32 — AVX2 static search tree
`src/stree.rs` (`STree64`, `i64`, `B=8`) and `src/stree32.rs` (`STree32`, `u32`,
`B=16`, Ragnar's original width). An implicit cache-line-sized S(+)-tree: children by
arithmetic (no pointers), a branchless SIMD `count_lt` node compare, and a
software-pipelined prefetched batch (`batch_stream::<P>`) that hides the miss latency.
Two **concrete** types, never a generic `STree<T>`. Full architecture in
[`stree-design.md`](stree-design.md).
## decompressors — the parallel (de)compressor family
Workspace members `lgz` (multi-member gzip), `lbzip2` (bzip2), `linflate` (DEFLATE
core), `ljar` (JAR), `lzip-parallel` (ZIP). Each parallel-decodes a **stock** archive
format across every core through the `gatling` + `chunk_revolver` pipeline. These are
the subjects of the bake-off (see [`benchmarks.md`](benchmarks.md)); their shapes are
in [`decompressors-design.md`](decompressors-design.md).
## chunk_revolver — zero-allocation slot pool
`gatling/src/revolver/` (`revolver.rs`, `coalesce.rs`, `ring.rs`). Two modes.
Fixed-slot: `ChunkRevolver::new(chunk_size, total_chunks, num_workers)` owns per-worker
rings of fixed-size slots; `try_get_chunk()` hands the reader a free slot, workers borrow
it as `Chunk<'a>` (`&[u8]`), and `return_chunk(ring, index)` recycles it. Coalescing:
`ChunkRevolver::with_coalescing(num_slots, slot_size, slice_size)` packs many
variable-width payloads into ONE slot and reference-counts it, so a memory budget buys an
order of magnitude more small files in flight. Either way, 1-reader/N-worker streaming
with no per-chunk `Vec<u8>` allocation. Details in
[`chunk_revolver-design.md`](chunk_revolver-design.md).
## scanners — vtd / xml / json
`src/vtd.rs` (VTD-style parallel OSM-XML → compact byte-offset `ElemIndex` + zone-map
summaries), `src/xml.rs` (the tag-agnostic scan machinery underneath), `src/json.rs`
(zero-copy parallel NDJSON scan + escape-aware `find_key`). All three compute
sub-chunk borders up front, then map them across `gatling_forkjoin::gatling_for_each`
and merge per-chunk results in index order.
## psort — parallel sample sort
`src/psort.rs`: `samplesort_aos_by_i64_key(&mut [[u8; RECORD_SIZE]])` (`:62`) sorts
fixed 16-byte array-of-structs records by their leading little-endian `i64`, all cores
at once; `radix_sort_aos_by_i64_key` (`:257`) is the reference LSD radix.
## vann — exact vector index
`src/vann/mod.rs`: `VectorIndex` is an **exact** brute-force f32 nearest-neighbour
oracle (100 % recall). `search_i8_rerank` (`:266`) runs a provably-lossless int8/VNNI
cosine **prefilter** (`score_i8_batch`, `vpdpbusd`) then an f32 **rerank** that is
byte-identical to the flat oracle; `search_batch` (`:353`) overlaps prefetch across
queries. Runtime SIMD detection via `active_simd()` (`:518`). Details in
[`vann-design.md`](vann-design.md).
## LAW 6 — the canonical branch, and the guard that makes the trap impossible
**Ask the repository which branch is canonical. Never assume, never hardcode, never
memorise a table.** Several repositories in this constellation carry BOTH a `main` and
a `master` on the remote and they do **not** agree on which is canonical — one
repository's trunk is the next one's abandoned branch. Enumerate them; do not learn
them:
```sh
for r in ~/git/*/; do
both=$(git -C "$r" ls-remote --heads origin main master | wc -l)
[ "$both" = 2 ] && printf '%-20s canonical=%s\n' "$(basename "$r")" \
"$(git -C "$r" symbolic-ref --short refs/remotes/origin/HEAD)"
done
```
On 2026-08-30 that listed exactly three, and the answer was **not** the same for all
three:
| repository | canonical | the trap |
|---|---|---|
| `znippy` | **`master`** | `main` exists and is **153 commits behind** — stale for months |
| `gunnar` | **`master`** | `main` exists; a lane pushed to it at 18:50 on 2026-08-30 |
| `znippy-zoomies` | **`main`** | `master` is the stale one — the opposite way round |
Two lanes pushed to a non-canonical trunk in one week. The gunnar one was harmless
**only by luck** — that tip happened to be an ancestor of `master`, so nothing was
stranded; the znippy one was not.
**What the hook refuses:** a push whose destination is a *trunk* name (`main`,
`master`, `trunk`, `develop`) that is not this repository's canonical trunk. A
deliberate push to a feature branch is untouched — `roadmap/obruten-kedja`,
`lane/whatever`, tags: all pass. The trap is the second default-looking branch, not
branches in general.
**How it knows:** `git symbolic-ref refs/remotes/<remote>/HEAD` — the remote's own
answer — falling back to asking the server with `git ls-remote --symref`. The
canonical name is **never written in the hook**; the same bytes therefore give
opposite answers in `znippy` and `znippy-zoomies`, which is the whole design. A
fourth place for the repositories to disagree would be the bug, not the fix. If
neither source answers, the hook says so and **allows**: a hook that blocked every
push on a repository it could not resolve would be deleted within the day, and then
nothing would guard anything.
The refusal names the branch that *is* right, the command that pushes there
(`git push <remote> HEAD:<canonical>`), the enumeration above, and the deliberate
override (`GIT_ALLOW_NONCANONICAL_TRUNK_PUSH=1`) for the one honest case — maintaining
or retiring the non-canonical trunk on purpose.
**Install** (hooks live outside the working tree, so a clone does not get them):
```sh
cp .githooks/pre-push .git/hooks/pre-push && chmod +x .git/hooks/pre-push
```
The hook chains to `git lfs pre-push` itself when `.gitattributes` tracks LFS paths
(it buffers stdin and replays it), so it *replaces* the LFS hook rather than fighting
it.
**Proven both ways in all three repositories** (LAW 2 — never accept a green you have
not seen red), 2026-08-30:
| repository | refused (exit 1) | allowed (exit 0) |
|---|---|---|
| `gunnar` | `main` | `master`, `roadmap/obruten-kedja` |
| `znippy` | `main` | `master`, `lane/x` |
| `znippy-zoomies` | `master` | `main` |
and end to end: `git push --dry-run origin HEAD:main` is refused in gunnar and in
znippy, while `git push --dry-run origin master` / `origin main` still runs the LFS
half.
## Which recipe reader this repo uses, and why (lane AJ, 2026-08-30)
**Reader:** `nornir-build-thing` **0.1.12**, via `xtask`'s `[patch.crates-io]` onto the edda
path.
**Why:** in-constellation; recipe is schema 1. The requirement had been the floating
`"0.1"` while the lock still pinned 0.1.4 — seven generations stale, with nothing
to force it forward. The `[patch]` (not a direct `path =`) is deliberate and
documented in the manifest: it is overridable per-checkout and it keeps the graph
on ONE copy.
The rule itself — `[nornir-build-thing] schema = N`, a refusal that names the
version to upgrade to, and a loader that refuses to DROP a key it cannot read —
lives once, in **edda `.nornir/build-design.md`, "THE COMPATIBILITY RULE"**. The
cross-version guard that would have caught the 2026-08-30 skew is
`edda/nornir-build-thing-skew`.