Gunnar

nordisk/znippy-zoomies

.nornir/znippy-zoomies-unsorted.md

# znippy-zoomies — crate card

## Reverse-engineered (from source — verify, don't trust the name)
- **One-line pitch:** A grab-bag of "fast-as-hell" building blocks (parallel XML/JSON scanning, static search trees, a no-barrier streaming engine, a parallel sort, and a vector-ANN index) extracted from katana-osm so multiple downstream repos can share them.
- **Does:** Provides `vtd`/`xml` (VTD-style parallel OSM-XML element scanners producing compact byte-offset `ElemIndex`), `stree`/`stree32` (Ragnar Groot Koerkamp's AVX2 static search tree over sorted i64/u32 keys, incl. mmap-backed leaf layer), `gatling` (generic no-barrier worker-pool engine: split → N decode → in-order collect → sink, with an async `gatling::io` sibling on tokio), `chunk_revolver` (zero-alloc slot pool for 1-reader/N-worker streaming), `psort` (parallel sample sort over 16-byte AoS records), `json` (zero-copy parallel NDJSON scanner), and `vann` (exact brute-force f32 nearest-neighbour index with SIMD/int8-VNNI cosine kernels).
- **Key deps:** rayon (data parallelism), memchr (delimiter scan), memmap2 (mmap stree/vann blobs), bytes+futures (async gatling::io payloads/in-flight set), libc on Linux; dev: criterion, tokio, tempfile, rand, serde_json.
- **Shape:** lib only (~9k LoC across 15 modules); biggest are stree (1653), vann (1469), vtd (1132), stree32 (942), gatling (866), xml (691). Multiple criterion benches (xml_scan, json_scan, gatling_roundtrip). (The `vann-mmap` feature named here gated zero `cfg` sites and was deleted 2026-08-22; the row blob is read, not mapped.)
- **Observed role in the workspace:** The workspace root crate and shared foundation. `osm-katana` depends on it and re-exports `vtd as xml_vtd` and `stree as stree64`. Doc comments say both skade (stree/gatling) and znippy (codecs/gatling) depend on it; extracted into its own leaf repo to avoid a release-cascade cycle.

## Fill in (one line each — only the human knows these)
- Primary consumer(s): 
- Why it exists (vs the obvious alternative): 
- Hard constraint (airgap / no-C-deps / no-FFI / pure-Rust / etc.): 
- Release cadence & independence (rides a parent? own repo someday?): 
- Status (production / prototype / experimental / stubbed): 
- Must-NOT-change invariant: 
- Biggest known gap or TODO: 
- Anything a newcomer would get wrong about it: