# Gatling — reasoning
> **Class:** `reasoning`. The *why*, the trade-offs, and the road here. The descriptive
> shape we ship is in [`gatling-design.md`](gatling-design.md), which by rule mentions
> none of this. If you want to know "why not X?" — X is here.
## The cardinal sin — the "1-ends cook" bug
The disaster mode is doing heavy work on the **reader** or the **collector** instead of
in the workers. There is only one of each, so the moment you sort/parse/assemble there,
one core does everything and the pool starves. The kitchen picture (see
[`gatling-for-idiots.md`](gatling-for-idiots.md)): the waiter and the plater never cook.
This is not hypothetical. It once dropped `osm-katana` from "all 32 cores" to ~3: the
ends had been made to cook — read the whole 8 GB then sort it one-at-a-time, walk the
whole file to find boundaries, assemble the final result at the pass. The cruel twist:
while broken, the **old rayon path looked faster**, and people almost blamed gatling.
Gatling was fine; we'd told the waiter to cook. Moving the cooking back into the workers
restored a flat 100 % on all cores — even on the entire planet's map (10.58 **billion**
points). The whole engine exists to make that failure mode structurally hard.
## Why "no rayon" (the constellation law)
rayon is mature and ubiquitous, so the ban needs a reason. Its **global thread pool** is
exactly what hid the osm-katana serialization: shared, implicit, and invisible when work
lands on the wrong side of a fork-join barrier. `std::thread::scope` + one atomic cursor
has no hidden global state, no barrier, and lets worker closures borrow caller stack data
directly (no `'static`/`Arc`). The crate carries **no `rayon` dependency** in its
`Cargo.toml` and none in its shipped tree; the last two holdouts (`src/json.rs` NDJSON
line-scan, `src/vtd.rs` sub-chunk parse) were ported off `rayon::into_par_iter` onto
`gatling_forkjoin::gatling_for_each`, and on the `json_scan` bench the ported scan is if
anything *faster* than the rayon version it replaced. The law is enforced by
[`tests/rayon_free_law.rs`](../tests/rayon_free_law.rs). (The one honest exception —
`ljar`/`lzip-parallel`'s *dedicated* non-global pools, and criterion's dev-only
transitive rayon — is documented in [`reasoning.md`](reasoning.md).)
## Why balanced scheduling is a *separate* body, not the default
Plain `gatling_for_each` claims `0..n` in order, one index at a time — optimal when
units cost about the same, and cheaper (no weight computation, no batch bookkeeping).
It is **not** optimal when a handful of units dwarf the rest: a monster claimed last
leaves one worker grinding alone while its siblings drained the cheap tail and went idle
(the classic long-pole). Rather than tax every caller with LPT machinery, the skew fix
is a distinct body (`gatling_for_each_balanced` / `gatling_map_balanced`) you opt into
with a real `weight`. Keeping the common case lean and the skew case explicit beats one
always-balanced engine that pays for scheduling nobody needed.
## Why `gatling_map_balanced` exists next to `gatling_for_each_balanced`
`gatling_for_each_balanced` schedules a bare `0..n`. nornir's flattened knowledge scan
doesn't hold a count — it holds a `Vec` of `(member_idx, file, crate, section, size)`
tags unioned across every repo, and wants to map each to an owned per-file scan then
**group back by `member_idx`**. Forcing that through `…_balanced(tags.len(), …, |i|
f(&tags[i]))` works but pushes an `i → tags[i]` re-index into the hot closure and makes
the weight close over the slice. The item-ref body (`f(i, &item)`, `size_of(&item)`)
reads one borrow, keeps distribution zero-copy, and let nornir **delete** the private
`gatling_map_balanced` mirror it had carried and call the canonical primitive.
## Why sync threads *and* an async sibling
The streaming engine is CPU-bound (one thread per core) — right for decode/parse. For
I/O-bound work (S3 PUT/GET, fsync) the cost is per-op round-trip latency, not CPU, so a
blocked OS thread is pure waste. `gatling::io` keeps every gatling invariant but swaps
threads for ≤ `max_in_flight` tokio futures. Same soul, different substrate — chosen by
whether the bottleneck is a core or a socket, not by taste.
## The rayon regression + the fix (the real history)
The "no rayon" law was, for a while, *asserted crate-wide but enforced narrowly* — and
the codecs slipped through the gap. The forensic record (crates.io tarballs + full git
pickaxe):
- **The codecs were rayon from first publish, never gatling.** crates.io is immutable, so
it settles it: `lgz` 0.2.0 (2026-05-21), `lbzip2` 0.1.0 (2026-05-15), `ljar` 0.1.0 and
`lzip-parallel` 0.1.0 (2026-05-21) *all shipped `rayon = "1"` at their earliest version*.
There is **no gatling version of any codec to "restore"** — the shared engine
(`znippy-zoomies::gatling`, extracted from katana 2026-05-27) was only ever wired into
`json`/`vtd`/the streaming compressor and osm-katana, never the codecs or skade.
- **The partial port declared complete.** `43c571a` (2026-07-03, "Port json/vtd scanners
off rayon; make the 'rayon-free' law true") touched **only** `src/json.rs` + `src/vtd.rs`
and dropped the *root* `Cargo.toml` rayon dep — **zero codec files**. `lgz`/`lbzip2` had
independently moved to a *private* scoped-thread fork-join upstream (a hand-rolled clone
of `gatling_for_each`, not the shared engine); `ljar`/`lzip-parallel` kept live
`OnceLock<rayon::ThreadPool>` singletons — the exact global-pool shape that bit
osm-katana.
- **The law narrowed by decree, the guard baked to match.** `769363f` rewrote the law's
prose from "no rayon … the whole crate" to "no rayon in the *shipped, non-dev* tree
(only criterion's dev-dep left)", quietly reframing it around the root crate;
`c3d5512` then created `tests/rayon_free_law.rs` **born scoped to `src/`** with an
explicit written carve-out excusing `ljar` + `lzip-parallel`. So the guard was written
to *not notice* the two shipped crates that still carried rayon.
**The fix (what shipped here).** The law is now real for the whole workspace, not the
root alone. `ljar` and `lzip-parallel` were ported off their rayon pools onto
`gatling::gatling_forkjoin::gatling_for_each` (their `rayon` deps dropped; `lgz`/`lbzip2`
folded their private clones onto the shared engine too — every codec now routes through
the one pool). And `tests/rayon_free_law.rs` was widened from the `src/`-only scan to
`whole_workspace_is_rayon_free()` (`workspace_src_roots()` walks **every** member's
`src/` + `Cargo.toml`), and its forbidden-token set now also **bans private pools** —
`rayon::ThreadPool` and the pool builders — so a codec cannot quietly stand up its own
pool without tripping red. The carve-out is deleted. This is why the law now holds
crate-wide and not by footnote.
## The writer story — ring-slots refuted, parallel writers win
The headline of this session. The codecs saturated cores very unevenly —
**lgz ~3.1, ljar ~3.5, lzip ~4.0 of 12, while lbz2 held ~10.4** — and the obvious first
guess was *slot starvation*: widen `ring_slots` so more chunks are in flight. That
hypothesis is **REFUTED**. Widening the ring moved nothing. The bottleneck is not the
pipeline width; it is the **single ordered writer/collector at the tail**. Both the
streaming (`gatling::run`) and fork-join engines funnel every decoded unit through *one*
collector draining an mpsc and *one* writer thread — a serial drain that copies and
writes gigabytes while N-1 cores idle. lbz2 hides it only because its per-block
BWT/MTF/Huffman decode is heavy enough to dominate the serial tail; the DEFLATE codecs,
whose inflate is cheap (see [`decompressors-design.md`](decompressors-design.md)), are
left writer-bound.
The real fix is to **remove the single writer**: let each worker write its own output.
That is the [`parwrite`](gatling-design.md) layer — the same no-barrier, atomic-cursor,
`std::thread::scope` pool, but with **no collector and no writer thread on the critical
path**. Two disjoint-output shapes make it safe without ordering:
- **Independent files** (`extract_entries_unordered`) — ZIP/JAR members are independent,
so each worker `pread`s its entry from the shared archive and writes its **own** output
file. No two workers touch the same path; no ordering needed.
- **Independent regions of one stream** (`write_segments_positional` /
`write_slices_positional`) — the caller pre-computes each segment's byte offset (a
prefix-sum of known output sizes), pre-sizes the file, and each worker `pwrite`s its
segment at its offset. Positional writes to **disjoint** regions never conflict, so
exact stream byte-order is preserved with *zero* ordering on the hot path — the
offset-`pwrite` breakthrough.
Measured cores_busy on the writer-lift branch:
| codec | before (single writer) | after (parwrite) | note |
|---|---|---|---|
| **ljar** | 3.5 / 12 | **11.5 / 12** | both regimes (entries + positional) |
| **lzip-parallel** | 4.0 / 12 | **11.5 / 12** | both regimes |
| **lgz** | 3.1 / 12 | **7.8 / 12** | offset-`pwrite` path on `-o` files only |
| **lbz2** | ~10.4 / 12 | ~10.4 / 12 | already CPU-bound; writer was never its limiter |
**The honest limits.** lgz reaches only 7.8, not 11.5, and only on `-o file` output: a
multi-member gzip stream still has a **single-stream boundary-scan tail** (find the next
flush boundary before the offsets are known), and **stdout cannot be offset-written** —
`pwrite` needs a seekable file, so a piped `lgz input.gz | …` falls back to the ordered
writer. `parwrite` also assumes the output sizes are known up front (ZIP/JAR central
directory, or a prior decode pass), which is why it is a *sibling* layer the codec opts
into, not a silent replacement for `gatling::run`.
## Why `parwrite` is a separate layer, not folded into `gatling::run`
`gatling::run`/`run_typed` guarantee **ordered** output through one collector — that
ordering is the contract callers rely on (bz2→XML reassembly, deep_scan record order).
`parwrite` trades that away: it only works when the output units are *provably disjoint*
(independent files, or pre-offset regions of a pre-sized stream) so **no ordering is
needed at all**. Baking disjoint-output assumptions into the ordered engine would either
weaken its guarantee or bolt a mode-flag onto the hot loop. Keeping it a distinct body —
the parallel-writer sibling of `gatling_for_each`, exactly as `gatling_for_each_balanced`
is the skew sibling — keeps each engine's contract clean and lets a codec pick the writer
that matches its output shape.
## The road, briefly
Extracted from `katana-osm` into this leaf crate (see [`reasoning.md`](reasoning.md) for
the release-cascade rationale). The engines gained bodies as consumers needed them: the
streaming byte/typed pair for OSM decompress→parse, the fork-join family for fixed work
sets, the balanced bodies for nornir's skewed member-scan, the async sibling for
znippy's I/O fan-out, `gatling::ordered` for znippy's io_uring batched small-file
reader (produces `(path, bytes)`, wants strict producer-order archive writes through a
bounded reorder buffer), and finally the **`parwrite`** layer (landed on `master`)
for the codec extraction/writer tail — the last un-parallelised stage, removed by letting
every worker write its own disjoint output.