<!-- nornir:class: reasoning -->
# gatling — the reusable list of tricks
**Written 2026-08-03**, from reading the whole engine (`gatling/src/*.rs`, published as
`rotaryengine` 0.1.10) and from building `ldeflate` on top of it. Deliberately written
**independently of that compressor**: this is the pattern list, not a design doc for one
consumer.
Everything here is either read from the engine source or measured on oden. Where a
commonly-repeated claim turned out not to hold, it says so — three of them did not.
---
## The soul, in one line each
**No barrier.** A worker that finishes unit *i* does not wait for a sibling, a scheduler,
or a stage boundary. It takes the next unit itself.
**Zero copy.** Bytes stay where the caller put them. Work items carry *indices, ranges and
pointers* — never payloads.
**Zero wait.** Nothing on the critical path blocks on something that is not the work
itself: not a lock hand-off, not a scheduler round, not an allocator.
---
## Trick 1 — Self-dispatch off one atomic, not a work queue
The cheapest possible hand-off. `gatling_forkjoin` is, at its core, four lines:
```rust
loop {
let i = next.fetch_add(1, Ordering::Relaxed);
if i >= n { break; }
f(i);
}
```
No channel send per unit, no central scheduler, no queue to drain. `Ordering::Relaxed` is
correct here because the *counter* carries no data — the data hand-off is the disjoint
index, and the scope join is the synchronisation edge.
**Why it beats a static carve-up.** Give each of 32 workers a fixed 1/32 of the range and
the whole run is as slow as the unluckiest slice. With a shared cursor, a worker that draws
cheap units simply draws more of them. On skewed unit costs — which is nearly all real work
— this is the entire difference.
**Where it lives:** `gatling_forkjoin::gatling_for_each`, `gatling_run`,
`gatling_for_each_balanced`, `gatling_map_balanced`, `gatling_map_owned`,
`gatling_scanlines`, and `parwrite`'s two entry points. Same four lines every time.
**MEASURED (oden, 2026-08-03, 32 workers):** one empty unit costs **62 ns** at claim 1 and
**28 ns** at claim 16, over 1 M units. At 100 k units it is 67/32 ns; at 10 k units it is
~290 ns, because pool spin-up (~340 µs for 20 workers, per skade's own note in
`read.rs`) has not yet amortised. **The pool has a floor: below ~50 k units of trivial work,
you are timing thread creation.**
## Trick 2 — Disjoint output slots, so no worker ever locks
Every fan-out in the engine pre-allocates one output slot per unit and lets each worker
write only the slot it uniquely claimed:
```rust
let mut out: Vec<MaybeUninit<T>> = Vec::with_capacity(n);
out.resize_with(n, MaybeUninit::uninit);
// ... worker writes out[i] for its own i only ...
unsafe { Vec::from_raw_parts(out.as_mut_ptr() as *mut T, len, cap) }
```
No `Mutex`, no `Arc<Mutex<Vec>>`, no per-worker `Vec` + final concat + sort. And because
slot *i* holds unit *i*'s result, **the output comes back in input order for free** — no
reduction pass, no re-sort, and a caller can index straight into it.
The soundness argument is always the same and always stated at the site: *each index is
claimed exactly once by the atomic, so the pointer writes never alias; the scope joins
before anyone reads.* The `OutPtr<T>` / `RowBase<P>` wrappers exist only to carry a raw
pointer across the scoped-thread boundary, and both are `Copy` with a hand-rolled impl so
they do not require `T: Copy`.
**The edition-2024 gotcha, and it is a real one.** Disjoint closure capture will grab the
raw pointer *field* rather than the `Send` wrapper struct, and the code stops compiling —
or worse, stops being sound. The engine works around it by rebinding the whole struct
inside the closure:
```rust
s.spawn(move || {
let out_ptr = &out_ptr; // whole-struct capture, so the Send impl applies
...
});
```
If you copy this pattern anywhere, copy that line too.
## Trick 3 — LPT: claim the heaviest units first
`gatling_for_each_balanced` / `gatling_map_balanced` sort a *schedule* of indices by
descending caller-supplied weight, then hand out schedule positions. Results are still
written to `out[original_index]`, so the reordering is invisible.
This is longest-processing-time-first, and it fixes the specific failure where the last
unit claimed is the most expensive one: a monster drawn at the end runs alone while 31
workers sit idle. Drawn first, its cost is hidden under everything else.
**Two constraints that are not obvious.** `weight` runs **serially on the calling thread**
for all *n* units before any work starts, so it must be O(1) and side-effect-free. And a
**panic inside a worker leaks** the `Vec<MaybeUninit<T>>` — there is no `catch_unwind`
anywhere in the engine. Return `Result`, never panic.
## Trick 4 — Batched claims, when units are cheap
`claim` (the `batch` parameter) takes `k` indices per `fetch_add` instead of one. It is the
knob for when the atomic itself starts to cost more than a unit.
**MEASURED, and the received wisdom is workload-specific.** The house note says claim 1 is
a 2× regression (from `znippy-zoomies/src/stree.rs`, on very cheap units). On empty units
that reproduces: 62 ns → 28 ns, a 2.2× difference. But on a **DEFLATE per object** — where
a unit costs microseconds — claim 1 / 4 / 16 measured **within 0.3% of each other** across
45 877 real git objects. The knob is real; its importance is entirely a function of how
cheap your unit is. Do not carry the 2× across workloads.
## Trick 5 — The two-queue split: coalesce the small, split the large
The shape `ljar` gestures at and that `ldeflate` implements: classify each item by size and
put it in a different queue.
- Items **below** a threshold are **coalesced** — one task drains many, so per-unit fixed
costs are paid once per batch rather than once per item.
- Items **above** a threshold are **split** — one item across many tasks, so a single
monster cannot serialise a worker.
- Everything between is one task, one item.
**Be precise about what ljar actually does, because it is less than it looks.** Its 10 MB
constant (`ljar/src/lib.rs:48`) picks between two *entry points* for a whole archive, not a
per-item size class. Its batch of 64 (`ljar/src/reader.rs:20`) is a **RAM bound** — "only
one batch of decompressed output is live at a time" — not dispatch amortisation. And it
dispatches **one gatling unit per entry with claim 1** (`parallel.rs:32`, `batch.rs:109`);
it never coalesces. So take the silhouette from ljar and build the small queue yourself.
**The measurement that decides the threshold is not the one you expect.** The obvious model
is "coalesce until dispatch cost stops mattering". On a real workload the dispatch was
never what mattered: 62 ns of gatling against ~4400 ns of per-object cost. The win came
from **reusing the expensive per-item object** (a zlib compressor: `reset` 1567 ns vs
`new` 4082 ns) inside the batch. **Measure what the per-item fixed cost actually is before
assuming it is the scheduler.**
## Trick 6 — 31+1: give the device its own core
N−1 workers compute; one thread does nothing but stream results to the device. Compute
never blocks on the device, and the device never idles waiting for a compute round to end.
In this engine the shape falls out of `gatling::ordered::run_ordered_sink`, whose
**collector runs on the calling thread**. So the caller's own thread *is* the writer, and
asking for `available_parallelism() - 1` workers gives 31+1 on a 32-core box literally.
**Why one writer and not 32, on local storage.** A write to NVMe is a memcpy-class syscall
with no round-trip latency to hide; one uncontended writer already saturates the device and
extra writers only buy lock, seek and allocator contention. This is documented reasoning,
not folklore — `skade/.nornir/s3-ingest-concurrency.md`, *"Why local stays sequential (and
must)"*. **The inverse holds for object storage**, where the cost is per-op latency that
only concurrency hides; that case wants `gatling::io` (the async sibling, bounded and
ordered over tokio tasks), not this.
> ### ⚠ Correction: skade is NOT the reference implementation of 31+1
>
> This trick is often attributed to skade "doing 200 M rows/s on 31 cores of Parquet while
> one core streams to NVMe". **That is not in skade.** Searched the whole repo including
> `git log --all --grep`: no "200 M", no "31+1", no "31 workers", and **no `cores - 1`
> arithmetic anywhere**. Every call site asks for `available_parallelism()`, all cores.
>
> What skade actually implements is **N workers → barrier → 1 sequential writer**:
> `gatling_map_owned` is a fork-join, `std::thread::scope` joins every worker before the
> results vec is read, and `write.rs:944` says outright that "the encode phase holds all
> files in memory before the write phase". Encode and write are **not** overlapped in
> `ingest_parallel` or `ingest_pipelined`.
>
> The "31" almost certainly comes from a *cores-busy measurement* —
> `s3-ingest-concurrency.md:143` reports zstd:9 NVMe ingest at **"31-32 / 32 cores busy,
> 97% CPU"**. That is 32 workers of which 31-32 were observed busy, not a 31-worker
> topology. skade's own highest rows/s figures are 92.7 M (uncompressed NVMe, and
> memory-bandwidth-bound at ~4/32 cores) and 103 M.
>
> The 31+1 pattern is still right, and the engine supports it directly. It just is not
> copied from skade, and the number should not be quoted.
## Trick 7 — Slot pool + carry: streaming without a copy
For data that *arrives* rather than already existing, `gatling::run` / `run_typed` fill
pre-allocated fixed-size slots on a reader thread and hand workers raw pointers into them.
Two details make it zero-copy in the common case:
- **Carry headroom.** Each slot reserves space at its *front*, so an unconsumed tail from
the previous chunk is written into the headroom and the worker sees one contiguous
`carry ++ read` slice with no concatenation. When a single unit outgrows the headroom
there is a correct fallback to a dedicated buffer — the headroom is a fast path, not a
cap on unit size.
- **`SlotFill::Incremental` vs `Big`.** Incremental grows a slot to exactly the bytes read
and never faults the unused tail (a 5 MB file used to zero 6 × 232 MB before reading a
byte); `Big` sizes once and reuses with no re-zeroing, which amortises to nothing when
every chunk is full. Small inputs want the first, planet-scale streams the second.
The slot is held alive in the collector's `InFlight` until every segment of that chunk is
done, which is what makes the workers' raw pointers sound.
## Trick 8 — Recover order at the edge, never in the middle
Ordering is a **sink** property in this engine, never a barrier.
- Fork-join: order is free (Trick 2 — disjoint slots are already indexed by input).
- Streaming: a collector thread holds out-of-order results and flushes strictly by
`chunk_id`, so workers never wait for their turn.
- `gatling::ordered::run_ordered_sink`: a bounded reorder buffer plus a **counting-permit
pool** primed with `cap` tokens. The producer must hold a permit to pull; the collector
releases one after each in-order emit. That single counter bounds the sum over
{ work channel, workers, result channel, reorder buffer } — memory is `O(cap)`, never
`O(stream length)`.
- `parwrite`: order recovered by **positional `pwrite` to pre-computed offsets**. No
collector at all, no serial concatenation — N workers write disjoint regions of one
pre-sized file and the byte order is exact.
**The choice to make:** if output sizes are known in advance, use `parwrite` and delete the
writer from the critical path entirely. If they are not, use the ordered sink and accept
one writer (which Trick 6 says you wanted anyway on NVMe).
## Trick 9 — Per-worker scratch, reused across every unit it claims
`parwrite` gives each worker a `comp` and an `out` `Vec` outside the claim loop, so after
the first (largest) unit the hot loop allocates nothing. Cheap, and it composes with
Trick 5: a coalesced batch is exactly the place to hold reusable per-worker state.
The general form: **anything expensive to construct should be constructed per worker or
per batch, never per unit.** Measured instance: a zlib compressor is 4082 ns to build and
1567 ns to reset, against ~95 ns of actual coding for a small object.
## Trick 10 — `background::Job` is the one sanctioned raw thread
The rayon-free law forbids `thread::spawn` outside the engine, which leaves producer /
consumer *overlap* (as opposed to fan-out) with nowhere to go. `background::Job` is that
home: one closure, one thread, `join` for the result. Depth-1 pipeline, not a pool. Use it
for a sampler, a spill-writer, a double-buffer — and nothing that wants work stealing.
---
## How the law is actually enforced (and it works)
`tests/rayon_free_law.rs` scans the root crate's `src/` plus **every member's `src/`, read
from the `members = [...]` line of the root `Cargo.toml`** — so a newly added crate is
covered automatically, with no carve-out to remember. Four checks: no rayon tokens, no
private pool (`thread::spawn` / `thread::Builder` / `crossbeam` / `ThreadPoolBuilder`), no
reinvented `thread::scope` outside the engine, and no manifest declaring rayon.
> **Correction, verified 2026-08-03 on `main` at `0e2fec4`:** this guard is often described
> as broken — "it `include!`s a missing file and cannot compile". **It compiles and all four
> tests pass.** There is no `include!` in the file. Verified further by injecting
> `rayon::current_num_threads()` into a new member crate: the guard went red and named the
> exact file and line. It is live and it does see new members.
## The three things this list corrects
1. **skade is not the 31+1 reference** (Trick 6). The pattern is sound; the attribution and
the 200 M number are not.
2. **ljar does not coalesce** (Trick 5). It has the silhouette and none of the small queue.
3. **The claim-size 2× is workload-specific** (Trick 4), and the rayon guard is not broken.
If a fourth is needed: **the dispatch cost is usually not your problem.** Measure the
per-item fixed cost first. Twice now the answer has been "the expensive object you rebuild
per item", not the scheduler.