# lzip-parallel — design
> **Class:** `design` · **Function:** `lzip-parallel` (parallel ZIP decode). Strict: the
> shape we ship, every claim carrying a real `file:symbol`. How to run it in
> [`lzip-parallel-guide.md`](lzip-parallel-guide.md); why the balanced fan-out earns its
> keep in [`lzip-parallel-reasoning.md`](lzip-parallel-reasoning.md). The family view is
> [`decompressors-design.md`](decompressors-design.md); the engine is
> [`gatling-design.md`](gatling-design.md). This is the **near-verbatim twin** of
> [`ljar-design.md`](ljar-design.md) — JAR ⊂ ZIP, so ljar is the same pipeline over the
> subset container.
## What it is
A pure-Rust parallel decoder for the *stock* `.zip` container (wheels, `.jar`, plain
`.zip`), decoded across every core. The CLI is `lzip-parallel/src/bin/lzip.rs`; the library
twin is `lzip_parallel::decompress_zip` (`lzip-parallel/src/lib.rs:58`) with the `_filter`
(`:76`), `_filter_set` (`:94`), `_filter_suffixes` (`:111`) and `_stream` (`:126`) family,
all returning `Vec<ZipEntry>`.
Every ZIP entry is its **own** DEFLATE (or STORE) stream with its **own** CRC-32, indexed
by a Central Directory at the file tail — so the parallelism is per-entry, and the only
serial cost is the one-time Central Directory parse.
## Path 1 — in-memory small file
`parallel::decompress_parallel` / `_filter` (`lzip-parallel/src/parallel.rs:17` / `:25`),
plus the `_filter_set` (`:126`) and `_filter_suffixes` (`:157`) variants for wheels. Each
reads the Central Directory once (`central_dir::read_central_directory`), filters out
directories and non-matching names **before** any inflate, then fans the matched entries
through `gatling::gatling_forkjoin::gatling_for_each_balanced` (call sites
`lzip-parallel/src/parallel.rs:32`, `:138`, `:167`) — weighted by `uncompressed_size`.
Results come back in **Central Directory order**. `decompress_zip` picks this path below
`SMALL_FILE_THRESHOLD` = 10 MB (`lzip-parallel/src/lib.rs:41`).
## Path 2 — streaming large file
`reader::StreamingZipRead` (`lzip-parallel/src/reader.rs:38`) with `new` / `with_filter`
/ `with_filter_set` / `with_filter_suffixes`. Each seeks to the tail, parses the Central
Directory, filters, then **sorts entries by `local_header_offset`**
(`lzip-parallel/src/reader.rs:55`) for sequential disk reads. Decoding runs in batches of
`BATCH_SIZE` = 64 (`:20`): `fill_batch` (`lzip-parallel/src/reader.rs:122`) reads each
entry's compressed bytes sequentially (seek to local header, skip name + extra, read
`compressed_size` bytes), then parallel-decodes via `gatling_for_each_balanced` (call site
`:175`) weighted by `uncompressed_size`. It implements
`Iterator<Item = Result<ZipEntry, ZipError>>`, yielding in file-offset order. **Only one
batch of decompressed output is live at a time** → bounded peak memory; the file is never
fully loaded. One micro-opt: the compressed read buffer is uninitialised
(`Vec::with_capacity` + `set_len`, `lzip-parallel/src/reader.rs:154-160`) because
`read_exact` fully overwrites it — the zero-fill was wasted work on the hot path.
## The saturation fix — LPT balanced fan-out
Weighting each entry by `uncompressed_size` makes the schedule **heaviest-first** (LPT): a
skewed archive — one fat entry among many tiny ones — can't tail-choke a single core,
because the monster is claimed first and the cheap tail backfills the idle workers. The
rebalance changes only *which worker claims which entry*, never the bytes or the ordering.
Proven by `streaming_skewed_roundtrip_byte_identical` (`lzip-parallel/src/reader.rs`).
## Correctness
Decode is `entry::decompress_entry_raw(compressed, method, expected_size)`
(`lzip-parallel/src/entry.rs:93`): method 8 → `linflate` DEFLATE, method 0 → STORE copied
verbatim — CRC-32 verified. A mismatch fails that entry; its siblings keep running.
## CLI extract path
`lzip-parallel/src/bin/lzip.rs` extracts each entry to its own output file via
`gatling::parwrite::extract_entries_unordered` (`gatling/src/parwrite.rs:85`): each worker
`pread`s its entry, decodes with `linflate`, writes its own file — no ordered collector,
since independent output files need no global order.
## Whereabouts
| Thing | file:symbol |
|---|---|
| CLI entry / extract | `lzip-parallel/src/bin/lzip.rs` (`extract_entries_unordered` at `:146`) |
| library API | `lzip-parallel/src/lib.rs:58` / `:76` / `:94` / `:111` / `:126` |
| in-memory parallel decode | `lzip-parallel/src/parallel.rs:17` / `:25` / `:126` / `:157` |
| streaming reader | `lzip-parallel/src/reader.rs:38` (`StreamingZipRead`) |
| batch fill (sequential reads) | `lzip-parallel/src/reader.rs:122` (`fill_batch`, `BATCH_SIZE`=64 at `:20`) |
| balanced fan-out call site | `parallel.rs:32` / `:138` / `:167`; `reader.rs:175` |
| balanced engine | `gatling/src/gatling_forkjoin.rs:209` (`gatling_for_each_balanced`) |
| per-entry decode + CRC | `lzip-parallel/src/entry.rs:93` (`decompress_entry_raw`) |
| skew correctness test | `lzip-parallel/src/reader.rs` (`streaming_skewed_roundtrip_byte_identical`) |
## Saturation state
Every parallel hot path fans per-entry DEFLATE across all cores through the one shared
engine; the LPT weight keeps a skewed archive from stranding cores on the tail. Honest
`cores_busy` numbers land in the [`README.md`](README.md) A/B/C mashup on the next
heavy-bench wave — this doc records no fabricated figures.