# lbzip2 library readers — design
> **Class:** `design` · **Function:** `lbzip2-readers` (the bzip2 library twins of the
> `lbunzip2` CLI). Strict: shipped shape, every claim a real `file:symbol`. Run them via
> [`lbzip2-readers-guide.md`](lbzip2-readers-guide.md); the whole-slice-vs-stream
> tradeoff in [`lbzip2-readers-reasoning.md`](lbzip2-readers-reasoning.md). The CLI is
> [`lbunzip2-design.md`](lbunzip2-design.md).
## What they are
Three callable (not-a-binary) bzip2 decoders in the `lbzip2` crate, each fanning
independent BWT blocks across every core through `gatling_forkjoin::gatling_for_each`.
They differ only in *how much* input they hold and *how* they hand output back. All
three front their decode with the same **parallel block-magic scan** —
`block_scan::find_all_blocks_parallel` (`lbzip2/src/block_scan.rs:143`) — which splits
the buffer into N chunks (each scanned with an 8-byte overlap so a boundary-straddling
magic is not missed), fans the scan across cores via `gatling_for_each`, then merges +
sorts + dedups by `bit_offset` to produce the **identical boundary set** the serial
`find_all_blocks` (`block_scan.rs:129`) would — proven by
`parallel_scan_matches_serial_multiblock` (`block_scan.rs:353`). The point: the
whole-stream magic walk no longer runs as a 1-core serial prefix.
## The three readers
### 1. `parallel::decompress_parallel(data) -> Vec<u8>` (`lbzip2/src/parallel.rs:15`)
Whole-slice, whole-output. Parses the `BZh` header, `find_all_blocks_parallel`, then
**groups blocks into `n_segments = (n_threads * 4).min(n_blocks)` contiguous segments**
(`parallel.rs:61`) — ~4 segments/thread so `gatling_for_each`'s self-dispatch can
balance uneven block sizes while amortizing per-block allocation. Each segment worker
runs `chunk::decode_segment_checked` (`lbzip2/src/chunk.rs:117`), appending every block
in its range into **one growing buffer** via `decode_block_into` (no per-block `Vec`).
Output is assembled by concatenating only `n_segments` buffers (`parallel.rs:114`).
Integrity: every block's stored CRC-32 is folded (rotate-left-1-then-XOR) into a
combined stream CRC and checked against the stream trailer via
`block_scan::find_stream_crcs` (`block_scan.rs:82`) for a single stream; concatenated
(pbzip2) inputs rely on the per-block CRCs (`parallel.rs:109`).
### 2. `reader::StreamingBz2Read<R: Read>` (`lbzip2/src/reader.rs:30`)
`Read`-trait streaming for a `Read` source. Reads ~100 MB compressed chunks
(`CHUNK_BYTES`, `reader.rs:18`), runs `find_all_blocks_parallel` per chunk
(`reader.rs:111`), **skips the last block when not at EOF** (it may span into the next
chunk), decodes the complete blocks across cores with `gatling_for_each`
(`reader.rs:143`) calling `block::decode_block`, then drains the decoded region from
`comp_buf` while keeping the undecoded tail. Reading the next chunk overlaps consuming
the current decoded output; only one chunk's decoded output is live at a time.
### 3. `reader::ParallelBz2Read<'a>` (`lbzip2/src/reader.rs:205`)
`Read`-trait over an in-memory `&[u8]` (e.g. an mmap'd file). Scans **all** boundaries
up front (`find_all_blocks_parallel`, `reader.rs:237`), then decodes in
`BATCH_SIZE = 64`-block parallel batches (`fill_batch`, `reader.rs:253`;
`gatling_for_each` at `reader.rs:267`), serving each batch through `read()`.
## Whereabouts
| Thing | file:symbol |
|---|---|
| parallel block-magic scan | `lbzip2/src/block_scan.rs:143` (`find_all_blocks_parallel`) |
| serial reference scan | `lbzip2/src/block_scan.rs:129` (`find_all_blocks`) |
| stream-CRC scan | `lbzip2/src/block_scan.rs:82` (`find_stream_crcs`) |
| whole-slice decode | `lbzip2/src/parallel.rs:15` (`decompress_parallel`) |
| Read-trait streaming | `lbzip2/src/reader.rs:30` (`StreamingBz2Read`) |
| Read-trait mmap/slice | `lbzip2/src/reader.rs:205` (`ParallelBz2Read`) |
| CRC-checked segment decode | `lbzip2/src/chunk.rs:117` (`decode_segment_checked`) |
| fan-out primitive | `gatling/src/gatling_forkjoin.rs:60` (`gatling_for_each`) |
## Saturation state
These readers are the natural consumers behind e.g. `osm-katana`'s bz2 path. The
scan-then-fan shape keeps cores fed; the honest ceiling matches `lbunzip2` (bzip2's
block independence is the whole win — see
[`lbunzip2-reasoning.md`](lbunzip2-reasoning.md)). No `cores_busy` is fabricated here;
real numbers land in the [`README.md`](README.md) mashup on the heavy-bench wave.