# archive — reasoning
> **Class:** `reasoning`. Why the archive layer is the way it is, and what we rejected. The
> descriptive shape is in [`archive-design.md`](archive-design.md).
## Why tar+zstd is the default, not znippy
The default engine has to be the one that **always works, always offline, with no feature flags**.
`tar` + `zstd` are pure Rust, universally understood, and need nothing beyond the always-on dep
set — so a `cargo build` with no features still packs and unpacks a bundle. znippy is *better*
(dedup, delta, O(1) read), but it is an optimisation, and an optimisation must never be the thing
standing between you and a working offline build. So znippy lives behind a feature and tar+zstd is
the dependable fallback that the pipeline can always reach for.
## Why a sidecar manifest instead of an in-archive index
The manifest is a separate `<bundle>.manifest.json`, not a header inside the tar. That keeps the
manifest readable and verifiable **without decompressing the archive** — the far side can inspect
what *should* be there, and the whole-file out-of-band SHA can be checked, before any extraction.
For tar (a linear format with no seekable index) an in-archive manifest would have to be read by
streaming anyway; a sidecar is simpler and engine-independent. znippy *does* additionally carry a
seekable in-archive index — that is what its O(1) read needs — but the portable contract is the
sidecar.
## Why verify rejects extra entries, not just changed ones
A naive verify checks "is every manifest entry present and unchanged?". That misses the attack
where a bundle *gains* an entry (a planted file) the manifest never listed. So `verify_entries`
matches the sets both ways: an archive entry absent from the manifest is as bad as a manifest
entry absent from the archive. This closes the extra-entry gap (regression-guarded by
`verify_rejects_extra_entry_absent_from_manifest`).
## Why the znippy engine consumes the real znippy crates (L5 dedup)
The `znippy` feature now links the REAL znippy stack (`../znippy`): `znippy-compress::compress_dir`
seals the bundle and `znippy-common` (`ZnippyArchive::open`/`ZnippyReader`, `get_file`,
`decompress_archive`, `verify_archive_integrity`) reads/verifies/unpacks it. A Skidbladnir bundle IS
a genuine `.znippy` archive (Arrow-IPC sub-index, blake3-checked chunk blobs) — `znippy` CLI /
DuckDB / Polars readable. There is ONE canonical archive format in the constellation, not a
look-alike twin (L5).
The earlier design re-implemented a **skidbladnir-native** store (magic `SKIDCAS1`, a zstd-JSON
index) to dodge znippy's arrow/rayon/CLI footprint — but that was a twin whose doc falsely claimed
znippy-compatibility. It has been deleted. The dependency cost (arrow + OpenZL + the gatling pool)
is acceptable because the `znippy` feature is **off by default and never shipped**: the lean offline
core and the embedded client stay on the always-Rust `TarZstd` fallback and never compile znippy.
History: [`znippy-core-consolidation-unsorted.md`](znippy-core-consolidation-unsorted.md).
## Why O(1) read matters enough to carry an index + footer
Reading one file out of a multi-gigabyte bundle by decompressing the whole thing is unacceptable
when the target only needs one artifact (a single product's binary out of a whole-lake bundle).
znippy's lookup sub-index + Arrow-IPC index let `read_entry` (`znippy_common::get_file`) locate the
entry and `pread` only its chunk blobs (blake3-verified), never touching the rest. Selective read
and the content-addressed delta both fall out of that same index.