Gunnar

mikronova/modgunn

.nornir/warehouse-reasoning.md

# warehouse — reasoning

> **Class:** `reasoning`. Why the warehouse seam is shaped this way, and the forks not taken.
> The descriptive shape is in [`warehouse-design.md`](warehouse-design.md).

## Why over skade, never nornir

modgunn needs the shared warehouse (advisories, snapshots, SBOM) but must not depend on
nornir — that edge is the cycle the whole crate exists to break (see [`reasoning.md`](reasoning.md)).
skade is an embedded-Iceberg leaf, so `modgunn → skade` is cycle-free, and skade re-exports
arrow/iceberg so modgunn needs no direct arrow/iceberg deps. modgunn reaches nornir's data by
opening the **same physical catalog** and reading nornir's `sbom_components` table directly —
data coupling through a shared table, not code coupling through an API. That is the only way
holger + nornir can both consume modgunn while modgunn stays a leaf.

## Why the whole seam is behind a feature (off by default)

skade pulls arrow/iceberg/redb — a heavy tree holger's lean gate and static-holger must never
carry. So `warehouse` is opt-in; the default `scan` build has no warehouse at all, and Engine
A takes an in-memory `AdvisoryDb` instead. The world-delta site and nornir opt in. The
DataFusion fleet-join is a *second* wall (`warehouse-sql`) because DataFusion (+ the C liblzma
via async-compression) is heavier still and only pays off at S3/S4 scale where loading the
whole advisory db into RAM stops being affordable.

## Why a read-only copied-snapshot open

redb is single-writer. A live `nornir-server` holds the write lock, so a modgunn reader that
tried to open read/write would block or fail. `open_read_only` copies `catalog.redb` into an
auto-cleaned temp dir and reads that — a consistent point-in-time snapshot with zero
contention. It works because skade stamps **absolute** `file://` data paths, so the copied
catalog still points at the live data files; only the tiny metadata db is copied. This mirrors
nornir's own `open_read_only`, so the two agree on the pattern.

## Why range-native rows and drift-tolerant reads

Advisories are stored as ranges (see [`feeds-reasoning.md`](feeds-reasoning.md)); the warehouse
round-trips the range JSON plus alias/withdrawn/source/CVSS/CWE columns. Reads go through
`str_col`, which tolerates missing columns, so schema v2 could add columns **additively**
without a migration — an old reader ignores new columns, a new reader defaults absent ones.
Iceberg appends are additive by nature, which is what makes `db_rev` generations cheap: each
import is an append, old generations stay for time-travel/audit (A31/A34), and `load_advisory_db`
simply returns the latest rev.

## Why modgunn consumes CDC rather than computing it

iceberg-rust 0.9.1 has no native incremental/changelog scan — only time-travel by snapshot id.
Rather than reconstruct CDC itself, modgunn leans on skade, which already walks the snapshot
lineage (`parent_snapshot_id`) and returns a `DeltaPlan` (added data files = inserts,
equality-delete files = deletes). modgunn just polls `commit_seq`, then `table_delta(from)`.
When upstream iceberg-rust ships a native changelog scan, skade swaps its internals and
modgunn's API does not change — the seam is deliberately thin so that swap is invisible here.

## Why the incremental scan exists at all

At S3/S4 scale recomputing the whole fleet on every advisory import is wasteful — only the
neighborhood a `read_delta` touched actually changed. `scan_sbom_delta(from)` scans only the
components added since a watermark and returns the next one, so a monitor polls
`commit_seq` → `scan_sbom_delta` and does O(delta) work, not O(fleet). This is the same reason
the heavy Engine-B materializations (blast-radius, owner-clusters) are designed to refresh off
the CDC delta rather than from scratch.