# skade — system design (f0)
> **Class:** `design` · **Function axis:** `f0` (the whole) — the per-function
> designs are [`catalog-design.md`](catalog-design.md),
> [`read-path-design.md`](read-path-design.md),
> [`commit-design.md`](commit-design.md),
> [`time-travel-design.md`](time-travel-design.md),
> [`builder-design.md`](builder-design.md).
> Strict: this file states **only the design that runs**. Every *why* and every
> rejected alternative lives in [`reasoning.md`](reasoning.md); the planning spine
> is [`plan.md`](plan.md).
## f0 — the whole: one file *is* your Iceberg catalog
`skade-katalog` is an embedded, single-file, ACID Apache Iceberg catalog: it
implements [`iceberg::Catalog`] against **one** `redb` database file. All catalog
metadata — namespaces, namespace properties, table-pointer rows, the immutable
commit log — lives inside that file (`src/store.rs`). The Iceberg table metadata
JSON / manifests / Parquet data live in whatever `FileIO` the builder is given
(local fs, S3, GCS). The public surface is five re-exports (`src/lib.rs:64`):
`RedbCatalog`, `RedbCatalogBuilder`, `RedbCatalogError`, and two cache-default
constants.
The system is five functions wired into one catalog:
| function | what it is | source | design |
|---|---|---|---|
| **catalog** | the `iceberg::Catalog` surface — namespaces + tables CRUD | `catalog.rs`, `keys.rs` | [catalog-design](catalog-design.md) |
| **read-path** | the warm read stack — L0 metadata cache, L1 pointer mirror, L1.5 handle cache | `meta_cache.rs`, `pointer_cache.rs`, `table_cache.rs` | [read-path-design](read-path-design.md) |
| **commit** | the durable write path — single commit, group-commit, atomic multi-table release | `catalog.rs`, `atomic.rs`, `store.rs` | [commit-design](commit-design.md) |
| **time-travel** | snapshot-id history — the Ragnar `STree64` index over the commit log | `static_index.rs`, `store.rs` | [time-travel-design](time-travel-design.md) |
| **builder** | construction + configuration (durability, cache budgets) | `builder.rs` | [builder-design](builder-design.md) |
### The redb tables (the substrate every function shares)
One `redb::Database` holds five tables (`src/store.rs:68`), all mutated inside a
single `WriteTransaction` per commit (one fsync):
| redb table | key → value | purpose |
|---|---|---|
| `tables` | `catalog\x1f table_id` → `metadata_location` | the live table pointer, CAS-advanced on commit |
| `commits` | `catalog\x1f table_id\x1f snapshot_id:u64_be` → `metadata_location` | append-only **commit log** — the history the Ragnar index is built from |
| `namespaces` | `catalog\x1f ns` → marker | namespace existence |
| `namespace_props` | `catalog\x1f ns\x1f key` → value | namespace properties |
| `meta` | key → `u64` | monotonic `commit_seq`, `update_counter`, reserved `live_tail_misses` |
redb is the **source of truth and durability layer**; every in-memory layer
above it is an accelerator, keyed by an immutable, content-addressed identifier
so an entry can only be *evicted*, never *stale*.
### The reuse graph (what each function `uses`)
```
iceberg::Catalog (trait)
│
┌──────────────────────┴───────────────────────┐
│ RedbCatalog │
│ catalog ── uses ──▶ read-path ──▶ store │
│ commit ── uses ──▶ read-path, store │
│ time-travel ── uses ──▶ store (commits) │
│ builder ── constructs ──▶ all of the above │
└───────────────────────────────────────────────┘
│ depends on
▼
redb · znippy-zoomies::STree64 · moka / arc-swap / imbl
```
- **read-path** `uses` **store** (the redb read txn is its miss fallback).
- **commit** `uses` **read-path** (loads the current table) and **store** (the
group-commit txn + commit-log append).
- **time-travel** `uses` **store** (the `commits` table it indexes and falls back
to) and shares nothing mutable with the latest-state read-path.
- **builder** constructs everything and owns the config knobs.
### Storage layout
```
<warehouse-dir>/
catalog.redb # the entire catalog (5 redb tables)
<namespace>/<table>/
metadata/<uuid>.metadata.json # immutable, content-addressed
data/<uuid>.parquet # Iceberg data files
```
One directory is the whole lakehouse: copy it, snapshot it, rsync it.
### Concurrency model
All redb access funnels through a `tokio::sync::Mutex<Database>` (`store.rs:137`)
so the async `Catalog` trait never blocks a runtime thread on redb's sync API.
redb itself allows one writer at a time; the mutex makes that queue explicit and
non-blocking. Latest-state reads (`table_exists`, `resolve_metadata`, warm
`load_table`) and time-travel index lookups take **no** lock — they read the
`ArcSwap`-published mirror / static index directly.
## Which recipe reader this repo uses, and why (lane AJ, 2026-08-30)
**Reader:** `nornir-build-thing` **0.1.11**, from crates.io, **no path** — dev-dependency only
(`tests/build_variants.rs`).
**Why:** skade is the ONE deliberate registry reader. A hardcoded `../nornir/...` path
broke any nornir git-worktree whose directory name was not `nornir`, so this edge
stays published-only. Its floor must therefore be a version that EXISTS on
crates.io: `0.1.11`, not the constellation's local 0.1.12. `0.1.4` had been
resolving to 0.1.11 in silence. skade's recipe is schema 1, which is what makes a
published reader correct for it.
The rule itself — `[nornir-build-thing] schema = N`, a refusal that names the
version to upgrade to, and a loader that refuses to DROP a key it cannot read —
lives once, in **edda `.nornir/build-design.md`, "THE COMPATIBILITY RULE"**. The
cross-version guard that would have caught the 2026-08-30 skew is
`edda/nornir-build-thing-skew`.