Gunnar

nordisk/skade

.nornir/commit-design.md

# The commit / write path — design

> **Class:** `design` · **Function:** `commit` · **Uses:** `read-path`, `store`.
> Only the shape that runs; the *why* (group-commit vs REST, optimistic CAS,
> the raw-triple workaround) is in [`commit-reasoning.md`](commit-reasoning.md).

![skade write path: FileIO metadata write, one redb transaction, L1 write-through](assets/skade_flow.svg)

## f0 — the commit shape

Every mutation of a table pointer follows one shape:

1. Write the new `…/<uuid>.metadata.json` blob via `FileIO` — immutable,
   content-addressed, done **outside** the redb lock (no network I/O under the
   lock).
2. One redb `WriteTransaction`: compare-and-swap the `tables` pointer against the
   base location it was read at, append to the `commits` log, bump the `meta`
   counters — one `commit()`, one fsync.
3. L1 pointer-mirror write-through, then maybe trigger a background Ragnar rebuild
   ([time-travel](time-travel-design.md)).

`record_commit` (`store.rs:111`) does step 2's log-append + counter bump inside the
caller's transaction, so history and pointer advance atomically.

## f1 — single-table commit

| entry | source | notes |
|---|---|---|
| `update_table(TableCommit)` | `catalog.rs:939` | the trait method: load current, `commit.apply`, persist |
| `commit_table(ident, reqs, updates)` | `catalog.rs:272` | the public primitive for callers holding raw requirement/update vecs (the `TableCommit` builder is crate-private upstream): checks every requirement against current metadata, folds updates into a builder, stages a `with_next_version()` location, persists |
| `persist_commit` | `catalog.rs:317` | shared tail: write blob → `store::group_commit` optimistic CAS → returns the staged `Table`; conflict → `CatalogCommitConflicts` (retryable) |

## f2 — group-commit (`store.rs:233`)

`Store::group_commit` coalesces concurrent commits into one redb transaction.
Each caller enqueues a `CommitFn` (`store.rs:54`) then races for the db lock;
whoever wins **drains the whole queue** and runs every op in a single write txn
with a single `commit()` — one fsync amortised across the batch. Each `CommitFn`
checks its own optimistic precondition and **writes nothing on `Err`**, so a
failing op cannot taint the others' commit. After the group `commit()` lands, each
op's `CommitOutcome` (`store.rs:34`) publishes its L1 mirror insert/remove and the
caller is woken via a `oneshot`. Losers of the race find their work already done.

## f3 — multi-table atomic release (`atomic.rs`)

`atomic_release` (`atomic.rs:46`) commits N `TableCommit`s all-or-nothing — either
every table head advances or none does. Three phases:

1. **Stage** — load each table, apply its commit in memory, capture
   `(base, staged)` metadata locations. Any apply failure aborts before any I/O.
2. **Write** — persist every staged metadata blob via `FileIO`, outside the redb
   lock. (These object writes are not transactional; a failure here leaves orphan
   blobs for Iceberg orphan-file cleanup.)
3. **Commit** — one redb txn: read every current pointer, verify each still equals
   its stage-time base (else `CatalogCommitConflicts`, no pointer moves), then flip
   them all, append every advanced table to `commits`. L1 write-through publishes
   all pointers under the write lock, so the batch is visible to readers
   all-or-nothing.

`atomic_release_raw` (`atomic.rs:157`) is the same all-or-nothing swap for callers
that hold raw `(ident, requirements, updates)` triples instead of `TableCommit`s.

## f4 — durability

Every mutation txn calls `write.set_durability(self.store.durability)`. The knob
is `WriteDurability` (`builder.rs:47`): `Immediate` (fsync per commit, the
default), `Eventual` (persist shortly after `commit`), `None` (no fsync, bulk
import). See [builder-design](builder-design.md).