# Git-like refs (`skade_katalog::git`) — design & guide
*Branches, tags, WAP publish, and rollback for the warehouse — 100% within the
Iceberg spec.* Every Iceberg table already carries a `refs` map (movable
**branches**, immutable **tags**); this surfaces it at the catalog level and
adds the git control-plane operations on top, all expressed as standard Iceberg
`SetSnapshotRef` / `RemoveSnapshotRef` updates driven through the existing
atomic commit path. Tables stay pure Iceberg — any reader works at a given ref.
See the capability it lands from: `.nornir/git-like-catalog.md` (Tier 1) and
`.nornir/warehouse-capability-gaps.md`.
---
## 1. Why (the gap it closes)
skade-katalog already had the versioned-metadata spine a git-like catalog needs
— **atomic multi-table commit** (`atomic.rs`), a **current-pointer mirror**
(`pointer_cache.rs`), a **durable commit log**, a **Ragnar snapshot index**
(`static_index.rs`), and **time-travel by snapshot-id / timestamp**
(`load_table_at` / `load_table_as_of`). What it did not expose was Iceberg's
**git surface**: you could not create a branch, tag a release, run
write-audit-publish, or roll a table back to a known-good state through the
catalog — even though iceberg-rust models all of it natively
(`SnapshotReference` / `SnapshotRetention`). This closes that gap without a new
dependency, without a JVM (a Nessie-without-the-JVM start), and without touching
any existing read/write path.
## 2. The idea (refs are just snapshot pointers with a policy)
An Iceberg **branch** is a *movable* named pointer to a snapshot; a **tag** is an
*immutable* one. Both live in the table metadata's `refs` map. Every git-like
operation is therefore a metadata edit that (a) validates a precondition and
(b) sets or removes one ref — exactly what `TableUpdate::SetSnapshotRef` /
`RemoveSnapshotRef` do. So the whole feature is a thin, correctness-focused
layer over the catalog's existing `commit_table` pointer-advance:
```
create_branch / create_tag → SetSnapshotRef{ ref, SnapshotReference{ snap, retention } }
drop_ref → RemoveSnapshotRef{ ref }
fast_forward / publish → SetSnapshotRef{ branch, descendant-snap } (ancestry-checked)
rollback_to → SetSnapshotRef{ main, ancestor-snap } (ancestry-checked)
```
Each goes through one redb write transaction (one fsync), appends the commit
log, and updates the L0/L1 caches — the same all-or-nothing discipline the rest
of the catalog uses. Reads (`list_refs`, `ref_snapshot_id`,
`resolve_metadata_for_ref`) hit the lock-free L1 pointer mirror + L0 metadata
cache; no redb transaction on the warm path.
## 3. The shape that runs
All methods are additive on `RedbCatalog` (`src/git.rs`):
| method | git analogue | what it does |
|---|---|---|
| `create_branch(ident, name, snap?, retention)` | `git branch` | mint a movable branch on `snap` (or the current snapshot); rejects an existing name |
| `create_tag(ident, name, snap?, max_ref_age_ms)` | `git tag` | mint an immutable tag; rejects an existing name |
| `drop_ref(ident, name)` | `git branch -d` / `git tag -d` | delete a branch or tag |
| `list_refs(ident)` | `git show-ref` | every branch/tag + its snapshot, sorted by name |
| `ref_snapshot_id(ident, name)` | `git rev-parse` | resolve a ref to its snapshot id |
| `fast_forward(ident, branch, to)` | `git merge --ff-only` | advance a branch to a **descendant** snapshot |
| `publish_branch(ident, from, to)` | WAP publish | fast-forward `to` (e.g. `main`) to the head of `from` (an audited branch) |
| `rollback_to(ident, snap)` | `git reset --hard <ancestor>` | move `main` back to an **ancestor** snapshot |
| `load_table_for_ref(ident, name)` | `git checkout <ref>` | read the table as of a branch/tag (routes through `load_table_at`) |
| `resolve_metadata_for_ref(ident, name)` | — | metadata-only ref read |
### The two safety invariants (the correctness core)
Both mutation families are guarded so history can never silently diverge — the
same rules `git push` and `git reset` enforce:
1. **Fast-forward only.** `fast_forward` / `publish_branch` walk the
`parent_snapshot_id` chain from the target back toward the branch's current
head; the move is allowed **only if the current head is an ancestor of the
target** (or equal — a no-op). A non-fast-forward move is rejected with a
retryable `CatalogCommitConflicts`. This is what makes WAP safe: a `main`
that advanced under an audit run refuses to publish rather than rewind.
2. **Rewind only.** `rollback_to` allows moving `main` **only to an ancestor**
of its current head — you can rewind along existing history, never jump to an
unrelated line. Nothing is deleted (the newer snapshots stay in the table),
so the rollback is reversible with a later `fast_forward`.
Every move additionally carries a `TableRequirement::RefSnapshotIdMatch`
optimistic precondition on the head it validated, so a concurrent writer that
moved the ref out from under us aborts the commit instead of clobbering it.
These invariants are covered by **RED-when-broken** tests
(`tests/git_refs.rs`): remove the ancestry walk and `fast_forward_and_non_ff_guard`
/ `rollback_ancestor_guard` fail, because a backward fast-forward or a rollback
to a sibling line would wrongly succeed.
## 4. What is NOT here (deferred, honestly)
- **Data production on a branch.** Landing *new* snapshots onto an audit branch
(the "write" half of WAP) is the writer's job — it needs iceberg-rust's
append/manifest transaction actions, which 0.9.1 keeps crate-private. This
module operates on snapshots that already exist; it is the ref *control plane*.
- **Cherry-pick / merge / conflict detection** (design-doc Tier 1 tail / Tier 2).
Cherry-pick needs snapshot re-parenting; catalog-level (cross-table) branches
and merge are the Nessie model — a later ladder rung, tracked in
`.nornir/git-like-catalog.md`.
- **Ref retention expiry.** The retention policy is stored on the ref
(`BranchRetention`, `max_ref_age_ms`); the expiry sweep that acts on it is not
implemented here.
## 5. Idiot guide (zero-knowledge)
Think of a table's history as a chain of saved states (snapshots), newest last.
- A **branch** is a sticky note that says "this is the current one" and moves as
you save. `main` is the default branch. `create_branch` puts a new sticky note
on any saved state.
- A **tag** is a sticky note that never moves — you slap it on a release so you
can always find it: `create_tag(t, "v1.0", None, None)` tags the current state.
- **Fast-forward** slides a branch's sticky note *forward* to a newer state, but
only if the newer state grew out of where the note is now — you can't skip to
an unrelated line, and you can't slide it backward. That is how
**write-audit-publish** works: writers build new states on an `audit` branch,
someone checks them, then `publish_branch(t, "audit", "main")` slides `main`
forward onto the checked state — atomically, or not at all.
- **Rollback** slides `main` *backward* to an older state it grew from
(`rollback_to`), like an undo. The newer states aren't thrown away, so you can
redo by fast-forwarding again.
- **Reading a ref**: `load_table_for_ref(t, "v1.0")` gives you the table exactly
as it was at that tag — time-travel by name instead of by number.
Nothing here changes how tables are stored: they remain ordinary Iceberg tables,
so any Iceberg reader still works. skade just turns on the git buttons.
## 6. Benchmarks
`skade_git_refs` (in-proc embedded, `bench/examples/nornir-bench.rs`) times the
git surface over a table with a linear snapshot history and many
branches/tags: the lock-free **ref-read** hot path (`ref_snapshot_id`,
`list_refs`) and the **ref-move** write path (create-tag/drop cycles, one redb
commit + fsync each). Heavy sweep via `BENCH_GIT_REFS` / `BENCH_GIT_READ_ITERS`
/ `BENCH_GIT_WRITE_ITERS`.
<!-- nornir:class: benchmarks -->
### Measured — HEAVY, host loki (12 cores, 64 GiB)
Table with a 64-snapshot linear history and 256 seeded branches/tags;
`BENCH_GIT_READ_ITERS=2000000`, `BENCH_GIT_WRITE_ITERS=5000`.
| op | result |
|---|--:|
| `ref_snapshot_id` — lock-free ref resolve (L1 mirror + L0 metadata) | 3720754 ops/sec (269 ns/op) |
| `list_refs` — build + sort the whole 256-ref map | 65706 ops/sec (15219 ns/op) |
| ref move — `create_tag` + `drop_ref` (one redb commit + fsync each) | 2760 commits/sec |
Ref reads ride the same lock-free warm path as `resolve_metadata`, so they cost
a metadata-map read, not a redb hit. Ref moves pay one durable redb transaction
(fsync) per commit — the same write cost as any other catalog pointer advance,
so the git control plane adds no new hot-path overhead to existing reads/writes.