Gunnar

nordisk/skade

.nornir/catalog-design.md

# The Iceberg Catalog surface — design

> **Class:** `design` · **Function:** `catalog` · **Uses:** `read-path`, `store`.
> Only the shape that runs; the *why* is in [`reasoning.md`](reasoning.md).

![skade-katalog embedded Iceberg catalog](assets/skade_katalog.webp)

## f1 — the `Catalog` trait over redb

`RedbCatalog` (`catalog.rs:33`) implements the async
[`iceberg::Catalog`](https://docs.rs/iceberg) trait. It is a cheap-to-clone
handle over four fields: the catalog `name`, the `warehouse_location`, a `FileIO`,
and a shared `Store` (plus the L0/L1.5 caches — see [read-path](read-path-design.md)).

### Namespaces

| trait method | impl | behaviour |
|---|---|---|
| `create_namespace` | `catalog.rs:468` | insert a marker row in `namespaces` + each property in `namespace_props`; errors `NamespaceAlreadyExists` |
| `get_namespace` | `catalog.rs:499` | existence check + range-scan its properties |
| `namespace_exists` | `catalog.rs:536` | single `namespaces` point lookup |
| `list_namespaces` | `catalog.rs:409` | prefix range-scan; `parent` filters to direct children (one more `.`-delimited component) |
| `update_namespace` | `catalog.rs:544` | full-set replace: wipe existing props, write the new map |
| `drop_namespace` | `catalog.rs:596` | rejected if any table remains in it; then remove marker + orphan props |

### Tables

| trait method | impl | behaviour |
|---|---|---|
| `create_table` | `catalog.rs:700` | resolve location (explicit → namespace `location` prop → `warehouse/ns/table`), write `00000-<uuid>.metadata.json`, insert the `tables` pointer under a write lock (TOCTOU recheck), record the commit, L1 write-through |
| `load_table` | `catalog.rs:790` | location-first: L1 pointer → L1.5 handle-cache clone, else L0 parse + build + cache |
| `list_tables` | `catalog.rs:662` | prefix range-scan of `tables` within a namespace |
| `table_exists` | `catalog.rs:827` | L1 mirror hit answers with no lock/redb; a miss confirms against redb |
| `drop_table` | `catalog.rs:807` | remove the `tables` pointer + L1 write-through (data files are left to Iceberg orphan cleanup) |
| `rename_table` | `catalog.rs:843` | move the pointer src→dest in one txn (dest namespace must exist, dest must be free); L1 remove+insert |
| `register_table` | `catalog.rs:891` | adopt an existing `metadata_location` under a new identifier; records a commit |
| `update_table` | `catalog.rs:939` | apply a `TableCommit` → see [commit-design](commit-design.md) |

### Key encoding

Every redb key is `{catalog}\x1f{ns_path}\x1f{leaf}` (`keys.rs`). Namespace parts
join with `.`; the `\x1f` (ASCII Unit Separator) delimiter cannot appear in an
Iceberg identifier, so keys are unambiguous. Range scans use `prefix_upper`
(`keys.rs:56`) to compute an exclusive upper bound by incrementing the last byte —
this is how `list_*` and namespace/table prefix sweeps stay contiguous and
catalog-scoped.

### What is not (yet) implemented

Schema-evolution `TableUpdate`s are limited to what iceberg-rust 0.9.1 exposes
publicly; `commit_table` / `atomic_release_raw` ([commit-design](commit-design.md))
exist because the `TableCommit` builder is crate-private upstream. Rationale and
the upstream-gap timeline are in [`reasoning.md`](reasoning.md).

### Errors

Every redb sub-error is mapped to an `iceberg::Error` via
`error::map_redb` (`src/error.rs`); the not-found / already-exists constructors
live at `catalog.rs:372`. `RedbCatalogError` is the crate's own error type,
re-exported at `lib.rs`.