Gunnar

nordisk/skade

.nornir/read-path-guide.md

# The read path — guide

> **Class:** `guide` · **Function:** `read-path`. The *what* is in
> [`read-path-design.md`](read-path-design.md); the *why* in
> [`read-path-reasoning.md`](read-path-reasoning.md).

## Pick the lightest read that answers your question

The caches are transparent — you never call them directly — but which catalog
method you call decides how much work a warm hit does:

| you need | call | warm hit does |
|---|---|---|
| "does this table exist?" | `table_exists(&ident)` | one L1 mirror lookup, no lock, no redb |
| the current metadata location | `resolve_metadata(&ident)` then `.metadata_location` path | L1 mirror |
| schema / snapshots / props | `resolve_metadata(&ident)` | L1 → L0 (`Arc` clone) |
| a full `Table` to scan | `load_table(&ident)` | L1 → L1.5 (`Table` clone) |

Rule of thumb: **don't call `load_table` if `resolve_metadata` answers you.**
Building a `Table` constructs iceberg's per-`Table` object cache; the handle cache
amortises it, but resolving metadata skips it entirely.

## Sizing the caches

Set budgets at build time ([builder-guide](builder-guide.md)):

```rust
RedbCatalogBuilder::default()
    .metadata_cache_bytes(256 * 1024 * 1024)   // L0: 256 MiB of parsed metadata
    .table_handle_cache_bytes(512 * 1024 * 1024) // L1.5: byte budget for resident Table handles
    // …
```

- **L0 (`metadata_cache_bytes`)** is a soft byte budget over parsed
  `TableMetadata`; snapshots + schemas dominate the weight. Raise it if you have
  many large tables and see repeated metadata parses.
- **L1.5 (`table_handle_cache_bytes`)** is a byte budget, NOT a handle count.
  Entries are NOT cheap: a handle pins its parsed metadata AND iceberg's own
  per-`Table` `ObjectCache`, which is why a count cap of 512 could pin ~16 GiB.
  Each handle's object cache is pinned to `TABLE_OBJECT_CACHE_BYTES` (4 MiB) and
  charged at that ceiling, so the budget is a real upper bound.
- `0` on either disables that layer — every read then falls through to the layer
  below (parse, or build). The L1 pointer mirror is always on.

## What you never have to worry about

There is **no cache invalidation to call, no TTL to tune, no staleness window.**
Because each entry is keyed by its immutable metadata location, a commit that
changes a table writes a new file at a new key — the old entry is simply never
looked up again and eventually evicted. Reads after a commit see the new pointer
immediately via the write-through mirror.