Gunnar

nordisk/dwarves

.nornir/auto-materialize-for-idiots.md

<!-- nornir:class: guide -->
# cache-hit auto-materialize — for idiots

> **Class:** `guide` (beginner) · The zero-knowledge walkthrough.
> What it IS: [auto-materialize-design.md](auto-materialize-design.md); how to use it: [auto-materialize-guide.md](auto-materialize-guide.md).

## The problem, in one picture

The forge is smart: if it already built something, it doesn't build it again. It says "cache hit — I know the answer, here's the id tag `sha256:abc…`" and moves on.

But the NEXT step in the pipeline often needs the *actual file* the skipped step made. On the machine that built it the first time, the file is still lying around. But on a **fresh checkout** or a **different machine**, the skipped step never ran here — so the file was never written. The next step reaches for it and finds an empty folder.

> "I skipped compiling (cache hit) — but the test step needs the compiled binary on disk, and it isn't there."

## The fix

We already push every built file's bytes into **holger** (the shared store) when a step runs — that's the other half of this system. So the bytes DO exist somewhere reachable. Auto-materialize just brings them back:

1. A step is a cache hit — skipped, not re-run.
2. Auto-materialize asks holger for that step's bytes (by the id tag).
3. It **writes them onto disk** in the work folder.
4. The next step reaches for the file — and now it's there.

The skipped step's output is restored *as if* it had run, without actually re-running it. Fast (no rebuild) AND correct (the file is on disk).

## Two flavours of "the file"

- Some steps make **one** file → restored under the step's name.
- Some steps make **several** files (like holger's embedded / mini / full binaries) → each restored under its own real name, because we kept a little list (a "manifest") of them.

## What if it isn't set up?

Nothing changes. If you don't set the work folder (`DWARVES_MATERIALIZE_DIR`), a cache hit just carries the id tag forward like before — no files written. If there's no store to fetch from, there's nothing to restore, so it quietly does nothing. And if a fetch ever fails, the build doesn't break — that step just gets rebuilt.

## The three words to remember

- **skip** — a cache hit means the step didn't run this time.
- **restore** — bring the skipped step's file back from holger onto disk.
- **no-op** — with nothing configured, the whole thing quietly does nothing (safe default).