# Holger — design (the whole system)
> **Class:** `design` · **Function axis:** `f0` (the whole) + `registry` · `serve` ·
> `storage` · `auth` · `agent` · `proxy` · `embed`.
> Strict: this file states *only the system we ship*. Every *why*, every rejected
> alternative and every not-yet-built direction lives in the reasoning book
> ([`reasoning.md`](reasoning.md), [`registry-reasoning.md`](registry-reasoning.md),
> `promotion-design.md`, `static-dynamic-profile-design.md`, …).
## f0 — one engine, three build variants
Holger is a **secure, airgap-first artifact server**: it serves Maven, Python, Rust,
npm, NuGet, Go, Gem, Deb, RPM, Helm, Docker, Conda, Composer — and znippy's own native
package — from immutable, content-addressed [znippy](https://gunnar.rs/nordisk/znippy)
archives. It replaces Nexus/Artifactory where **auditability and airgap compliance** are
the point: move packages across a network boundary as one signed `.znippy` file, then
serve every ecosystem's *native* protocol out of it.
The system is **one serve engine wired seven ways**. Each capability is a `##` section
below; here is how they fit:
| function | one line | source |
|---|---|---|
| `registry` | one trait, many backends; a RON config graph → an O(1) routing table | `traits/src/lib.rs`, `server/lib/src/lib.rs` |
| `serve` | two front doors (native-HTTP + gRPC) bottoming out in one `backend.fetch` | `server/lib/src/exposed/http.rs`, `grpc.rs` |
| `storage` | immutable znippy archives + writable `file-*` backends | `server/lib/src/storage/`, `server/repository/*` |
| `auth` | mTLS/OIDC authN + RBAC authZ + append-only audit, gating writes | `server/lib/src/auth.rs` |
| `agent` | internet-side connectors pack artifacts into `.znippy` for airgap carry | `agent/lib`, `agent/connector/*` |
| `proxy` | pull-through remote cache over pluggable Nexus/Artifactory/Rust upstreams | `server/lib/src/proxy.rs`, `upstream/*` |
| `embed` | the parsed config *is* a functional API — link the lib, run in-process | `server/lib/src/lib.rs`, `object.rs` |
**The reuse graph.** `serve` uses `registry` (dispatch) and `storage` (extract). `auth`
gates `serve`'s write path. `proxy` is a `registry` backend that uses `storage`
(write-back) and an upstream. `embed` exposes `registry` + `storage` with no `serve`.
`agent` is a sibling process that produces what `storage` serves. `traits/` is the floor
all of them stand on.
## The three build variants (the split nornir's build-thing models)
The same engine ships three ways; the boundary is the **crate graph**, not a runtime flag.
| variant | artifact | form | status |
|---|---|---|---|
| **embedded** | none (host binary) | `holger-server-lib` linked in-process by a consumer (nornir's `embed-holger`) — no socket, no serialization | ✅ built |
| **mini** | `static-holger` *(planned)* | stripped, read-only, **zero egress crate linked** — provable from `cargo tree` | 🧭 design (crate-split refactor) |
| **full** | `holger-server` (pkg `holger`, `server/cli`) | writable backends + proxy + remote upstreams + full RON graph | ✅ built (un-split) |
The companion airgap-transfer binary **`holger-agent`** (`agent/cli`) is a *sibling* to
the server, not a server variant — it runs internet-side (see `agent`). What each variant
compiles in, and why the crate boundary (not `#[cfg]`) is the airgap guarantee, is the
reasoning book's territory.
**These three tiers are declared as data and consumed through nornir's build-thing.**
The `holger-build-space` crate (`build-space/src/lib.rs`) fills nornir-build-thing's
`BuildVariantSet`/`BuildSurface` with holger's real `(package × bin × features ×
profile × target)` points — grounded in the actual `server/lib` + `server/cli` Cargo
features (`sign`, `testmatrix`, `served`, `gui`), no invented UI. The bytes are the
on-disk `.nornir/build-variants.toml`, `include_str!`-embedded so descriptor and file
can never drift; `holger_build_space::build_variant_set()` parses them and
`build_surface()` projects the three rows (each with its materialized `cargo` command
and a `nornir build --variant <tier>` action) a Build tab/CLI renders.
holger is the **first real consumer** of build-thing (task #36), and it *presses the
button* end-to-end: `holger_build_space::press_tier(name, repo_root, outcome, artifacts)`
(`build-space/src/lib.rs:press_tier`) validates a tier against the set, then captures a
content-addressed build-provenance `Attestation` binding the verbatim tier descriptor,
the materialized argv, the VCS rev + working-tree freshness, the detected toolchain +
host (cores/memory), the outcome, and the SHA-256 of every produced artifact — folded to
a stable digest. The record is **unsealed** (`is_sealed()` is `false`) — holger never
fabricates a signature; the real seal is znippy's detached-CMS signer applied behind
nornir's `build-provenance` feature. This is the dep-light *describe/attest* seam (it
does not itself shell out to cargo); the heavy end-to-end press — spawn `cargo build
--message-format=json`, hash the executables, attest in one shot — is
`nornir_build_thing::build_variant`, driven by `nornir build --variant <tier>`. The
worked demo `build-space/examples/press_build_thing.rs` (`cargo run -p
holger-build-space --example press_build_thing`) runs the full *describe → press →
ledger round-trip*: reflect the surface, press `full`, persist to a dep-light
`JsonFileSink` ledger, reload, and prove the digest still verifies.
---
## registry — one trait, many backends
A *repository backend* is a plugin. Each maps an `ArtifactId` (namespace / name /
version) to a **path inside a znippy archive**, and speaks its ecosystem's **registry
protocol** over HTTP so a stock `cargo`/`pip`/`helm`/`docker` client talks to holger
unmodified. Every backend implements **one trait** — `RepositoryBackendTrait`
(`traits/src/lib.rs`):
| method | purpose |
|---|---|
| `name() -> &str` | the repo's configured name (the routing key) |
| `format() -> ArtifactFormat` | which ecosystem (`Rust`, `Maven3`, `Docker`, …) |
| `is_writable() -> bool` | znippy backends are read-only; `file-*` are writable |
| `fetch(id) -> Option<Vec<u8>>` | resolve an `ArtifactId` to bytes (direct/gRPC path) |
| `put(id, data)` | store bytes (writable backends only) |
| `handle_http2_request(suburl, body) -> (status, headers, bytes)` | serve the native HTTP registry protocol |
| `fetch_many_with_upstreams(upstreams, ids)` | batched fetch with upstream fall-through |
The engine holds backends only as `Arc<dyn RepositoryBackendTrait>` and dispatches
through the vtable — it never matches on the concrete type. `ArtifactFormat::znippy_type_id()`
pins each format to znippy's DenseUnion discriminant; a test in `traits/src/lib.rs`
asserts the numbers never drift (**znippy is the single source of truth**).
**Wiring.** Config is a **directed graph** in RON (`holger.ron`): `Repository` nodes
reference `StorageEndpoint` and `ExposedEndpoint` nodes by name. At startup
`wire_holger()` (`server/lib/src/lib.rs`) resolves it in passes — index nodes →
`instantiate_backends()` (`backend_from_config()` in `server/lib/src/repository.rs`
matches `ron_repo_type` to a concrete backend and boxes it; *this match is the registry*)
→ wire each repo's IO to storage + exposed endpoints → reverse-link → build a
`FastRoutes` table per exposed endpoint. `FastRoutes`
(`server/lib/src/exposed/fast_routes.rs`) buckets routes by the first byte of the repo
name into two `[usize; 256]` arrays, so `lookup(name)` is O(1) and lock-free on the hot
path. The same `FastRoutes` is shared by the gRPC service and `LocalHolger`.
### backend families
All backends share the trait and the storage floor; the only per-family code is the
`ArtifactId`→path mapping and the protocol framing.
- **A · real registry protocol** — `handle_http2_request` synthesizes the index on
demand: `znippy-{python,npm,nuget,go,docker,helm,composer,rust,gem}` (Gem = real
RubyGems compact-index; Conda = real generated `repodata.json`).
- **B · path passthrough** — the client knows the layout; holger serves bytes verbatim:
`znippy-{maven,deb,rpm,conda}`.
- **C · native** — `znippy-package` serves znippy's own `{name}-{ver}.znippy`.
`file-rust` and `file-oci` are the only **writable** backends (the "dynamic"/development
shape); every `znippy-*` backend is read-only by construction (the "static"/drift shape).
- **D · virtual aggregate** — `GroupBackend` (`server/lib/src/group.rs:GroupBackend`) is
a read-only "group repo" (the Nexus/Artifactory concept): it owns no storage and
resolves each request across an ordered list of MEMBER backends, first hit wins, so a
team exposes e.g. `maven-all = [maven-releases, maven-snapshots, central-proxy]` as one
URL. Config declares it with `ron_repo_type: "group"` + `ron_members` (member repo
names); `wire_holger` resolves the members in a second pass and
`backend_from_config()` (`server/lib/src/repository.rs`) boxes the `GroupBackend`. It is
distinct from a proxy — no writable primary, caches nothing, `put` is refused; `format`
is the members' shared format. Tests: `fetch_returns_first_member_hit_in_order`,
`group_is_read_only_and_takes_member_format` (`server/lib/src/group.rs`).
**Maven metadata + checksums are synthesized** (not just passed through). `MavenRepoZnippy`
(`server/repository/znippy-maven/src/lib.rs`) synthesizes the native
`maven-metadata.xml` on demand from the index columns the packer extracted — both the
GA-level version listing and, for a `-SNAPSHOT` GAV, version-level snapshot metadata
(`synthesize_snapshot_metadata` → `build_snapshot_metadata`). holger is an immutable
airgap mirror storing snapshots under their literal `-SNAPSHOT` name (no deploy
timestamp), so it advertises a **non-unique ("local") snapshot**
(`<snapshot><localCopy>true</localCopy>`). Every served path also answers `.sha1`/`.md5`
requests by hashing exactly the bytes it serves (`sha1_hex`/`md5_hex`), so stock Maven
clients verify downloads. Passthrough still wins when the archive already carries upstream
metadata. Tests: `snapshot_metadata_declares_local_copy_and_lists_classifiers`,
`maven_backend_synthesizes_checksums_and_snapshot_metadata`
(`server/repository/znippy-maven/`).
---
## serve — two doors, one backend
There are **two front doors** onto the same backend:
- **`handle_http2_request(suburl, body)`** (`server/lib/src/exposed/http.rs`) — the
native-protocol door. A stock `cargo`/`pip`/`npm`/`docker` client hits the HTTP
gateway; the backend parses the sub-URL into a protocol verb and returns
`(status, headers, bytes)`.
- **`RepositoryService`** (gRPC, `server/lib/src/grpc.rs`) — the holger-native door.
`FetchArtifact` returns the whole blob; `StreamArtifact` chunks it at 64 KB;
`ListArtifacts` / `PutArtifact` round out the surface. Two more services —
`ArchiveService` and `AdminService` — live in `proto/holger.proto`.
Both bottom out in `backend.fetch(id)`, which maps the `ArtifactId` to an archive path
and calls `ZnippyReader::extract_file(path)` — **random-access extraction**, no
full-archive decompress, blake3-verified. The proto Rust is generated **without
`protoc`** (pure-Rust `protox` via `cargo xtask gen-proto`, committed to
`server/lib/src/generated/holger.v1.rs`).
---
## storage — immutable znippy + writable file backends
- **znippy archives** — immutable, content-addressed (blake3 per chunk, verified on
read), one archive per repository version, built internet-side and carried across the
airgap, served read-only via `ZnippyReader::extract_file` (O(1) random access, no full
unpack).
- **filesystem** — a `{repo}/{path}` tree for `file-rust` (writable Cargo sparse
registry) and `file-oci` (writable OCI v2 CAS).
`StorageEndpoint` (`server/lib/src/storage/mod.rs`) names a location (`znippy` |
`rocksdb`) and a path; repos reference it in `ron_in` / `ron_out`.
**`file-rust`** (`server/repository/file-rust/src/lib.rs`, `RustRepoFile`) serves a real
Cargo sparse registry out of a `{ns}/{name}/{version}` disk tree via three routes:
`/{repo}/index/config.json` (JSON, advertising the `dl` URL), `/{repo}/index/{p1}/{p2}/{crate}`
(NDJSON, one line per version, **`text/plain`**), `/crates/{crate}/{version}/download`
(the exact stored bytes). The `dl` URL is the `ron_base_url` config knob — there is **no
`127.0.0.1` fallback**. Checksums are the live SHA-256, **streamed** in 64 KiB chunks
(`crate_sha256_hex_streaming`) and memoized by `(len, mtime)`; stored artifacts are
immutable (atomic write-then-rename), so a matching stamp reuses the cached digest.
---
## auth — authenticate, authorize, audit (write path)
A write goes: agent → **auth gate** → **route by repo name** → **backend.put()** →
**storage**. Reads are **not** auth-gated.
- **Authenticate** (`server/lib/src/auth.rs`, `validate_request()`): an mTLS CN (chain
verified once at the TLS handshake by rustls `WebPkiClientVerifier`, `exposed/tls.rs`;
per request only a leaf-CN parse) **or** an OIDC bearer token (validated against the
issuer's `/userinfo` through one pooled `reqwest::Client` + a TTL token cache keyed by
`(issuer, blake3(token))`, default 120 s). Empty method list = open; configured-but-invalid = 401.
- **Authorize** (RBAC): both write paths run a role check — gRPC `put_artifact` →
`authorize_write` (`grpc.rs`) and the HTTP/OCI gateway → `authorize_http_write`
(`exposed/http.rs`). `Role = Reader | Writer | Admin`; DELETE needs `admin`, other
writes need `writer`-or-`admin`; **fail-closed**. RBAC is **opt-in** — off until a
`roles`/`default_role` policy is set (`rbac_enabled()`); policy lives in RON, not redb.
- **Ingest cap**: `put` payloads over `ron_max_body_bytes` (default **1 GiB**, one knob
for gRPC `put` and the HTTP gateway) are rejected (413 / `ResourceExhausted`) before
the backend sees them.
- **Audit** (on by default): an append-only Arrow-IPC log records identity (`sub`/CN) +
repo + artifact + action + source IP + timestamp; replay skips a corrupt segment.
**How much** it records is `audit.mode` in RON, and each setting gives up
something named:
- `Everything` (default, and what holger always did) — one row per call, blake3-chained
to its predecessor and flushed to the OS before the response returns. The chain
covers every call.
- `Aggregated` — identical calls fold into `(ident, action, repo, artifact, ip, status)
-> count, bytes, first_seen, last_seen`, written every `audit.flush_secs` (default 30).
MEASURED on a 64-address / 16-artifact mirror workload: **19.5× fewer rows and 19.0×
fewer segment bytes** for 20 000 fetches. The chain then covers the FOLDED rows, and
whatever has not been flushed is unwitnessed process memory, lost on a crash. The
per-principal and per-IP totals (`audit sent-to`) stay **exact** — the fold key keeps
every column that tally groups on — but the per-call timeline does not survive, and a
`--since/--until` window cuts at fold boundaries rather than at calls.
- `Off` — nothing is recorded and **there is no tamper-evidence at all**; the chain
claims above do not apply. `audit query` and `audit sent-to` then REFUSE
(`holger-audit-8`) instead of printing an empty result, on both the gRPC and the CLI
door, because an empty answer that means "not recorded" reads as "nothing happened".
`SecurityProfile::Critical` refuses to boot on it.
The UI is not audited separately: the SERVER audits the call, whoever made it — the
console, or a plain `cargo` / `pip` / `docker`.
- **Path-traversal guards**: `check_component` on the `file-*` paths rejects empty / `.`
/ `..` / `/` / `\` / NUL; the agent download path uses `safe_join`.
---
## agent — the airgap producer
The **agent** runs internet-side and never needs the server to be reachable. It moves
artifacts between systems through the `ConnectorTrait` seam (`traits/src/lib.rs`):
`list_repositories`, `list_assets`, `download_asset`, `upload_asset`. Sources/targets:
crates.io, Nexus, Holger, a `.znippy` archive, a directory. Each connector carries a
`ConnectorAuth` (`None | Basic | OidcToken | Mtls`). Concrete connectors:
`agent/connector/nexus` (Basic auth) and `agent/connector/world` (crates.io / PyPI, https
host allow-list). The agent packs the downloaded set into one signed `.znippy` and the
operator carries it across the boundary on USB/optical; the server serves it read-only.
Binary **`holger-agent`** (`agent/cli`).
---
## proxy — pull-through remote cache
`ProxyBackend` (`server/lib/src/proxy.rs`) is a registry backend that tries `primary`
then each upstream and **write-through-caches** a hit. The upstream is pluggable via the
`RemoteUpstream` trait (`traits/src/lib.rs`) — a read-oriented backend carrying an
upstream base URL + the shared `UpstreamAuth` enum. Concrete impls: `NexusUpstream` and
`ArtifactoryUpstream` (`server/lib/src/upstream/{nexus,artifactory}.rs`) alongside the
crates.io-shaped `RustRegistryUpstream`, wired through the `nexus-remote` /
`artifactory-remote` repository types (`ron_base_url` + `ron_upstream_repo`,
`server/lib/src/repository.rs`). So holger targets holger / Nexus / Artifactory
interchangeably as a cache backend. Proxy is a **full-variant** capability (it links
egress crates).
---
## embed — the in-process functional API
The parsed RON config *is* the `Holger` value (`server/lib/src/lib.rs`); it carries direct
method bindings so a **linking consumer** drives a live registry with no gRPC socket and
no serialization:
| method | does |
|---|---|
| `Holger::fetch(repo, &ArtifactId) -> Option<Vec<u8>>` | resolve bytes from a named repo's backend |
| `Holger::put(repo, &ArtifactId, &[u8])` | store bytes (rejects read-only repos) |
| `Holger::list_repositories() -> Vec<&str>` | the configured repo names |
These call straight through `Arc<dyn RepositoryBackendTrait>`. A consumer wanting a repo's
*contents* reaches `holger.repositories[i].backend_repository` (the same trait object) and
calls `list` / `archive_files` / `fetch`; the trait + `ArtifactEntry` / `ArtifactId` types
are re-exported from `holger-server-lib` so no separate `holger-traits` dep is needed.
`Holger::dev_pair_ron` emits the RON for the **dev release pair** — `/cache` (a
writable-znippy crates.io mirror that fills write-through on miss) and `/sparring` (a
writable release-rehearsal registry). The reference consumer is nornir's `embed-holger`
feature (which lives on the nornir side, not here).
## Which recipe reader this repo uses, and why (lane AJ, 2026-08-30)
**Reader:** `nornir-build-thing` **0.1.12**, PATH (three entries: `build-space`, `xtask`,
`holger-build`).
**Why:** in-constellation. The pin said `0.1.0` while the lockfile compiled 0.1.8 — a
version claim no build ever tested. holger's recipe is schema 1. NOTE the open
gap: holger's graph still holds a SECOND, registry-sourced copy pulled by
published transitives (`modgunn`, `nornir`, `nornir-config`); that closes on the
release cascade, not in this manifest.
The rule itself — `[nornir-build-thing] schema = N`, a refusal that names the
version to upgrade to, and a loader that refuses to DROP a key it cannot read —
lives once, in **edda `.nornir/build-design.md`, "THE COMPATIBILITY RULE"**. The
cross-version guard that would have caught the 2026-08-30 skew is
`edda/nornir-build-thing-skew`.