<!-- nornir:class: history -->
# airgap-install — design + guide (crate `nornir-airgap`, codename *Skíðblaðnir*)
> **What it is (one line):** a **100% Rust** framework to **pack** a product (or a whole product
> lake) into an air-gapped bundle, **move** it across the gap, **unfold** the target (existing
> server via Redfish, cloud/IaC via Terraform, or a KVM VM for test), and **install + configure +
> start** every product in dependency order — no shell glue.
>
> **Motif — Skíðblaðnir:** the Norse ship big enough to carry *all* the gods and their war-gear,
> yet folds up like cloth into a pouch. That is exactly the job: fold the entire stack (binaries,
> deps, even the "digger and cement-mixer" tooling) into one portable archive, carry it over the
> airgap, unfold it on the far side, and raise the house. Source analysis (Rickard's prompt) +
> prior-art repos are in the appendices — they are the **normative source** for this design.
Status: **EPIC AIRGAP** in [nordisk_plan.md](nordisk_plan.md). Source repos: `dist`,
`airgap-kubernetes-main`, `airgap_llm`, `tunnr`, `msi-agent`, `bifrost`.
---
## 1. The model — pack → move → unfold → install → start
```
[build host / internet] ║ AIRGAP ║ [target: bare metal / VM / cloud]
pack ──► archive (znippy, cache+delta) ─╫────────╫─► unfold ──► install ──► configure ──► start(ordered)
║ ║ │ │ │ │
║ ║ redfish/ per-product data path dep-ordered,
║ ║ terraform/ user+systemd + ports services may
║ ║ kvm +unpack point at each other
```
The "moving house" analogy (from `airgap-kubernetes-main`): load the truck with **belongings
*and* the tools to build the house** (digger, cement mixer — slimmable later behind a feature),
drive across the gap, dig the hole, pour the foundation, **then** unpack the belongings.
## 2. Phases (the `dist` contract, made pure-Rust)
**Phase 0 — pack (build side).** Collect each product's artifacts + *all* target-identical
dependencies into a versioned bundle. Sometimes a component must be built so its deps are 100%
target-alike → build in a per-target container (this is *why* `dist` builds in containers; the
silly-looking `wget`-in-container is just dep capture). The bundle is a **znippy archive** with
**content cache + delta** so the *next* version ships only what changed (the `airgap-kubernetes`
speed story). A `VERSION` file lives outside the bundle for easy bumping (from `dist`).
**Phase 1 — platform prep (target side).** Per-platform specialization (e.g. optionally disable
SELinux), fetch + verify the airgap tarball, **unpack into a software directory**, and:
- **PKI:** call an **existing Vault** if present, else **generate the PKI ourselves** (pure-Rust,
or a spun-up temporary Vault). Pluggable `PkiSource`.
- **Layout:** prepare a **data directory** (fast-moving data) and a **software directory**; software
is partitioned by **airgap version** (date-stamped, e.g. `software/20250505/`), and **each product
gets its own subdir** under the version. Future: znippy lets a product read straight from the
archive — no full decompression.
**Phase 2 — per-product install.** For each product: create the **service user**, write
**systemd/init** units, **unpack** its software from the decompressed software path (Phase 1),
**configure** it to use the data path (Phase 1), and **open its firewall ports**.
**Phase 3 — ordered start.** Start all services **in dependency order** — products may point at
each other, so this is a topo-sorted bring-up (nornir already owns dep-graph machinery — reuse it).
## 3. Unfold backends (the target abstraction)
One `Unfold` trait, three impls (from `airgap-kubernetes`):
- **`redfish`** — provision/boot an **existing physical server** via its BMC (Redfish REST).
- **`terraform`** — drive IaC for cloud/VM fleets (wrap/emit plans; keep CLI parity).
- **`kvm`** — spin a local **KVM VM** (mostly test convenience; `tunnr` already has a pure-Rust KVM
boot harness to reuse).
Targets also span **bare-metal notebooks/servers AND a container reachable over RDP/Windows**
(`tunnr` + `msi-agent`) — the `Target` enum carries that breadth.
## 4. Crate module layout (`nornir-airgap`)
```
pack — collect artifacts + target-alike deps → manifest (container build when needed)
archive — znippy bundle writer/reader, content cache + delta (no full decompress on read)
fetch — move + verify the bundle across the gap (checksums, signatures)
platform — per-OS specialization (selinux toggle, prereqs)
pki — PkiSource: ExistingVault | SelfGenerated | TempVault
layout — data dir + version-dated software dir + per-product subdirs
product — Product spec: user, units (systemd/init), unpack, configure(data path), ports
unfold/ — Unfold trait + redfish · terraform · kvm
orchestrate— dependency-ordered install + start (reuse nornir dep graph / topo sort)
config — declarative lake/product descriptor (replaces the ansible playbooks)
```
## 5. Cross-cutting laws (apply here too)
- **100% Rust, no shell** — replace `dist`'s ansible + `airgap_llm`'s `*.sh` with Rust (LAW: containers/installers controlled in Rust, no `*_up.sh`).
- **znippy** archive everywhere (pack + delta + read-without-full-decompress); **zero-copy**.
- **CLI + MCP parity** — every action (`nornir airgap pack|fetch|unfold|install|start`) mirrored as an MCP tool.
- **`.nornir` design + guide** (this file) + an **idiot guide** to follow.
- Persist install/airgap events to the **warehouse** (a `airgap_events` table) — same DAG pattern as `release_events`.
---
## 6. Implementation status (AIRGAP1–9, 2026-06-15)
The pure-Rust offline core is **implemented + inject-assert tested**; the heavy
external integrations are **honest trait seams** with explicit TODOs (a clean
seam beats a hollow fake). Per item:
| item | status | where | test |
|---|---|---|---|
| AIRGAP1 pack | **DONE** | `pack.rs` `Staging` + `pack` + `write_version_file` (VERSION beside the bundle); **real `PodmanBuild: ContainerBuild`** (podman build → create → cp; renders the Containerfile, honest error if podman absent) | `pack_collects_staging_tree_and_version_file_is_outside`, `podman_build_renders_containerfile_and_is_honest_when_absent` |
| AIRGAP2 archive | **DONE** (tar+zstd default; **real znippy engine** behind `znippy`) | `archive.rs` `Archive` trait + `TarZstd` + `BundleManifest`; **`ZnippyArchive`** = content-addressed cache (dedup by SHA-256) + `delta`/`apply` (ship only changed blobs) + O(1) `read_entry` (footer→index→seek one blob, no full decompress). `Patch` type. Wired via the nornir `airgap-znippy` feature (`nornir-airgap/znippy`). | `pack_then_extract_roundtrip_with_manifest_checksums`, `tampered_manifest_fails_root_verify`, `znippy_content_cache_dedups_identical_entries`, `znippy_selective_read_does_not_decompress_whole_archive`, `znippy_delta_round_trip_is_byte_identical_and_ships_only_changes`, `znippy_full_extract_round_trips` |
| AIRGAP3 platform | **DONE** | `platform.rs` `Platform::detect`/`specialize` (SELinux) + `unpack`; `fetch.rs` `verify_on_receipt` (zero network) | `os_release_detects_family_and_selinux`, `unpack_extracts_bundle_into_software_dir`, `faithful_bundle_verifies_on_receipt`, `tampered_manifest_is_rejected` |
| AIRGAP4 PKI | **DONE** (SelfGenerated real; Vault seam) | `pki.rs` `MachineCa` (ed25519) `issue_leaf`/`verify_leaf`; `VaultClient` seam | `self_generated_leaf_is_a_real_verifiable_cert`, `vault_sources_are_explicit_seams` |
| AIRGAP5 layout | **DONE** | `layout.rs` `Layout` version-dated dirs + atomic `activate()` symlink swap | `prepare_makes_version_dated_per_product_dirs`, `atomic_symlink_swap_on_upgrade` |
| AIRGAP6 install | **DONE** | `product.rs` `ComponentSpec` + `Installer`/`SystemdInstaller` (reuses `systemd::install_systemd`, opens firewall ports) | `component_spec_renders_layout_driven_systemd_spec`, `component_spec_to_product_is_consistent` |
| AIRGAP7 unfold | KVM real argv; Redfish/Terraform seams; MSI guid real | `unfold.rs` `KvmTarget::qemu_argv` (tunnr-identical), `redfish`/`terraform` seams; `msi.rs` `WixBuilder` seam + real `guid_v5` | `kvm_qemu_argv_is_built_like_tunnr`, `terraform_resource_renders_nornir_component`, `external_backends_are_explicit_seams`, `guid_v5_is_deterministic_and_well_formed`, `sha1_matches_known_vector` |
| AIRGAP8 ordered start | **DONE** | `orchestrate.rs` Kahn topo `start_order`/`start_waves`; `events.rs` `AirgapEvent`/`JsonlSink`/`EventEmitter`; warehouse `airgap_events` Iceberg table (`src/warehouse/airgap_events.rs` + `iceberg_schema::airgap_events`) | `topo_waves_from_fixture_dep_graph`, `cycle_is_rejected`, `emits_install_dag_to_warehouse_jsonl_and_reads_back`, `jsonl_decodes_airgap_events` |
| AIRGAP9 CLI+MCP | **DONE** | `nornir airgap pack\|fetch\|unfold\|install\|start\|events` (`bin/nornir.rs` `run_airgap`); MCP `airgap_pack`/`airgap_verify`/`airgap_unfold_plan`/`airgap_events` (`bin/nornir-mcp.rs`) | `mcp_every_tool_exercised_and_asserted` (4 airgap rows) |
| INSTALL-3/4/5 adoption | **DONE** (holger/tunnr specs + `nornir airgap component add\|upgrade\|remove\|status`); **delta wired** via the znippy engine (`upgrade --delta --from <old>` reports the diff savings; `delta_supported()` == the `airgap-znippy` feature) | `product.rs` `ComponentSpec::{holger,tunnr,nornir_server}` + `registry`/`lookup` + `delta_supported`; `bin/nornir.rs` `ComponentOp::{Add,Upgrade,Remove,Status}` + `run_install_*` + `report_delta_savings` | `registry_enumerates_and_looks_up_components`, `holger_add_renders_exact_unit_text`, `tunnr_spec_renders_appliance_unit_and_delta_tracks_feature`, `upgrade_is_idempotent_at_spec_level` |
**Real now (this branch `feat/airgap-archive-engine`):** the **znippy archive
engine** (`ZnippyArchive` — content cache + delta + O(1) selective read, behind
the `znippy` feature / nornir's `airgap-znippy`), the **per-target container
dep-capture** (`PodmanBuild: ContainerBuild` — real `podman` subprocess), and the
**tunnr delta upgrade** (`airgap component upgrade --delta`). The znippy *format* is
self-contained (blob region + seekable index + footer), faithful to ../znippy's
on-disk philosophy, with no rayon (the lean budget) — it does not link the
`znippy` crate (which pulls `znippy-cli`).
**Still seam + TODO (deliberately not faked):** the Vault PKI backends
(`VaultClient`), the Redfish + Terraform *provisioning* (their plans/bodies ARE
rendered), and the WiX-in-podman MSI *build* (the UUID-v5 GUID logic IS real).
KVM builds the real qemu argv but only *boots* when `qemu-system-x86_64` is
present (test/staging convenience).
## 7. Idiot guide (the five verbs)
```bash
# ── on the build host (internet OK) ────────────────────────────────
# 1. Lay out a staging tree exactly as it should appear in the bundle:
# stage/linux-x86_64/holger, stage/manifests/holger.json, …
# 2. Pack it into a signed bundle (+ a VERSION file beside it):
nornir airgap pack ./stage --out lake-20250505.tar.zst --version 20250505
# → writes lake-20250505.tar.zst + lake-20250505.tar.zst.manifest.json + VERSION
# ── carry lake-*.tar.zst + its .manifest.json over the gap (USB/relay/age) ──
# ── on the air-gapped target (ZERO network) ───────────────────────
# 3. Verify on receipt (recomputes manifest + every entry hash):
nornir airgap fetch lake-20250505.tar.zst # ✓ or refuses to install
# 4. (optional) inspect the unfold plan for a target:
nornir airgap unfold kvm oden-test --disk lake.img # prints the qemu argv
# 5. Install one component (root; reads its ComponentSpec JSON):
sudo nornir airgap install lake-20250505.tar.zst --spec holger.json \
--software-dir /opt/lake --data-dir /var/lib
# 6. Bring the lake up in dependency order + record the DAG to the warehouse:
nornir airgap start lake.json # lake.json = a LakeSpec
nornir airgap events # read the bring-up DAG back
```
Every verb has an MCP twin (`airgap_pack`, `airgap_verify`, `airgap_unfold_plan`,
`airgap_events`) — CLI/agent parity (the LAW). PKI: pass `"pki":"SelfGenerated"`
in the LakeSpec for pure-Rust ed25519 per-machine certs (no Vault needed).
### 7b. `nornir airgap component` — the registry-driven generic component path (INSTALL-3..6)
`nornir airgap install` (above) installs a component from a **spec-JSON inside a
bundle**. `nornir airgap component add/upgrade/remove/status` is the friendlier
twin: it resolves a **canonical `ComponentSpec` by name** from the built-in
registry (`nornir_airgap::product::registry()` → `holger` · `tunnr` ·
`nornir-server`), so no hand-written JSON is needed. **This is the adoption of
`nornir-airgap` — ONE install system, not a second `nornir-install` crate.**
```bash
nornir airgap component status # list the registry + active state
nornir airgap component status holger # one component
nornir airgap component add holger --bundle lake.tar.zst --dry-run # render the unit, no root
sudo nornir airgap component add holger --bundle lake.tar.zst # unpack + unit + ports
sudo nornir airgap component upgrade holger --bundle lake-new.tar.zst # atomic version swap
sudo nornir airgap component upgrade tunnr --bundle lake-new.tar.zst --delta --from lake.tar.zst # znippy delta
sudo nornir airgap component remove holger # stop + remove the unit (idempotent)
```
- `add` and `upgrade` share the **same idempotent atomic path** (`SystemdInstaller::install` → layout version dir + `current` symlink swap); the rendered unit is version-independent (ExecStart goes through `current`), so an upgrade is *just the swap*.
- `--delta` requests the binary-delta upgrade via the znippy zero-copy engine (`archive::ZnippyArchive::{delta,apply}`). Build nornir with `--features airgap-znippy` (→ `nornir-airgap/znippy`) and `product::delta_supported()` is `true`: `upgrade --delta --from <old-bundle>` reports how few bytes the diff ships (reuse N, ship M) before the atomic swap. Without the feature it prints a clear note and falls back to a full re-install.
- **holger/tunnr are NOT nornir code dependencies** — they are *described* by registry specs, never linked. This keeps nornir lean while `nornir airgap component add holger` works end-to-end.
- nornir's own server has its bespoke `nornir install add` / `update` / `remove` (the canonical path for the monitored server with its env file + SSH key + monitored-workspace flags). It renders the **same `nornir-server.service` unit** as `airgap component … nornir-server` — ONE unit name — but `install` is the everyday fast path; the generic component verbs are for the wider lake (holger/tunnr/bundles).
- **holger self-installs too (2026-06-21).** Two ways to put holger on a box:
- *From nornir (top-down):* `nornir airgap component add holger --bundle …` — registry/bundle-driven, nornir orchestrates the whole lake (above).
- *From holger (self):* `holger-server install {add | update | remove | status | render}` — holger drives the **same `nornir-airgap::systemd` engine** itself, no bundle/registry needed (you just have the holger binary). `render` is the no-root dry-run. holger owns its own `SystemdServiceSpec` (verb `start --config`, RON config) — the engine only applies it. This needed the new **`remove_systemd` / `status_systemd`** verbs so the spec layer has the full add·update·delete·status lifecycle (see [`airgap-installer.md`](../crates/nornir-airgap/.nornir/airgap-installer.md)).
---
## Appendix A — SOURCE PROMPT (Rickard, normative; the earlier agent failed to filter this in)
> nornir will have rust 100% framework/working crate for airgap and installation: redfish,
> terraform, kvm, existing servers. Here are fragments and repos touching this — perhaps different
> solutions but all similar: **pack and move and install.**
>
> **repo `dist`** — has airgap in containers. Why? Because sometimes it needs to build components
> that must be 100% target-dependencies-alike. Sometimes it's just `wget` in the container — looks
> silly, yes, but container all over the board. A version file is also used outside so it's easy to
> bump version. `dist` also has installation ansible scripts for a lake which consists of many
> products; the story is the same for each product. **1)** make some per-platform specialization,
> perhaps disable SELinux. **2)** fetch airgap tarball, unpack it in a directory; generate a PKI —
> we must support an existing Vault to call, or generate the PKI ourselves (using Rust or calls to a
> temporary Vault); also prepare a **data** directory (fast-moving data) and a **software**
> directory (software dir has a subdir per software-generation/airgap version, e.g. `20250505` like
> a date), where each product creates a subdir for itself. **3)** install each product: create the
> service user, create systemd/init, unpack the software from the airgap directory decompressed into
> the software path (step 2), and configure the product to use the data path (step 2); open firewall
> ports per product. In the end **start all services in correct order** — note they might point at
> each other. (Using znippy here in future removes the need for total decompression.)
>
> **repo `airgap-kubernetes-main`** — the essence by analogy: imagine moving in the future — get a
> big-ass truck, pack it with all your belongings and furniture, also pack sand, cement, a digger, a
> cement-mixer, all kinds of tools (might be slimmed as a feature later, but now all machines
> needed). Drive away, roll out the digger, dig a hole, fill it with cement, build the house, then
> lastly unpack your belongings. So airgap-kubernetes does airgap stuff (VERY QUICKLY using cache +
> delta for next versions — this will be a **znippy archive**), then it unfolds by selecting
> **redfish**, **terraform**, or **kvm** (mostly for test convenience).
>
> **repo `tunnr`** — a startup trying to distribute a Rust application **without an OS**: works for
> bare-metal notebooks/servers AND a container solution reachable from **RDP/Windows**.
## Appendix B — prior-art map
| repo | contributes | note |
|---|---|---|
| `dist` | 3-phase install, ansible lake, container dep-capture, VERSION file | replace ansible/shell with Rust |
| `airgap-kubernetes-main` | pack-everything, cache+delta speed, redfish/terraform/kvm unfold | → znippy archive |
| `airgap_llm` | `prepare_airgap.sh`/`verify_airgap.sh` workflow | port to Rust `pack`/`fetch` |
| `tunnr` | Rust-as-PID1, distroless/bare-metal, KVM boot harness, RDP/container | reuse KVM + Target breadth |
| `msi-agent` | Windows MSI (wix) packaging | the Windows/RDP target |
| `bifrost` | pure-Rust deploy/rollback companion (nornir-server on oden) | the Rust-deploy precedent |