Gunnar

nordisk/dwarves

.nornir/dwarves-lifecycle-and-ui.md

<!-- nornir:class: reference -->
# dwarves — fat node, thin UI, lifecycle, uninstall & deploy (operational reference)

> ## ⚠ AMENDED 2026-08-03 — the FIVE-ATOM LADDER, and MCP moved into the server
>
> This document describes **two** binaries. The declaration now has **five atoms**, and
> `.nornir/build-variants.toml` is the authority, not this file:
>
> | atom | role | linkage | bin |
> |---|---|---|---|
> | `dwarves-server` ⭐ | server | fat | `dwarves-server` |
> | `dwarves` | client | fat | `dwarves` |
> | `dwarves-cli` | client | thin | `dwarves-cli` |
> | `dwarves-ui` ⭐ | client | thin | `dwarves-ui` |
> | `dwarves-solo` ⭐ | solo | fat | `dwarves-solo` |
>
> Two changes an operator will actually notice:
>
> * **`dwarves-mcp` is gone as a binary.** The MCP tools are `dwarves::mcp` in the
>   library and are served by **`dwarves-server mcp`** on stdio. An MCP client config
>   that said `"command": "dwarves-mcp"` becomes
>   `"command": "dwarves-server", "args": ["mcp"]`. Nothing about the tools changed.
> * **`dwarves-solo` is new** — the UI with the forge compiled in. One binary, nothing to
>   install, no `DWARVES_SERVER` to set. `dwarves-ui` stays thin: *"dwarves UI skall
>   ALDRIG default byggas med servern compiled in"* is untouched, because solo is a
>   different file, not a different mood.
>
> `dwarves-ui-fat` and `dwarves-default` were also removed. The fat-read UI IS the solo
> shape; the default-features compile guard was never a shippable artifact and that
> surface is compiled by every plain `cargo build -p dwarves` / `cargo test --workspace`.

> ## ⚠ AMENDED 2026-08-02 — `dwarves-server` FOLDED INTO `dwarves-server`
>
> Everything this document calls **`dwarves-full`** is now the **`dwarves-server`** bin.
> There is ONE server binary, ONE `[[bin]]`, ONE variant and ONE systemd unit; the
> `dwarves-full` bin, its `[[bin]]` entry and its build variant are deleted. The
> occurrences below were renamed in place, because it is the same product under one name.
>
> What the fold fixed: the repo held three disagreeing declarations of the same thing —
> `.nornir/build-variants.toml` called `dwarves-server` a thin `role = "tool"` that is
> "NOT the server", `crates/dwarves/src/service.rs` (the ONE `ServiceSpec` both bins
> installed) called it a `Fat` state-holding forge, and the live unit was named
> `dwarves-server.service` with `ExecStart=/usr/local/bin/dwarves-full`. Rickard settled
> it: **dwarves-server IS the dwarves server** — role=server, linkage=fat, state-holding,
> Skidbladnir compiled in so `dwarves-server --install` creates its own unit.
>
> The fat/thin axis is now a FEATURE choice on one source, not a second file:
> `--features embed-all,read-fat` is THE server (`:7878` + `:7900`, the declared atom);
> `--features server` is the lean forge daemon (`:7900` only, reaches a remote nornir).
>
> **The `file:line` citations below are as-of 2026-07-30 and were already drifting; the
> dated SIZE/mtime measurements are kept as the history they are.**

State of the forge's runtime shapes and service lifecycle as of **2026-07-30**. Every
claim below is cited to `file:line` + commit hash and was verified against the tree.
Related design doc: [service-lifecycle-L1](service-lifecycle-L1.md) (the skidbladnir
service seam); [ui-embedding-and-standalone](ui-embedding-and-standalone.md) (the
thin/fat gRPC split rationale).

## 1. Two binaries: `dwarves-server` (THE SERVER, fat) vs `dwarves-ui` (THIN)

### `dwarves-server` — THE FAT NODE
ONE headless process serving nornir's full gRPC stack on **`:7878`** (via
`nornir::server_grpc::run`, the same entrypoint the standalone `nornir-server` uses)
side-by-side with the forge's own `dwarf.v1` on **`:7900`**, on one tokio runtime
(`crates/dwarves/Cargo.toml:35-54`, `crates/dwarves/src/bin/dwarves-server.rs`).

**As of commit `4b2e19a`** ("feat(dwarves): default features embed nornir + skade…",
2026-07-30 00:02) the crate `default` now includes `embed-nornir` + `embed-skade`, so a
plain `cargo build` yields the full binary — **no more `--features embed-nornir` trap**:
- `default = ["cli", "vault", "embed-nornir", "embed-skade"]` (`Cargo.toml:81`;
  was `["cli", "vault"]` before `4b2e19a`).
- `embed-nornir = ["dep:nornir", "dep:mimalloc", "server", "grpc-web", "warehouse"]`
  (`Cargo.toml:299`).
- `embed-skade = ["dep:skade"]` (`Cargo.toml:304`).
- The `dwarves-server` `[[bin]]` declares `required-features = ["server"]` (amended
  2026-08-02 by the fold — it was `["embed-nornir"]` on the retired `dwarves-full` bin).
  The crate default satisfies it, so `cargo build -p dwarves` builds it, and the default
  also carries `embed-nornir`, so the plain build is the FAT shape.
- **Lean embedders** opt out with `--no-default-features` / `default-features = false`
  (e.g. `dwarves-gui` pulls `dwarves` with `default-features = false`, features
  `["facett-adapter","dupes","coverage","mimir"]` — `dwarves-gui/Cargo.toml:345`).
- Caveat still in force: `required-features` targets are **silently skipped** by
  `cargo build --all-targets`; the default now covers the plain `-p dwarves` build.

### `dwarves-ui` — THE THIN CLIENT (HOST-ONLY)
The `[[bin]] dwarves-ui` of the **`dwarves-gui`** crate (`dwarves-gui/Cargo.toml:18-19`;
renamed from `dwarves-gui` for a uniform `<repo>-ui` launch name matching
`nornir-ui`/`korp-ui`). egui/eframe front-end that knows **exactly ONE server address —
the dwarves forge (`:7900`)** — and dials nothing else. It embeds **NO nornir-server and
NO skade**, and depends on `dwarves` lean (`default-features = false`, `Cargo.toml:345`).

**The suite-host-embedded-auth LAW.** nornir is EMBEDDED in the dwarves server (the host);
the UI defers to that host and reaches nornir/Mímir data THROUGH the host, never by
dialing nornir itself. Concretely:
- The UI reads **only `DWARVES_SERVER`** (or `--server <url>`). There is **NO `NORNIR_SERVER`**
  knob in the client — it does not know and never opens a socket to nornir `:7878`.
  **The `:7878` port is for EXTERNAL nornir consumers only.** (`dwarves-gui/src/main.rs`
  resolves the single forge endpoint; `build_native_app` takes NO nornir argument —
  `dwarves-gui/src/lib.rs`.)
- The **Funnel is always EMBEDDED / in-process** (renders with nornir-server DOWN, no
  network dial) — the old `$NORNIR_SERVER` thin-funnel path is gone.
- **SERVED-VIA-HOST (RESOLVED — the warehouse proxy landed).** The warehouse/Mímir-backed
  panes (Warehouse navigator · Data view · Dupes · Coverage · Lenses/Mímir · Binary ·
  Coderoom · the warehouse selector's registry list + Add/Populate) are served THROUGH the
  host. The dwarves forge now re-serves the **`nornir.v1` READ surface**
  (`Warehouse.{Tables,Scan}` · `Workspaces.{List,Get,Register,Remove,Fetch,PopulateStatus}`
  · the eleven `Mimir.*` lens folds) on its OWN `:7900` socket, forwarding each call to its
  embedded nornir and stamping dwarves' own host bearer on the hop (`crate::warehouse_proxy`).
  The single wiring seam `warehouse_endpoint` in `build_native_app` is now the **forge
  endpoint**: the panes' `nornir-warehouse-embed` thin client speaks the same
  `/nornir.v1.Warehouse/Scan …` paths, but to the dwarves host, which proxies. The `:7878`
  socket never enters the UI. A dead / embedded-nornir-less host makes a proxied read return
  an honest `Unavailable`, so the panes still degrade HONESTLY (never a fabricated empty) —
  they just never dial `:7878`. **The hop is a localhost gRPC forward, not a literal Rust
  call** (a direct call would need `dwarves` to link `nornir` — a dependency cycle); auth is
  still host-governed. A genuine in-process reader trait injected through `dwarves-server` is
  the logged follow-up.
- **THIN BY DEFAULT is a rule, not a tuning.** Startup banner: `dwarves-gui: HOST → dwarves {forge}`.
  Default forge grpc-web endpoint = `https://cicd.g11.se:7900` (wasm web build).
- Size: ~76 MB thin vs ~234 MB fat (the on-disk `dwarves-server` is ~234 MiB — see §5).
- Build: `cargo build -p dwarves-gui --bin dwarves-ui`.
- **Run (the ONE launch command): `DWARVES_SERVER=http://oden:7900 dwarves-ui`**
  (add `WAYLAND_DISPLAY=…`/`XDG_RUNTIME_DIR=…` as your session needs). Do NOT set
  `NORNIR_SERVER` — the client ignores it.

## 2. Lifecycle is Skidbladnir's, not dwarves'
dwarves declares a `ServiceSpec` and drives a `skidbladnir::service::Lifecycle`; it does
NOT hand-roll unit files or shell `systemctl` (zero-shell law).
- `dwarves::service::lifecycle(exec, port, scope) -> Lifecycle`
  = `Lifecycle::new(service_spec(...))` (`crates/dwarves/src/service.rs:202-203`).
- Unit name is **`dwarves-server.service`** — asserted at `service.rs:740`
  (`unit_name_is_dwarves_server_service`).
- systemd control is **native D-Bus via `zbus`** over `org.freedesktop.systemd1`, the
  replacement for shelling `systemctl` (`Skidbladnir/src/systemd.rs:1426`, struct
  `Systemd { conn: zbus::blocking::Connection }`, manager calls at `systemd.rs:1462-1464`).
- The `["systemctl", "--user"?, <verb>, <unit>]` argv in `Skidbladnir/src/install.rs:699`
  is a **pure offline classifier** (`classify` a verb argv — `install.rs:709`), NOT a
  live shell-out.
- Bus scope: `Systemd::connect_scoped` uses `zbus…Connection::session()` for user scope,
  `…::system()` for system scope (`systemd.rs:1445-1450`).

## 3. The uninstall fix — Skidbladnir `9456079` (NOT pushed)
Branch **`fix/uninstall-data-nuke-bus-optional`** in `/home/rickard/git/Skidbladnir`,
head `9456079` ("idempotent / self-healing over HALF-REMOVED state…", 2026-07-30 00:23).
The branch has **no upstream — it is local-only, not pushed** (predecessors `998f3ac`
scope auto-detect, `ca131b9` refuse-on-mismatch). All in
`systemd::remove_systemd_scoped_with` (`Skidbladnir/src/systemd.rs:1767-1920`):
- **Auto-detects scope from the installed unit**: system = `/etc/systemd/system/<unit>`,
  user = `~/.config/systemd/user/<unit>` (`systemd.rs:1774-1795`). `--scope` is needed
  ONLY when a unit exists at **both** scopes (the tiebreaker); "no system install +
  running as user" resolves to user scope on its own.
- **System scope needs root** — an informative refusal that **erases nothing**
  (`systemd.rs:1801-1810`), never an opaque mid-erase EPERM.
- **Bus required only when a live unit exists to stop**: connects the bus only if a unit
  is present at the resolved scope (`unit_present`, `systemd.rs:1818-1845`). A
  **half-removed** state (unit gone, data residue) needs **no bus** and **finishes the
  erase idempotently** — repeated uninstalls converge to clean. A present unit whose bus
  is **unreachable REFUSES and erases nothing** (never wrong-scope-delete).
- **Always keeps KEYS** (`.ssh`, token) per the uninstall LAW — erase the declared data
  root, keep the identity; account removal (opt-in) comes last and is skipped if the
  identity wipe removed nothing (`systemd.rs:1868-1918`).
- **Earlier root cause**: a `--user` scope hits `Connection::session()`
  (`systemd.rs:1447`), which fails deterministically when there is **no login-session
  bus** (service account / `sudo`) — not random.
- dwarves' declared data root = `<base>/.dwarves` (`DWARVES_DATA` override), refusing a
  root that IS an account home (`service.rs:19-22`, `declared_data_root`).

## 4. Deploy / bootstrap on oden
The running server is a **SYSTEM unit** `User=dwarves` (+ `Group=dwarves`; asserted for
system scope at `service.rs:749`/`751`, and `User=root` is negated at `service.rs:754`),
warehouse at
**`/home/dwarves/.dwarves/warehouse`** (resolver ladder `$DWARVES_WAREHOUSE` →
`$HOME/.dwarves/warehouse` → system `/home/dwarves/.dwarves/warehouse` —
`crates/dwarves/src/store.rs:40-41`), binary **`/usr/local/bin/dwarves-server`**.
- **Swapping the binary by hand**: stop the service AND kill any stray `dwarves-server`
  process first (otherwise `cp` fails with **`Text file busy`**), then `cp` + start.
- **Going forward**: `dwarves-server --refresh [bundle]` drives the whole
  build→install→migrate→restart dance as ONE idempotent, rollback-safe call via
  `skidbladnir::install::refresh_product` (`service::run_refresh`, invoked at
  `crates/dwarves/src/bin/dwarves-server.rs`) — atomic generation swap, no manual cp.
- **One-time bootstrap done today**: the `4b2e19a` fat binary is live at
  `/usr/local/bin/dwarves-server`, **mtime 2026-07-30 00:32**, size 245,232,848 B
  (~234 MiB). Backups from prior deploys sit beside it (`*.bak-*`, `*.prev`).

## 5. (RESOLVED by the host-only change) add-workspace no longer dials nornir from the UI
The old bug: in `dwarves-ui`, **Add workspace → `Workspaces.Register`** failed with
**`Unauthenticated: missing or invalid bearer token`** because the THIN client dialed
nornir `:7878` **directly** (token-gated) while carrying no bearer — a direct-nornir
client, the very violation the host-only law forbids.

**Now (LIVE via the proxy):** the UI holds no nornir address and never dials `:7878`. The
warehouse-write affordance (Add/Populate) — and the whole warehouse/Mímir read surface — is
served THROUGH the host: the dwarves forge re-serves `nornir.v1.Workspaces` /
`nornir.v1.Warehouse` / `nornir.v1.Mimir` on `:7900` and forwards each call to its embedded
nornir, stamping dwarves' OWN bearer on the hop (`crate::warehouse_proxy`, the `SuiteHost`
authority — one token, no second `~/.nornir/token`). So `Add → Workspaces.Register` now
travels UI → dwarves host → embedded nornir, carrying the dwarves bearer the whole way; the
`Unauthenticated` failure class is gone because there is no thin UI→nornir socket left, and
the register is authenticated by the host, not an absent nornir token. **Honesty flag:** the
host→embedded-nornir hop is a localhost gRPC forward (dial URL `$NORNIR_SERVER_ADDR`, default
`http://127.0.0.1:7878`), not a literal in-process Rust call — a direct call would require
`dwarves` to link `nornir` (a dependency cycle). A genuine `SuiteHost` reader trait injected
through `dwarves-server` (where nornir IS linked) is the logged follow-up.

## 6. Robot-test coverage GAP
The robot-UI hard gate is real — the facett `Severity` a11y contract
(`facett/facett-core/src/a11y.rs:49-112`; `Severity::{Error,Warning,Info}`). A low-cost
continuous robot-UI test runner for oden was built today, but it lives in the **korp**
repo, NOT dwarves: branch `fix/w102-oden-robotui-runner`, commit **`c4601bf`**
("tools(w102): low-cost continuous robot-UI test runner for oden", 2026-07-29 21:56, in
`/home/rickard/git/korp`). W097/W102 are worklist ids tracked in
`workspace_nordisk/.nornir/` (not code constants).

**The host-only law IS robot-gated** (`dwarves-gui/tests/robot_host_only.rs`): a headless
robot builds the SHIPPED shell given ONLY the dwarves host, with `NORNIR_SERVER` set to a
poison `:7878` value, and asserts over `state_json` that the full roster mounts, the Forge
dials the dwarves host, and **no `:7878`/nornir address appears anywhere in the shell** —
the client can never silently dial nornir. The Funnel is asserted embedded (up with nornir
DOWN). The former gap — a live end-to-end "served-via-host warehouse read" — is now
CLOSED: `dwarves-gui/tests/robot_warehouse_proxy_live.rs` stands up the REAL dwarves
warehouse proxy in front of a mock nornir and drives the SHIPPED shell given ONLY that
dwarves host, asserting the Warehouse navigator's world tier renders LIVE rows
(`synthetic_functions=false`, real node/edge counts, `real_source_desc` names the dwarves
host) with no `:7878` anywhere. The proxy's own forwarding is proven at the gRPC level by
`crates/dwarves/tests/warehouse_proxy_live.rs` (Warehouse.Scan + Workspaces.Register/List
through the host return the upstream's live data; a dead upstream is honestly `Unavailable`).

### 6. The ⚒ Build-Thing + Mímir live-demo surface is robot-gated end to end

The panes that fail live (the ⚒ BUILD-THING roster, the 🔍 Lenses, the 🌐 Warehouse
navigator) now have honest-state robot coverage on BOTH the display and the EFFECT axes —
each with a RED-when-broken peer, so none can go green through an outage:

* **Roster, both fixtures.** `robot_build_thing_roster_host.rs` drives the SHIPPED
  host-only shell over a *hermetic staged* constellation (a temp TOML) — non-empty +
  honest, plus an EMPTY root that reads as the named "NO CHECKOUTS" state, never blank.
  `robot_build_thing_roster_live.rs` drives the SAME shell over the REAL
  `ConstellationBuilds::discover` path on the paved checkout dir (`$DWARVES_CONSTELLATION`,
  else the parent of the dwarves repo): on oden that is genuinely non-empty (17 declared
  repos, ~89 build cells, a real headline); on a bare checkout it is the honest NO CHECKOUTS
  empty state. Adaptive but TYPED either way — never a false-green blank pane (the t14s bug).
* **Launch + build EFFECTS, headless.** `robot_build_thing_launch_effects.rs` selects the
  newer `rdp-korp-thin-container` cell (the korp-thin-rdp-viewer target) and clicks ▶ Run
  with an INJECTED `ContainerLauncher`: the run FIRES (boot.cell/kind), the readiness line
  reaches the console, and it ends `phase=passed` — with a RED peer where the launcher
  returns `Err` and the pane ends `failed`, never a fabricated green. It also drives the
  `bootable_only` ("endast bootbara") filter and asserts the container cell's visibility
  genuinely FLIPS (hidden ON, shown OFF) while the bootable vm cell stays — a filter that
  filters — and clicks ▶ Build on a cargo variant to run a REAL `cargo build` that resolves
  to an HONEST `green=false`/"FAILED" report (never `report=null`), so the build effect
  surfaces the true outcome. No feature flag, no socket, no QEMU — runs in stock `cargo test`.
* **Stable error CODES on the failure paths.** Every non-UI dwarves error now carries a
  frozen `dwarves-<subsystem>-<n>` code beside its human message
  (`dwarves::errcode`, registry at `.nornir/dwarves-error-codes.md`), so the failure tests
  assert the CODE and stop being blind to rewording. The recurring three are keyed:
  an empty roster is `dwarves-buildthing-1`, a container run that cannot start is
  `dwarves-launch-2`, a dead warehouse upstream is `dwarves-proxy-2` (the proxy's own
  diagnosis, which travels up into the Lenses pane). Each has a peer asserting the code is
  ABSENT on the healthy path, so a code can never be a constant that always passes.
  `robot_error_codes.rs` proves the code reaches `state_json`, a gRPC `Status` message and
  the `x-dwarves-code` metadata, and re-asserts the registry's wired/reserved split.
* **Mímir, live + dead.** `robot_lenses_live.rs` fronts a LIVE `MockMimir` upstream with the
  REAL dwarves proxy and drives the shipped host-only Lenses scan: `scanned=true`, status
  NOT UNAVAILABLE, `thin=true`, and the REAL bloat finding (a named monomorphization)
  survives the proxy round-trip into the pane state. Its twin `robot_warehouse_dead_upstream.rs`
  proves the same pane reports `UNAVAILABLE`/`scanned=false` with no fabricated findings on a
  dead upstream — an outage reads as an outage. The Warehouse navigator has the matching
  live (`robot_warehouse_proxy_live.rs`) + dead pair.