<!-- nornir:class: guide -->
# gunnar — fast setup
<img src="assets/gunnar-setup.svg" alt="one main appliance, a twin beside it, a browse replica, and the console" width="1020">
New here? **Start here** below: clone the source, build the server with one
`cargo` line, run it on your own computer and push to it. Nothing else is needed
for that. The five numbered lists after it are the appliance route (a bootable
medium, a twin, the console, the metered medium) — the advanced path.
Addresses are examples (RFC 5737): `192.0.2.10` is the main, `198.51.100.10` the
twin, `203.0.113.10` the browse replica. Substitute yours.
Every command below was run on one laptop before it was written down: §1 and §3
against three appliances installed from the medium §4 builds and booted in KVM
(`cargo xtask install-test pair`), §2 against the console in podman, pointed at
one of them.
---
## Start here — clone, build, run
The first way, and the simplest: Gunnar is a Rust program. You build it from
its source and run it on any Linux computer — no installer, no medium, no
virtual machine.
**What you need:** a Linux computer (x86-64), about 10 GB of free disk and
20–40 minutes for the first build.
```
1 sudo apt install git build-essential cmake pkg-config protobuf-compiler
2 curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
. "$HOME/.cargo/env"
3 mkdir ~/nordisk && cd ~/nordisk
4 export GIT_LFS_SKIP_SMUDGE=1
git clone ssh://git.gunnar.rs:2222/nordisk/gunnar
git clone ssh://git.gunnar.rs:2222/nordisk/gunnar-ui
git clone ssh://git.gunnar.rs:2222/nordisk/monetize
5 git clone ssh://git.gunnar.rs:2222/nordisk/edda
git clone ssh://git.gunnar.rs:2222/nordisk/Skidbladnir
git clone ssh://git.gunnar.rs:2222/nordisk/znippy
git clone ssh://git.gunnar.rs:2222/nordisk/znippy-zoomies
git clone ssh://git.gunnar.rs:2222/mikronova/draupnir
6 cd gunnar
cargo build --release -p gunnar-server -p gunnar-cli
7 ./target/release/gunnar-server serve
```
Leave `7` running. Open a **second terminal**:
```
8 cd ~/nordisk/gunnar && mkdir -p ~/gunnar
9 ssh-keygen -t ed25519 -N "" -f ~/gunnar/admin
10 ./target/release/gunnar-cli control status --at 127.0.0.1:50051
11 ./target/release/gunnar-cli control bootstrap alice --key @$HOME/gunnar/admin.pub \
-i ~/gunnar/admin --at 127.0.0.1:50051
12 ./target/release/gunnar-cli control repo create alice/hello \
-i ~/gunnar/admin --at 127.0.0.1:50051
13 export GIT_SSH_COMMAND="ssh -i ~/gunnar/admin -o IdentitiesOnly=yes"
14 cd ~ && git clone ssh://git@127.0.0.1:2222/alice/hello.git
15 cd hello && git commit --allow-empty -m first && git push origin HEAD:refs/heads/main
```
What the lines mean, one clause each:
* `1` installs the tools the build calls: a C compiler, `cmake`, and `protoc`
(the protobuf compiler that turns Gunnar's `.proto` files into Rust). The
line is for Debian and Ubuntu; on Fedora it is
`sudo dnf install git gcc gcc-c++ cmake pkgconf protobuf-compiler`.
* `2` installs Rust (rustup, the stable toolchain). Accept the defaults.
* `GIT_LFS_SKIP_SMUDGE=1` (in `4`): the rendered PDFs under `docs/` are Git LFS
objects, and the build needs none of them. The clones take their small
pointer files instead, so a clone works whether or not `git-lfs` is installed
and whatever the server holds of them.
* `4` and `5` put eight checkouts **side by side** in one directory. Gunnar's
workspace names the other seven by path (`../edda/…`), so they must sit
beside it with exactly these names. They are compiled from source, not
downloaded as crates. `cargo xtask one-clone` (inside `gunnar/`) names any
sibling that is missing.
* `6` is the whole build. It writes `target/release/gunnar-server` (the server)
and `target/release/gunnar-cli` (the client you talk to it with). The first
build compiles about 800 crates; later builds are quick.
* `7` starts the server with its defaults: git over SSH on port `2222` (every
interface), the control plane on `127.0.0.1:50051` (this computer only), and
the data in `~/.local/share/gunnar` (`--data-root DIR` puts it elsewhere).
Stop it with Ctrl-C; start it again with the same line, and your data is still
there.
* `10` asks the new server who it belongs to. It answers `bootstrap OPEN` until
`11`.
* `11` makes `alice` the administrator, with the key from `9`. It works **once**.
* `12`–`15` create a repository, clone it and push the first commit to it.
* To let other computers push, they need port `2222` of this one. The control
plane stays on `127.0.0.1`. From another computer, reach it with
`ssh -L 50051:127.0.0.1:50051 you@this-computer`.
When this works, the numbered lists below turn the same server into an
appliance of its own: a bootable medium (§4), a twin that takes over (§3), and
the web console (§2).
---
## 1. One gunnar server
<img src="assets/gunnar-setup-1.svg" alt="one appliance and one person: control over :50051, push over :2222">
```
1 cargo xtask iso-installer --variant gunnar-installer-open \
--opt dhcp=on --opt wireguard=off
2 cargo xtask install-test pair --members main --keep
3 ssh-keygen -t ed25519 -N "" -f ~/gunnar/admin
4 gunnar-cli control status --at 192.0.2.10:50051
5 gunnar-cli control bootstrap alice --key @~/gunnar/admin.pub \
-i ~/gunnar/admin --at 192.0.2.10:50051
6 gunnar-cli control user add bob --key @~/gunnar/bob.pub \
-i ~/gunnar/admin --at 192.0.2.10:50051
7 gunnar-cli control repo create alice/hello \
-i ~/gunnar/admin --at 192.0.2.10:50051
8 export GIT_SSH_COMMAND="ssh -i ~/gunnar/admin -o IdentitiesOnly=yes"
9 git clone ssh://git@192.0.2.10:2222/alice/hello.git
10 cd hello && git commit --allow-empty -m first && git push origin HEAD:refs/heads/main
```
What the lines mean, one clause each:
* `1` writes `.iso-out/gunnar-installer-open.iso`. Boot it on the machine
(`dd` it to a stick, or attach it as boot media) and the machine **is** a
gunnar appliance; `2` does that in KVM on this box instead and prints the
forwarded ports. On real hardware, drop `2` and use the machine's own address.
Use `--build kernel` instead of `--opt wireguard=off` when the pinned kernel
toolchain image is on the build box — see §4.
* `4` needs no credential and is the one question an unclaimed server answers.
It says `bootstrap OPEN` until `5`, and `closed` for ever after.
* `5` works **once, ever**. It prints `enrolled alice` and `role gunnar-admin`.
* `7` creates sha256 repositories. Stock `git init` is sha1 — clone the empty
repository as in `9` rather than pushing an existing sha1 one, or create it
with `--hash sha1`.
* `--at` has an environment twin: `export GUNNAR_CONTROL_ADDR=192.0.2.10:50051`
and drop it from every line below.
* The control plane binds **loopback** by default. The appliance medium binds it
to the private interface (`--grpc-bind eth1:50051`); on a host you configured
yourself, reach it with `ssh -L 50051:127.0.0.1:50051 192.0.2.10`.
---
## 2. The console beside it
<img src="assets/gunnar-setup-2.svg" alt="gunnar-ui pointed at the appliance over gRPC :50051">
```
1 cd ../gunnar-ui
2 cargo build --release -p gunnar-ui-server --target x86_64-unknown-linux-musl
3 (cd crates/gunnar-ui-web && trunk build --release)
4 ssh-keygen -t ed25519 -N "" -f ~/gunnar/ui-id
5 ssh-keygen -t ed25519 -N "" -f ~/gunnar/ui-guest
6 GUNNAR_UI_CONTROL=192.0.2.10:50051 GUNNAR_UI_IDENTITY=~/gunnar/ui-id \
GUNNAR_UI_STATE=~/gunnar/ui-state.redb \
target/x86_64-unknown-linux-musl/release/gunnar-ui-server \
enroll --admin-key ~/gunnar/admin --name gunnar-ui
7 gunnar-cli control user add gunnar-ui-guest --key @~/gunnar/ui-guest.pub \
--kind machine -i ~/gunnar/admin --at 192.0.2.10:50051
8 mkdir -p ~/gunnar/console && cd ~/gunnar/console
9 cp ~/git/gunnar-ui/target/x86_64-unknown-linux-musl/release/gunnar-ui-server .
10 cp -r ~/git/gunnar-ui/crates/gunnar-ui-web/dist .
11 printf 'FROM scratch\nCOPY gunnar-ui-server /gunnar-ui-server\nCOPY dist /dist\nENTRYPOINT ["/gunnar-ui-server"]\n' > Containerfile
12 podman build -t gunnar-console:local .
13 export TOKEN=$(openssl rand -hex 16)
14 podman run -d --name gunnar-console --network=host \
-v ~/gunnar:/keys:Z \
-e GUNNAR_UI_CONTROL=192.0.2.10:50051 \
-e GUNNAR_UI_IDENTITY=/keys/ui-id \
-e GUNNAR_UI_GUEST_IDENTITY=/keys/ui-guest \
-e GUNNAR_UI_WEBAUTHN_RP_ID=localhost \
-e GUNNAR_UI_WEBAUTHN_ORIGIN=http://localhost:3210 \
-e GUNNAR_UI_ADMIN_TOKEN=$TOKEN \
-e GUNNAR_UI_BIND=127.0.0.1:3210 \
-e GUNNAR_UI_DIR=/dist \
-e GUNNAR_UI_STATE=/keys/ui-state.redb \
-e GUNNAR_UI_GIT_URL=ssh://192.0.2.10:2222 \
gunnar-console:local
15 curl -s http://127.0.0.1:3210/auth/status
16 curl -s -H "Authorization: Bearer $TOKEN" \
"http://127.0.0.1:3210/auth/passkey/register/start?user=alice"
17 open http://localhost:3210/login # finish the passkey in the browser
18 curl -s -X POST -H "Authorization: Bearer $TOKEN" \
-H 'content-type: application/json' \
-d '{"role":"operator","user":"bob"}' \
http://127.0.0.1:3210/auth/passkey/invites
```
The six that matter, and what each is:
| variable | what it takes |
|---|---|
| `GUNNAR_UI_CONTROL` | the server's control plane, `host:port` |
| `GUNNAR_UI_IDENTITY` | path to an **ed25519 private key**; enrolled by `5` as a `Machine` principal with a server-wide admin grant |
| `GUNNAR_UI_GUEST_IDENTITY` | a **second** key, enrolled with **no** grant, for visitors with no session. Unset means nothing is served anonymously. Never the key above |
| `GUNNAR_UI_WEBAUTHN_RP_ID` | a **domain**. An IP address is refused at start-up; in development the domain is `localhost` |
| `GUNNAR_UI_WEBAUTHN_ORIGIN` | the exact browser origin, scheme and port included. It is also what invite links are built from |
| `GUNNAR_UI_ADMIN_TOKEN` | break-glass bearer for passkey **number one**. It stops working the moment an admin passkey exists |
* `6` is the enrolment: it makes three calls with your admin key
(`AddPrincipal{Machine}` → `AddKey` → `Grant{server-wide admin}`) and prints
`enrolled gunnar-ui` and the console's fingerprint.
* `11`–`12` produce an 18.6 MB image on `scratch` — the musl binary and the
built page, no base image, no shell. `roles` for `18` are `viewer`,
`operator`, `admin`; anything else is refused by name.
* `15` answers `{"authenticated":false,…,"open":false,"passkey_enabled":true,…}`.
`"open":true` there means the console is running with no credential at all —
which it refuses to do on any non-loopback bind.
* `16` returns `200` and a WebAuthn challenge. The ceremony itself needs a
browser; that is `17`.
* Browse `localhost:3210`, not `127.0.0.1:3210`: the RP ID has to match.
* `GUNNAR_UI_DIR` unset gives you a console that answers its API and serves no
page, with one `warn` line to say so. `GET /` is `401` until you have a
session; `GET /login` is the page.
---
## 3. A twin, and the two flips
<img src="assets/gunnar-setup-3.svg" alt="the main, the twin, the browse replica, and the two flips">
§2 is unchanged and applies to the pair: the console is one process pointed at
one control plane, and the only thing a failover changes about it is
`GUNNAR_UI_CONTROL`.
### 3a. Install the twin and start replicating
```
1 cargo xtask install-test pair --members main,twin,spare --keep
2 gunnar-cli control bootstrap alice --key @~/gunnar/admin.pub \
-i ~/gunnar/admin --at 198.51.100.10:50051
3 gunnar-cli control replication standby -i ~/gunnar/admin --at 198.51.100.10:50051
4 gunnar-cli control replication add twin ssh://git@198.51.100.10:2222 \
-i ~/gunnar/admin --at 192.0.2.10:50051
5 gunnar-cli control user add primary --key '<the line step 4 printed>' \
--kind machine -i ~/gunnar/admin --at 198.51.100.10:50051
6 gunnar-cli control grant primary admin --namespace alice \
-i ~/gunnar/admin --at 198.51.100.10:50051
7 gunnar-cli control replication resync twin -i ~/gunnar/admin --at 192.0.2.10:50051
8 gunnar-cli control replication list -i ~/gunnar/admin --at 192.0.2.10:50051
```
* `3` makes the twin read-only to people: it takes its primary's replication
push and refuses a `git push` from anyone else with *"push to the primary"*.
* `4` mints the main's own deploy key if it has none — `/data/replication_key`
on an appliance, and `replication list` names the path — keeps the private
half on the box for ever, and prints the public line **and the two commands to
run on the twin**. `5` and `6` are those commands. `admin`, not `write`: the
twin has to *create* each repository. You never place a key by hand; a box
with no shell could not be handed one.
* `7` is not optional. Adding a standby copies **nothing**; replication rides the
push bus, so without a backfill the twin holds only what is pushed from now on.
`--repo <ns>/<repo>` does one; `--except <ns>/<repo>` (repeatable) does all but.
* `8` is the proof. Before `7` it reads
```
twin pending 0 failures 0 last success NEVER
NO COPY alice/hello [twin]
```
and after it, once the queue drains,
```
replication 1 target(s) · synced 1 · failed 0 · bus gaps 0
twin pending 0 failures 0 last success <ms> ms idle
copied alice/hello [twin] 1 refs there
```
`last success NEVER` with `pending 0` means the twin has never taken a byte
and nothing is asking it to — usually `7` not run. `failures` climbing means
`5`/`6` not run, or run with `write` instead of `admin`.
### 3b. The twin becomes the writable main
```
1 gunnar-cli control replication promote -i ~/gunnar/admin --at 198.51.100.10:50051
2 gunnar-cli control replication demote -i ~/gunnar/admin --at 192.0.2.10:50051
3 gunnar-cli control replication standby -i ~/gunnar/admin --at 192.0.2.10:50051
4 git remote set-url origin ssh://git@198.51.100.10:2222/alice/hello.git
5 git push origin HEAD:refs/heads/main
```
* `1` on the **twin**: `promoted — this box is now a writable primary and takes
client writes`.
* `2` and `3` on the **old main**, and they are two different things. `2` clears
its standby roster — without it a recovered old primary resumes pushing its
stale state and force-clobbers the new primary's writes — and says so:
```
demoted — 1 standby target(s) cleared; this box ships no second copy
It STILL TAKES CLIENT WRITES. If this is a failover and another box is
the primary now, freeze this one too or it will accept a push nobody
replicates:
gunnar-cli control replication standby
```
`3` is that freeze:
```
standby — this box is now read-only to clients; it accepts only its primary's replication push
role standby (read-only to clients)
```
A client still pointed at it is then refused by name — `fatal: remote error:
this box is a standby; push to the primary` — instead of leaving a commit
nobody replicates. `demote` alone leaves the box **writable**, and says so.
* A *planned* hand-off is one command on the healthy old main instead of `1`–`3`,
and it does not lose a write — freeze, flush every standby, demote, then
promote the named box:
```
gunnar-cli control replication abdicate --promote 198.51.100.10:50051 \
-i ~/gunnar/admin --at 192.0.2.10:50051
```
* Then move the traffic, which is infrastructure and not gunnar's to move: DNS
(or your load balancer) to `198.51.100.10`, and the console's
`GUNNAR_UI_CONTROL=198.51.100.10:50051`.
### 3c. The main was nuked
```
1 cargo xtask iso-installer --variant gunnar-installer-open \
--opt dhcp=on --opt wireguard=off
2 gunnar-cli control bootstrap alice --key @~/gunnar/admin.pub \
-i ~/gunnar/admin --at 192.0.2.10:50051
3 gunnar-cli control replication standby -i ~/gunnar/admin --at 192.0.2.10:50051
4 gunnar-cli control replication add rebuilt ssh://git@192.0.2.10:2222 \
-i ~/gunnar/admin --at 198.51.100.10:50051
5 gunnar-cli control user add survivor --key '<the line step 4 printed>' \
--kind machine -i ~/gunnar/admin --at 192.0.2.10:50051
6 gunnar-cli control grant survivor admin --namespace alice \
-i ~/gunnar/admin --at 192.0.2.10:50051
7 gunnar-cli control replication resync rebuilt -i ~/gunnar/admin --at 198.51.100.10:50051
8 gunnar-cli control replication list -i ~/gunnar/admin --at 198.51.100.10:50051
```
* Every `--at 198.51.100.10:50051` line runs against the **survivor**, which is
the primary now; every `--at 192.0.2.10:50051` line runs against the **fresh**
box. `8` ends at `synced 1 · failed 0` and `copied alice/hello [rebuilt]`,
and a `git clone` off the rebuilt box returns the history with `fsck --full
--strict` clean.
* The surviving twin is the main now; the fresh box is its standby. That is the
finished state — the pair is whole again, replicating the other way.
* To put the rebuilt box back in charge afterwards, that is §3b with the
addresses swapped.
* Install the fresh box the same way you installed the first: `1`, then boot it.
The rig in `3a` step `1` lays down a third appliance up front (`spare`) so the
drill can be run without waiting for an install.
### 3d. A browse replica, in one line
```
1 gunnar-cli control replication add public ssh://git@203.0.113.10:2222 \
--scope public-only --control 203.0.113.10:50051 \
-i ~/gunnar/admin --at 192.0.2.10:50051
2 gunnar-cli control user add primary --key '<the line step 1 printed>' \
--kind machine -i ~/gunnar/admin --at 203.0.113.10:50051
3 gunnar-cli control grant primary admin --namespace alice \
-i ~/gunnar/admin --at 203.0.113.10:50051
4 gunnar-cli control replication browse -i ~/gunnar/admin --at 203.0.113.10:50051
```
* **`4` comes last, and that is not a preference.** A browse replica refuses
every control-plane mutator, itself included:
```
this box is a browse replica: it holds public repositories read-only and cannot be
promoted or changed here — Access.AddPrincipal is refused; run it against the primary
```
so `2` and `3` are unrunnable on a box that has already been flipped. Authorise
the key first, flip second. `replication standby` is the way back.
* Public repositories only; one that goes private is deleted there, which is why
`--scope public-only` also needs `--control`. It cannot be promoted — not by
`promote`, not by `abdicate --promote`, not by the passive heartbeat.
---
## 4. Building gunnar
<img src="assets/gunnar-setup-4.svg" alt="the medium is built on a laptop and boots into the appliance">
**The advanced route.** The plain `cargo build` in *Start here* gives you the
server as a program. The lines below build the **appliance medium** instead: an
installer image that turns a whole machine into a Gunnar server, composed by the
Build Thing in a pinned container.
```
1 dwarves buildthing gunnar --variant gunnar-installer
2 dwarves buildthing gunnar --variant gunnar-installer-open --opt dhcp=on
3 dwarves buildthing gunnar --variant gunnar-installer-metered
4 cd ../gunnar-ui && cargo build --release -p gunnar-ui-server \
&& (cd crates/gunnar-ui-web && trunk build --release)
```
and the one equivalent from inside the gunnar checkout, which finds the dwarves
sibling and the kernel for you:
```
cargo xtask iso-installer --variant gunnar-installer-open --opt dhcp=on
```
* Run `1`–`3` from the directory that holds the checkouts (`dwarves` resolves
`gunnar` as a child of the working directory, then `$NORDISK_ROOT/gunnar`, then
`../gunnar`, and prints which rule it used). Add `--list` to any of them to
resolve the variant and build nothing.
* `1` is the keyed appliance medium, `2` the open (trust-on-first-use) one, `3`
the metered one. All three declare `target_disk_gib = 64` and require UEFI.
* `--opt <name>=on|off` only accepts a toggle the variant **declares**; an
undeclared name is refused by name. `--list-options` prints every one.
* `--build kernel` compiles the appliance kernel from source in the pinned
container instead of carrying one off the build box, and it is the way the
law wants. Without that flag the medium takes `$KORP_KERNEL` (or
`$TUNNR_KERNEL`) — a kernel you built and can name. With **nothing declared
the verb REFUSES by name** (`[host-kernel-not-declared]`) rather than sealing
whatever this box happens to be running into a medium that installs an
appliance somewhere else; `GUNNAR_HOST_KERNEL=1` is the named exception that
takes the newest readable `/boot/vmlinuz*`, and the verb then says CARRIED OFF
THIS BOX on every line that mentions the kernel. This is not theory: the O1
run in `open-build-capacity.md` composed with no `KORP_KERNEL`, silently
carried `/boot/vmlinuz-7.0.0-30-generic`, and the installed appliance stopped
at `md: personality for level 0 is not loaded!` because that kernel has raid0
and xfs as modules. Whichever kernel is named, it is **checked against every
symbol the variant's options require** before a byte is composed. A carried kernel with `CONFIG_TUN=n` is
refused by name for the `wireguard` option (boringtun is a userspace data
plane and opens `/dev/net/tun`); `--opt wireguard=off` drops the requirement
along with the feature, and `--build kernel` satisfies it properly.
* `4` is the console, and it is the one that is **not** a declared build
variant: `gunnar-ui` carries no `.nornir/build-variants.toml`, so there is no
`dwarves buildthing` line for it and these two commands are the build.
---
## 5. The metered medium, and the money half
**Skip this section entirely unless you built `gunnar-installer-metered` (§4, line
`3`).** The open medium has no money code in it at all: no plans, no quotas, no
buying, storage unlimited. Everything below describes the OTHER medium.
gunnar never takes money. It does three things and stops:
* **reports** what each namespace uses, over `Usage.Read` / `Usage.Activity`;
* **accepts** a signed entitlement and enforces it — `Entitlement.Set`, verified
against the Ed25519 key named by `--monetize-pubkey`, and refused if the
signature does not check;
* **mints an actor ticket** on request, `Billing.MintActorTicket`, which is the
appliance vouching that a console session really is a namespace admin.
The money half is a **separate product in a separate repository**, `monetize`,
running as its own binary and talking to this one over that signed seam. It is
not part of this checkout and is not built by §4. An appliance from the metered
medium with no monetize server in front of it enforces the entitlements it was
last given and issues no new ones.
```
1 gunnar control appliance-identity
2 gunnar control plan show <namespace>
3 gunnar control plan list --expiring
```
* `1` prints the appliance's ticket public key — the value monetize must be
configured with before it will believe a ticket this box minted.
* `2` and `3` are read-only. There is no `gunnar control entitlement set`: the
only way a cap changes is a fact signed by monetize's private key, which this
box does not hold and must never hold.
* `Entitlement` and `Billing` exist **only** in the metered build. On an open
appliance the symbols are not compiled in and the services are not mounted.
### Growing the data set while it serves
This part is gunnar's own, and it works the same whoever asks for it.
```
4 gunnar control capacity
5 gunnar control grow status
```
* `4` answers `total` / `servable` / `used`, and says whether this boot stack can
grow at all. A medium that reports `growth=SEALED` will take the data and
refuse the resize — only a medium built with the runtime grow engine says
`growth=AT RUNTIME · hotplug=yes`.
* A growth is ordered when the set crosses **80 % used**, and it grows down to
**65 %** — one growth, sized to the low mark, not one per customer. There is a
minimum of **one hour** between growths unless the set is over **90 %**.
* The sequence drains the gate, waits for in-flight transfers to finish, unmounts
`/data`, waits for the members to come back larger and equal, grows the array
and remounts. **No reboot.** Pushes wait; nothing is killed.
* It will not start unless a replication twin, if one is configured, is caught
up — and it sizes that twin to what the primary is about to hold, **before**
the primary grows. A twin too small to hold the primary can never catch up,
and every future growth would then be refused.
* Nothing here resizes a cloud volume. The appliance asks for members that are
already bigger; making them bigger is the outside actor's job, and on a real
cloud that actor is monetize with an explicitly enabled write credential.