<!-- nornir:class: design -->
# gunnar — details
Reference for `gunnar 1.5.0`. Every number carries the date it was measured and
the box it was measured on. A dash means no measurement exists.
## 1. Storage
One trait, `git-storage-trait::GitOps`, with two implementations picked at
runtime by `GUNNAR_ENGINE` (`crates/gunnar-server/src/storage.rs`). Same server,
same wire, same process, different store.
| | znippy | gix |
|---|---|---|
| container | Apache Arrow IPC inside a znippy archive, one archive per repository; Apache Iceberg as the cluster backend | loose objects and packfiles on a filesystem |
| implementation | `znippy/plugins/native/znippy-plugin-git` | `edda/crates/storage-git-gix` over `gix-odb` / `gix-ref` |
| pack entry bytes | stored as the client deflated them; not re-compressed, not re-encoded | as received |
| index | Arrow index tables, fronted by a `stree` static search tree, redb as the tail | one `.idx` per pack, probed in sequence |
| point lookup | `O(log n)` | `O(P·log n)`, `P` = pack count |
| cargo feature | `engine-znippy`, default; the only engine the musl appliance ships | `engine-gix` |
Storing pack entries inflated per oid measured 3.4× larger than the input over
60 000 small objects and removes the pack-copy path.
gix is used as a codec by both engines: pack headers, varints, delta ops, object
grammar. It owns no handle and decides no byte placement.
### What one repository costs on disk — XFS, 2026-10-06
`crates/gunnar-server/tests/repo_overhead.rs` lists every file of a repository with
its apparent and its allocated size, and holds a small repository under 100 KiB.
| | redb 2.6.3 tail (before) | redb 4.3.0 tail (after) |
|---|---:|---:|
| small repository: one tiny push, closed, mean of 100 | 1 654 784 B | 69 632 B |
| empty repository, closed | 1 634 304 B | 53 248 B |
| `gunnar-git.znippy/objects.tail` alone | 1 617 920 B allocated, 3 686 400 B apparent | 36 864–45 056 B allocated |
The whole fixed cost was the one redb file per repository: redb 2 allocates
540 KB creating a database and 1.6 MB once it has been closed. The other six
files take one 4 KiB block each. The tail runs on redb 4 through
`znippy-plugin-git` 0.1.8 (the change, published); an older store's
redb 2 tail is copied into redb 4 the first time the store opens
(`tests/old_layout_store.rs` opens a data directory the old build wrote). The
transaction log is ~500 B allocated per entry with no preallocation; the load
test's "~180 KiB of disk per txlog entry" was this tail divided by the ~7.7
entries one repository's create and push write.
## 2. Write path
| step | property |
|---|---|
| receive | pack bytes fsynced, then the journal row that references them fsynced |
| ack | sent after both fsyncs |
| index | built after the ack, over a channel, one indexer per account |
| read before the index is ready | falls back to a scan |
| crash marker | the same per-pack bit that marks "indexed" |
A crash between the two fsyncs leaves object bytes that no row references. It
does not leave a row referencing absent bytes.
### Writer bench — the desktop, 2026-08-07
Real packfiles, one distinct pack per iteration, `/dev/md1` raid0 NVMe, real
fsyncs, load average recorded per cell. The 8 KiB rung, index included.
| arm | 8 KiB push | durability on return |
|---|---:|---|
| `SafeWriter` | 147.7 µs | bytes on the platter, journal row on the platter |
| `UringWriter` | 137.9 µs | the same, via one linked `io_uring` chain |
| `git index-pack` | 1 993 µs | fsynced |
| `gix` library writer | 50 573 µs | none — zero fsync in 200 calls |
| `FastWriter` | 3.9 µs | none — page cache; a ceiling, not a shipping option |
Cost of durability by pack size: 123× at 200 B, 34× at 8 KiB, 4.3× at 512 KiB,
2.4× at 32 MiB.
Method note: the five arms do not produce the same artifact. `git` and the `gix`
library resolve every object and emit a per-object `.idx` before returning;
gunnar's arms store the bytes verbatim and hand an extent to the indexer, and
its index is six columns per *pack*. `io_uring` is 1 `io_uring_enter` against 4
syscalls, measured, and buys 4–7% at mid size while losing at both ends: about
95% of a durable append is the device answering the fsync.
Read side, same day: server CPU on a clone 0.0840 s → 0.0637 s (−24.2%); the
client is 9.43× stock git after a `TCP_NODELAY` fix.
## 3. Engine matrix — znippy against gix
The desktop, 2026-09-01. 256 / 2048 / 16384 commits, 5 repetitions plus warm-up,
both engines in one binary on the same corpus. Load-1 mean 13.4 during the run:
per-cell spreads are tight and non-overlapping, so the ratios hold; the absolute
times are provisional.
### End to end, 16384 commits
| dimension | znippy | gix | ratio |
|---|---|---|---|
| clone, serve wall | 7.23 s | 17.7 s | 2.4×, widening with size |
| push, receive-pack | 1.54 s | 0.80 s | 1.9× |
| RSS floor | 29.2 MB | 13.6 MB | 2.15× |
| pack fidelity | byte-identical | byte-identical | 0 objects recompressed either engine |
| `fetch_divergent` | passes | fails its own correctness guard | pack-copy does not engage on incremental fetch under gix |
### Per operation
| operation | faster | ratio |
|---|---|---|
| `reachable`, 16384 commits | znippy | 5.6 ms against 14.9 s, ~2700× |
| `put_refs`, batch CAS | znippy | 30–500× |
| `refs` | znippy | 30–50× |
| `size`, `extents`, `read` | znippy | 1.3–3× |
| `header` | gix | ~5× |
| `update_ref`, single ref | gix | ~2× |
| `has` | gix | 1.3–1.7× |
gix has no reachability bitmap until `gc` runs, which is the mechanism behind
the `reachable` row.
### Optimizations this suite produced
Both landed in `znippy` (`plugins/native/znippy-plugin-git/`), each proven
byte-identical to the previous path and across a fold-changing push.
| change | before | after |
|---|---|---|
| `reachable` memoized: two `O(C)` maps rebuilt per negotiation, now folded once and invalidated by the same token as `commit_raw` | 1.258 ms | 0.73 µs at C = 20 000, ~1720×, flat across a 417× range of C |
| `head` and single-ref `update_ref` keyed by `RefLog::get(name)` | `O(r)` clone of the ref map | `O(log r)` |
A gix read figure taken before 2026-08-11 16:12 UTC is void: edda `05be139`
writes a real EWAH `.bitmap` at gc and reads it in `select`, and `de58ccc`
routes `GitOps::reachable` through the same seam. Select went 19.070 → 0.110
CPU-s.
## 4. Forge matrix
The desktop, 32 cores, 2026-08-13, 3 interleaved rounds, load average 4.39. Same
client (`git 2.53.0`), 8.00 CPUs and 8192 MB per engine read back out of each
container's own cgroup. Byte counts taken by a counting relay in front of every
engine, never from an engine's own accounting. `gunnar` is the znippy engine;
the comparison column is the better of gitea and Forgejo.
| arm | server→client bytes | net CPU s | peak RSS MB |
|---|---|---|---|
| clone (small) | 1.78 MB / 1.78 MB · 0.998× | 0.010 / 0.033 · 3.42× less | 2308 / 113 · 20.49× more |
| fetch (divergent) | 344.0 kB / 296.6 kB · 1.16× more | 0.037 / 0.055 · 1.47× less | 156 / 109 · 1.44× more |
| push (base) | 1.4 kB / 2.1 kB · 1.45× less | 0.302 / 0.944 · 3.13× less | 620 / 125 · 4.95× more |
| ls-remote (small) | 1.2 kB / 1.2 kB · 1.04× less | 0.001 / 0.004 · 6.29× less | 331 / 144 · 2.30× more |
| clone, ramped ×16 | 392.97 MB / 392.83 MB · 1.000× | 3.179 / 3.956 · 1.24× less | 1004 / 172 · 5.85× more |
| clone, ramped ×16, pinned | 392.97 MB / 392.82 MB · 1.000× | 2.290 / 3.814 · 1.67× less | 924 / 115 · 8.05× more |
Peak anonymous RSS on a clone is flat across a 108× range of work while both Go
forges scale with the repository size. The level is not stable: it was measured
three times in two days with a different result each time, and the ramped arm
holds a fraction of it for the identical corpus. What is established is that
gunnar spends an order of magnitude more resident memory than gitea or Forgejo
on every single-client arm. The cause is not established.
### Population, the desktop 2026-08-12
| measurement | gitea | gunnar |
|---|---|---|
| provision 100 000 users | 77 min, projected | 6.1 s |
| ls-remote p50 at 100 000 users | 27.0 ms | 10.0–10.5 ms, flat from 100 users |
| ls-remote p99 at 100 000 users | 39–43 ms | 102–168 ms |
| ls-remote throughput | 1 140 req/s | 2 060–2 320 req/s |
Open-loop arm `gunnar.population_day`, the desktop 2026-08-12: 100 000 users at
full identity weight, 50 arrivals/s held 300 s, every operation a fresh process
paying a real TLS handshake against `--http-tls-self`. 49.56/s achieved, 14 893
of 15 000 completed, 107 shed, 0 failures, on 2.55 of 32 cores.
## 5. Server features
| feature | behaviour |
|---|---|
| replication | a standby is a full gunnar: it serves reads, takes the primary's replication and can be promoted. `gunnar control replication abdicate --promote <standby>` freezes the primary's writes, drains every queued push, demotes itself, then promotes the standby. A standby refuses a direct client push until promoted; an abdicated box reboots frozen. The traffic switch is the reverse proxy's. |
| provenance | `gunnar-server verify` walks a source line back to the commit that wrote it, checks that commit's own `sshsig` against the key table, names the push that landed it, and shows it under a signed tree head, then forward to the build record and the artifact the registry admits. An unsigned commit or a push absent from the log renders as a named hole. Each link is `sshsig` under its own namespace, checkable with `ssh-keygen -Y verify`; no external trust root. |
| review | a merge is refused unless a verified approval names the exact commit the branch is at; moving the branch invalidates the approval. A protected branch can forbid self-approval and require N distinct approvers. An approval signed by a key that is not the claimed reviewer's is refused. |
| admission | one gate — per-principal fair share, a server-wide ceiling, a memory budget — over SSH, over smart HTTP (over budget → `429`) and over the one control-plane RPC that streams bytes, `ArchiveOut` (→ `RESOURCE_EXHAUSTED`). |
| dependency CVEs | `cargo deny` runs as a deploy door and as `cargo xtask cve`; a new advisory or a yanked crate fails the build by RUSTSEC id. The appliance carries no operating system packages. |
| operator verbs | `blame`, `search`, `grant list`, `revoke`, `health`, `stats`, `storage`. |
## 6. Transports and features
| feature | default | effect |
|---|---|---|
| `engine-znippy` | on | Arrow IPC storage engine |
| `engine-gix` | off | filesystem object store |
| `http` | off | smart HTTP transport, authenticated by a principal-bound access token as the HTTP Basic password or as `Bearer` |
| `systemd` | off | unit installation; on for the service shape only |
| `replication` | — | send-pack, the SSH dialler and the ref plan: +4 packages, measured 2026-08-30 |
`git://` is never built. gunnar terminates no TLS: it refuses to read a
credential off a plain connection unless the terminator sends
`X-Forwarded-Proto: https`, or `--http-allow-cleartext-credentials` is passed. A
repository marked `Public` is readable with no key, via the reserved principal
`anonymous`, answered by the same authorisation table as every other principal.
Secrets at rest are sealed with AES-256-GCM (aws-lc-rs). The key-encrypting key
lives in `secrets.kek`, mode 0600, beside the control store, or comes from
`GUNNAR_SECRET_KEK` and is never written. This addresses exfiltration of the
data without the disk — a cold-tier archive copied to object storage, a backup,
a replica's `secrets.arrow`. It does not address possession of the disk, root on
the running appliance, or its memory.
Concurrency is `gatling`. rayon is absent from the resolved graph and the
absence is asserted by `cargo xtask guard`.
## 7. Delivery
| target | artifact |
|---|---|
| appliance ISO | one binary as PID 1, no operating system under it |
| UpCloud | installer ISO; re-imaged by `cargo xtask reinstall-appliance` |
| Azure / AWS | container image |
| existing host | systemd unit |
Build variants are declared in `.nornir/build-variants.toml`.
**Fast setup — [`gunnar-quickstart.md`](gunnar-quickstart.md).** Four numbered
lists of commands and nothing else: one appliance; the console beside it; a twin
and the two failover flips; the four build lines. Every command in it was run
against appliances booted in KVM on the day it was written.
## 8. Repository tasks
| command | what it does |
|---|---|
| `cargo xtask guard` | resolves the cargo graph per feature set and asserts the dependency rules |
| `cargo xtask cve` | `cargo deny` advisories and bans over `Cargo.lock` |
| `cargo xtask ci` | fmt, bench-build, clippy, preflight, tests, appliance build, guard, cve |
| `cargo xtask bench-build` | compiles every detached bench manifest with `--all-targets` |
| `cargo xtask trailer-guard` | reports attribution trailers in commits and tracked files |
| `cargo xtask publish` | writes the public tree as one orphan commit on branch `public` |
| `cargo xtask svg` | regenerates the two architecture diagrams from their models |
| `cargo xtask docs` | assembles the PDF books under `docs/` |
## 9. Licensing
Source code is AGPL-3.0-only. Original documentation is CC-BY-4.0. The gunnar
name and the badger are trademarks owned jointly by Vetra Automation AB and
Ignalina ApS and are not licensed. The code is owned jointly by the two companies
in equal shares (50/50), and they together offer it under separate commercial
terms (contact gunnar@gunnar.rs). Authoritative text
and the third-party inventory: `LICENSE`, `NOTICE`,
`THIRD_PARTY_LICENSES.md`, `TRADEMARKS.md`, `COMMERCIAL-LICENSING.md`.
## 10. Building against sibling checkouts
A fresh clone builds on its own:
```
git clone ssh://git.gunnar.rs:2222/nordisk/gunnar
cd gunnar
cargo build --release -p gunnar-cli
```
Nothing has to sit beside it. Every crate this workspace does not own comes from
crates.io, including the fourteen that were `path` dependencies into `../edda`,
`../znippy` and `../monetize` until 2026-09-09:
| crate | version | repository |
|---|---|---|
| `gunnar-bitmap` | 1.1.1 | edda |
| `gunnar-delta` | 1.1.1 | edda |
| `gunnar-pack` | 1.1.1 | edda |
| `gunnar-pktline` | 1.1.0 | edda |
| `gunnar-sendpack` | 1.1.0 | edda |
| `gunnar-stage` | 1.1.0 | edda |
| `nornir-build-thing` | 0.1.14 | edda |
| `nornir-git` | 0.1.9 | edda |
| `nornir-testmatrix` | 0.2 | edda |
| `storage-git-gix` | 0.1.1 | edda |
| `wg-appliance-core` | 0.1.4 | edda |
| `git-storage-trait` | 0.1.2 | znippy |
| `znippy-plugin-git` | 0.1.2 | znippy |
| `monetize-embed` | 0.1.0 | monetize |
The one path dependency left is `vendor/gix-features`, which is inside this
repository — see the `[patch.crates-io]` table in `Cargo.toml` for why it is
vendored and when it goes.
### Working on gunnar and a sibling at the same time
To build gunnar against a local checkout of one of those crates, point cargo at
it through **your own** cargo configuration — never through this repository's
`Cargo.toml`:
```toml
# ~/.cargo/config.toml (or $CARGO_HOME/config.toml)
[patch.crates-io]
gunnar-pack = { path = "/home/you/git/edda/crates/gunnar-pack" }
git-storage-trait = { path = "/home/you/git/znippy/git-storage-trait" }
monetize-embed = { path = "/home/you/git/monetize/crates/monetize-embed" }
```
Three things about that placement, each of which has cost someone a day:
* **Not in this repository's `Cargo.toml`.** A `[patch.crates-io]` entry pointing
outside the repository puts the sibling checkout back as a hard requirement for
everyone who clones, which is exactly the property this change bought.
* **Not `.cargo/config.toml` in this repository either** — that file is tracked
(it carries the `xtask` alias and the `fmt` guard), so an entry there is the
same thing with extra steps. And cargo reads no `.cargo/config.local.toml`: the
only files it merges are `config.toml` and the legacy extensionless `config`,
walking up from the current directory and then `$CARGO_HOME`. A "local" file
next to the tracked one is simply ignored, silently.
* **Absolute paths.** `~/.cargo/config.toml` is read from wherever you run cargo,
so a relative path resolves against your shell's directory, not the repository.
A patch is a *candidate*, not an override: the resolver still takes the greatest
compatible version, so a local checkout whose version is behind the registry loses
to the registry copy and cargo says `patch ... was not used in the crate graph`.
Bump the local version, or you will debug the wrong source.
To go back, delete the entries. `cargo xtask guard` and `cargo xtask cve` both
read the resolved graph, so run them with the patches OFF before believing them.