Gunnar

nordisk/Skidbladnir

.nornir/testmatrix-arms.json

{
  "_comment": [
    "Rescue arms for the gated-tests guard (nornir_testmatrix::gatedtests).",
    "ENFORCED since 2026-08-01 by tests/silenced_tests_guard.rs — this file is no",
    "longer a note nobody reads: a plain `cargo test` now fails if a test target",
    "goes dark and is not listed here. Adding an arm that does not actually",
    "compile would be silence wearing a badge, so every arm below was built.",
    "",
    "── arm 1 · install_matrix_live (declared 2026-07-21) ──",
    "",
    "tests/install_matrix_live.rs carries `#![cfg(feature = \"matrix\")]`. `matrix`",
    "(= [\"kvm\"]) is NOT in default (default = [\"pki\"]), so a plain `cargo test`",
    "compiles it to ZERO tests and prints green-by-vacuum.",
    "",
    "WHAT THE ARM PROVES (host oden, 2026-07-21): the file COMPILES under",
    "`--features matrix` (verified with `cargo test --features matrix --test",
    "install_matrix_live --no-run`). That kills the zero-tests-by-vacuum silence",
    "and catches bit-rot in the skidbladnir::matrix / OsImage / install path.",
    "",
    "WHAT THE ARM does NOT prove: the test BODY is `#[ignore]` and boots real KVM",
    "guests, resolving ISOs/VMs from `$SKID_ISOS` / `$SKID_VMS` under /mnt/work4t.",
    "That /mnt/work4t path does NOT exist on oden. The claim that followed it — \"this box is not the KVM host\" — was WRONG and is retracted (2026-08-03): oden has /dev/kvm, the user is in group `kvm`, and the new",
    "vm_systemd_suite arm below boots two real guests here. Stage images under",
    "/home/rickard/scratch/skidbladnir/isos (md1) and set $SKID_ISOS/$SKID_VMS.",
    "run is NOT possible here. Run it on a KVM-capable box with the ISOs staged:",
    "  cargo test --features matrix --test install_matrix_live -- --ignored --nocapture",
    "The feature gate is correct: `matrix` drags in the heavy `kvm` backend the",
    "hermetic default deliberately refuses.",
    "",
    "── arm 2 · promote_contract (declared 2026-08-01) ──",
    "",
    "tests/promote_contract.rs carries `#![cfg(feature = \"promote\")]` and hides 15",
    "`#[test]` fns — the PUBLIC-API contract of `skidbladnir::promote` as a consumer",
    "(Njord's deploy tool) sees it. `promote` (= [\"dep:serde_yaml\"]) is not in",
    "default, so every one of those 15 assertions had been reporting",
    "`running 0 tests ... ok` since the file was written.",
    "",
    "WHAT THE ARM PROVES (host oden, 2026-08-01, MEASURED not inferred): the file",
    "compiles under `--features promote` AND all 15 tests RUN and PASS —",
    "  cargo test --features promote --test promote_contract",
    "  → test result: ok. 15 passed; 0 failed; 0 ignored",
    "Nothing was broken; the gap was pure reporting. It is a real rescue all the",
    "same: this is a compile-time API-shape contract, so keeping it building on",
    "every declared arm is exactly what stops a downstream break going unseen.",
    "",
    "WHAT THE ARM does NOT prove: nothing about the LIVE promote path. These are",
    "offline, in-process assertions over the schema, the slot registry fence, the",
    "two-key guard and the bundle envelope. The kubernetes half (`promote-k8s`:",
    "set-image, rollout wait, ingress switch, Helm-secret reads) gates no test file",
    "and is NOT covered by any arm here — see the open gap in `promote::release`'s",
    "module docs.",
    "",
    "One arm per file, not a fat `--features matrix,promote`: `matrix` drags the",
    "whole KVM backend and `promote` is pure serde_yaml, and an arm naming a",
    "combination nobody actually runs would be a declaration that is not true.",
    "",
    "── arm 3 · vm_systemd_suite (declared 2026-08-03) ──",
    "",
    "tests/vm_systemd_suite.rs carries `#![cfg(feature = \"matrix\")]`, so like arm 1 it",
    "compiles to ZERO tests under a plain `cargo test`.",
    "",
    "WHY THE FILE EXISTS: Skidbladnir owns service creation, and a unit renderer",
    "tested by string-comparing its own output asserts what it WROTE, never what",
    "systemd DID with it. On 2026-08-03 korp-server --install --system rendered",
    "User=korp, printed success, and ran as `rickard`, because five stale drop-ins in",
    "/etc/systemd/system/korp-server.service.d/ overrode it. No rendering test could",
    "have caught that. This suite boots a guest and grades the APPLIED state:",
    "`systemctl show`'s resolved properties, `systemctl cat` (fragment PLUS drop-ins),",
    "/proc/<MainPID>/environ, and a fresh boot's activation ordering.",
    "",
    "WHAT THE ARM PROVES (host oden, 2026-08-03, MEASURED not inferred): the file",
    "compiles under `--features matrix`, and the 20 PURE host-side judge tests in",
    "src/vmsuite.rs RUN and PASS on that arm —",
    "  cargo test --features matrix --lib vmsuite",
    "  → test result: ok. 20 passed; 0 failed",
    "Every one of them drives a guard RED with synthetic facts, so each guard is",
    "proven able to say no.",
    "",
    "MEASURED CORRECTION (2026-08-03): those 20 did NOT run in a plain `cargo test` —",
    "src/vmsuite.rs was itself behind `#[cfg(feature = \"matrix\")]`, so the default run",
    "was 267 tests and the judges were among the 37 gated out. That was a real gap, not",
    "a nuance: the judges are pure, dep-free and have nothing to do with KVM, so they",
    "SHOULD sit in the lean core where every run exercises them.",
    "",
    "THAT FOLLOW-UP IS CLOSED (2026-08-25, `94ca0f9`). `pub mod vmsuite;` is no longer",
    "gated (src/lib.rs:393) and only `boot_once`/`run_suite` still carry",
    "`#[cfg(feature = \"matrix\")]`. MEASURED on this box the same day, from the test",
    "roster rather than from a diff:",
    "  cargo test --lib -- --list | grep -c '^vmsuite::'   → 23",
    "So the 23 judges now execute on every plain `cargo test`, and this arm no longer",
    "carries them. What `--features matrix` still adds to the LIBRARY target is SIX",
    "tests, all in src/machine.rs behind `kvm` (matrix = [\"kvm\"]) — measured by",
    "differencing the `--list` rosters:",
    "  machine::tests::{build_fat_seed_image_roundtrips, extra_qemu_args_wire_data_serial_and_guest_agent,",
    "   guest_agent_reply_parsers, kvm_isolated_boot_spec_has_airgap_net, nocloud_seed_files_render,",
    "   persist_disk_defaults_off_and_is_carried_onto_the_draupnir_bootspec}",
    "They are hermetic (they pass here in 0.00 s, no KVM touched) but dependency-bound:",
    "`kvm` pulls libc + draupnir/seed + base64, which is the gate's actual purpose. That",
    "is a different thing from the vmsuite gap and must not be written up as one.",
    "WHAT THE ARM does NOT prove: the live two-boot run is `#[ignore]`. Run it with",
    "  cargo build --release --target x86_64-unknown-linux-musl --manifest-path probe/Cargo.toml",
    "  cargo test --features matrix --test vm_systemd_suite -- --ignored --nocapture",
    "It needs qemu, /dev/kvm and a cached cloud qcow2 (see the file's module docs).",
    "",
    "The live run is ALSO self-guarding: several expectations declare expect_pass=false",
    "and the suite fails unless those guards go red against a deliberately-broken",
    "guest, and a case that reports nothing fails every one of its checks. Silence is",
    "not success in either direction.",
    "",
    "── the gap arm 1 did NOT close (found 2026-08-03) ──",
    "",
    "Arm 1 verified install_matrix_live with `--no-run`: it proved the INTEGRATION",
    "test compiles under `--features matrix`, and nothing more. The unit tests inside",
    "src/matrix.rs were never run under that feature by anything. One of them,",
    "`unrendered_backend_is_skipped`, had been FAILING (not merely vacuous): it",
    "asserted Backend::WindowsService has no renderer, which stopped being true when",
    "the last renderer landed and made `renderer_for` total. Replaced by",
    "`every_declared_backend_has_a_renderer` (the invariant that made it vacuous,",
    "stated positively) plus a direct assertion that the skip outcome is still",
    "labelled SKIP and carries no ProbeReport.",
    "",
    "The lesson for every arm here: `--no-run` proves compilation, not agreement. An",
    "arm that stops at --no-run should say so — arm 1 did, honestly — but a feature",
    "gate with unit tests behind it needs an arm that RUNS them. Measured after the",
    "fix: `cargo test --features matrix` → 303 passed, 0 failed.",
    "",
    "── arm 4 · vm_korp_provision (declared 2026-08-03) ──",
    "",
    "tests/vm_korp_provision.rs carries `#![cfg(feature = \"matrix\")]`, so like arms 1",
    "and 3 it compiles to ZERO tests under a plain `cargo test`.",
    "",
    "WHY THE FILE EXISTS: arm 3 asks \"did systemd resolve what the renderer wrote?\".",
    "This asks the question one layer out — does the INSTALLED SERVICE finish its own",
    "job? On oden 2026-08-03 korp-server cut Sweden (42.7 min), tiled it, and uploaded",
    "28 974 tile objects into its own store — and then answered every client",
    "`0 region asset(s)` with an EMPTY error, for minutes, because",
    "`korp::control::Control` resolves its map roster once at construction and replays",
    "that answer forever. Only a manual restart made the map visible. Every host-side",
    "check (S3, ~/.korp/work, the region registry, $KORP_STATE) was green throughout:",
    "the defect exists only on the client side of the wire, on a machine where systemd",
    "started the service. Neither property is available to a host `#[test]`.",
    "",
    "The suite boots a guest holding a korp-server, a thin korp-probe client, a map",
    "source and a region config, runs `korp-server --install --system` ONCE, and then",
    "only OBSERVES. A unit test pins that the acceptance path contains exactly one",
    "korp-server invocation and no `restart`, so the acceptance cannot quietly weaken.",
    "",
    "WHAT THE ARM PROVES (host oden, 2026-08-03, MEASURED not inferred): the file",
    "compiles under `--features matrix`, its two cheap tests RUN and PASS on that arm,",
    "and the 30 PURE judge tests in src/korpprov.rs run and pass —",
    "  cargo test --features matrix --lib korpprov",
    "  → test result: ok. 30 passed; 0 failed",
    "",
    "SUPERSEDED THE SAME WAY (2026-08-25, `94ca0f9`): `pub mod korpprov;` is no longer",
    "gated either (src/lib.rs:412) — only `boot_once`/`run_korp_suite` still carry",
    "`#[cfg(feature = \"matrix\")]`. MEASURED: `cargo test --lib -- --list | grep -c",
    "'^korpprov::'` → 31. The judges run in every default build; the arm covers only",
    "the live guest run below.",
    "Every korp judge is driven RED there with synthetic captures: a footerless",
    "parquet, a store that took fewer objects than the tiler claimed, a roster that came",
    "back empty, a changed MainPID, a disk refusal with no numbers, and the memory gate",
    "refusing a box that cannot spare the guest.",
    "",
    "WHAT THE ARM does NOT prove: the live two-boot run is `#[ignore]`. It needs qemu,",
    "/dev/kvm, a cloud image whose glibc can run the built korp-server, a built",
    "korp-server + korp-probe, and the cached map fixtures. See the file's module docs.",
    "",
    "SCALE: the live run uses a 1:31 000 scale model of Rickard's design (liechtenstein",
    ".osm.bz2 as the planet, Oberland admin_level=6 as Skåne, Vaduz admin_level=8 as",
    "Lund) so a cut is 0.27 s instead of 42.7 min. Same code, same `.osm.bz2` reader,",
    "same PolygonMask, same tiler. It proves the MECHANISM and says nothing about 173 GB.",
    "",
    "MEMORY: oden lost its tmux server to a global OOM the same day. The guest ceiling",
    "is explicit and `run_korp_suite` refuses to boot unless the host has that plus an",
    "8 GiB floor free. `the_memory_gate_is_falsifiable_on_this_host` proves the gate can",
    "still say no on THIS box, and runs on a plain `--features matrix` test pass.",
    "",
    "── arm 5 · service_kind_live (declared 2026-08-15) ──",
    "",
    "tests/service_kind_live.rs carries `#![cfg(feature = \"service-drive\")]`, which is",
    "not in default (default = [\"pki\"]), so a plain `cargo test` compiles it to ZERO",
    "tests.",
    "",
    "WHY THE FILE EXISTS: the suite's chain matrix could not drive the `service` kind at",
    "all — 12 of 44 rows, MEASURED 2026-08-15 at",
    "edda/crates/nornir-testmatrix/tests/chain_matrix.rs:190 (\"draupnir has NO systemd",
    "backend\"). It cannot get one: Skidbladnir/Cargo.toml:87 already declares",
    "draupnir = { path = \"../draupnir\" }, and cargo forbids a package cycle even",
    "through an optional dep. So the systemd shape lives here, in src/service_drive.rs,",
    "and REUSES draupnir::Seen rather than twinning it — which is why the feature exists",
    "and why it is off (it is the only thing that would pull draupnir into the LEAN core",
    "rather than behind a heavy backend).",
    "",
    "WHAT THE ARM PROVES (this box, 2026-08-15, MEASURED not inferred): the file compiles",
    "under `--features service-drive` AND its three RED arms RUN and PASS against a real",
    "`systemd --user` — no sudo, no system unit, no privilege escalation:",
    "  cargo test --features service-drive --test service_kind_live",
    "  -> test result: ok. 3 passed; 0 failed; 1 ignored",
    "The three are all falsifiability arms, which is the point: a service that exits",
    "immediately reports Seen::Exited carrying its real code 1; a crash-looping",
    "Restart=on-failure service (whose ActiveState NEVER rests in `failed`) is caught in",
    "the SubState=auto-restart window carrying its real code 7; and a genuinely ACTIVE,",
    "serving process watched for a marker it does not print reports Seen::StillRunning",
    "with its real output. That last one is the whole thesis: active is not answering.",
    "",
    "EACH WAS DRIVEN RED BY SABOTAGE, 2026-08-15, not assumed to be falsifiable:",
    " * `let dead = at_rest || restarted` (auto-restart term deleted) ->",
    "   a_crash_looping_service_... FAILS: Exited { code: None, after: 5.08s } instead of",
    "   { code: Some(7), after: 4.15s }. The NRestarts backstop still catches the death,",
    "   one restart cycle late and unable to attribute a code -- honest, but weaker.",
    " * the deadline arm returning Seen::Marker { line: \"unit is <state>\" } (i.e. grading",
    "   readiness on ActiveState) -> an_active_service_that_never_prints_the_marker FAILS:",
    "   `expected StillRunning, got Marker { line: \"unit is active\", after: 4.02s }`.",
    " * removing this arm from the array below -> silenced_tests_guard FAILS naming the",
    "   file: `1 SILENCED (4 hidden test fns) ... tests/service_kind_live.rs`.",
    "All three sabotages were reverted; src/service_drive.rs restored to sha256",
    "17976d6540ddfc32d2f54b74a41c24845e308e0c3164556c3111b3b93c2d6452.",
    "",
    "ONE MEASURED CORRECTION recorded here because it nearly shipped as a lie: the",
    "crash-loop test originally used `/bin/false` + RestartSec=1, and it PASSED with the",
    "death-detection sabotaged away -- systemd's default start rate limit (5/10s) drove",
    "that unit into `failed` anyway, so the plain ActiveState branch caught it and the",
    "test never exercised the thing its name claimed. It now runs a 4 s lifetime, which",
    "keeps the restart rate under the burst limit.",
    "",
    "The GREEN arm is #[ignore]d because it needs a real gunnar-server on the box:",
    "  cargo test --features service-drive --test service_kind_live -- --ignored --nocapture",
    "MEASURED 2026-08-15, one pass: unit installed by the ONE writer into",
    "~/.config/systemd/user, started over D-Bus, marker `gunnar ssh listener bound",
    "local_addr=127.0.0.1:22321` seen after 14.8 ms, a real TCP client read",
    "`SSH-2.0-gunnar_1.1.3 v1.1.3-58-g99f8de0-modified` off the wire, torn down, port",
    "dead, unit file gone.",
    "",
    "WHAT THE ARM does NOT prove: anything about SYSTEM scope. A `systemctl --user` unit",
    "needs no root, so ensure_system_user, /etc/systemd/system and the User=/Group=",
    "directives (which the renderer OMITS in user scope) are untouched by every test",
    "here. It also proves nothing about gunnar beyond its greeting: the SSH",
    "identification string is the first thing the server writes, not a transaction — no",
    "key was authenticated, no namespace resolved, no repository served.",
    "",
    "── arm 6 · stderr_reaches_the_boot_log (declared 2026-09-28, release integration) ──",
    "",
    "crates/installer/nordisk-pairing/tests/stderr_reaches_the_boot_log.rs carries",
    "`#![cfg(all(unix, feature = \"stderr-capture\"))]` (bootlog::capture_stderr, cbbbfab);",
    "`stderr-capture` (= [\"dep:rustix\"]) is not in default. MEASURED oden 2026-09-28:",
    "  cargo test -p nordisk-pairing --features stderr-capture --test stderr_reaches_the_boot_log",
    "  → test result: ok. 1 passed; 0 failed; 0 ignored",
    "",
    "── arms 7–9 · nordisk-narrate loopback, nordisk-pairing installer_link_goes_quiet + two_readers_one_link (declared 2026-10-10, integrate/r5-3-3) ──",
    "",
    "crates/installer/nordisk-narrate/tests/loopback.rs is `required-features = [\"dev-narrate\"]`",
    "(36d1a9a); crates/installer/nordisk-pairing/tests/{installer_link_goes_quiet,two_readers_one_link}.rs",
    "carry `#![cfg(feature = \"installer-wireguard\")]` (0d2a690, bbb8dd7). Neither feature is in",
    "default, and no arm re-invoked them: silenced_tests_guard RED since they landed. MEASURED on the",
    "build box 2026-10-10:",
    "  cargo test -p nordisk-narrate --features dev-narrate --test loopback → ok. 2 passed",
    "  cargo test -p nordisk-pairing --features installer-wireguard --test installer_link_goes_quiet",
    "    --test two_readers_one_link → ok. 0 passed; 1 ignored / 0 passed; 2 ignored",
    "",
    "WHAT THE ARMS do NOT prove: the pairing tests are #[ignore] (they need a TUN in a private netns:",
    "`podman unshare unshare -n <test bin> --ignored`); these arms prove they COMPILE, not that the",
    "link stays quiet. The loopback arm runs both of its tests."
  ],
  "arms": [
    {
      "package": "skidbladnir",
      "features": [
        "matrix"
      ],
      "test": "install_matrix_live"
    },
    {
      "package": "skidbladnir",
      "features": [
        "promote"
      ],
      "test": "promote_contract"
    },
    {
      "package": "skidbladnir",
      "features": [
        "matrix"
      ],
      "test": "vm_systemd_suite"
    },
    {
      "package": "skidbladnir",
      "features": [
        "matrix"
      ],
      "test": "vm_korp_provision"
    },
    {
      "package": "skidbladnir",
      "features": [
        "service-drive"
      ],
      "test": "service_kind_live"
    },
    {
      "package": "nordisk-pairing",
      "features": [
        "stderr-capture"
      ],
      "test": "stderr_reaches_the_boot_log"
    },
    {
      "package": "nordisk-narrate",
      "features": [
        "dev-narrate"
      ],
      "test": "loopback"
    },
    {
      "package": "nordisk-pairing",
      "features": [
        "installer-wireguard"
      ],
      "test": "installer_link_goes_quiet"
    },
    {
      "package": "nordisk-pairing",
      "features": [
        "installer-wireguard"
      ],
      "test": "two_readers_one_link"
    }
  ]
}