Heimdall: specification (v0.1, 2026-10-30)

Heimdall is a Rust sandbox for running untrusted work: parsing, verifying or scanning data that came from somewhere you do not trust. It is generic. Holger is the first user, then Gunnar, and tunnr/rusternetes can use it as well. No Go and no gVisor.

The core idea: the software is probed while it runs, and the kernel-call allowlist is generated from what it actually did. That profile is checked in, enforced at runtime, and re-checked in CI.

1. Goals and non-goals

Goals

• Run a job (a worker program or a wasm guest) on untrusted input, so that a compromised job can touch nothing but its own input and output.
• Give the kernel the smallest possible attack surface: only the system calls the worker really uses, with fixed arguments where possible.
• Generate those allowlists automatically, by probing the worker at runtime. Ship them as reviewed, checked-in code, and fail CI when the worker's behaviour drifts.
• Return a signed result, so that the caller trusts only Heimdall's verdict and never parses the untrusted bytes itself.
• Pure Rust, static binaries, usable on a normal Linux host, inside the OS-less appliance (korp/tunnr as PID 1) and in Kubernetes (rusternetes).

Non-goals

• Not a general container platform; it is a job runner.
• No user-space kernel (gVisor-style). The protection comes from the smallest syscall set, the broker and wasm.
• No KVM requirement. A microVM may be added later as an optional fourth layer.

2. Three layers, stacked

┌─ Layer 1: youki container ───────────────────────────────────┐
│  generated seccomp allowlist (default action KILL_PROCESS)   │
│  Landlock: only the job's in/ (read) and out/ (write)        │
│  no-new-privs, all capabilities dropped, user namespace,     │
│  read-only root, tmpfs /tmp, no network (except fetch jobs), │
│  cgroup v2 limits (cpu, memory, pids, io), rlimits           │
│                                                              │
│  Layer 2: the syscall broker (seccomp USER_NOTIF)            │
│   the few risky calls left (openat, socket, connect, …)      │
│   pause the worker; a Rust supervisor checks the call        │
│   against the job policy and answers or opens on its behalf  │
│                                                              │
│  ┌─ Layer 3: wasmtime (optional per job) ──────────────────┐ │
│  │  the parser/verifier compiled to wasm32: NO WASI fs/net │ │
│  │  bytes in through memory, verdict + files out           │ │
│  │  fuel + epoch interruption + memory limit + timeout     │ │
│  └─────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘

• A wasm job: wasmtime is the only native code inside the container. Its own syscall profile is tiny and generated once for the host binary.
• A native job (code that cannot be wasm, such as ClamAV in C): layers 1 and 2 only, with a profile generated for that binary.

3. The profile generator ("probe, then generate")

This is the part tunnr/rusternetes asked about. It is generic: profiles belong to each workload, the generator to Heimdall.

3.1 Workflow (a Build Thing / Runa pipeline step)

1. Build the worker, release profile, for the target (x86_64 and aarch64; musl and glibc are separate profiles, because the libc changes the calls).
2. Probe: run the worker's probe corpus (its integration tests plus a fuzz corpus that drives error paths, large inputs, timeouts and signal handling) under Heimdall's tracer in learning mode.
   • The tracer uses seccomp USER_NOTIF with a "log and allow" policy for every syscall number, in Rust. There is no strace dependency and no ptrace by default; ptrace is only a fallback for kernels before 5.0.
   • For each call it records: the syscall number, the arch, and the argument facts that matter. These are the socket domain/type/protocol, openat flags (O_RDONLY vs write/create), ioctl request numbers, mmap/mprotect protection bits (refuse PROT_EXEC|PROT_WRITE together), clone flags, prctl options and fcntl commands.
   • vDSO calls (clock_gettime and friends) never reach seccomp. That is fine, but they are noted.
3. Merge and minimise: take the union over all probe runs, then reduce to the smallest rule set. That means exact argument matches where the observed values form a small set, masks for flag fields, and a bare allow only where arguments vary freely (read/write on already-open fds).
4. Generate code:
   • profiles/<workload>/<arch>-<libc>.rs: a Rust const table the runtime compiles into a BPF filter (crate seccompiler, or Heimdall's own BPF builder);
   • the same profile as reviewable JSON (.json), for humans and diffs.
   • Default action: SECCOMP_RET_KILL_PROCESS, plus an audit log entry. The few calls the broker should see become SECCOMP_RET_USER_NOTIF instead of a plain allow (see §4).
5. Review: the generated diff is committed by a person, never auto-merged into a release branch.
6. Drift check in CI: every CI run re-probes in learning mode and fails if the observed set is not a subset of the checked-in profile. The failure names the new calls and the test that triggered them. The fix is regenerating and reviewing; the profile is never widened by hand without a probe.

3.2 What "hard-coded for our purpose" means

The profile is fixed per workload version. It changes only when the workload's code or toolchain changes, and CI forces that. Nothing is learned at runtime in production: production always runs in enforcing mode.

3.3 Coverage risk and how it is handled

A path the probe corpus never ran is missing from the profile, so production kills the worker on it. That fails closed, which is the safe direction. Mitigations:

• fuzz the worker during probing (cargo-fuzz corpus replay) so error paths run;
• a "near-miss" report: calls seen only in error paths are listed separately for review;
• a staging run in audit mode (SECCOMP_RET_LOG for unknown calls instead of kill), only on test boxes, never in production.

4. The syscall broker (layer 2)

• It uses SECCOMP_RET_USER_NOTIF (kernel ≥ 5.0) and SECCOMP_IOCTL_NOTIF_ADDFD (≥ 5.9).
• Brokered calls: openat/openat2, socket, connect, bind, and optionally execve (normally denied outright).
• The policy per job: allowed path prefixes (the job's in/ read-only, out/ write), allowed address/port for a fetch job, and nothing else.
• TOCTOU-safe: the broker never lets the worker's own call proceed after a check on a pointer the worker can change. It reads the arguments, checks NOTIF_ID_VALID, opens the file or socket itself and injects the fd with NOTIF_ADDFD. The worker gets the fd; the kernel never acts on the worker's racy pointer.
• Every decision is logged (job id, call, args, verdict). A refused call answers with EACCES, or kills the worker, per policy.

5. The container (layer 1)

• The runtime is youki (crate libcontainer) used as a library; no podman required. A crun back end is optional, for hosts that already have it.
• Namespaces: user, mount, pid, ipc, uts, net (an empty network namespace except for fetch jobs), and a cgroup namespace.
• Filesystem: a read-only root image pinned by digest and verified (sha256) at every start, not only the first, plus tmpfs /tmp. The input is bound read-only and the output write-only into a fresh directory per job.
• Landlock (ABI v1–v4 where available): read access to /in and the image, write access to /out and /tmp, and for ABI v4 the network ports of a fetch job.
• Privileges: no-new-privs, all capabilities dropped, rlimits (nofile, nproc, fsize, core = 0), and cgroup v2 limits on memory, cpu, pids and io.
• One fresh container per job, destroyed after it; no shared long-running daemons inside the sandbox.
• On the appliance: the image lives on a read-only, verified medium, and environment overrides like HOLGER_SANDBOX_MEDIA_DIR are ignored once korp booted the box.

6. The wasm host (layer 3)

• wasmtime with no WASI filesystem or network. Job I/O goes through a small WIT interface (component model): run(input: list<u8>, params) -> result<output, refusal> and emit-file(name, bytes).
• Limits: fuel, epoch interruption with a wall-clock timeout, a memory limit, a table/instance limit, and a stack limit.
• Guests are signed (the same signing as Holger's wasm plugins: .wasm.p7s + .wasm.blake3) and verified before load.
• A trap or a limit hit gives refused: guest trapped (<reason>), never a host crash.

7. Job API and the signed result

struct Job { kind: JobKind, input_dir: PathBuf, params: Params, policy: Policy }
enum JobKind { Wasm { module: GuestRef }, Native { workload: WorkloadRef } }
struct Policy { read: Vec<PathBuf>, write: Vec<PathBuf>, net: Option<NetAllow>, limits: Limits }
struct JobResult { job_id, verdict: Accepted | Refused { reason }, outputs: Vec<(name, sha256)>,
                   events: Vec<BrokerEvent>, started, finished }
struct SignedResult { result: JobResult, sig: Ed25519Signature, key_id }

• The result is signed with a per-boot Ed25519 key. Its public key is handed to the caller at boot, out of band (a file the caller pins, or korp's boot record).
• The caller accepts only a SignedResult whose signature verifies with the pinned key, and only the outputs it lists with matching hashes.
• heimdall-cli:
  • heimdall run --job <kind> --in <dir> --out <dir> [--policy <file>]
  • heimdall verify-result <file>
  • heimdall profile probe <workload> / heimdall profile gen <workload> / heimdall profile check <workload>

8. Repository and crates

git.gunnar.rs/nordisk/heimdall, public, AGPL-3.0-only plus commercial (Ignalina ApS and Vetra Automation AB 50/50, like holger), CLA like holger's.

9. Consumers

• Holger:
  • verify-bundle (a wasm guest: the bundle signature Ed25519/X.509, hashes, size caps, the znippy index, DSSE before payload parsing);
  • the scanners (ClamAV native, the others as wasm where possible);
  • the fetch job of the fetching downstream (network only to its one upstream; writes into the inbox). The server never parses untrusted bytes.
• Gunnar (later): pushed pack parsing, LFS object checks, the encrypted-repository policy checks.
• tunnr / rusternetes:
  • the probe-then-generate step as a Build Thing / Runa pipeline step for any service: build, probe with the test corpus, generate the profile, review, then enforce at deploy;
  • a Kubernetes RuntimeClass heimdall that runs pods through heimdall-run (youki + profile + Landlock);
  • the profile shipped with the image (an OCI annotation or a sidecar file pinned by digest).

10. Kernel requirements and fallbacks

Heimdall reports the features it actually got on every start, and each signed result names them.

11. Tests (test-first, every one RED before GREEN)

• Wasm: a good input gives a signed Accepted; a tampered input gives a signed Refused with the reason; a malformed input traps the guest, and the host returns Refused with no crash; fuel and timeout limits hold.
• Seccomp: a call outside the profile kills the worker, and the event is logged.
• Landlock: writing outside /out is refused; reading outside /in is refused.
• Broker: an open outside policy is refused; a TOCTOU race test (the pointer changes after notify) never opens the wrong file.
• Fetch job: it reaches only its allowed address; any other connect is refused.
• Signatures: a forged or replayed SignedResult is refused by verify-result.
• Probe: generating twice from the same corpus gives the same profile (deterministic); adding a new syscall to the worker fails profile check and names the test that triggered it.
• Image: a changed rootfs image is refused at the next start.
• Per arch: x86_64 and aarch64 profiles both enforce.

12. Milestones

1. The repository, heimdall-core, heimdall-wasm and the verify-bundle guest. Pure Rust, no syscall work: the most value, the least risk.
2. heimdall-container: youki, Landlock, cgroups, image verify, with a fixed hand-reviewed profile.
3. heimdall-probe: learning mode, the generator, the CI drift check, as a Runa pipeline step.
4. heimdall-broker: USER_NOTIF + ADDFD.
5. Consumers: Holger verify and scan, then the fetch job; the tunnr/rusternetes RuntimeClass; Gunnar later.

13. Rules for whoever builds it

• Rust only; static binaries; no Go.
• Test-first, RED then GREEN commits. Author rickard@ignalina.dk, no AI trailers on git.gunnar.rs repositories, never force-push.
• Every brief and README states the purpose plainly: Heimdall is a defensive sandbox. Its tests exercise its own containment, never anything outside it.