OpenZL inside the verify-bundle guest
Heimdall is a defensive sandbox. Holger packs its update bundles with znippy,
and znippy compresses members with OpenZL, a C library. The pure-Rust guest
used to refuse every such member (heimdall-bundle-3), so a real Holger bundle
could not be verified in Heimdall. This spike (lane zlwasm, 2026-10-30) compiles
OpenZL's DECODER into the guest's wasm32 module.
Result: it works. Real Holger-packed bundles are accepted inside the wasm
host with the same report Holger's native verifier gives, and the guest still
imports nothing but the WIT emit-file.
How
| Piece | What |
|---|---|
| Feature | openzl in guests/verify-bundle, on by default. --no-default-features keeps the old refusal by name. |
| C source | guests/verify-bundle/openzl/: 115 .c + 268 .h from openzl-sys-rs 0.4.1+openzl.0.3.0 (OpenZL 0.3.0, zstd 1.5.7, lz4 1.10.0), exactly the units wasm-ld --trace pulls in for ZL_getDecompressedSize + ZL_decompress. vendor.sh re-creates the tree from the crate. |
| Patch | One: OpenZL static-asserts a 64-bit size_t only to keep its heap-arena header 16 bytes; wasm32 pads the header instead (openzl/wasm32.patch). |
| Compiler | Debian clang 19 (in the toolchain base image), --target=wasm32-wasip1 -O2 -DNDEBUG, against wasi-libc 0.0~git20241209.574b88d-2 HEADERS only (ci/ci.Containerfile). No wasi-libc object is linked. |
| libc | malloc/calloc/realloc/free = the guest's Rust allocator (src/zl.rs, 16-byte header, NULL on failure); memcpy/memmove/memset = Rust compiler-builtins; fprintf/vfprintf/fputc/stderr/vsnprintf write nothing and abort traps (openzl/shim/hzl.c, 90 lines). No stdio, threads, time, or WASI. |
| ABI | hzl_frame_size/hzl_decode return scalars only (ZL_Report is a struct by value; Rust never sees it). |
| Native builds | (tests, fuzz) link openzl-sys-rs 0.4.1 — the library Holger itself links — instead of the vendored copy. |
| Proof of no imports | wasm_e2e::the_guest_module_imports_only_the_wit_emit_file_even_with_openzls_c_linked_in parses the core module: exactly $root::emit-file. |
Frame checks (src/znippy.rs)
- Member caps, the archive's 100:1 expansion ratio and the release manifest's member size are checked from the index BEFORE anything is read (unchanged).
- The member buffer is reserved once at its declared size (
try_reserve_exact; a size the guest cannot address or allocate isholger-updatelimit-exceeded). - Per compressed chunk: the frame header's decompressed size must EQUAL the
chunk's
uncompressed_sizefrom the index, before the decoder runs; a frame that claims more (a hidden bomb) or less is refused without decoding. - The decoder writes into exactly that slice and must fill all of it.
- blake3 of the decoded bytes must equal the index checksum (znippy hashes the uncompressed bytes), then the release manifest's SHA-256/BLAKE3 as before.
Decode failures are holger-update-28 (container: "cannot extract bundle
member"), the code Holger's own verifier gives for the same files.
Results
| Measure | Value |
|---|---|
Guest tests (cargo test --release, guests/verify-bundle) | 173 (157 before + 16), all green on oden (scripts/ci.sh); on dwarf all green except znippys_real_writer_splits_a_big_member_into_several_chunks..., which also fails on unmodified main on dwarf (znippy picks its chunk size from host memory; host-dependent, not this change) |
| New tests | tests/openzl.rs 13 (Holger-packed accepted with Holger's report, multi-chunk, format 27, every-byte bit flips agree with Holger, truncation, size lies, hidden bomb, honest bomb, stored-flagged-compressed, 200 garbage frames, the fuzz find natively and as a zero-length tail chunk); wasm_e2e 4 (no imports, Holger-packed accepted in the host = library output, 4 MB bundle, 11 hostile frames incl. the fuzz find refused, never trapped) |
| Pinning tests flipped | good_bundles (now Accepted like Holger), archive (stored-flagged-compressed is a container refusal), wasm_e2e (Accepted in the sandbox) |
Guest .wasm (release, opt-level = "s", LTO) | 1,330,909 B without openzl → 2,421,038 B with (+1.09 MB) |
| ~8 MB bundle, full verification, native vs wasm | stored 23.0 / 174.7 ms, OpenZL 31.5 / 187.7 ms (oden) |
Fuzz openzl_frame, 30 min oden, ASan C | ~47.9 M execs, 0 crashes (after one allocation finding, below) |
| scripts/ci.sh on oden (own image tag heim-zlwasm) | green, 801 tests passed, 0 failed |
Speed
Measured with cargo test --release --test wasm_e2e -- --ignored --nocapture measure on oden (quiet, 5 runs each, fuel raised). Payload: 8,000,000 bytes
of word-salad text (OpenZL packs it about 3:1). Times are the WHOLE verification
(index, signatures, digests, emit), mean of 5.
| Bundle | File size | Native (library) | wasm (in the Heimdall host) |
|---|---|---|---|
| stored members | 8,018,372 B | 23.0 ms | 174.7 ms |
| OpenZL members (Holger's packer) | 2,641,773 B | 31.5 ms | 187.7 ms |
| difference ≈ OpenZL decode of 8 MB | ≈ 8.5 ms | ≈ 13 ms |
So the C decoder in wasm costs about 1.5x its native time; the guest's total is dominated by hashing in wasm (about 7.5x native), not by OpenZL. On loaded dwarf: stored 33.5 / 310.8 ms, OpenZL 67.4 / 349.1 ms.
The default RunLimits fuel (2e9) is spent by an 8 MB bundle even when it is
STORED (hashing in wasm); bundles beyond a few MB need a larger fuel budget in
the job policy regardless of OpenZL.
Fuzz
guests/verify-bundle/fuzz target openzl_frame (header size, then a bounded
decode), native build with the C under ASan + libFuzzer coverage
(CC=clang CFLAGS="-fsanitize=address,fuzzer-no-link"), oden, nice 19:
| Run | Time | Execs | Result |
|---|---|---|---|
| 1: ASan defaults (a huge malloc aborts) | 11 s | 294,165 | 1 finding, below |
2: malloc returns NULL above 2 GiB, as the guest's allocator does; -fork=4 | 1,816 s | ~47.9 M | 0 crashes, 0 OOM, 0 timeouts; cov 3911, features 11477, corpus 2763 |
The finding (kept as fuzz/seeds/openzl_frame/regress-constant-stream-alloc
and three regression tests): a 36-byte frame whose header declares 0 bytes, so
it passes the header-size check, but whose constant codec makes OpenZL ask
malloc for ~2^64 bytes (DI_constant_typed -> DCTX_newStream -> STREAM_reserve). Under ASan's defaults that is allocation-size-too-big; with
a real malloc it returns NULL, OpenZL reports Allocation (70), and the member
is refused (holger-update-28, Holger agrees). It is not a memory-safety bug in
OpenZL, our glue, or the size checks. In the guest the Rust-backed malloc
returns NULL (size overflow, or past the memory limit), so it can only be a
refusal; it is never a trap.
What it shows: OpenZL's INTERNAL stream sizes come from the frame's graph, not from the header size the reader bounds. Inside the guest the job's memory limit (256 MiB default) caps them. In Holger's NATIVE verifier nothing caps them, so a frame could make Holger allocate (and fill) up to what the host grants; that is worth a follow-up in znippy (run OpenZL with a bounded allocator, or verify bundles only through Heimdall).
Risk that stays (owner decision)
The C decoder shares ONE linear memory with the Rust verifier. wasm keeps every C bug inside the guest instance (no host access, no code injection: a bad access past memory traps), but a C memory-corruption bug that writes inside linear memory could in principle alter the verifier's own data (for example a digest it is about to compare) and so its verdict, which Heimdall then signs. Mitigations, in order of strength:
- Separate instance (recommended next step): build the decoder as its own
core module with its own memory, exporting
decode(frame) -> bytes, and compose it with the verifier (wit-component/wac) so the verifier calls it through the canonical ABI. A corrupted decoder can then only return wrong bytes, which the blake3 + manifest digests catch. The composed component still imports nothing but the WIT. This is a design change (two modules in one guest component), so it is left to the owner. - Fuzz the 32-bit build itself through the wasm host (the native fuzz runs a 64-bit build of the same C; the one patch is the 32-bit difference).
- The fallback below removes the C entirely.
Fallback (not built)
If OpenZL in the guest is not wanted: zstd for bundles — Holger packs bundle
members with C zstd (znippy codec switch for .holger.znippy only), and the
guest unpacks with the pure-Rust ruzstd decoder. No C in the guest, at the
cost of a second codec in znippy and a bundle format flag.