<!-- nornir:class: reasoning -->
# End-to-end conformance testing — real client ⇄ real holger (containers)
> Status: **DESIGN, for review.** The authoritative way to test that a znippy
> package actually serves: a real package-manager client pulling from a real
> holger, over the real wire protocol. mannequin is the mocked *world*, not the
> thing under test.
## The shape
```
┌─────────────────────────┐ ┌──────────────────────────────────┐
│ client container │ wire │ holger container │
│ "pretending to be a dev"│ ──────▶ │ holger serve (real binary) │
│ real cargo / npm / pip │ proto │ serves from a sealed .znippy │
│ / gem / conda │ │ [+ mannequin inside, if needed: │
│ registry → holger │ │ OIDC / PKI / upstream world] │
└─────────────────────────┘ └──────────────────────────────────┘
```
- **Client container** runs the *real* tool (e.g. `rust:slim` → `cargo fetch`,
`node` → `npm install`, `python` → `pip install`, `ruby` → `gem install`,
`conda`), with its registry/index pointed at the holger container. The test
asserts the install/resolve **succeeds** and yields the right
package/version/bytes. This exercises the full stack: `.znippy` → holger backend
→ typed view → wire protocol → the real client's parser — exactly where the
view fixes matter (npm scope, maven `-SNAPSHOT`/classifier, conda build string).
- **holger container** runs the real `holger serve` over a sealed `.znippy`
fixture. **mannequin runs inside/alongside it** only when the scenario needs the
outside world (OIDC token issuer, PKI CA, an upstream to proxy). For a pure
airgap read test, no mannequin is needed — holger just serves the sealed archive.
- **Why not mannequin for this:** mannequin mocks *upstreams/IdP/Nexus*. Package
*serving* conformance is about holger's own wire output vs a real client, so the
client and holger must be real. mannequin stays the mocked world (and the
human-dev "fire it up while testing" workflow stays valid).
## Who drives it
| Runner | Question | Cell |
|---|---|---|
| **test matrix** (`xtask/tests/holger_matrix.rs`) | *correctness* — does `npm install` (etc.) actually work against holger? | heavy `#[ignore]` cell per ecosystem, in the `--ignored` pass |
| **nornir bencher** (`xtask/examples/nornir-bench.rs`) | *throughput* — how fast does holger serve real clients at concurrency? | already containerised vs Nexus; extend to real-client pulls |
Both already use **`testcontainers`** — the Nexus comparison bench fires a real
Sonatype Nexus 3 (`bench-scenarios/src/nexus.rs`). Same machinery spins the
holger + client containers. **Fixture-gated:** when no Docker/Podman, record
**SKIP** (`is_skip` pattern), never fail off-CI. Heavy-on in the release gate.
## Layering (don't put everything in containers)
1. **Light pass — in-process, no Docker (every run):** typed-view round-trips
(`plugin_read_api_test`), auth logic via mannequin in-process (mTLS verify,
offline-OIDC JWT verify, RBAC `role_for`), the CPU multicore sanity cell.
2. **Heavy pass — containers, gated (release gate):** real-client ⇄ real-holger
conformance per ecosystem; real Keycloak (online OIDC) / real Nexus
(proxy/import parity).
3. **No-container gate:** `cargo tree -p static-holger` asserts no egress crate
linked (airgap-provable).
## Per-ecosystem client cells (the matrix grid to grow)
| ecosystem | client image | command | assert |
|---|---|---|---|
| rust | `rust:slim` | `cargo fetch` w/ `source.crates-io.replace-with` → holger | resolves + downloads the crate |
| npm | `node:slim` | `npm install <pkg>` w/ registry → holger | installs scoped + unscoped |
| python | `python:slim` | `pip install --index-url holger <pkg>` | wheel + sdist |
| maven | `maven` | `mvn dependency:get` → holger | GAV incl. `-SNAPSHOT` / classifier |
| gem | `ruby:slim` | `gem install --source holger <gem>` | compact-index resolves |
| conda | `continuumio/miniconda3` | `conda install -c holger <pkg>` | repodata.json resolves |
Each starts as one heavy gated cell; SKIP without a container runtime.
## Open
- Whether to bake holger+mannequin into one image or compose two; one image is
simplest for the matrix (holger serve + optional mannequin sidecar process).
- Fixture provenance: build the sealed `.znippy` in the light pass, mount it into
the holger container for the heavy pass (no rebuild).