# knut-popviz — guide
> **Class:** `guide` · audience: a techie running the population loader or driving
> `run_load` / the introspection oracle. The *what* is in
> [`popviz-design.md`](popviz-design.md). This file is the workflow.
>
> **Reuse:** `popviz.uses = [falkordb, graphar, popsim]` — see [falkordb](falkordb-design.md), [graphar](graphar-design.md).
## Run the GUI
```sh
# FalkorDB target (default build has no display-less deps beyond egui):
cargo run -p knut-popviz --features gui -- \
--target falkor --scale 1.0 [--drop] [--autostart]
# Iceberg target (needs the iceberg feature + a warehouse dir):
cargo run -p knut-popviz --features gui,iceberg -- \
--target iceberg --warehouse /tmp/wh --scale 1.0 [--autostart]
```
You get a progress bar per label (the 12 register labels A–L), overall `rows/s`,
and a Start button. `--autostart` (or `KNUT_POPVIZ_AUTOSTART`) kicks the load off
on the first frame — no click — which is exactly what the headless smoke tier
uses. `--start-db` (or `KNUT_FALKORDB_AUTOSTART`) boots a FalkorDB container first
for the `falkor` target.
| flag | does |
|---|---|
| `--target falkor\|iceberg` | the sink (`Target::Falkor` / `Target::Iceberg{warehouse}`) |
| `--warehouse DIR` | Iceberg warehouse dir (iceberg target) |
| `--scale F` | population scale — per-label totals are `LABELS[i] × scale` |
| `--drop` | drop the target graph/tables before loading |
| `--autostart` | load on first frame (robot tier) |
## The headless `run_load` API
The whole load logic is in the default build with no egui — call it directly:
```rust
use knut_popviz::{run_load, LoadParams, PopvizState, Target};
use std::sync::{Arc, Mutex};
let params = LoadParams {
url: "redis://127.0.0.1:6379".into(),
graph: "pop".into(),
scale: 1.0,
drop: true,
target: Target::Falkor, // or Target::Iceberg { warehouse }
};
let state = Arc::new(Mutex::new(PopvizState::new(params.scale))); // lib.rs:90
// run_load is blocking — put it on its own thread; `wake` repaints a UI (no-op here):
let st = state.clone();
std::thread::spawn(move || run_load(params, st, || {})); // lib.rs:250
// poll the shared state:
loop {
let s = state.lock().unwrap();
if s.phase == knut_popviz::Phase::Done { break; }
if s.phase == knut_popviz::Phase::Failed { panic!("{:?}", s.error); }
// drop(s); sleep …
}
```
`PopvizState::new(scale)` sizes every `LabelProgress` (`total = LABELS[i] ×
scale`, `loaded = 0`); `run_load` connects, generates every label's Arrow batches
in parallel (rayon `par_iter`), then loads each label **concurrently** via
`insert_vertices_multi_par(…, concurrency, on_progress)` (up to
`available_parallelism()` queries in flight), a barrier, the edges via
`insert_edges_keyed_par`, then `create_index(label, "_gar_id")` per label, ending
in `Phase::Done` (or `Phase::Failed` with `error`). Both the sim (gen) and the
sink (insert) are multicore — the fix for the old ~1-core serial ingest.
## The introspection oracle
`PopvizState::state_json()` (`lib.rs:132`) is the external-state oracle — assert
on it with no display. The GUI writes it to `$KNUT_POPVIZ_STATE` every ~300 ms
(default `<tmp>`; overwritten in place). It reports:
```json
{ "phase": "loading vertices", "target": "falkor",
"total": 11000000, "loaded": 250000, "fraction": 0.022,
"rows_per_s": 24800.0, "elapsed_s": 10.1,
"labels": { "Person": {"loaded": 250000, "total": 8000000}, "Address": {…}, … },
"error": null, "placeholders": [] }
```
Poll the file (or the shared state) as your post-condition: `phase: "done"` with
every label `loaded == total`. `placeholders` carries `"not started"` while idle —
assert it empty once data is expected.
## The robot-test matrix (`tests/robot.rs`)
Every tier asserts on the **same** `state_json()` oracle through the shared
`assert_done_state` bar (`robot.rs:30`), so asserting the JSON is asserting what
the GUI shows. Three of the four tiers run **unattended in CI — no container**:
| tier | what | how to run |
|---|---|---|
| **data layer** (no display) | `state_json` unit tests over `PopvizState` — sizing, fraction, phase transitions, `placeholders` | `cargo test -p knut-popviz` |
| **synthetic, container-free** | `robot_synthetic_load_reaches_done` / `_surfaces_failure` (`robot.rs:56`,`:116`) — field-pokes `PopvizState` through idle → vertices → indexing → done (and the failure path), no `run_load`, **no FalkorDB, no Podman** | `cargo test -p knut-popviz --test robot` |
| **no-DB end-to-end** | `robot_iceberg_load_reaches_done_no_db` (`robot.rs:147`, `#[cfg(feature = "iceberg")]`) — drives the *real* `run_load` into a scratch Iceberg warehouse (skade, `tempfile::tempdir`) and asserts `done`. Genuine end-to-end, **still no DB / no container** | `cargo test -p knut-popviz --features iceberg --test robot` |
| **live FalkorDB** | `robot_load_matrix_reaches_done` (`robot.rs:174`, `#[ignore]`) — the *real* `run_load` over a scale matrix into a live FalkorDB | needs FalkorDB at `$KNUT_FALKORDB_URL`; `cargo test -p knut-popviz --test robot -- --ignored` |
The no-DB Iceberg tier is the one that exercises `run_load` **end to end without
any external service** (the synthetic tier pokes state; the live tier needs a DB) —
so CI holds the full driver, not just the UI contract, to the `done` bar.
The live-GUI smoke variant runs the binary with `--autostart` + `KNUT_POPVIZ_STATE`
set and polls the JSON file — no window needed.
## The popsim model as usage (determinism note)
`knut-popviz` re-exports the shared `knut-popsim` model (`LABELS`, `geo`,
`EKONOMISKT_BISTAND_RATE`). Generate a graph directly when you want the model, not
the load:
```rust
use knut_popsim::{simulate, schema, SimParams};
let g = simulate(SimParams { n_persons: 1_000, n_kommun: 20, seed: 0xC0FFEE });
// g.nodes / g.edges — a generic PopGraph, no backend types
let s = schema(); // the 12-label schema graph (11 hub + 4 cross edges)
```
| item | signature |
|---|---|
| `simulate` | `fn simulate(SimParams) -> PopGraph` |
| `schema` | `fn schema() -> PopGraph` |
| `SimParams` | `{ n_persons: u32, n_kommun: u16, seed: u64 }` |
| `LABELS` | `const LABELS: [(&str, i64); 12]` |
| geo | `geo::{address_for, is_ekonomiskt_bistand, cluster_hotspots, CITIES, SWEDEN_BBOX}` |
**Determinism is a contract:** `simulate` uses a seeded LCG (no `rand`), so the
same `SimParams` yields a byte-identical graph; geo placement is splitmix64 seeded
by node id, so a given person always lands in the same place. That is what lets
the loader and the browser demo agree, and what makes the robot matrix
reproducible.
## Notes
- Throughput is bounded by FalkorDB's online write path, but the client no longer
is: `run_load` now generates batches on all cores (rayon) and keeps up to
`available_parallelism()` `UNWIND-CREATE` queries in flight (`insert_*_par` over
the multiplexed link) instead of one-at-a-time — lifting the ingest off ~1 core.
The graph produced is byte-identical to the serial path. See
[`falkordb-design.md`](falkordb-design.md) (f3a — the parallel bulk path).
- The viz schema is generic (`_gar_id` + a couple columns) — it watches
*throughput*, not the faithful per-label schema the `population` example carries.