# knut — guide (f0, the whole workflow)
> **Class:** `guide` · **Function axis:** `f0` — the end-to-end session that spans several
> functions. Each function has its own `*-guide.md` for the detail; this file is the spine that
> runs them in order. The *what* is in [`design.md`](design.md); the *why* in
> [`reasoning.md`](reasoning.md).
This is the 10-minute path from nothing to a live, queryable graph that stays fresh under the
query. Every step links to the function chapter that goes deep.
## The whole session, in order
### 1 — Start the backing services
**First, work out how little you actually need.** Not every knut path wants a service; the
Spark-free Iceberg read wants *none*. This map is the answer to "what do I have to start
before this works", per path:
The short version: an **embedded** box is a crate you link or a directory on disk — nothing to
start. A **service** box is a process that must already be running. Reading Iceberg with the
`skade` feature needs zero services; adding the FalkorDB graph sink needs exactly one; only the
Spark Connect lane needs a JVM. Transfer methods into the graph are detailed in
[`falkordb-guide.md`](falkordb-guide.md), the Iceberg lanes in
[`bifrost-guide.md`](bifrost-guide.md).
knut talks to FalkorDB (on **Valkey 9**) for the graph and, for the lakehouse path, a
Spark+Iceberg endpoint. The Containerfiles that back the live test matrix are in
`containers/` (`falkordb-valkey9`, `spark-iceberg`):
```bash
podman build -t falkordb-valkey9 containers/falkordb-valkey9
podman run -d --name falkordb -p 6379:6379 falkordb-valkey9
```
Or let a `--features autostart` build bring FalkorDB up from Rust (testcontainers → Podman).
### 2 — Get a graph in (bulk load)
Two easy on-ramps, both **knut-native** (no `falkordb_bulk_loader`):
```bash
# The synthetic 12-label Swedish-population graph, straight to FalkorDB:
cargo run -p graphar-falkordb --release --example population
# The same load with a live per-label progress GUI + headless introspection:
cargo run -p knut-popviz --features gui -- --target falkor --scale 1.0 --autostart
```
Under the hood this is Arrow batches → `FalkorDbLoader::insert_vertices/_edges` (one
`UNWIND … CREATE/MERGE` per batch). Detail: [falkordb-guide.md](falkordb-guide.md),
[popviz-guide.md](popviz-guide.md). To persist the same batches as files first, write GraphAr
chunks with `graphar::io::write_chunk` — [graphar-guide.md](graphar-guide.md).
### 3 — Serve the graph over Arrow Flight SQL
Front the graph with a Flight SQL server so any ADBC/ODBC client (including Power BI) can read
it. Name the queries you want to expose as "tables":
```rust
let mut registry = SchemaRegistry::new();
registry.register_named("people", "MATCH (p:Person) RETURN p._gar_id, p.name", schema);
run_server(redis_url, "pop", registry, "0.0.0.0:50051").await?;
```
Add TLS / bearer / Basic / mTLS with `run_server_with(.., ServerOptions { .. })`. Full API +
the bearer-rotation and per-query-authz workflow: [flight-guide.md](flight-guide.md).
### 4 — Query it (any Arrow client)
```bash
# The thin Python client → pandas / NetworkX:
python -m knut.query "SELECT * FROM people"
```
A generic `SELECT * FROM people` is rewritten to the registered Cypher server-side; the rows
stream back as Arrow. Power BI connects with the stock ADBC/Flight SQL driver.
To pull a graph straight into **NetworkX** for analysis in Python (shortest path, centrality, …),
`KnutClient.to_networkx(vertex_query, edge_query)` walks FalkorDB → Arrow → pandas → NetworkX in one
call — see [networkx-export.md](networkx-export.md) and `python/examples/social_graph.py`.
### 5 — Keep it fresh from the lakehouse (Bifröst CDC)
Point Bifröst at a file-drop directory and it materialises rows into the same graph on a poll
loop, idempotently:
```bash
bifrost gen-sample --files ./drop --table lake.people --rows 100000 # try it with no lakehouse
bifrost run --mapping mapping.yml --files ./drop --falkor redis://localhost:6379 --graph pop --interval 30
```
The `MERGE` sink means the live query in step 4 keeps working while the graph updates.
Mapping YAML, CDC, the log contract, and the state ledger: [bifrost-guide.md](bifrost-guide.md).
> **The Iceberg sources are library-API today, not CLI.** The shipped `bifrost` binary wires
> `FileSource` only — there is no `--warehouse` / `--spark` flag. To watch an Iceberg table you
> call `knut_bifrost::listener::watch_iceberg_delta(...)` (Spark-free, `--features skade`) or
> assemble `Listener::new(spec, reg, SparkConnectSource::connect(..), sink, state)` from your own
> `main`. Which of those needs a service running is lane A vs lane C on the map in step 1.
> Both lanes end to end, the transfer-method menu into FalkorDB, and the no-pushdown caveat:
> [iceberg-falkordb-guide.md](iceberg-falkordb-guide.md).
### Alongside — author + watch the compute
The compute that *fills* the lake is a pipeline. Watch a Spark Declarative Pipeline or any
Spark Connect job with `knutpipe` ([pipelines-guide.md](pipelines-guide.md)):
```bash
knutpipe watch sc://localhost:15002 --sql "SELECT count(*) FROM lake.people"
```
Or author the pipeline itself in Þund's one IR and run it on the native or Spark backend —
[thund-guide.md](thund-guide.md).
## Where each capability lives
| you want to… | function | chapter |
|---|---|---|
| write/read columnar graph files (5 formats, local or S3) | graphar | [graphar-guide.md](graphar-guide.md) |
| bulk-load Arrow into a live graph | falkordb | [falkordb-guide.md](falkordb-guide.md) |
| serve Cypher over Flight SQL / export a query to Iceberg | flight | [flight-guide.md](flight-guide.md) |
| stream a lakehouse table into the graph by CDC | bifrost | [bifrost-guide.md](bifrost-guide.md) |
| author + run a batch/streaming dataflow | thund | [thund-guide.md](thund-guide.md) |
| view a Spark pipeline / job | pipelines | [pipelines-guide.md](pipelines-guide.md) |
| generate + load the demo population with a live GUI | popviz | [popviz-guide.md](popviz-guide.md) |
Never done this before? [for-idiots.md](for-idiots.md) spells out every keystroke.