Gunnar

mikronova/funnel-core

.nornir/guide.md

<!-- nornir:class: guide -->
# funnel-core — guide

> How to drive the crate. Everything is a pure function over a `Funnel` you build
> by applying events.

## 1. Build a funnel from events

```rust
use funnel_core::{Funnel, Event, ItemKind, IdeaId, PlanId, NodeId};
use chrono::Utc;

let mut f = Funnel::default();
let now = Utc::now();
f.apply(&Event::IdeaSubmitted {
    id: IdeaId::seq(1), source: "human:rickard".into(),
    text: "Add a #board view".into(), refs: vec![],
    item_kind: ItemKind::Idea, ts: now,
})?;
```

`apply` validates (unknown id, duplicate edge, self-loop, **or a cycle-closing
edge** ⇒ `Err`) and mutates in-memory state. The caller appends the accepted event
to its own log. The cycle guard matters: a plan whose edges form a loop can never
drain (`topo_ready` / `simulate_flow` skip it), so a `EdgeAdded from → to` is
rejected when `to` already reaches `from` — the plan stays a DAG by construction.

## 2. Project the views

| call | returns | use |
|------|---------|-----|
| `history(&f, kind, status)` | `Vec<HistoryItem>` | the newest-first intake list |
| `cards_from_funnel(&f)` | `Vec<Card>` | every idea/error/test + every task node as a card |
| `board(&cards)` | `Board` | the five fixed lanes (Inbox/Accepted/Proposed/Planned/Dropped) |
| `board_to_json(&b)` | `serde_json::Value` | the headless test / viz `state_json` |
| `query_cards(&cards, q)` | `Vec<Card>` | filter + sort by a smart query |
| `topo_ready(&mut f)` | `Vec<NextStep>` | actionable nodes, dependencies first |
| `simulate_flow(&mut f, steps)` | `FlowSim` | time-step the DAG to completion |
| `PlanDag::from_funnel(&f, sel)` | `PlanDag` | nodes + edges + topo order for the viz |
| `dag.critical_path()` | `Vec<String>` | the longest dependency chain (bottleneck lineage) |
| `dag.critical_path_len()` | `usize` | its edge count = min sequential waves to finish |
| `dag.critical_path_deps()` | `Vec<String>` | Dep-only twin — longest task→task chain, mother lineage ignored |
| `dag.critical_path_deps_len()` | `usize` | its edge count = pure dependency wave count (≤ `critical_path_len`) |
| `dag.depths()` | `BTreeMap<String,u64>` | per-node longest-path depth = layout layer / wave |
| `dag.layers()` | `Vec<Vec<String>>` | the layered layout — node ids bucketed by depth (index = render column) |
| `dag.width()` | `usize` | peak layout width = the most cards in one column = peak concurrent work |
| `dag.slack()` | `BTreeMap<String,u64>` | per-node float — waves a node may slip; `0` = critical |
| `dag.critical_nodes()` | `Vec<String>` | every zero-slack node (⊇ a single `critical_path`) |
| `dag.bottlenecks()` | `Vec<Bottleneck>` | nodes ranked by downstream reach + fan-in/out — "which node unblocks the most work" |
| `dag.diff(&baseline)` | `PlanDiff` | added/removed nodes+edges + changed nodes between two revisions |
| `DagGrowth::from_events(&evs, sel)` | `DagGrowth` | birth-stamped growth timeline (batch) — when each node/edge first entered the DAG |
| `DagGrowth::new(sel)` + `g.extend(&ev)` | `DagGrowth` / `()` | streaming growth — push one event per tick, get the new births with no full re-fold |
| `g.node_count_at(step)` / `g.edge_count_at(step)` | `usize` | the monotone growth curve — DAG size as of a step |

## 3. The smart query (YouTrack/JIRA style)

`query_cards(&cards, "type:bug is:open for:me sort:newest")`. Operators
(case-insensitive): `type:`/`kind:`, `lane:`/`status:`, `is:` (`open`+lanes),
`reporter:`, `assignee:` (`me` ⇒ rickard), `for:` (reporter OR assignee),
`label:`, `has:plan`/`no:plan`, `sort:` (`newest`/`oldest`/`id`). Any bare word is
a case-insensitive substring over id+title+body+reporter. Unknown operators
degrade to free text, so a query is never an error — it only narrows.

## 4. The prompt-decompose mode

An idea decomposed via `nornir funnel prompt` records how it ran with an
`Event::IdeaModeSet { mode }`. The token is kept free-form on the wire, but read
it typed:

```rust
use funnel_core::FunnelMode;
let m = idea.mode_kind();                 // Option<FunnelMode> (None = unset/unknown)
FunnelMode::Auto.child_lane();            // Lane::Planned  — auto tasks join the plan
FunnelMode::Manual.child_lane();          // Lane::Proposed — manual tasks stage for review
```

`FunnelMode::child_lane()` is the single source of the **mode gate** (which lane a
decomposed child task lands in). `FunnelMode::parse` is forgiving — an unknown or
future token degrades to `None`, never an error.

## 5. Critical path + layout depth

`PlanDag::from_funnel` gives boxes and arrows; the critical-path vocabulary tells
you which chain sets the schedule and how to lay the nodes out:

```rust
let dag = funnel_core::PlanDag::from_funnel(&f, None);
dag.critical_path();        // ["i-001","n-001","n-002","n-003"] — the longest chain
dag.critical_path_len();    // 3  — min sequential waves the plan can finish in
dag.critical_path_deps();   // ["n-001","n-002","n-003"] — Dep-only, mother dropped
dag.critical_path_deps_len(); // 2 — pure task→task depth (≤ critical_path_len)
dag.depths()["n-003"];      // 3  — this node's layout layer (deps always sit left)
dag.layers();               // [["i-001"],["n-001","n-002"],["n-003"],["n-004"]] — columns
dag.width();                // 2  — the widest column = peak concurrent work
dag.slack()["n-001"];       // 2  — the lone short branch can slip 2 waves
dag.critical_nodes();       // ["i-001","n-002","n-003","n-004"] — the zero-slack set
```

**Slack (critical-path tension).** `depths()` gives each node its *earliest*
layer; `slack(n)` is how many layers later it could run without pushing out the
plan's finish (`D - earliest(n) - longest-chain-from(n)`). Zero slack ⇒ the node
lies on *some* longest path — it is **critical**, and `critical_nodes()` returns
that whole set. Unlike `critical_path()` (one chain), `critical_nodes()` catches
*every* critical node even when several equally-long paths exist (a diamond has no
non-critical node, yet a single path names only one branch). A scheduler spends a
node's slack as dispatch freedom; a replan protects the zero-slack set.

`critical_path()` is the deepest *lineage* (it always begins with a mother→child
hop); `critical_path_deps()` is the deepest pure *dependency* chain, decoupled
from decomposition — so a viz can show "how deep is the idea" vs "how deep is the
work". `depths()` gives each node its column; `layers()` inverts that into the
per-column card lists a renderer draws (index = column, dependencies always to the
left), and `width()` is the widest column — the canvas width to budget and the
peak parallelism a scheduler can exploit (`layers().len()` is the height =
`critical_path_len() + 1`). All appear in `dag.state_json()` for the headless/viz
surface.

![layered layout](assets/plandag-layers.svg)

![slack and the critical set](assets/plandag-slack.svg)

### 5a. Bottlenecks — which node unblocks the most work

Where `critical_path()` answers "how *long* is the schedule", `bottlenecks()`
answers "*which node* should a human clear or parallelise next":

```rust
let bn = dag.bottlenecks();              // ranked, most-constraining first
bn[0].id;                                // "i-001" — the decomposition root gates it all
bn[0].downstream;                        // count of work stalled behind it
bn.iter().find(|b| b.id == "n-005").unwrap().fan_out; // local divergence width
```

Each `Bottleneck { id, fan_in, fan_out, downstream }` scores a node three ways:
**fan_in** (in-degree — a join everything waits on), **fan_out** (out-degree — the
successors immediately freed when it finishes, a divergence a human can fan out to
workers), and **downstream** — the size of its whole transitive-descendant cone,
i.e. *all* the work stalled behind it. Rows are ranked `downstream` desc, then
`fan_out`, then `fan_in`, then id — so `bottlenecks().first()` is the single node
whose completion unblocks the most work. Downstream reach beats local fan-out: a
narrow node heading a deep chain outranks a wide node feeding three leaves. Every
node appears (an isolated leaf ranks last, all-zero); a viz filters to
`group.is_some()` for task-only constraints and highlights the head. Spans both
edge kinds, so "downstream" is everything a node gates — the same lineage
`critical_path` walks. One reverse-topological descendant-set pass, O(V·E) on the
cold render path. Surfaces in `dag.state_json()["bottlenecks"]`.

![bottleneck ranking](assets/plandag-bottlenecks.svg)

## 5b. Replan diff — what changed between two revisions

`PlanDag::diff` is the DAG twin of the coverage / bench diff gates: give it the
old plan as `baseline` and it reports the structural delta a replan step shows.

```rust
let baseline = funnel_core::PlanDag::from_funnel(&before, Some("p-001"));
let revised  = funnel_core::PlanDag::from_funnel(&after,  Some("p-001"));
let d = revised.diff(&baseline);

d.added_nodes;     // ids in the new plan only        (e.g. ["n-004"])
d.removed_nodes;   // ids gone from the new plan
d.added_edges;     // new dependency / decomposition edges
d.removed_edges;   // edges the revision dropped       (e.g. n-002 → n-003)
d.changed_nodes;   // survivors whose payload/wiring changed → [{id, changed:[...]}]
d.is_empty();      // true ⇒ an unchanged replan (no delta)
```

A **survivor** is a `changed_node` when any of its content fields (`title`,
`status`, `group`, `is_bug`) differs, or its **deps** — the set of edges incident
to it — changed. The derived layout metrics (`staleness_hours`, `weight_bytes`,
`fanout`) are intentionally NOT diffed: staleness is a clock artefact and
fanout/deps are already reported by `added_edges`/`removed_edges` + the `"deps"`
aspect. Everything is id/tuple-sorted, so `diff` is deterministic and
`d.state_json()` is a stable replan-gate surface.

![replan diff](assets/plandag-diff.svg)

## 5c. Growth timeline — draw the DAG growing live with history

Where `diff` compares two arbitrary revisions, `DagGrowth::from_events` is the
running growth of a *single* log: replay the events and it stamps every node and
edge with the **step** (0-based event index) + timestamp at which it FIRST
appeared. The funnel event log is append-only and never removes a node or edge, so
the DAG only ever grows — this is the temporal seam the live funnel pane draws the
graph *growing* along, and the history scrubber reads to show the DAG *as of* any
point in the log.

```rust
let g = funnel_core::DagGrowth::from_events(&events, None); // whole funnel

g.nodes;              // [{id:"i-001", step:0, at}, {id:"n-001", step:3, at}, …] — birth order
g.edges;              // mother edges born with their child; dep edges at their EdgeAdded
g.steps;              // total events replayed
g.node_count_at(3);   // how many nodes exist as of step 3 (monotone non-decreasing)
g.edge_count_at(7);   // the edge-count growth curve — a sparkline / progress bar
```

A renderer **plays the birth marks forward** to reveal each element as its step is
reached (the "DAG growing live" animation); a scrubber filters the marks by step
(or reads `node_count_at`/`edge_count_at`) to size the graph at any moment. It is
built on the same `Funnel::apply` fold + `PlanDag::from_funnel` projection every
other query uses (so it agrees with them by construction) and runs on the cold
render/history path. Scoping mirrors `from_funnel`: `Some(pid)` restricts the
timeline to one plan (its mother is born when its plan is created, not at intake).
An invariant-violating event — one the store would never have appended — births
nothing and leaves the DAG standing. `g.state_json()` carries the birth marks for
the headless/viz surface.

![growth timeline](assets/plandag-growth.svg)

### 5c-bis. Streaming growth — one event per tick (`extend`)

`from_events` is the **batch** path (cold load, history scrub): it re-projects the
whole DAG after every event, so a live pane that called it *every tick* would
re-replay the entire log each time (O(steps²) over a session). For the **live**
"growing right now" animation, open a stream and push one event per tick:

```rust
let mut g = funnel_core::DagGrowth::new(None);   // empty stream, whole funnel
for ev in incoming_ticks {                       // one event as it lands
    g.extend(ev);                                // stamps only the NEW births
    render(&g);                                  // draw the freshly-revealed nodes/edges
}
```

`extend` keeps a running `Funnel` accumulator inside the stream, applies just this
one event to it, projects the frame once, and appends only the nodes/edges seen for
the first time — O(`from_funnel`) per tick, no full re-fold. The result is
**byte-identical** to the batch fold: streaming a log event-by-event yields the same
`nodes`/`edges` (same order, same birth steps) as `from_events` on that log — a
tested equivalence guard. Growth stays monotone, each element is stamped at the step
it first appears (a new dep edge at its own `EdgeAdded`, not at an endpoint's birth),
and a rejected event births nothing yet still advances a step. Batch and stream are
also composable: a `from_events` timeline retains its accumulator, so you can
**hydrate** a pane from the log so far and then `extend` new ticks onto it.

| path | when | cost |
|---|---|---|
| `DagGrowth::from_events(&log, sel)` | cold load / full history scrub | one pass over the whole log |
| `DagGrowth::new(sel)` + `extend(ev)` per tick | live pane, one event at a time | O(`from_funnel`) per tick, no re-replay |

## 6. Wiring the plan to the code (Plan-DAG ↔ Component-DAG)

The Plan-DAG says *what was planned*; the **Component-DAG** (crates/modules, from
Mímir) says *what the code is*. `funnel_core::wiring` joins them so you can ask
"which code does this plan touch, and what is the blast radius?".

funnel-core never fetches the component graph — you hand it one through the
`ComponentGraph` trait (nornir implements it over `Mimir.DepsOf`/`DependentsOf`;
`ComponentDag` is the in-memory/fixture one):

```rust
use funnel_core::{ComponentDag, ComponentEdge, ComponentNode, ComponentGraph, ComponentId};

let graph = ComponentDag::new(
    vec![ComponentNode::krate("funnel-core").with_path("funnel-core/src"),
         ComponentNode::krate("nornir"),
         ComponentNode::krate("nornir-viz")],
    vec![ComponentEdge::dep("nornir", "funnel-core"),   // from DEPENDS ON to
         ComponentEdge::dep("nornir-viz", "nornir")],
);
graph.blast_radius(&ComponentId::new("funnel-core")); // ["nornir","nornir-viz"] — dependents
graph.dependency_closure(&ComponentId::new("nornir-viz")); // ["funnel-core","nornir"]
```

> **Direction.** `ComponentEdge { from, to }` = "`from` depends on `to`" (the Cargo /
> `dep_graph_edges` reading). That is the **opposite** of `PlanDagEdge`, where
> `from → to` means "`from` must finish first". `blast_radius` walks *dependents*.

### 6a. Draw wires, then traverse

```rust
use funnel_core::{draw_wire, reject_wire, Wiring};

f.apply(&draw_wire("n-001", "funnel-core", Some("drag".into()), now))?; // the pane's gesture
let w = Wiring::from_funnel(&f);

let r = w.blast_radius("n-001", &graph);       // one plan node
r.touched;                                     // ["funnel-core"] — bound + present
r.affected;                                    // ["nornir","nornir-viz"] — the ripple
r.is_complete();                               // true ⇒ the radius is the whole truth

let dag = funnel_core::PlanDag::from_funnel(&f, Some("p-001"));
let cone = w.cone_blast_radius(&dag, "n-001", &graph); // n-001 + every plan node after it
cone.unmapped_nodes;                           // plan nodes in the cone with NO binding
w.plan_nodes_for(&ComponentId::new("funnel-core")); // reverse: who claims this crate?
```

`BlastRadius::affected` being empty is **ambiguous on its own** — always read
`unmapped_nodes` / `is_complete()` with it. Empty + complete = "nothing downstream".
Empty + incomplete = "we do not know, the plan is not wired".

### 6b. The unmapped report — the honest half

```rust
use funnel_core::{WiringReport, UnmappedReason};

let rep = WiringReport::of(&dag, &w, &graph);
rep.is_fully_mapped();      // the gate: is EVERY plan-DAG node bound?
rep.coverage();             // 0.0 ..= 1.0
rep.unmapped_ids();         // ["i-001","n-002"] — nothing is ever silently dropped
rep.unmapped[0].reason;     // UnmappedReason::NoWire | OnlyProposed | OnlyRejected | Dangling
rep.done_but_unmapped();    // the loud row: Done task nodes nobody wired
rep.unplanned;              // Option<Vec<ComponentId>> — components no plan claims
```

Every Plan-DAG node lands in exactly one of `mapped` / `unmapped`, so the unmapped
list is trustworthy. The reasons distinguish a *gap* (`no_wire`) from a *pending
decision* (`only_proposed`, `only_rejected`) from a *data error* (`dangling` — a
confirmed wire pointing at a component the graph does not hold; it **looks** mapped
and is the loudest). `unplanned` is `Option` on purpose: `None` means the graph
cannot enumerate its nodes (a live RPC graph), which is not the same as "every
component is claimed".

| wire status | binds? | how you get it |
|---|---|---|
| `confirmed` | **yes** | `draw_wire(..)`, or confirming a suggestion |
| `proposed`  | no | `infer_declared`, a Mímir suggestion, an empty status on a non-drawn wire |
| `rejected`  | no | `reject_wire(..)` — appended, never deleted, so it is not re-proposed |

An unknown/future `origin` or `status` token **never binds** — the seam fails closed.

### 6c. Free proposals from `targets`

```rust
let evs = Wiring::infer_declared(&f, &graph, now); // Vec<Event>, deterministic order
for e in &evs { f.apply(e)?; }                     // then append to your log
```

Matches each plan node's file-glob `targets` against the components' `path`/`name`
(a target *inside* a component's path counts). Emits **proposals**, never bindings,
and skips any pair that already has a wire — including a rejected one — so
re-running is idempotent and never resurrects a thrown-away suggestion.

![plan ↔ component wiring](assets/dag-wiring.svg)

Full rationale + the nornir-side handoff: `dag-wiring-design.md`.

## 7. Determinism

Every projection is BTree-ordered and clock-free (staleness is measured against the
newest card in the set, never `now`), so re-projecting the same funnel is
bit-identical. That is what makes the board and Plan-DAG golden-testable.