nordisk/facett

.nornir/databricks-spark-ui-parity-reasoning.md

# Databricks / Apache Spark UI parity — a facett build map

> **Status: RESEARCH + DESIGN (no code).** This doc catalogues *everything* in
> the open-source **Apache Spark Web UI** and the **Databricks** Spark/SQL
> observability surfaces, then maps each view onto facett's component family —
> what already covers it, what needs an extension, and what is net-new. The goal
> is "we want **that class of UI** in facett": a self-hosted, egui-native,
> headless-testable rebuild of the Spark/Databricks monitoring experience, fed by
> engine-agnostic view-models (the same source-agnostic seam as
> [`facett-pipeline`](facett-pipeline.md) and [`facett-grid`](facett-grid.md)).
>
> facett owns **only the picture**. A host (knut / korp / nornir viz) maps the
> Spark REST API / event log / Databricks `system.query.history` onto facett
> view-models; facett never learns what a "stage" is. This mirrors the existing
> [`PipelineAdapter`](facett-pipeline.md) split (engine → adapter → normalized
> model → view).
>
> Companion docs: [`components.md`](components.md) (the crate list this cites),
> [`facett-grid.md`](facett-grid.md), [`facett-pipeline.md`](facett-pipeline.md),
> [`facett-graphview.md`](facett-graphview.md), [`facett-syschart.md`](facett-syschart.md),
> [`roadmap.md`](roadmap.md).

---

## 0. Executive read

The Spark/Databricks UI is, structurally, **five visual primitives** repeated
across ~12 tabs:

1. **Big sortable metric tables** (Jobs, Stages, Tasks, Executors, Storage,
   Environment, Query History, JDBC/Connect sessions) — facett **already nails
   this** with [`facett-grid`](facett-grid.md) (1M-row virtualized, multi-sort,
   pinned cols, Σ footer, in-cell bars/sparklines/chips, conditional-format heat).
2. **Operator/RDD DAGs** (job DAG, stage DAG, SQL query plan, Databricks Query
   Profile graph) — partly covered by
   [`facett-pipeline::PipelineView`](../facett-pipeline/src/lib.rs) +
   [`facett-graph::DepGraphView`](../facett-graph/src/depgraph.rs); **the gap is
   per-node metric bodies + cost-heat + AQE initial/final + collapse**.
3. **Timeline / Gantt swimlanes** (Jobs event timeline, per-stage task timeline
   with stacked phase segments) — **net-new**; nothing in facett draws a
   horizontal time-axis Gantt today (`facett-plot` is x/y only).
4. **Percentile / distribution rows** (Summary Metrics for Tasks: Min/25/Median/
   75/Max; skew detection) — partly covered by
   [`facett-stat::StatGridView`](../facett-stat/src/lib.rs) sparklines; **needs a
   box/percentile cell**.
5. **Time-series line+histogram pairs** (Structured Streaming stats, Databricks
   compute metrics) — [`facett-plot`](../facett-plot/src/lib.rs) covers the line/
   bar half; **the paired histogram + stacked "operation duration" band is an
   extension**.

Plus two profiling artifacts (**thread dump → flame graph**, **plan text tree**)
that are minor/long-tail.

**The three highest-ROI builds** (see §3) are: a **`facett-timeline`** Gantt
(net-new, unlocks Jobs + Task timelines + Lakeflow run Gantt), a **plan-DAG
extension** to `facett-pipeline`/`DepGraphView` for **metric-bearing, cost-heated,
collapsible** nodes (the SQL plan / Query Profile graph), and **wiring the
existing `facett-grid`** to the Stage/Task/Executor tables (near-zero-cost, huge
coverage).

---

## 1. The full Spark / Databricks UI inventory

Legend for **visual form**: `table` · `gantt` (horizontal time-axis swimlane) ·
`dag` (node/edge operator graph) · `plan-tree` (collapsible text) · `percentile`
(Min/25/Median/75/Max distribution row) · `line+hist` (timeline chart + paired
histogram) · `bar` (progress / segmented bar) · `flame` (stacked-sample flame
graph) · `kv` (key/value table).

### 1A. Open-source Apache Spark Web UI

| View | Data it shows | Visual form | Why useful |
|---|---|---|---|
| **Jobs — landing** | header (User, Uptime, Scheduling Mode, Active/Completed/Failed counts); jobs table: Job Id, Description, Submitted, Duration, Stages succeeded/total, Tasks progress | `table` + `bar` | fastest triage — spot the failed/slow job, the stalled task bar |
| **Jobs — event timeline** | executor add/remove markers + each job as a bar on a shared wall-clock axis | `gantt` | correlate executor churn (dynamic allocation, lost executors) with job timing |
| **Job detail** | header (status, stage counts active/pending/completed/skipped/failed, SQL link); stages table (Stage Id, Desc, Submitted, Duration, Tasks bar, Input, Output, Shuffle R/W) | `table` + `bar` | which stage dominates; skipped = cache/shuffle reuse |
| **Job detail — RDD DAG** | RDD/DataFrame vertices, op edges, boxed by **stage** (boundaries = shuffles); green = cached, grey = skipped | `dag` | how the computation split into stages, where shuffles force boundaries |
| **Job detail — event timeline** | executor track + each **stage** as a bar | `gantt` | stage serialization vs overlap, gaps between stages |
| **Stages — landing** | per-status stage tables (same cols as job detail); Fair Scheduler Pools table (name, min share, weight, active stages, running tasks, mode) | `table` | cross-job stage overview; pool fairness |
| **Stage detail — header** | Total Time Across Tasks; Locality Summary (PROCESS/NODE/RACK/ANY); Input/Output/Shuffle R/W size+records; associated Job Ids | `kv` | stage-level totals + locality quality |
| **Stage detail — operator DAG** | physical operators in the stage (BatchScan, WholeStageCodegen[id], Exchange, HashAggregate, Sort, ShuffledRowRDD) | `dag` | ties the stage back to exact operators |
| **Stage detail — Summary Metrics for Tasks** | per-metric distribution at **Min/25th/Median/75th/Max**: Duration, GC, Scheduler Delay, (De)serialization, Getting Result, Peak Exec Memory, Input, Output, Shuffle Read (+ fetch wait, remote), Shuffle Write (+ time), Spill (mem), Spill (disk) | `percentile` | **the #1 skew/straggler tool** — Median≪Max ⇒ skew; high spill ⇒ low memory |
| **Stage detail — Aggregated Metrics by Executor** | per-executor rollup: Exec Id, Address, Task Time, Total/Failed/Killed/Succeeded Tasks, excluded flag, Input/Output/Shuffle R/W, Spill mem/disk | `table` | skew (one task) vs bad node (one executor slow across all tasks) |
| **Stage detail — Accumulators** | named accumulators + aggregated values | `table` | user counters |
| **Stage detail — Tasks table** | per-task: Index, ID, Attempt, Status, Locality, Exec Id, Host, Launch Time, Duration, Scheduler Delay, Deser Time, GC, Result Ser, Getting Result, Peak Exec Mem, Accumulators, Input, Output, Shuffle Read fetch-wait/size/remote, Shuffle Write time/size, Spill mem/disk, Errors | `table` | row-level forensics — the exact straggler, its host, its error |
| **Stage detail — task event timeline** | one bar per task, segmented by phase: Scheduler Delay, Task Deser, Shuffle Read, **Executor Computing**, Shuffle Write, Result Ser, Getting Result | `gantt` (stacked) | where time actually goes — lots of non-compute ⇒ over-partitioning/GC/scheduling |
| **Storage — landing** | cached RDD/DF: Name/Id, Storage Level, Cached Partitions, **Fraction Cached**, Size in Memory, Size on Disk | `table` | confirm full materialization; spotted spill |
| **Storage — detail** | per-partition: Block Name, Storage Level, Size mem/disk, hosting executor(s); data-distribution-on-cluster table | `table` | uneven partition distribution, eviction |
| **Environment** | Runtime Info (Java/Scala), Spark Properties, Hadoop Properties, System Properties, Classpath Entries, Resource Profiles | `kv` | ground truth for "what config was actually in effect" |
| **Executors** | per-executor + driver: Exec Id, Address, Status, RDD Blocks, Storage Mem (on/off heap), Disk Used, Cores, Active/Failed/Complete/Total Tasks, **Task Time (GC Time)**, Input, Shuffle R/W, Logs; per-row **Thread Dump / Heap Histogram / Flame Graph** | `table` (+ `flame`) | cluster health/balance; dead nodes, GC-bound execs, hot threads |
| **SQL/DataFrame — query list** | ID, Description, Submitted, Duration, Job IDs, status | `table` | bridge logical query → physical jobs |
| **SQL/DataFrame — query plan DAG** | operator boxes grouped in blue **WholeStageCodegen[id]** boxes; **per-node metrics**: output rows, peak memory, spill, data size, scan/metadata time, shuffle bytes/records/time, fetch wait, sort time, hash-probe, build-side size, commit time, Python UDF bytes | `dag` (metric-bearing) | where rows explode, where memory peaks/spills, which operator dominates |
| **SQL/DataFrame — plan text** | Parsed → Analyzed → Optimized logical → Physical plan (codegen `*(id)` markers, `AdaptiveSparkPlan`/`isFinalPlan`) | `plan-tree` | confirm Catalyst/AQE choices (pushdown, join type, coalescing) |
| **Structured Streaming — query list** | Name, Status, ID, Run ID, Submitted, Duration, **Avg Input/sec**, **Avg Process/sec**, Latest Batch, Error | `table` | which stream is failing / falling behind |
| **Structured Streaming — statistics** | Input Rate, Process Rate, Input Rows, Batch Duration (each line+hist); **Operation Duration** stacked (addBatch/getBatch/latestOffset/queryPlanning/walCommit); state metrics (total/updated rows, state memory, dropped-by-watermark, watermark gap) | `line+hist` + `gantt`(stacked) | input>process ⇒ backlog; batch-dur→trigger ⇒ falling behind; state growth ⇒ OOM risk |
| **Spark Connect** | sessions table (User, session id, Start/Finish, Duration, executed requests); request/SQL stats linking to jobs/SQL | `table` | attribute jobs back to a remote client session |
| **JDBC/ODBC (Thrift)** | sessions (User, IP, Session id, Start/Finish, Duration, Total Execute); SQL stats (User, Job id, Group id, Start/Finish/Close, Exec/Duration time, Statement, State, Detail) | `table` | BI admin: who ran what; compile vs exec vs fetch |
| **(legacy) DStream Streaming** | per-batch Scheduling Delay, Processing Time, Total Delay; batches table | `line+hist` + `table` | superseded by Structured Streaming tab |

### 1B. Databricks-specific additions / changes

| View | Data it shows | Visual form | Why useful |
|---|---|---|---|
| **Embedded Spark UI (compute page)** | the full Spark tab set, proxied through control plane; **live** on running clusters, **history-server replay** (~last 1k jobs) on terminated; executor **loss reason** (decommission); Master worker-decommission surface | (all of 1A) | same Spark UI without driver network access; gated by workspace ACLs |
| **Query Profile — graph** | operator DAG, leaves→results; each node shows switchable **Time / Memory peak / Rows**; highlight most-expensive nodes; zoom/search/filter; preview card | `dag` (metric-bearing, cost-heat) | visual hotspot finder for a single query |
| **Query Profile — tree** | condensed hierarchy ordered slowest-first | `plan-tree` | slowest operators at a glance |
| **Query Profile — overview** | status; user/compute/runtime; UUID; SQL; **wall-clock "where time went"** (waiting-for-compute / in-queue / scheduling / optimizing+pruning / executing / result-fetch); aggregated task time; rows returned/read; bytes read (+from cache); files/partitions read; **files/bytes pruned**; **bytes spilled to disk**; **% task time in Photon**; **Top operators** ranked by cost; Performance Insights (beta) | `bar` (segmented) + `table` | one-screen diagnosis: spill, skew, pruning effectiveness, Photon coverage |
| **Query Profile — per-node metrics** | Time spent, Memory peak, Rows, Cumulative time, Tasks total time, Source/Sink rows; scan/IO (files read/pruned, sizes, bytes read/cache, metadata/scan time, partitions); Bytes spilled; # tasks; verbose CPU/cloud metrics | `dag` node body | attribute cost to the relational operator |
| **Query History** | list (query text, status, start, duration, end, warehouse, user, rows, statement id); filters (user/date/compute/duration/status/type/tag); detail panel | `table` + detail | find slow/expensive/failed queries across a warehouse |
| **Query History — phase breakdown** | segmented bar: Waiting-for-compute / Waiting-at-capacity (queue) / Scheduling / Compilation+pruning / Execution / Result-fetch (`*_duration_ms`) | `bar` (stacked) | is slowness provisioning, queuing, planning, execution, or fetch? |
| **Photon indicators** | `Photon*` operator nodes + adapter boundary; **% task time in Photon**; **coloring (Spark UI: orange=Photon/blue=Spark; Query Profile: purple=Photon/grey=Spark)** | `dag` coloring + metric | is the query vectorized; where did it fall back to JVM |
| **AQE adaptive plan** | `AdaptiveSparkPlan` root w/ **`isFinalPlan`**; **initial vs current/final** both shown; DAG mutates live; runtime stats `isRuntime`; **`AQEShuffleRead`/`CustomShuffleReader`** (Coalesced / skewed partition counts); dynamic SMJ→BHJ visible as plan diff | `dag` (versioned) | proof AQE re-planned; quantifies coalescing/skew-split |
| **Lakeflow Jobs run views** | **Matrix** (runs×tasks status cells, bar-height=duration), **Graph** (task DAG, click=metrics+query-history links), **Timeline (Gantt)** (task bars + deps + perf insights), **List** (sortable) | `table`/`dag`/`gantt` | multi-task workflow monitoring; click-through to Query Profile |
| **Compute Metrics (replaces Ganglia)** | Hardware: CPU util+active nodes (stacked by mode), container memory, JVM heap, network rx/tx, free FS, mem+swap. Spark: **server-load distribution heatmap/tiles**, task metrics, shuffle bytes, task duration. GPU: per-GPU util/mem/fb/enc/dec | `line`/`bar`/`heatmap` | I/O-bound (iowait), memory pressure/swap, skew (uneven tiles), GPU saturation |
| **Thread dump → flame graph** | per-executor/driver JVM thread snapshot rendered as async-profiler stacked samples (frame width ∝ time) | `flame` | hot CPU paths / hung tasks without an external profiler |
| **Streaming (notebook live + UI)** | live line charts per micro-batch + Raw `StreamingQueryProgress` JSON; numInputRows, input/processed rows/sec, durationMs phases, stateOperators (rows/memory/rocksdb), per-source lag (Kafka offsets-behind, Kinesis ms-behind, Auto Loader files-outstanding) | `line+hist` + `kv` | backlog, batch-dur trend, state growth, per-source lag |

---

## 2. facett mapping — what covers it, the gap, what to build

Per Spark/Databricks view: the facett crate that covers it **today**, the **gap**,
and **what to build**. Crates cited are real (see [`components.md`](components.md)
and the workspace `Cargo.toml` members).

| Spark/DBX view | facett component today | Gap | What to build |
|---|---|---|---|
| Jobs / Stages list, Tasks table, Aggregated-by-Executor, Executors, Storage, Environment, Accumulators, Query History list, JDBC/Connect sessions | ✅ [`facett-grid`](facett-grid.md) `Grid` (virtualized, multi-sort, pinned cols, Σ footer, in-cell **progress bar** for Tasks-done, **sparkline**, **chips** for status/locality, **color-scale** heat for duration/spill); `facett-table` for tiny lists | **none structural** — pure wiring. Need a host adapter Spark REST/JSON → `RecordBatch`. Minor: a "Tasks succeeded/total" **dual-segment progress cell** (grid has single-value bar) | **Wire it** + a tiny `condfmt` "stacked progress" cell variant. ~zero new component. |
| Environment (kv), Storage detail, streaming Raw JSON | ✅ `facett-grid`/`facett-table`; [`facett-card`](../facett-card/src/lib.rs) `CardView` for the kv detail; [`facett-docview`](facett-docview.md) for big text | none | wire kv into a `CardView` or 2-col grid |
| **Job DAG**, **Stage operator DAG** | 🟡 [`facett-pipeline`](facett-pipeline.md) `PipelineView` (L→R layered DAG, status colour, animated flow edges, job-progress bar, throughput edge labels) + [`facett-graph::DepGraphView`](../facett-graph/src/depgraph.rs) (toposort columns, boxes/arrows, sublabel, **click→drill-down**) | DAGs are **stage-boxed** (nested grouping), nodes need green=cached / grey=skipped semantics, and the job DAG groups RDDs inside stage boxes | **Extend** `PipelineView`/`DepGraphView`: **node grouping boxes** (WholeStageCodegen / stage clusters) + per-node cached/skipped state colour. Most of the layout already exists. |
| **SQL query plan DAG**, **Query Profile graph**, **Photon coloured plan**, **AQE adaptive plan** | 🟡 `DepGraphView` gives boxes+arrows+drill-down; `PipelineView` gives themed status colour | **The big gap**: (a) **per-node metric body** (multi-line: rows / peak mem / spill / time), (b) **cost-heat** node fill (Time/Memory/Rows switch like Query Profile), (c) **collapsible** WholeStageCodegen/Photon groups, (d) **initial-vs-final / `isFinalPlan` toggle** showing two plan versions, (e) Photon orange/purple palette per node | **NET-NEW `facett-plandag`** (or a heavy `DepGraphView` mode): metric-bearing, cost-heated, collapsible, versioned operator-DAG. See §4. |
| Plan **text tree** (parsed/analyzed/optimized/physical), Query Profile **tree view** | 🟡 [`facett-jobview`](../facett-jobview/src/lib.rs) (hierarchical tree with **ASCII branch guides**, fold/unfold, depth) is the closest shape; [`facett-docview`](facett-docview.md) for static plan text | jobview rows are job entries, not generic indented metric rows; needs slowest-first ordering + per-node metric column | **Extend `facett-jobview`** into a generic **indented metric tree** (depth + guides + a value/heat column) — reuses its guide renderer + fold tween. |
| **Jobs event timeline** (executors + jobs/stages bars), **Job-detail timeline** | ❌ nothing — `facett-plot` is x/y line/bar only, no horizontal **time-axis Gantt** | **whole primitive missing**: wall-clock axis, stacked swimlanes (executor lifecycle / job / stage / task bars), zoom/pan, colour-by-status | **NET-NEW `facett-timeline`** — a swimlane Gantt. See §4. (Tab label already exists in [`facett-tabbar`](../facett-tabbar/src/lib.rs): "🧵 Timeline".) |
| **Per-stage task event timeline** (stacked phase segments per task) | ❌ none | same Gantt primitive, but each bar is **internally segmented** into phases (scheduler delay / deser / shuffle read / compute / shuffle write / ser / getting-result) with a phase legend | **`facett-timeline`** with **segmented bars** + legend (one component covers both timeline kinds). |
| **Lakeflow run Gantt + Matrix + Graph** | Matrix → 🟡 [`facett-matrix`](../facett-matrix/src/lib.rs) (repo×aspect coloured cells — same shape as runs×tasks!); Graph → `DepGraphView`/`PipelineView`; Timeline → `facett-timeline` (new); List → `facett-grid` | matrix bar-height-by-duration; timeline = the new component | reuse `facett-matrix` for the run matrix (cell colour + a height/intensity tweak); rest covered above |
| **Summary Metrics for Tasks** (Min/25/Median/75/Max percentiles) | 🟡 [`facett-stat`](../facett-stat/src/lib.rs) `StatGridView` (label + value + inline bar + **sparkline** + verdict) | no **box/percentile** cell; can't show 5-number distribution or the Median≪Max skew signal | **Extend `facett-stat`** with a **percentile/box-plot cell** (5-number horizontal box). Small, high value (skew detection). |
| **Structured Streaming statistics** (Input/Process Rate, Input Rows, Batch Duration line+hist) | 🟡 [`facett-plot`](../facett-plot/src/lib.rs) `PlotView` (line/bars via `egui_plot`) | missing the **paired histogram** beside each timeline, and the **stacked "Operation Duration"** band | **Extend `facett-plot`**: a `line+hist` paired layout + a **stacked-area** kind (egui_plot supports both). |
| **Compute Metrics** (CPU/mem/heap/network/FS line charts) | ✅ `facett-plot` line/area; ✅ [`facett-syschart`](facett-syschart.md) for node badges | server-load **heatmap/tiles** by node | `facett-plot` covers the lines; the per-node load **heatmap** → `facett-matrix` (node×time coloured cells) or a `facett-grid` heat column |
| **Streaming query list, Connect/JDBC sessions, state-operator metrics** | ✅ `facett-grid` | none | wire |
| **Thread dump → flame graph** | ❌ none | flame graph (stacked samples, width∝time) is its own primitive | **NET-NEW `facett-flame`** (lower priority) — recursive stacked rects, zoom-on-click. Could share the `facett-timeline` painter (both are nested horizontal rects). |
| **Executors thread dump (raw)**, **driver/executor logs** | ✅ [`facett-log`](../facett-log/src/lib.rs) (ring log, kind-coloured) + [`facett-console`](../facett-console/src/lib.rs) (monospace terminal, ANSI, scrollback) | none | wire stderr/stdout/thread-dump text in |
| **The whole multi-tab Spark UI shell** | ✅ [`facett-tabbar`](../facett-tabbar/src/lib.rs) (tab bar; already ships a "Timeline / DepGraph / Release" demo) + [`facett-core::FacetDeck`](../facett-core/src/lib.rs) (`ui` tabbed + `wall_ui` multi-pane wall, palette/zoom/clipboard routed) | none | compose: one `FacetDeck` per Spark tab, exactly like [`facett-warehousedeck`](../facett-warehousedeck/src/lib.rs) composes panes from existing widgets |

**Coverage tally:** of the ~25 distinct views, **~12 are already covered** by
`facett-grid`/`table`/`card`/`log`/`console`/`tabbar`/`deck` (pure wiring), **~7
need extensions** to existing crates (`facett-pipeline`/`DepGraphView`,
`facett-stat`, `facett-plot`, `facett-jobview`, `facett-matrix`), and **~3 are
net-new** (`facett-timeline`, `facett-plandag`, `facett-flame`).

---

## 3. Prioritized build roadmap (highest ROI first)

Ordered by `(views unlocked) × (how Spark-defining it is) ÷ (build cost)`.

### P0 — Wire the tables onto `facett-grid` (days, no new component)
The cheapest, biggest single win. A host adapter `SparkRestApi/EventLog → Arrow
RecordBatch` plus `Grid::from_record_batch(...)` immediately delivers: **Jobs,
Stages, Tasks, Aggregated-by-Executor, Executors, Storage, Environment, Query
History list, Connect/JDBC sessions** — ~half the entire UI. `facett-grid` already
has the in-cell **progress bar** (task progress), **sparkline**, status **chips**
(locality/status), and **colour-scale heat** (duration/spill) these tables want.
Only addition: a **dual-segment "succeeded/total" progress cell** (Spark's task
bar shows succeeded+running+failed). Deliverable: a `knut`-side `SparkGridAdapter`
+ a column-spec table. *No facett source change beyond an optional condfmt variant.*

### P1 — `facett-timeline` (net-new): the Gantt swimlane
The single most **Spark-defining** visual that facett lacks. One component covers
**three** views: the **Jobs event timeline** (executor lifecycle + job/stage
bars), the **per-stage task timeline** (segmented phase bars), and the **Lakeflow
run Gantt**. Wall-clock x-axis, stacked named lanes, **segmented bars** (per-phase
colours from the active `Theme`), drag-pan + scroll-zoom (reuse
[`facett-core::scroll_engine::SmoothScroll`](../facett-core/src/scroll_engine.rs)),
click-select a bar → detail, hover tooltip. `Facet` + `state_json` like everything
else. The "🧵 Timeline" tab already exists in `facett-tabbar`. See §4.

### P2 — `facett-plandag` (net-new) / heavy `DepGraphView` mode: metric-bearing operator DAG
The **SQL query plan / Databricks Query Profile graph** — the other signature
Databricks surface. Builds on the existing `DepGraphView` toposort + `PipelineView`
theming, adding: (a) **per-node metric body** (rows / peak mem / spill / time),
(b) **cost-heat fill** with a Time/Memory/Rows **switch** (Query Profile's core
interaction), (c) **collapsible WholeStageCodegen/Photon groups**, (d) **Photon
orange/purple node palette**, (e) **initial-vs-final `isFinalPlan` toggle** (two
plan versions). High value, higher cost — schedule after the Gantt. See §4.

### P3 — Targeted extensions (small, each unlocks one signature view)
- **`facett-stat` percentile/box cell** → *Summary Metrics for Tasks* + skew
  detection (Median≪Max). Smallest change, classic Spark tuning view.
- **`facett-plot` line+hist pair + stacked-area** → *Structured Streaming
  statistics* + *Operation Duration* band + compute-metrics charts.
- **`facett-jobview` generic indented metric-tree mode** → plan **text tree** /
  Query Profile **tree view** (reuses its branch-guide + fold tween).
- **`facett-matrix` height/intensity tweak** → Lakeflow **run matrix** + compute
  **server-load heatmap** (node×time).

### P4 — `facett-flame` (net-new, long-tail): the thread-dump flame graph
Lowest frequency, but the only true flame surface (Executors → Thread Dump →
Flame Graph). Recursive stacked rects, width∝samples, zoom-on-click. Cheap if it
**shares the `facett-timeline` nested-rect painter** (a flame graph is a
depth-stacked Gantt without a time axis) — which is a reason to build P1 first.

### P5 — Compose the shell
A `SparkUiDeck` à la [`facett-warehousedeck`](../facett-warehousedeck/src/lib.rs):
one `facett-tabbar` across Spark tabs, each tab a `FacetDeck` of the widgets above,
in `wall_ui` (multi-pane) or tabbed mode. **Zero new components** — pure
composition of P0–P4, exactly the warehousedeck pattern.

---

## 4. The 2–3 net-new facett components this implies

### NEW #1 — `facett-timeline` (a swimlane Gantt) — **P1, build first**
**Why new:** facett has no horizontal **time-axis** view. `facett-plot` is x/y
line/bar; `PipelineView`/`DepGraphView` are left-to-right *topological* (not
*temporal*) DAGs; `facett-jobview` is a vertical list. A Gantt — *bars positioned
and sized by wall-clock time across stacked lanes* — is a genuinely new primitive.

**Model (source-agnostic, mirrors the `PipelineAdapter` seam):**
```
TimelineModel { lanes: Vec<Lane>, t0, t1 }     // wall-clock window
Lane  { label, rows: Vec<Bar> }                 // e.g. "executor-3", "stage 7"
Bar   { start, end, status, label,              // status → Theme colour
        segments: Vec<Segment> }                // optional intra-bar phases
Segment { kind, fraction }                       // scheduler-delay/compute/shuffle…
```
**Covers:** Jobs event timeline, Job-detail timeline, **per-stage task timeline**
(segmented bars = the phase legend), Lakeflow run Gantt. **Reuses:** `Theme`
colours, `effects::glow_rect` for the selected bar, `scroll_engine::SmoothScroll`
for pan/zoom, the `Facet`+`state_json`+`FacetCaps` contract. **Caps:** selectable
+ themeable + resizable + scalable (zoom = time-axis zoom). **Headless-testable**:
inject bars → assert laid-out bar rects + `state_json`. A flame graph (NEW #3) is
the same nested-rect painter without a time axis, so build this first.

### NEW #2 — `facett-plandag` (metric-bearing, cost-heated, collapsible operator DAG) — **P2**
**Why new (vs extending):** the SQL plan / Query Profile graph wants four things
no current graph facet has at once: a **multi-line metric body inside each node**,
**cost-heat fill with a Time/Memory/Rows switch**, **collapsible operator groups**
(WholeStageCodegen / Photon), and a **two-version `isFinalPlan` (AQE) toggle**.
`DepGraphView` gives the toposort columns + boxes + drill-down; `PipelineView`
gives themed status. This composes both (the established "compose `facett_graph`,
paint our own" pattern from [`facett-pipeline.md`](facett-pipeline.md)) and adds
the metric/heat/collapse/version layer. **Could** ship as a heavy `DepGraphView`
mode instead of a crate — decide at build time; the *capabilities* are what
matter. **Covers:** SQL query plan DAG, Databricks Query Profile graph, Photon
plan, AQE adaptive plan. **For huge plans**, it is the natural first consumer of
[`facett-graphview`](facett-graphview.md)'s vello render engine + its stubbed
LOD/culling/clustering (collapse dense subgraphs into super-nodes).

#### NEW #2b — the **constraint overlay** (bottleneck badge · downstream emphasis · critical path)

**Design (what shipped).** The operator DAG is a DAG, so it has the same
constraint structure a **funnel Plan-DAG** has — a `PlanDagView` now surfaces
*where a plan is constrained*, additively, over the metric/heat/collapse/version
layer. A pure analysis on the resolved graph (`facett-plandag/src/model.rs`):

- `Resolved::bottlenecks() -> Vec<Bottleneck{ id, fan_in, fan_out, downstream }>` —
  ranks every node by its **transitive-descendant cone** (`downstream` = all work
  stalled behind it) → `fan_out` → `fan_in` → id, so `bottlenecks().first()` is the
  single biggest constraint.
- `Resolved::top_bottleneck()` — the highest-ranked node **with an incoming edge**
  (skips the pure-source root whose cone is trivially the *whole* plan — the generic
  analog of the funnel "mother idea" a renderer filters out). This is the "clear
  this next" node the view **badges**.
- `Resolved::critical_path()` — the longest producer→consumer chain (structural
  longest path, deterministic id tie-break).

This **mirrors `funnel_core::PlanDag::{bottlenecks, critical_path}()`** — *not* a
funnel-core dependency: `facett-plandag` renders a host-fed generic `PlanModel`
(≠ a funnel `Funnel`) and is pinned to the buildable dependency-island of
`facett-core`/`facett-graph` (the same reason it inlines `facett-pipeline`'s status
colours — see its `Cargo.toml`). The nornir-viz layer maps a funnel `PlanDag` →
this `PlanModel`; the ranking rule (reach → fan_out → fan_in → id) is byte-identical
so the two agree by construction.

**The picture** (`facett-plandag/src/lib.rs`, `render`), all gated on
`state.show_constraints` (subtle + **on by default**; a header `⚑ constraints`
toggle / `Msg::SetConstraints` turns it off — see the SVG below):

- a warm **⚑ badge** pins `top_bottleneck` on the node's top edge + a stronger
  border, with **its own AccessKit node** (`get_by_label("⚑ bottleneck · <title>")`)
  carrying the downstream reach as the numeric value;
- every node gets an **outer accent ring** whose weight/alpha grows with its
  `downstream` reach (how much work is stalled behind it);
- the **critical-path** chain rides the accent colour with a soft weight beneath the
  arrow.

**Guide (host / robot).** The overlay reads the SAME `resolve()` the painter and
`state_json` read, so *the pixels ARE the data*: `state_json` gains `bottlenecks`
(the full ranking), `top_bottleneck` (the badged id), `critical_path` (the chain)
and `show_constraints`; `top_bottleneck` + `show_constraints` also join
`portable_state`/`load_state`. A robot never touches egui — it asserts
`state_json["top_bottleneck"]`, or (render-proof) `get_by_label` the badge in the
AccessKit tree. `facett-plandag/tests/render.rs` does exactly that on a wide-vs-deep
DAG: the **Deep** node (fan_out 1, downstream 4) is badged over the **Wide** node
(fan_out 3, downstream 3) — RED-when-broken if the reach ranking ever flips to
fan-out-primary — plus overlay-off drops the badge.

**For-idiots.** *Which box do I fix first?* The one with the little ⚑ flag — it has
the most work piled up behind it. The brighter a box's outer ring, the more the
plan is waiting on it; the blue arrows trace the longest road through the plan (its
schedule length). Flip the `⚑ constraints` switch off if you just want the raw
metrics.

![constraint overlay](assets/plandag-constraints.svg)

### NEW #3 — `facett-flame` (flame graph) — **P4, optional / long-tail**
**Why new:** depth-stacked sample rectangles (width∝time) with zoom-on-click — no
facett component does this. **Cheap if** it reuses `facett-timeline`'s nested-rect
painter (a flame graph ≈ a depth-stacked Gantt with no time axis). **Covers:**
Executors → Thread Dump → Flame Graph (the only true flame surface). Lowest
priority — build only after the timeline painter exists to share.

**Net-new count: 2 essential (`facett-timeline`, `facett-plandag`) + 1 optional
(`facett-flame`).** Everything else is `facett-grid` wiring or a bounded extension
to `facett-stat` / `facett-plot` / `facett-jobview` / `facett-matrix` /
`facett-pipeline`+`DepGraphView`, all composed under the existing
`facett-tabbar` + `FacetDeck` shell.

---

## Sources

Apache Spark Web UI: `spark.apache.org/docs/latest/web-ui.html`;
`apache/spark` `core/.../ui/jobs/StagePage.scala` (verbatim task columns + the
`TIMELINE_LEGEND` phase labels); The Internals of Spark Core (japila).
Databricks: `docs.databricks.com` Query Profile, Query History (+
`system.query.history` schema), Photon, AQE (`optimizations/aqe`), cluster /
compute metrics, Spark-UI debugging guide, Lakeflow job monitoring; the Photon
SIGMOD 2022 paper (operator class names); Databricks engineering blogs (Structured
Streaming UI in Spark 3.0, DBSQL Query Profile, Faster SQL with AQE).
*Caveats carried from research:* Photon operator class names are from the SIGMOD
paper not the docs; two Photon palettes (Spark UI orange/blue vs Query Profile
purple/grey); AQE node naming drift (`CustomShuffleReader` in DBX docs vs
`AQEShuffleRead` in OSS); Query-History column set partly inferred from filters +
detail panel; ~1k-job replay limit is community-sourced, not an SLA.