<!-- nornir:class: design -->
# Data-fill submit RPCs — pushing leaderboard / test / release rows to a live server
> Design + guide + idiot-guide for **EPIC N-data**: the **write** counterpart to
> the thin viz read path ([viz-remote-rpcs.md](viz-remote-rpcs.md)). It closes the
> gap where a live server's 🏆 Leaderboard / 🧪 Test / 🚀 Release panes stayed
> empty because nothing could push `agent_model_runs` / `test_results` /
> `release_events` into the **locked** server warehouse. Mirrors `Bench.Submit`.
## 1. The problem
`nornir-server` holds the single-writer **redb catalog lock** on each served
workspace's Iceberg warehouse. A thin client (`nornir … --server`, `nornir viz`)
can't open that warehouse to write it — only the server process can. The viz
**read** path already solved the read half (`Viz.*` / `Warehouse.Scan`), and
`Bench.Submit` solved benches. But the three remaining data-bearing tables had
**no client→server write path**, so on the live server those panes were empty:
| Table | Pane | Produced by | Write path before |
|---|---|---|---|
| `agent_model_runs` | 🏆 Leaderboard | `nornir bakeoff run` (dev's LLMs) | local warehouse only |
| `test_results` | 🧪 Test | `nornir test run` (dev's hardware) | local warehouse only |
| `release_events` | 🚀 Release | `nornir release run` (dev's hardware) | local warehouse only |
Each is produced on the **developer's** machine (it needs their LLMs / CPU / repo
checkouts), so the data must be **shipped** to the server — exactly like benches.
## 2. The contract (mirror `Bench.Submit`)
> The CLI mints the **same row structs** the embedded path appends, ships them as
> proto messages over a `Telemetry.*` RPC, and the server handler maps them
> **1:1** back into those structs and calls the **same** `append_*` the embedded
> path calls. The thin write is byte-equivalent to the embedded write.
The mode switch is uniform: a `persist_*` helper checks `NORNIR_SERVER`
(`server_target()`). Set ⇒ submit over gRPC; unset ⇒ append the local warehouse.
Same shape as `persist_bakeoff` / `run_bench`.
## 3. The three RPCs (all on the `Telemetry` service)
| RPC | Request | Server handler | Appends | Client helper | CLI |
|---|---|---|---|---|---|
| `Telemetry.SubmitBakeoff` | `SubmitBakeoffRequest{AgentModelRunPb[]}` | `submit_bakeoff` | `append_agent_model_runs` | `remote::submit_bakeoff` | `nornir bakeoff run/demo` |
| `Telemetry.SubmitTestResults` | `SubmitTestResultsRequest{TestResultPb[]}` | `submit_test_results` | `append_test_results` | `remote::submit_test_results` | `nornir test run/all` |
| `Telemetry.SubmitReleaseEvents` ⭐NEW | `SubmitReleaseEventsRequest{ReleaseEventPb[]}` | `submit_release_events` | `append_release_events` | `remote::submit_release_events` | `nornir release run` |
All three return `SubmitCountResponse{accepted}`. The `nornir-workspace` header
selects which served workspace the rows land in (same as every other RPC).
**Status before this epic (honest reckoning):**
- **Bakeoff** was already end-to-end (RPC + handler + `persist_bakeoff` routing).
- **Test** had the RPC + handler + a `submit_test_remote` client *defined but
never called* — `run_test_one` always wrote the **local** warehouse. EPIC
N-data **wired** it via the new `persist_test`.
- **Release** had **no** submit RPC at all — only a `Viz.ReleaseEvents` **read**.
EPIC N-data **added** `Telemetry.SubmitReleaseEvents` + handler + client +
CLI wiring.
## 4. The `release_events` Option round-trip
`ReleaseEventRow` carries two `Option`s that must survive the wire **distinct
from their empty/zero forms** (the proto-3 "all scalars have a default" trap, the
same one `BenchRun.TestOutcome` solves with `has_duration`):
- `depends_on: Option<Vec<String>>` — `None` (a root op, no upstream) must NOT
read back as `Some(vec![])` (a real op that waited on nothing). Carried as
`repeated string depends_on` + `bool has_depends_on`.
- `elapsed_ms: Option<i64>` — only `phase = end` rows carry a value. Carried as
`int64 elapsed_ms` + `bool has_elapsed_ms`.
The CLI/server map `Option ↔ (value, has_*)` on both ends. The integration test
asserts all three shapes round-trip (None, non-empty list, **present-but-empty**).
## 5. How `release run` ships its events
`nornir release run` runs on the dev's hardware and its `wh` handle is opened
**read-only** (lock-tolerant, to coexist with a live server). So the per-emit
`append_release_events(&wh, …)` can't reach the server. Instead the
`ReleaseEventEmitter` now **records every minted row** into a shared buffer
(`recorded_rows()`); a drop-guard in `run_release_run` drains that buffer and
submits the whole run over `Telemetry.SubmitReleaseEvents` on **every** exit
path — success, `?`, or a phase `bail!` — so even a *failed/partial* release
fills the live 🚀 Release tab. Embedded mode is a no-op (the per-emit append
already wrote the local warehouse). Best-effort + non-fatal: a submit failure is
logged, never aborts the run.
## 6. Iceberg schema-evolution LAW
No table schemas changed — all three tables already exist with their full column
set, and the proto messages are **additive** (a new service method + new message
types, no field renumbering), so old↔new binaries stay wire-compatible. The
`append_*` paths already go through `ensure_table_schema`, so any future column
add auto-evolves.
## 7. Tests (inject real rows, assert real output)
`tests/data_submit_rpcs.rs` — for each of the three RPCs: spin ONE ephemeral
`nornir-server` over an **empty** workspace warehouse, **submit** real rows over
the submit RPC, then **read them back** over the thin RPCs the live panes use
(`Viz.BakeoffResults`, `Warehouse.Scan` of `test_results`, `Viz.ReleaseEvents`)
and assert the **exact values** landed (scores, error strings, statuses, failure
messages, and the `release_events` Option round-trip), not merely a non-empty
result. A unit test in `warehouse/release_events.rs` asserts the emitter's
recording buffer captures every minted row across clones, in `seq` order.
## 8. Idiot-guide — fill the live server
The fat CLI does it automatically once `NORNIR_SERVER` is set. Point at the
server (token + workspace), then run the producers exactly as you would locally —
they detect `NORNIR_SERVER` and submit instead of writing a local warehouse:
```bash
export NORNIR_SERVER=https://oden:7878
export NORNIR_SERVER_TOKEN="$(sudo sed -n 's/^NORNIR_TOKEN=//p' /etc/nornir/nornir.env)"
export NORNIR_WORKSPACE=holger # which served workspace to fill
# 🏆 Leaderboard — bake-off rows
nornir bakeoff demo # quick mock matrix, or:
nornir bakeoff run --task "…" --agents claude,kilo --models opus,llama3
# 🧪 Test — multi-aspect matrix rows
nornir test run <repo> # one repo, or:
nornir test all # every configured repo
# 🚀 Release — release-op DAG rows
nornir release run --workspace holger # ships events on completion
```
Each prints `→ submitted N row(s) to <server>`. Then open `nornir viz` (or the
panes directly) against the same server and the 🏆 / 🧪 / 🚀 tabs are populated.
Embedded (no `NORNIR_SERVER`) behaves exactly as before — local warehouse.