Skade
Pure-Rust Iceberg. The catalog rests on a static search tree — lookups in nanoseconds — and it embeds: one file, opened in-process.
On catalog reads it measures 492× Nessie and 677× Polaris. The benchmarks are below.
⚡ GATLING — all cores, no CPU wait · no copy · no allocation · no rayon · no Java, no Python
Mashup — an all-core data plane over an embedded catalog
Built to Brokkr's software criteria.
Concurrency is znippy-zoomies gatling.
SQL is DataFusion. The catalog index is
Ragnar Groot Koerkamp's static search tree —
9 ns lookups, 40× binary search.
Skade vs stock Iceberg vs raw Parquet — oxr, 1M rows, oden · 32 cores (tmpfs), 2026-07-24:
| data plane | write (M rows/s) | read (M rows/s) | read cores busy |
|---|---|---|---|
| Skade — gatling all-core encode + all-core scan | 44.4 | 234.1 | 9.5 / 32 |
| Skade — the same scan, single core | 42.6 | 51.2 | 1.0 / 32 |
Iceberg — stock iceberg-rust writer + scan, same catalog | 10.0 | 37.0 | 1.0 / 32 |
raw Parquet — ArrowWriter, no catalog / no snapshots (serial read) | 9.2 | 62.0 | 1.0 / 32 |
Skade writes 4.4× stock Iceberg and 4.8× raw Parquet on the same hardware. And it reads on all cores: the ONE gatling engine fans the manifest parse, footer load, and Parquet decode across the box, so 1M rows read single-core at 51.2 M/s (1 core) scale to 234.1 M/s at 9.5 cores — 4.6× — while a single-writer engine pins one core (Iceberg, raw Parquet: 1.0/32).
Honest ceiling, measured not asserted: skade's scan tops out at ~9 busy cores on this
UNCOMPRESSED 4-column shape because it is DRAM-bandwidth bound, not fan-out bound —
per-core throughput halves (51 → 25 M/s/core) as cores are added, and more work units
lift cores-busy but not total throughput. On compute-bound data (compressed,
high-entropy) the identical scan keeps ~21 of 32 cores at a 13× single→all-core speedup
(scan_core_saturation test). A raw row-group-parallel Parquet read (no catalog) goes
faster still — 394 M/s at 12.7 cores — so skade's read gap over it is the catalog/snapshot
cost, not decode; the full per-core breakdown and that control arm are in
the full overview.
The catalog is one redb file, opened in-process. Nessie and Polaris are servers — a JVM, a port, a process to keep alive:
table_exists | Skade | vs Nessie | vs Polaris |
|---|---|---|---|
| throughput | 2.0M ops/s | 492× | 677× |
| latency p50 | 0.31 µs |
The full leaderboard — every storage variant, all four catalog capabilities, the head-to-head chart — is in the full overview and the manual.
Serve it: skade-flight, the SQL door (new 2026-09-09)
Power BI, the Arrow Flight SQL ODBC driver, ADBC and DataFusion clients speak SQL —
skade-flight/ is that door: Apache Arrow Flight SQL over the skade
warehouse, DataFusion in, streamed Arrow out. One long-lived session, a plan cache,
per-partition parallel encode, the ODBC dialect shim (SELECT TOP n, {d …}, {fn UCASE(x)} …),
and the column metadata the ODBC driver needs to fold filters. Measured on 2M rows × 6 columns:
35–40M rows/s over the wire, 87M projected, plan latency 40–80 µs. Two test matrices ship with it:
every Iceberg datatype × 8 checks, and 129 Power Query statement shapes (38 verbatim from
Microsoft's diagnostics), all emitting nornir testmatrix rows.
skade-flight --warehouse /var/lib/skade/warehouse --listen 0.0.0.0:50055 --token "$T" --basic pbi:secret
(skade-katalog serve remains the catalog door — REST / gRPC for Iceberg-aware engines; skade-flight
is the SQL door for BI tools. One of them per warehouse directory.)
Read the real docs
- 📖 The complete manual —
docs/manual.pdf— the complete anatomy. - 📄 Full overview —
.nornir/README-full.md— design, quickstart, and every measured benchmark. - 📘 The idiot's guide —
docs/idiot_guide.pdf