# Skade > Pure-Rust Iceberg. The catalog rests on a static search tree — lookups in > nanoseconds — and it embeds: one file, opened in-process. > > On catalog reads it measures 492× Nessie and 677× Polaris. The benchmarks are below. ⚡ **GATLING** — all cores, no CPU wait · no copy · no allocation · no rayon · no Java, no Python ## Mashup — an all-core data plane over an embedded catalog Built to [Brokkr's software criteria](https://git.gunnar.rs/nordisk/dwarves). Concurrency is [znippy-zoomies `gatling`](https://git.gunnar.rs/nordisk/znippy-zoomies). SQL is [DataFusion](https://datafusion.apache.org/). The catalog index is [Ragnar Groot Koerkamp's static search tree](https://curiouscoding.nl/posts/static-search-tree/) — 9 ns lookups, 40× binary search. **Skade vs stock Iceberg vs raw Parquet — `oxr`, 1M rows, `oden` · 32 cores (tmpfs), 2026-07-24:** | data plane | write (M rows/s) | read (M rows/s) | read cores busy | |---|--:|--:|--:| | **Skade** — gatling all-core encode + all-core scan | **44.4** | **234.1** | **9.5** / 32 | | Skade — the *same* scan, **single core** | 42.6 | 51.2 | 1.0 / 32 | | Iceberg — stock `iceberg-rust` writer + scan, same catalog | 10.0 | 37.0 | 1.0 / 32 | | raw Parquet — `ArrowWriter`, no catalog / no snapshots (serial read) | 9.2 | 62.0 | 1.0 / 32 | Skade writes **4.4× stock Iceberg** and **4.8× raw Parquet** on the same hardware. And it reads on **all cores**: the ONE gatling engine fans the manifest parse, footer load, and Parquet decode across the box, so 1M rows read single-core at 51.2 M/s (1 core) scale to **234.1 M/s at 9.5 cores — 4.6×** — while a single-writer engine pins one core (Iceberg, raw Parquet: 1.0/32). Honest ceiling, measured not asserted: skade's scan tops out at **~9 busy cores on this UNCOMPRESSED 4-column shape because it is DRAM-bandwidth bound, not fan-out bound** — per-core throughput *halves* (51 → 25 M/s/core) as cores are added, and more work units lift cores-busy but not total throughput. On **compute-bound** data (compressed, high-entropy) the identical scan keeps **~21 of 32 cores at a 13× single→all-core speedup** (`scan_core_saturation` test). A raw *row-group-parallel* Parquet read (no catalog) goes faster still — 394 M/s at 12.7 cores — so skade's read gap over it is the catalog/snapshot cost, not decode; the full per-core breakdown and that control arm are in [the full overview](.nornir/README-full.md). The catalog is one redb file, opened in-process. Nessie and Polaris are servers — a JVM, a port, a process to keep alive: | `table_exists` | Skade | vs Nessie | vs Polaris | |---|--:|--:|--:| | throughput | 2.0M ops/s | 492× | 677× | | latency p50 | 0.31 µs | | | *The full leaderboard — every storage variant, all four catalog capabilities, the head-to-head chart — is in [the full overview](.nornir/README-full.md) and the manual.* ## Serve it: skade-flight, the SQL door (new 2026-09-09) Power BI, the Arrow Flight SQL ODBC driver, ADBC and DataFusion clients speak SQL — **[`skade-flight/`](skade-flight/)** is that door: Apache Arrow Flight SQL over the skade warehouse, DataFusion in, streamed Arrow out. One long-lived session, a plan cache, per-partition parallel encode, the ODBC dialect shim (`SELECT TOP n`, `{d …}`, `{fn UCASE(x)}` …), and the column metadata the ODBC driver needs to fold filters. Measured on 2M rows × 6 columns: 35–40M rows/s over the wire, 87M projected, plan latency 40–80 µs. Two test matrices ship with it: every Iceberg datatype × 8 checks, and 129 Power Query statement shapes (38 verbatim from Microsoft's diagnostics), all emitting nornir testmatrix rows. ```sh skade-flight --warehouse /var/lib/skade/warehouse --listen 0.0.0.0:50055 --token "$T" --basic pbi:secret ``` (`skade-katalog serve` remains the catalog door — REST / gRPC for Iceberg-aware engines; skade-flight is the SQL door for BI tools. One of them per warehouse directory.) ## Read the real docs - 📖 **[The complete manual — `docs/manual.pdf`](docs/manual.pdf)** — the complete anatomy. - 📄 **[Full overview — `.nornir/README-full.md`](.nornir/README-full.md)** — design, quickstart, and every measured benchmark. - 📘 **[The idiot's guide — `docs/idiot_guide.pdf`](docs/idiot_guide.pdf)**