<!-- nornir:class: guide -->
# Fast-commit populate — guide
> **Class:** `guide` · **Function:** `populate`. The *what/why* is in
> [`populate-fast-commit-design.md`](populate-fast-commit-design.md); the
> per-mechanism detail is in [`ingest-multi-guide.md`](ingest-multi-guide.md) and
> [`committer-overlap-guide.md`](committer-overlap-guide.md).
## The one-liner
To fill a table fast, call `Table::populate`. To fill **several** tables as one
atomic unit with a single fsync, call `Warehouse::ingest_multi`. To shed the
per-commit fsync entirely during a bulk load, open with
`Warehouse::open_with_durability(dir, WriteDurability::None)`.
## Single table — the recommended default
```rust
use skade::Warehouse;
let wh = Warehouse::open("/data/lake").await?;
let mut t = wh.create_table("events", &schema).await?;
// groups: Vec<Vec<RecordBatch>> — each inner Vec becomes one Parquet file.
let stats = t.populate(groups, /* files_per_commit */ 16).await?;
println!("{} rows, {} commits", stats.rows, stats.commits);
```
`populate` encodes across all cores, overlaps the FileIO writes, **and** overlaps
the commit fsyncs with the ongoing writes, at a machine-tuned channel depth
(`skade::default_channel_depth()`). Prefer it over the lower-level entries unless
you need a specific topology:
* `Table::ingest_parallel(groups, fpc)` — strict serial writer (simplest).
* `Table::ingest_pipelined(groups, fpc, depth)` — overlapped writes, **serial**
commit tail (the proven serial-commit contract).
* `Table::ingest_pipelined_overlap(groups, fpc, depth)` — what `populate` calls,
with an explicit depth.
All four produce a byte/row-identical table with the same snapshot grouping;
they differ only in scheduling.
## Many tables — one fsync for the whole flush
```rust
use skade::WriteProps;
let a = wh.create_table("bench_runs", &runs_schema).await?;
let b = wh.create_table("dep_graph", &dep_schema).await?;
let c = wh.create_table("components", &comp_schema).await?;
let (advanced, stats) = wh.ingest_multi(
vec![(a, runs_groups), (b, dep_groups), (c, comp_groups)],
&WriteProps::default(),
).await?;
assert_eq!(stats.commits, 1); // ONE fsync flipped all three tables
assert_eq!(advanced.len(), 3); // each gained exactly one snapshot
```
The batch is **all-or-nothing**: if any table's groups can't be encoded, nothing
commits — no earlier table advances. Each table still gets exactly one new
snapshot; only the commit *transactions* collapse from N to 1. Single-partition
per table, same rule as `append`.
## Bulk import — shed the fsync
```rust
use skade::{Warehouse, WriteDurability};
// The whole warehouse is rebuildable from source → skip the per-commit fsync.
let wh = Warehouse::open_with_durability("/data/lake", WriteDurability::None).await?;
// … populate / ingest_multi as above (now at zero per-commit fsync cost) …
drop(wh);
// Reopen at the crash-safe default for durable serving.
let wh = Warehouse::open("/data/lake").await?;
```
* `WriteDurability::Eventual` — persists shortly after commit; may lose the last
few commits on a hard crash, DB stays consistent. A safe middle ground.
* `WriteDurability::None` — no per-commit fsync, fastest. Use for a **bounded**
import (redb only reclaims pages at higher durability, so exclusive `None` grows
the file), then reopen at `Immediate`.
Combine the levers: `open_with_durability(None)` + `ingest_multi` gives you the
fewest commits *and* the cheapest commits.
## How do I know it worked?
`IngestStats.commits` reports the snapshot-commit count of a call; for the whole
process, `skade::skade_katalog::redb_commit_count()` is the durable fsync counter
— read a delta around a populate to confirm a multi-table flush cost exactly one.
The test suite (`skade/tests/populate.rs`, `skade/tests/ingest_multi.rs`) asserts
all of this RED-when-broken.