<!-- nornir:class: guide -->
# Table compaction — guide
> **Class:** `guide` · **Function:** `compaction`. The *what/why* is in
> [`compaction-design.md`](compaction-design.md).
## When to compact
A table that is appended to continuously (many small `append` commits) grows one
snapshot + one tiny file per commit, unbounded. When the snapshot count or
tiny-file count climbs (a few thousand snapshots, or metadata in the hundreds of
MB), compact it: the rebuild collapses the whole history to **one** snapshot in a
handful of large zstd files. There is no separate `expire_snapshots` step — the
rebuild is the expiry.
Compaction requires a quiescent writer for that table (call it from your single
append actor). Concurrent readers are fine.
## Compact a table
```rust
let report = warehouse.compact_table("events", None).await?;
println!(
"compacted {} rows; was {} snapshots",
report.rows, report.snapshots_before
);
// REQUIRED: reload any Table handle you were holding — the old one points at the
// retired metadata, and its next append would write into the dir you're about to
// GC.
let events = warehouse.table("events").await?;
// Delete the retired data dir only after a grace window longer than your longest
// in-flight query (readers resolved just before the swap still stream old files).
if let Some(old) = report.old_data_dir {
// e.g. schedule after N minutes:
tokio::fs::remove_dir_all(old).await.ok();
}
```
## Compact and prune in one pass
The full rewrite is the one safe moment to drop rows (single writer, total
rewrite). Pass a `prune` closure that transforms each batch — e.g. keep only rows
newer than a cold-storage watermark:
```rust
let cutoff: i64 = 1_000;
let prune = move |b: RecordBatch| -> skade::Result<RecordBatch> {
// return a batch with the unwanted rows removed (same schema)
filter_rows(b, |id| id >= cutoff)
};
let prune_ref: &(dyn Fn(RecordBatch) -> skade::Result<RecordBatch> + Send + Sync) = &prune;
let report = warehouse.compact_table("events", Some(prune_ref)).await?;
assert_eq!(report.rows_pruned, /* rows the closure dropped */ n_dropped);
```
`CompactReport.rows` counts what survived, `rows_pruned` what the closure dropped
(`0` without a hook).
## Guarantees
- **Byte-identical data.** Compaction rewrites *files*, never *values*; the
surviving rows read back identically (verified in `tests/compaction.rs`).
- **One snapshot out.** The compacted table has exactly one snapshot regardless of
prior history, so the growth trigger converges.
- **Atomic + gap-free swap.** `name` never resolves to nothing across the swap;
the compacted data survives a crash/reopen. A concurrent `SELECT` never 404s on
the transient scratch table.
## Limits
- **Unpartitioned tables only** — `compact_table` errors on a partitioned table.
- **No concurrent writer** to the same table during compaction (a racing append is
silently lost — it appended to the snapshot being retired).