<!-- nornir:class: reasoning -->
# skade pluggable object store — `skade::ObjectStore`
> `skade::ObjectStore` is the backend-agnostic bytes-IO abstraction for the
> Iceberg data/metadata files a `Warehouse` reads and writes. It decouples skade
> from any single S3 client so a consumer (e.g. an isolated **Njord**
> mail-server) can run skade → Iceberg on a **shared MinIO** with the backend of
> *its* choice. `skade/src/object_store.rs`, wired in `skade/src/warehouse.rs`.
## Why it exists
Before this, the published `skade` crate hardwired `LocalFsStorageFactory` in
`Warehouse::open` — local disk only. The real S3 work (the `s3` feature, the
configurable MinIO endpoint) lived in the **unpublished `bench/` crate**
(`bench/src/s3_storage.rs`, backed by our own `aegir` client). A direct
consumer of the crate had no way to point skade at an object store, and no way
to pick *which* S3 client to use.
The Njord mail-server is isolated and already links `aws-sdk-s3`; it must write
to a shared MinIO. Pulling a second S3 client (rust-s3) into that build is
wasteful, and rust-s3 doesn't even compile in every workspace (see below). So
skade needs a trait the consumer can satisfy with the SDK it already has.
## The trait (`skade/src/object_store.rs`)
```rust
#[async_trait]
pub trait ObjectStore: Debug + Send + Sync {
async fn put(&self, key: &str, bytes: Bytes) -> Result<()>;
async fn get(&self, key: &str) -> Result<Bytes>;
async fn list(&self, prefix: &str) -> Result<Vec<String>>;
async fn delete(&self, key: &str) -> Result<()>;
async fn exists(&self, key: &str) -> Result<bool>;
// Provided (overridable) — S3 backends do a ranged GET / HEAD.
async fn get_range(&self, key: &str, range: Range<u64>) -> Result<Bytes>;
async fn size(&self, key: &str) -> Result<Option<u64>>;
}
```
Keys are opaque, slash-separated path-like strings. `Result` is
`skade::Result` (one error type over the stack).
## Backends (feature-gated)
| backend | feature | crate | what |
|------------------|------------|------------------|-------------------------------------------------|
| `MemoryStore` | *(always)* | — | in-process map; tests / ephemeral |
| `LocalFsStore` | `rustfs` | std only | embedded zero-external-dep local filesystem |
| `RustS3Store` | `s3` | `rust-s3` 0.35 | MinIO/S3 via configurable endpoint (**default S3**) |
| `AwsS3Store` | `aws-s3` | `aws-sdk-s3` 1.x | MinIO/S3 reusing the consumer's own SDK |
`default = []` — **no S3 client is pulled unless asked for**, and
`Warehouse::open` is byte-for-byte unchanged (still `LocalFsStorageFactory`).
Existing users and the existing local-path path are untouched.
### Note on "rustfs"
The `rustfs` *crate* on crates.io (0.0.x) is a full **distributed
object-storage server**, not an embeddable client — useless as a library dep.
What "local object store" actually means for a consumer is a persistent store
with no S3 server and no extra crates, so the `rustfs` feature ships
`LocalFsStore`: keys → files under a root dir, std-only.
### Note on rust-s3 / maybe-async
`rust-s3` uses `maybe-async`. In a workspace that also links `gix` (which flips
`maybe-async` to `is_sync` *globally*) rust-s3 will **not compile** — this is
exactly why the bench crate used our own `aegir` client instead. The published
skade crate has its own detached workspace, so the `s3` feature builds cleanly
for direct consumers. A consumer who *does* hit the gix conflict (or who already
links aws-sdk-s3, like Njord) should use the **`aws-s3`** backend.
## How the warehouse uses it (the iceberg bridge)
iceberg-rust drives all blob IO through its own `Storage` / `StorageFactory`
extension points (`FileIOBuilder::new(factory)`). skade bridges the trait into
that machinery rather than re-plumbing the writer/reader:
```
ObjectStore (trait)
└─ ObjectStoreBridge : impl iceberg::io::Storage (read/write/list/delete → trait)
▲
ObjectStoreFactory : impl iceberg::io::StorageFactory (build() → bridge)
│ given to
RedbCatalogBuilder::with_storage_factory(...) → FileIO → every data/metadata blob
```
*The ObjectStore bridge: one trait, fronted into iceberg's `Storage` / `FileIO`, sends every data + metadata blob to a feature-gated backend while the redb catalog stays local.*
Two constructors (both additive — `open` is unchanged):
```rust
// 1. From a live store (in-process; e.g. Njord's own aws-sdk-s3 client wrapped).
Warehouse::open_with_store(db_path, "s3://bucket/warehouse", Arc::new(store)).await?;
// 2. From a serializable config (settings/env-driven; backend built lazily).
Warehouse::open_with_object_store_config(
db_path, "s3://bucket/warehouse",
ObjectStoreConfig::AwsS3(S3Config { endpoint: "http://minio:9000".into(), .. }),
).await?;
```
The redb **catalog file** stays on local disk at `db_path` (it is a single-file
ACID mmap, not an object); only the Iceberg **data + metadata blobs** go through
the `ObjectStore`. The `warehouse_uri` is the location iceberg stamps into table
metadata and hands back to the store on every IO.
### typetag / serialization caveat (honest)
iceberg's `StorageFactory` and `Storage` are `#[typetag::serde]` — a built
storage must survive a config round-trip. A live SDK client is **not**
serializable, so `ObjectStoreFactory` serializes as a small `ObjectStoreConfig`
enum and rebuilds the backend in `build()`, exactly as iceberg's own S3 FileIO
does. Consequence: `open_with_store` (live store) is for the in-process flow; if
iceberg ever serializes the factory and rebuilds it, the live store is dropped
and a `MemoryStore` is reconstructed. For a durable, restart-safe S3 warehouse,
prefer `open_with_object_store_config` (config-built, fully reconstructible).
## Pointing at MinIO / rustfs / aws
```rust
use skade::{Warehouse, ObjectStoreConfig, S3Config};
let cfg = S3Config {
endpoint: "http://minio:9000".into(), // empty → real AWS
region: "us-east-1".into(),
bucket: "njord-mail".into(),
access_key_id: "minio".into(),
secret_access_key: "minio123".into(),
path_style: true, // required for MinIO
};
// rust-s3 backend (feature "s3")
let wh = Warehouse::open_with_object_store_config(
"/var/lib/njord/catalog.redb", "s3://njord-mail/warehouse",
ObjectStoreConfig::RustS3(cfg.clone())).await?;
// aws-sdk-s3 backend (feature "aws-s3") — Njord reuses its own SDK
let wh = Warehouse::open_with_object_store_config(
"/var/lib/njord/catalog.redb", "s3://njord-mail/warehouse",
ObjectStoreConfig::AwsS3(cfg)).await?;
// embedded local store (feature "rustfs")
let wh = Warehouse::open_with_object_store_config(
"/var/lib/njord/catalog.redb", "file:///var/lib/njord/store",
ObjectStoreConfig::LocalFs { root: "/var/lib/njord/store".into() }).await?;
```
## Flow note — Njord mail-server on shared MinIO
```
Njord (isolated) shared MinIO
├─ aws-sdk-s3 (already linked) ───────┐
├─ skade [features = ["aws-s3"]] │
│ Warehouse::open_with_object_store_config(
│ catalog.redb (local disk), │ redb stays local
│ "s3://njord-mail/warehouse", │
│ AwsS3(S3Config{ endpoint: minio }))
│ └─ AwsS3Store ──put/get/list──▶ s3://njord-mail/warehouse/...
└─ table.append(batch) → parquet + metadata land in MinIO via the ONE SDK
```
One S3 SDK in the build, the catalog ACID on local disk, the lake on shared
MinIO. That is the unblock.
## Tests (inject-assert) — `skade/tests/object_store.rs`
No live MinIO required (CI-safe):
- `memory_store_roundtrips` / `local_fs_store_roundtrips` — inject 3 objects
(incl. an empty one and a 4 KiB body), assert `get` returns the **exact**
bytes, `get_range` slices exactly, `list(prefix)` returns exactly the matching
keys, `size` matches the injected length, `delete` is precise + idempotent.
- `warehouse_writes_and_reads_through_memory_store` — a real Iceberg
`create + append(3 rows) + read` where **every blob lands in the injected
`MemoryStore`** (asserts a `.parquet` and a `metadata` key are present), and 3
rows come back through the trait.
- `warehouse_roundtrips_through_local_fs_config` (`rustfs`) — same end-to-end
table round-trip via a serializable `ObjectStoreConfig::LocalFs`, asserting a
parquet file landed under the **configured** store root (endpoint/root
honored) and 2 rows read back.
```bash
cargo test --test object_store # MemoryStore + warehouse-over-memory
cargo test --features rustfs --test object_store # + LocalFsStore + config round-trip
```
The live-MinIO S3 backends (`s3`, `aws-s3`) are build-verified in CI; their wire
behavior is exercised against a real MinIO out of band (they share the same
`ObjectStoreBridge` + trait contract the memory/local tests pin).