Gunnar

nordisk/monetize

.nornir/04-estates-call-monetize.md

# Estates call monetize: one monetize, pull only, repair that survives without it

*Design note, 2026-10-10. Owner decisions made on this date; the code below is
the 5.4 target, not what ships in 5.3.x. gunnar is the worked example. **holger
is next** and follows the same pattern with its own implementations of the same
traits.*

## 1. The rule

**Monetize never calls into an estate.** That covers private instances,
gunnar.rs and holger.rs alike. Every estate calls monetize; monetize answers.

Monetize dials out to exactly two kinds of party:

- **cloud providers**, through the cloud trait (UpCloud today), and
- **payment vendors**, with one account per product (gunnar's Stripe for gunnar,
  holger's Stripe for holger).

Nothing else. Monetize holds no WireGuard peer, no tunnel and no credential into
any estate.

Why: every private instance has its own WireGuard and they share nothing.
Reaching into each one would mean one tunnel and one key per customer, all held
by one public machine, so a single compromise would open every customer's
private network. Monetize used to work by sitting *inside* the estate's network
on the front. That is exactly the placement 5.4 removes.

## 2. One monetize

- **One monetize machine** with a public address. `monetize-ui` and
  `monetize-server` are **separate processes** on that machine.
- It serves every product through plugins (`monetize-product`), gunnar first and
  then holger.
- **An account is `(product, tenant)`.** Members of a tenant are not accounts. A
  Private Gunnar instance bought by tenant X is **another row on X's gunnar
  account**, not a new account. It is billed through gunnar's payment vendor.
- No legacy shapes: monetize is still a test system, so the per-front monetize
  (each estate front running its own `monetize-server` and ledger) is replaced,
  not kept alongside.

## 3. The pull channel

Each estate (the front, the primary and the twin) keeps an outbound, mutually
authenticated channel to monetize, either long-poll or stream:

| the estate sends | monetize answers |
|---|---|
| heartbeat + status (role, head, disk fill, grow phase, twin health) | nothing, or queued orders |
| "any orders for me?" | **signed orders**: grow go-ahead, entitlements, plan change, stop, adopt twin at X |
| a request: "twin dead, need a new box", "data lost, recharge running" | a case on that estate in monetize-ui |

- The appliance verifies every order against the `monetize_pubkey` it already
  carries (gunnar: `gunnar-common/src/cli.rs`, `monetize_pubkey`). An unsigned
  or wrongly signed order is refused.
- An instance that is offline simply doesn't collect its orders; they wait.
  Expiry and grace are the existing `monetize_vital` rules.
- Cloud work (create server, resize, attach volumes) is monetize → cloud API.
  The estate only reports "volumes free" and waits for "go".
- **First install is the exception.** A box that doesn't exist yet cannot call
  home. The installer stays a temporary WireGuard peer during install (the
  nordisk-pairing `installer-wireguard` link, udp/51821). After the install
  closes, the new box calls monetize like every other estate.

### Where today's code still dials in (to be turned around)

| today | where |
|---|---|
| the front dials the product's control plane | `crates/monetize-front/src/self_register.rs` (`block`: "vm1 has nowhere to dial it") |
| grow / restore drive the appliance directly | `crates/monetize-server/src/grow.rs` (`restore_decision`), `products/gunnar/src/restore.rs` |
| adopting a twin dials the primary | `ApplianceTwinship::adopt_twin(twin_addr)`, `crates/monetize-product/src/lib.rs:1233` |
| entitlement push | the per-estate monetize-server pushes to its product |
| a box's WireGuard tunnel address | gunnar `Bootstrap.AssignTunnelAddress` (only used by the mock in `products/gunnar/src/testing.rs` today) |

Each of these becomes an order the estate pulls, or a report the estate sends.

## 4. Repair: one implementation, three callers

The owner's choice is **C + B**:

- **C:** the estate repairs what it can by itself, reusing the twin logic that
  is already built and tested.
- **B:** monetize hosts an **estate guardian**: a plugin per product type, behind
  a shared trait, that adds what needs the cloud (a new box) and gives the
  operator buttons.

The result is resilient. If monetize is down, an estate can still recharge from
its twin; only creating a new machine needs monetize.

### The verbs

| verb | what it does | needs the cloud? |
|---|---|---|
| **review twin** | twin health, its head vs the primary's, last heartbeat, what a recharge would lose (the §2.1 list in `DATA-SET-GROWTH-FLOW.md`) | no |
| **recharge from twin** (PRIO) | the primary's data is dead or empty: re-lay the set and replay the twin's log into it | no |
| **new twin** | the twin is gone: create a box, lay it, the primary adopts it | yes |
| **new server** | the primary's box is gone: create a box, lay it, recharge it from the twin | yes |

### No duplicated code (hard rule)

There is **one implementation of each verb**, and three callers use it:

1. **the estate itself**: the boot check, when the server comes up and finds its
   data set dead;
2. **the local admin UI** on the estate (gunnar-ui-server for gunnar), which
   *always* shows **[Recharge from twin]** and **[Review twin]** to its admin,
   not only when something is broken;
3. **the monetize estate guardian**, through the product's trait
   implementation, which shows the same actions as a case with buttons in
   monetize-ui.

Both screens render the same verb results through the same API shape.

### What gunnar already has (the seed)

| piece | where | state |
|---|---|---|
| recharge: replay a twin's log into a fresh data root | gunnar `crates/gunnar-server/src/twin.rs` (`restore`), the `gunnar control twin restore` verb | built, tested (`twin/tests.rs`: a recharge rebuilds every index on the new main) |
| a standby that notices a dead primary and promotes itself | gunnar `crates/gunnar-server/src/standby_watch.rs` | built, **off by default** ("naming a new primary is the operator's decision") |
| abdicate / promote | `gunnar control replication abdicate --promote`, `replication promote` | built |
| the per-product traits | `crates/monetize-product/src/lib.rs`: `Product::restore_lane` (:972), `Product::twinship` (:980), `ApplianceRestore` (:1134), `ApplianceTwinship { twin_health, adopt_twin }` (:1219) | defined |
| gunnar's restore impl | `products/gunnar/src/restore.rs:168` (`impl ApplianceRestore for GunnarProduct`) | built; driven only by a failed growth (`restore_on_failure`, off by default) |
| new twin from monetize | `crates/monetize-server/src/config.rs:966` `rebuild_twin_when_lost` | **configured but read by nothing**, and **no product implements `ApplianceTwinship`** |
| the human path | private-gunnar-ops `cargo xtask restore main`, `cargo xtask elect twin --apply` | built |

Today the same "recharge from twin" logic lives in at least three places: gunnar's
`twin restore`, monetize's `products/gunnar/src/restore.rs`, and the
private-gunnar-ops xtask. The 5.4 work picks the one that survives (the gunnar
side, next to gunnar's own verbs) and makes the other two thin callers.

### Safety for the buttons

- **Recharge refuses while the primary's data is still live and serving**,
  unless it is explicitly overridden. The old primary is fenced first, so there
  are never two writable boxes.
- Pressing a repair button takes strong admin auth (passkey / SSHSIG); the
  principal is recorded.
- Monetize does **not** act automatically by default. The guardian detects
  missing heartbeats, a lagging twin, or a primary that listens but is wedged,
  and then raises a **case** on the estate:
  `detected → investigating → approved → running → done | failed`.
  A human looks into it and presses the button. Automatic action can become a
  per-estate opt-in later, off by default, like `standby_watch`.

### Who does what when monetize is down

| situation | estate alone | needs monetize |
|---|---|---|
| primary's data dead, twin alive | **recharge from twin** (boot check or local admin button) | no |
| twin dead | review shows it; the estate keeps serving | **new twin** (cloud) |
| primary box gone | the twin keeps the data; `standby_watch` if opted in | **new server** (cloud) + recharge |

## 5. The cloud trait

Monetize creates boxes through a cloud trait, so a provider can be swapped
later; that swap isn't a priority yet. UpCloud is the only implementation today
(`monetize-cloud`, the cloud guest plugin, `mock-upcloud` for tests). The
guardian's "new twin" and "new server" use that trait and never call a
provider's API directly.

## 6. holger is next

holger follows the same pattern, with nothing copied from gunnar's code:

- holger implements the **same** `monetize-product` traits (`ApplianceRestore`,
  `ApplianceTwinship`) with its own verbs (holger already has a twin,
  `promote`, and an estate replication lane);
- the local admin UI (holger-ui) gets the same **[Recharge from twin]** and
  **[Review twin]** panel, rendered from the same API shape;
- holger estates call monetize the same way, holger signup goes through its own
  product plugin, and holger's payment vendor is holger's own Stripe.

holger is on hold in production. Its list of what to bring in line lives in
holger's own plan (`holger/.nornir/HOLGER-PLAN.md`, the "keep up with gunnar"
section). Only the trait implementation may start early.

## 7. Open

- The pull channel's exact wire (long-poll vs stream; which service owns it).
- Who may press which button (an operator principal vs the tenant's own admin
  for a Private Gunnar instance).
- Whether `adopt_twin` becomes an order the primary pulls ("adopt twin at X") or
  a step the new twin drives by calling the primary inside the estate.
- `standby_watch`'s probe is a TCP connect, so a primary that is wedged but
  still listening reads as alive. A signed liveness call would prove more.