Gunnar

mikronova/modgunn

.nornir/ioc-design.md

# ioc — design (threat-intel indicators, the garm-side axis)

> **Class:** `design` · **Axis:** `i0` the model · `i1` per-source parsers · `i2` the warehouse table ·
> `i3` the match predicate. Strict: the shape we run. The flagship-join use is
> [`baddie-detection-design.md`](baddie-detection-design.md) **b6**.

Where [`feeds-design.md`](feeds-design.md) imports *advisories* about a known-vulnerable **package**,
`ioc` ([`src/ioc.rs`](../src/ioc.rs)) imports the **garm-side** signal: indicators that a host or
artifact is *actively malicious* — a malware-serving URL, a botnet-C2 IP, a known-bad file hash.
modgunn owns the corpus for BOTH dwarves (supply-chain verdict) and garm (detection), so it lands in
modgunn's warehouse (`ioc_indicators`) for either consumer. Feature `scan`; pure parse, no network.

## i0 — the model

[`Ioc`](../src/ioc.rs)`{ kind, value, threat, source, tags }`, `kind` = [`IocKind`] ∈
`url | ipv4 | cidr | domain | sha256 | sha1 | md5`. Values are **normalized at ingest** through ONE folder ([`normalize_value`], used by
every parser via `Ioc::new` AND by the [`IocIndex`] lookup, so the two can't drift) so the b6 join keys
cleanly: hashes **and DNS domains** are case-folded (both case-insensitive), IPs canonical dotted-quad
(`is_ipv4` rejects leading-zero / non-digit octets), URLs kept verbatim (a URL path is case-sensitive). Each parser is tolerant of the feed's comment/header rows and **validates the indicator
shape**, so a stray header/garbage line is skipped rather than landing a bogus indicator.

## i1 — per-source parsers

| source | entry | format | kind(s) |
|---|---|---|---|
| abuse.ch **URLhaus** | `parse_urlhaus_csv` | quoted CSV (`#` header); url @col 2, threat @5, tags @6 | url |
| abuse.ch **Feodo** | `parse_feodo_csv` | CSV; dst_ip @col 1, malware @5 | ipv4 |
| abuse.ch **SSLBL** | `parse_sslbl_csv` | CSV; SHA1 @col 1 | sha1 |
| abuse.ch **ThreatFox** | `parse_threatfox_csv` | quoted CSV; ioc_value @2, ioc_type @3, fk_malware @5, tags @12 | url/ipv4/domain/hash (per `ioc_type`) |
| abuse.ch **MalwareBazaar** | `parse_malwarebazaar_csv` | CSV; sha256 @col 1, signature @8 | sha256 |
| **Tor** exit list | `parse_tor_exit_list` | plain IPv4 list (`parse_ip_blocklist`) | ipv4 |
| **Emerging Threats** compromised | `parse_emergingthreats_ips` | plain IPv4 list | ipv4 |
| **MISP** event | `parse_misp_event` | JSON (`Event.Attribute[]` + `Object[].Attribute[]`); the TI sharing standard | per attribute `type` |
| AlienVault **OTX** pulse | `parse_otx_pulse` | JSON (`results[].indicators[]` or a bare pulse) | per indicator `type` |
| **Spamhaus DROP** | `parse_spamhaus_drop` | `network/prefix ; SBLref` list | cidr |
| **GreyNoise** | `parse_greynoise` | JSON (GNQL `{data:[…]}` / bare array / single IP-context) | ipv4 — **only** `classification ∈ {malicious, suspicious}` |

Shared helpers: [`split_csv`] (RFC-4180 quoted fields — `pub(crate)`, so it is also the ONE
quoted-CSV splitter [`feeds::parse_exploitdb_csv`](../src/feeds.rs) uses rather than a second,
subtly-different implementation), [`data_lines`] (skip blank/`#`),
[`parse_ip_blocklist`] (the plain-IP shape behind Tor/ET). A CSV column order is **verified against the
live feed** before use (e.g. ThreatFox `tags` @12 confirmed with `is_compromised` @10 present).

**GreyNoise is the inverted case — most of its document must NOT be imported.** Unlike every other
source here (a bad-list where every row is bad), a GreyNoise result classifies internet background
noise as `malicious` / `suspicious` / `benign` / `unknown`, and also flags **RIOT** entries
(*Rule It Out* — Google DNS, CDN health-checkers, Microsoft update ranges). Because a b6 hit is an
**unconditional refuse** with no reachability gate, importing the benign half would blocklist
`8.8.8.8` and refuse a large slice of the fleet. So [`parse_greynoise`] admits only `malicious` and
`suspicious`, and drops anything `riot: true` regardless of classification. `unknown` is dropped too —
"not yet classified" is not evidence of malice. The red-when-reverted test
`greynoise_imports_only_malicious_never_benign_or_riot` pins exactly that.

**Anti-silent-empty:** a parser is pure and may return an empty `Vec` for a header-only doc; the
importer (`warehouse import-iocs`) treats a *provided-but-empty* feed as a MISSING-FEED error — a feed
that fetched nothing is never a fake pass.

## i2 — the warehouse table

`ioc_indicators` ([`Warehouse::append_iocs`](../src/warehouse.rs) / `read_iocs`) — snapshot-keyed,
append-only: `(snapshot, kind, value, threat, source, tags)`. CLI:
`warehouse import-iocs --snapshot <id> [--urlhaus|--feodo|--sslbl|--threatfox|--malwarebazaar|--tor|
--et-compromised|--spamhaus|--misp|--otx|--greynoise <file>]` (several may combine into one snapshot); read back with
`warehouse ioc-log --snapshot <id> [--kind <k>]`.

## i3 — the match predicate (b6)

modgunn owns the **match**, not the render (the Mímir/garm lens renders). [`IocIndex`](../src/ioc.rs)
is the b6 analogue of `Advisory::matches`: `build(corpus)` indexes by `(kind, normalized value)`,
`matches(kind, value) -> Option<&Ioc>` is an O(1) exact bad-list lookup with the same normalization —
**plus**, for an `Ipv4` query, a **CIDR containment** scan (`ip & mask == net & mask`), so a workspace
egress IP inside a bad network (a Spamhaus DROP block) matches even without an exact-IP entry. A hit ⇒
🔴 **refuse** (active malware, no reachability gate). The consumer keys a workspace observable — a
resolved dependency's download URL, a released artifact's hash, a workspace egress IP — through this
predicate; see [`baddie-detection-design.md`](baddie-detection-design.md) b6 for the join + verdict.

**The join is now end-to-end (2026-07-27), not just a lookup.** Three pieces closed it:

- [`IocKind::infer`](../src/ioc.rs) classifies a **raw** observable by shape (scheme ⇒ url; 64/40/32
  hex ⇒ sha256/sha1/md5; dotted-quad ⇒ ipv4; `net/prefix` ⇒ cidr; dotted labels with an alphabetic
  TLD ⇒ domain), so a consumer hands over strings instead of pre-classified pairs. An unclassifiable
  observable yields `None` — skipped, never guessed into a bucket where it would silently never match.
- [`IocIndex::verdict_for`](../src/ioc.rs) folds infer + match + [`baddie::classify_ioc`](../src/baddie.rs)
  into one call, so a b6 hit comes back as the SAME `BaddieVerdict` shape the CVE axes produce (🔴
  refuse, axis `ioc`, rank 9) and one renderer colours every node.
- [`Warehouse::ioc_index`](../src/warehouse.rs) rebuilds the predicate from a persisted snapshot and
  [`Warehouse::ioc_scan`](../src/warehouse.rs) runs the sweep. A stored row whose `kind` label does not
  round-trip through [`IocKind::from_label`] is **skipped and counted** (never coerced into a default
  kind — a mislabelled row would either miss or mass-match), and the CLI reports the count: a shrunken
  bad-list must not read as a clean sweep.

CLI: `warehouse ioc-scan --snapshot <id> --observables <file>` (one observable per line, `#` comments
skipped). Each hit prints the decided verdict and the indicator that matched — which for an IPv4 inside
a bad CIDR is the **network**, not the IP, so the operator sees which block. Exits non-zero on any hit;
an empty observables file is an error, because an empty sweep is not a clean sweep.