The Canonical Model (implementation)
The relational implementation of Source, Observation, Entity, Event, Case, Report — schema, bitemporality, provenance signing, and partitioning.
The relational implementation of Source → Observation → Entity → Event → Case → Report. Product doc
02-canonical-model.mdexplains the concepts; this file is the schema, the bitemporality, the provenance signing, and the partitioning you build.
The six nouns as tables (PostgreSQL 16)
-- Every table is tenant-scoped. tenant_id is present even though each tenant has
-- its own database, because (a) defense-in-depth, (b) it lets RLS policies be uniform,
-- (c) it eases a future move to a pooled model. See security/43.
CREATE TABLE sources (
id text PRIMARY KEY, -- 'ais.spire', 'ofac.sdn'
tenant_id uuid NOT NULL,
family text NOT NULL, -- MARITIME|ENERGY|SANCTIONS|NEWS|...
connector text NOT NULL, -- connector type/driver
config jsonb NOT NULL DEFAULT '{}', -- non-secret config (secrets live in Secret Manager)
status text NOT NULL DEFAULT 'healthy',
created_at timestamptz NOT NULL DEFAULT now()
);
CREATE TABLE observations (
id bigint GENERATED ALWAYS AS IDENTITY, -- surfaced as OBS-<id>
tenant_id uuid NOT NULL,
source_id text NOT NULL REFERENCES sources(id),
observed_property text NOT NULL, -- 'vessel_position', 'designation', ...
observed_at timestamptz NOT NULL, -- EVENT time (when it was true)
ingested_at timestamptz NOT NULL DEFAULT now(), -- SYSTEM time (when we learned it)
entity_ref text, -- resolved entity id (nullable until resolved)
geom geography(Point,4326), -- PostGIS; nullable
payload jsonb NOT NULL, -- normalized fields
classification text NOT NULL DEFAULT 'OPEN',
provenance jsonb NOT NULL, -- {signature, signer, source_hash, received_at}
PRIMARY KEY (id, observed_at)
) PARTITION BY RANGE (observed_at); -- monthly partitions
CREATE TABLE entities (
id text PRIMARY KEY, -- 'VESSEL-SF12'
tenant_id uuid NOT NULL,
type text NOT NULL, -- vessel|cable|org|person|channel|...
canonical_name text NOT NULL,
attrs jsonb NOT NULL DEFAULT '{}', -- resolved/merged attributes
risk text, -- designated|exposed|clear|null
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now()
);
CREATE TABLE entity_aliases ( -- DETERMINISTIC resolution keys ONLY
entity_id text NOT NULL REFERENCES entities(id),
key_type text NOT NULL, -- 'mmsi','imo','lei','isin', gov ids — UNIQUE keys only
key_value text NOT NULL,
confidence real NOT NULL DEFAULT 1.0,
PRIMARY KEY (key_type, key_value) -- deterministic keys are globally unique → can't fork
);
-- NOTE: `name_norm` and other PROBABILISTIC signals are NOT stored here — a normalized name
-- is not unique (two real entities can share one) and would violate this PK. They live as
-- similarity FEATURES used only in Stage 2 matching (see 21), e.g. a pg_trgm index on
-- entities.canonical_name, never as a unique alias.
CREATE TABLE events (
id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
tenant_id uuid NOT NULL,
entity_id text REFERENCES entities(id),
kind text NOT NULL, -- 'ais_gap','throughput_drop','designation'
occurred_at timestamptz NOT NULL,
detected_at timestamptz NOT NULL DEFAULT now(),
confidence real,
summary text NOT NULL,
obs_refs bigint[] NOT NULL, -- the observations that fused into this
attrs jsonb NOT NULL DEFAULT '{}'
);
CREATE TABLE cases (
id text PRIMARY KEY, -- 'CASE-014'
tenant_id uuid NOT NULL,
title text NOT NULL,
status text NOT NULL DEFAULT 'OPEN', -- OPEN|WATCH|CLOSED
priority text NOT NULL DEFAULT 'MEDIUM',
hypothesis text,
owner_id uuid,
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now()
);
CREATE TABLE case_bindings ( -- events + entities bound to a case
case_id text NOT NULL REFERENCES cases(id),
ref_type text NOT NULL, -- 'event'|'entity'
ref_id text NOT NULL,
PRIMARY KEY (case_id, ref_type, ref_id)
);
CREATE TABLE reports (
id text PRIMARY KEY, -- 'RPT-038'
tenant_id uuid NOT NULL,
title text NOT NULL,
status text NOT NULL DEFAULT 'DRAFT', -- DRAFT|PUBLISHED
origin text NOT NULL, -- derived-from-case|stand-alone|imported
derived_from text REFERENCES cases(id),
body jsonb, -- structured doc; cites obs/events
signed_by uuid, signed_at timestamptz,
created_at timestamptz NOT NULL DEFAULT now()
);Bitemporality (the key design choice)
Every observation carries two clocks:
observed_at— when the fact was true in the world (event time).ingested_at— when the system learned it (system time).
This powers the AsOf time-travel control: "what did we know as of last Tuesday" is WHERE ingested_at <= :asof, while "what was true last Tuesday" is WHERE observed_at <= :asof. Entities and events keep updated_at/detected_at for the same reason.
Corrections never overwrite. A correction arrives as a new observation carrying a supersedes pointer to the one it replaces; resolution marks the prior row superseded but keeps it. This is a committed part of the schema (not deferred):
ALTER TABLE observations ADD COLUMN supersedes bigint; -- the obs id this one corrects (nullable)
ALTER TABLE observations ADD COLUMN superseded_by bigint; -- set on the prior row when corrected
-- AsOf rule: the "current" fact at time T is the latest non-superseded observation
-- whose ingested_at <= T (system time) for the property/entity; superseded rows remain
-- queryable for audit and for reconstructing what we believed at a past T.Provenance & signing
Each observation's provenance is a signed envelope written at ingest:
{
"signer": "ingestor:ais.spire@v1.4",
"source_hash": "sha256:…", // hash of the RAW payload as received
"record_hash": "sha256:…", // hash of the CANONICAL record (see below)
"received_at": "2026-06-03T14:02:11Z",
"alg": "ed25519",
"signature": "…" // sign(record_hash) with the per-tenant Ed25519 key
}Decided (reconciles 02): sign with Ed25519 (per-tenant key in Secret Manager) over a record_hash computed across the canonical record — the normalized, deterministically-serialized tuple (source_id, observed_property, observed_at, entity_ref, geom, payload, classification) — not just the raw-payload hash. Signing the normalized payload is what lets you prove the normalized fields weren't altered, which the raw-only hash cannot. source_hash is kept separately so the raw payload (in Mongo/GCS, keyed by source_hash) is also verifiable. Product doc 02's HMAC note is superseded by this Ed25519 decision — update 02 to match.
Referential integrity (a deliberate choice)
events.obs_refs bigint[] and case_bindings.ref_id text are soft references — they cannot use a foreign key into the composite, partitioned PK (id, observed_at). This is intentional: it keeps event/case binding cheap across monthly partitions. The cost is that integrity is enforced in application code (and a periodic checker), not by the DB. The build agent should treat these as soft refs by design, not a missing constraint.
Indexing & partitioning
- Partition
observationsby month onobserved_atusingpg_partman(with a scheduledrun_maintenanceto pre-create upcoming partitions and detach old ones); attach/detach for retention keeps the hot set small. - Indexes:
(entity_ref, observed_at desc),(observed_property, observed_at desc), GIST ongeom, GIN onpayloadfor ad-hoc field queries. events (entity_id, occurred_at desc);case_bindings (ref_id).- Consider BRIN on
observed_atfor very large partitions.
Migrations
- Tool:
node-pg-migrate(decided — matches the TypeScript backend), keep migrations inpackages/db. insigzctl migrate --tenant=<slug> --to=latestruns them per tenant (the provisioning step in §17). Migrations must be idempotent and forward-only; every tenant is at a known version (the control plane's release pinning tracks it).- RLS policies (see
security/43) ship as migrations too.
The grounding contract (DB side)
The AI never asserts; it cites [OBS-####]. A server-side validator resolves every cited id against observations for the tenant before a drafted claim ships (see ai/31). The DB guarantees those ids are real, tenant-scoped, and signed — that's why grounding can be trusted.
Gaps to close
- Finalize the entity type taxonomy and the
observed_propertyvocabulary (controlled lists). - Decide retention windows per source/class and wire partition detach + GCS archival.
- Set the signing key rotation cadence (coordinate with
42-secrets-and-kms.md; historical verification keys retained).
Data Stores (the polyglot decision)
What persists where and why; the build starts single-store Postgres with pgvector, with polyglot Mongo as the documented migration target.
Entity Resolution
How raw observations become attached to the right Entity — the anti-fork problem — via a two-stage deterministic then probabilistic matcher.