Engineering
CI/CD & Release Management
Build, scan, test, deploy - and how releases roll out across a fleet of single-tenant services with per-tenant version pinning.
Build → scan → test → deploy, and how releases roll out across a fleet of single-tenant services with per-tenant version pinning (product §17 Infrastructure/Releases).
Pipeline (Cloud Build, triggered from GitHub)
push / PR
→ install + typecheck + unit tests
→ SAST (CodeQL/Semgrep) · SCA (OSV/Snyk) · secret scan (gitleaks) [block on high]
→ build container image
→ container scan (Trivy) [block on critical]
→ sign image (cosign) [Binary Authorization gate]
→ integration tests (ephemeral PG + Mongo) incl. cross-tenant negative tests
→ IaC scan (tfsec/Checkov) on terraform
→ on main: push image to Artifact Registry, tag a release version (vX.Y.Z)
→ DAST (ZAP) against stagingOnly signed, scanned images are deployable (Binary Authorization). See security/44.
Releases & channels
- A release is an immutable, versioned image (
app:vX.Y.Z) on a channel (stable / legacy). - The control plane tracks per-tenant pinned version and rolls out:
- fleet-wide rollout of a stable version deploys to every tenant not explicitly pinned, one rolling Cloud Run revision at a time, respecting pins;
- a tenant can be pinned to a version (regulated customers who validate upgrades) or upgraded individually.
- Schema migrations ship with the image;
insigzctl migrateruns per tenant as part of its deploy. Migrations are forward-only and must be backward-compatible across one version (deploy code, then migrate, or expand→migrate→contract).
Environments
- staging: a non-prod tenant for DAST + manual QA + migration dry-runs.
- prod: per-tenant services; canary the rollout (1 low-risk tenant → wait → fleet).
- Rollback = redeploy the previous revision (Cloud Run keeps revisions); DB migrations use expand/contract so rollback never needs a destructive down-migration.
Decided
- Cloud Build (D9).
- expand→migrate→contract is THE migration rule (not a suggestion): deploy code that tolerates both shapes, migrate, then remove the old shape — so rollback never needs a destructive down-migration (D13).
- Canary cohort: one low-risk tenant first → wait/observe → fleet. Fleet-wide rollout is an Owner/Ops action with an approval step (product §17).
Gaps to close
- Define the exact canary tenant + soak window per release.