Configuration and deployment

CR0 recovery — 2026-10-22

CR0 is completed as of 2026-10-23: PR #132 main qualification/deploy succeeded, the user accepted desktop and report behavior, and a live backend fallback to the previous qualified image and return passed while preserving SQL/schema/storage. At CR0 closure, the backend was restored to d9afef7. See the final evidence and exact scope. CR1 subsequently completed on 2026-10-27. On 2026-10-29, CR2 main 4fafb84 passed Quality and deployed on GKE with two all-role backend replicas and a zero-replica dedicated worker; the operator accepted desktop/report behavior. Dependencies, ingress, volumes and recorded migration list were preserved. PR #140 subsequently fixed the namespace patch and qualified the split plus a real split → all → split rollback with installed desktop acceptance. The final compatibility release 274499b (PRs #141/#142) then passed exact-main Quality and controlled rollout on 2026-10-30: two HTTP replicas and one dedicated worker ready, PostgreSQL/MinIO retained, configuration/migrations/PVC/ingress and aggregate counts preserved. CORS was applied only on the shared report/screenshot bucket; its temporary grant was revoked and IAM bindings match baseline. See final live evidence. Actual browser acceptance is explicitly deferred to destination qualification; phase-marker advancement remains to be consolidated. Real Cloud Run application/data deployment and traffic cutover are not performed by this release. The earlier all-rollout evidence retains its historical transient observations.

PR #131 merged as 98f4527. Its main Quality and image builds succeeded; deployment initially exceeded the 45-minute wait during slow APT transfers. A retry of only failed jobs recovered deployment. PR #132 then published the bounded installer and Astro patch; its main run completed deployment in 12m58s, with native installation in 59 seconds on the static job. The installer repair evidence preserves the earlier failure and local fixture.

The Cloud Run migration plan is active at CR2; the following CR0 recovery details remain historical. PRs #128 and #130 are merged and deployed on GKE: dedicated source-built MonStaff MinIO/client, repaired backup toolchain, storage-before-seed ordering and image builds overlapping exact-SHA Quality. The latter run completed in 34m29s with 11m51s of image work overlapping Quality. MinIO’s full PVC was expanded online from 30 to 60Gi; a real backup, isolated SQL restore and standard CronJob execution succeeded. The independent GCS recovery is now qualified: all 3709 current object names/sizes, six small/large sample hashes and a SQL restore from GCS passed, with the key preserved outside GKE. Recurring offsite backup, point-in-time consistency and application/rollback acceptance remain separate. The restore script requires source DATABASE_URL, an empty RESTORE_DATABASE_URL, and a completed new-format backup with checksum. Historical unmarked dumps remain preserved. Cloud SQL testudodev is PostgreSQL 18; GKE is 17.11.¹

The GKE overlay reconciles the existing prebound MinIO PV/PVC at 60Gi after actual expansion. Its immutable StatefulSet claim template retains the original 30Gi seed. For future resizing, increase the PVC request first, verify actual CSI/filesystem expansion, then reconcile recorded PV capacity; do not preemptively enlarge the PV capacity field. The completed manual backup does not attest that the next scheduled run has already executed.

Bucket monstaff-backups-dev is provisioned through isolated root infra/recovery/. Temporary copy credentials/resources were removed after qualification. See the GCS evidence for the prefix, protection settings and exact limits, and the earlier timing/capacity evidence for the GKE repair.

Terminal retirement — publication boundary, 2026-10-23

Backend/Admin terminal retirement is deployed on GKE: new interactive terminals and relay operation are disabled, while command/script Remote Actions and protected read-only history remain. The repository also removes the native PTY runner; no new installed-desktop release was published as part of CR0. Explicit Redis resource cleanup remains separate. See the retirement plan and publication evidence; earlier local implementation descriptions below are historical.²

Configuration groups

The backend validates environment variables through apps/backend/src/config/env.ts. Major groups are:

• PostgreSQL: DATABASE_URL;
• GitHub OAuth: client ID, client secret and callback URL;
• MonStaff auth: JWT secret and expiry;
• encryption: 32-byte hex key for stored third-party tokens;
• S3: endpoint, credentials, region, path style, screenshot, desktop-update and managed-AI-pack buckets;
• Local AI release trust: desktop builds pin the JSON MONSTAFF_AI_PACK_PUBLIC_KEYS; the corresponding private key remains outside Git and backend storage;
• web origins: ADMIN_URL, canonical PUBLIC_API_URL and explicit TRUST_PROXY in production;
• desktop admission: optional inclusive DESKTOP_MIN_VERSION and DESKTOP_MAX_VERSION semantic bounds; a blank bound is open and there is no per-cohort gate since ADR-024;
• desktop build: VITE_API_URL.

Example files must contain placeholders only. Production secrets belong in a secret manager or Kubernetes Secret populated outside Git.

Remote Actions release trust

Desktop releases with MONSTAFF_CORE_REMOTE_ACTIONS_ENABLED=1 must embed a nonempty MONSTAFF_REMOTE_ACTION_PUBLIC_KEYS JSON map of key IDs to Ed25519 SPKI public PEM keys. The backend signs with REMOTE_ACTION_SIGNING_PRIVATE_KEY; deployment sources it from the GitHub Actions secret K8S_REMOTE_ACTION_SIGNING_PRIVATE_KEY. The configured key ID is remote-actions-v1, the backend default. Only the public key belongs in desktop build variables. Release validation and extracted CLI/Core qualification check that enabled builds contain the expected trust keys.³

The 0.3.1 release preparation fixes missing release trust. It does not enable macOS remote execution: the current Core rejects that platform because it lacks an accepted detached-process containment boundary. Terminal and elevation capabilities remain separately gated, as do organization policy and execution approval.³

Desktop platform set

Quality and Release now enable macOS Apple Silicon, macOS Intel, Linux x64 and Windows x64. Release and promotion set DESKTOP_RELEASE_PLATFORMS empty, so installed qualification, supply-chain verification and candidate validation require every platform. Both native matrices and both workflow environments must remain aligned. The previous Apple-Silicon-only development restriction is superseded by the user request of 2026-09-09. The 0.3.40 run exposed Linux/Windows Clippy errors in macOS-only URL scheme definitions. Their correction passed Linux Clippy in 0.3.41. Windows package creation also exposed CRLF version parsing and masked validation failure; both corrections passed in 0.3.42, which completed four-platform qualification, promotion and GKE deployment. Public verification exposed incorrect x64 CLI version labels; 0.3.43 repaired metadata generation and candidate validation, then passed all four native qualifications, promotion and GKE deployment. All eight public GUI/CLI packages match their published hashes and sizes.⁴⁵⁶⁷⁸

The bilingual download page lists published artifacts from the live manifest and offers explicit architecture selection for the GUI and CLI/Core. It does not infer Mac CPU architecture from the browser. Production EN/IT selections were verified against the 0.3.43 artifacts. Windows publisher trust remains outside the configured support boundary; updater signing is enabled.⁶⁴

Local deployment

docker/docker-compose.dev.yml supplies PostgreSQL and MinIO and initializes private development buckets. Optional profiles add digest-pinned ClickHouse plus OpenTelemetry Collector (telemetry) and JWT-enabled OnlyOffice Document Server plus the internal renderer (reports). Verification proves OTLP ingestion/late revision/tenant filtering, a 20,000-row ClickHouse backup-delete-restore, real LibreOffice fixture conversion and healthy renderer/OnlyOffice containers. Neither profile is a production deployment contract. docker/docker-compose.test.yml provides isolated PostgreSQL and MinIO on dynamic host ports; the integration runner now dumps the exercised database, restores it into a second empty database and verifies representative counts.⁹¹⁰¹¹

Production containers

Production Compose, Dockerfiles, Kubernetes bases/overlays and OpenTofu are checked by pnpm config:check and pnpm infra:check. The Node 26 backend, Admin and website build stages install the repository-pinned pnpm version explicitly because the selected base image no longer provides the corepack executable; configuration validation protects this bootstrap contract.¹² The repaired images built and published remotely, the migration Job passed and GKE completed the commit-addressed rollout.¹³ Application containers run as non-root; Kubernetes applies seccomp, dropped capabilities, read-only application filesystems and explicit resources. MinIO and Ollama images are digest-pinned, required buckets are initialized without anonymous access, and schema migration runs as a dedicated pre-rollout Job.

Kubernetes

The backend is horizontally scaled and has resources, separate liveness/readiness probes, an HPA, disruption budget and ingress/egress NetworkPolicies. The GKE overlay also deploys one bounded LibreOffice-backed report renderer and one JWT-enabled OnlyOffice Document Server behind office.monstaff.io; both have dedicated probes, resources, services and network-policy boundaries. PostgreSQL and MinIO remain single-replica stateful workloads, now protected by disruption budgets, encrypted scheduled backup, versioned object storage and scripts/restore-drill.sh. Local PostgreSQL dump/restore is verified; this does not substitute for an isolated production-equivalent database/object-store drill, approved RPO/RTO or managed high availability.¹⁴

The Prometheus alert bundle under k8s/observability/ is optional because PrometheusRule requires the operator CRD; it is intentionally not part of the base Kustomize resources.

Release flow

A desktop release starts with a version bump on main: when apps/desktop/package.json changes version, “Tag desktop release” creates desktop-v<version> on that exact commit and dispatches “Release MonStaff” on the tag; a version already tagged elsewhere is refused, so a broken release is fixed by bumping again, never by moving a tag. The version line is four files that verify-desktop-version.mjs requires to agree: apps/desktop/package.json, apps/desktop/src-tauri/tauri.conf.json, apps/desktop/src-tauri/Cargo.toml and version under [workspace.package] in the root Cargo.toml, which the Core and CLI crates inherit. Release and deploy wait for the exact-SHA Quality run (require-quality-run.mjs --wait, 45-minute budget) instead of failing fast; the former “Resume Quality consumers” workflow is gone. The release builds its native lane while Quality runs and only assembles the promotion candidate once Quality has passed; the Quality jobs it requires are derived from the lanes quality.yml declares. Since the product-owner decisions of 2026-09-05, promotion is automatic: the last job of “Release MonStaff” dispatches “Promote qualified MonStaff candidate” for its own run id, tag and commit with the repository token (ADR-026), and the promotion preflight waits up to 15 minutes for that run to complete before running the same candidate validation and post-approval revalidation as a manual dispatch. The earlier workflow_run trigger (ADR-025) never fired, because GitHub emits no such event for a release the tagging workflow created with GITHUB_TOKEN; desktop-v0.3.10 was promoted by hand before the repair. pnpm release:promote remains the manual path for re-promoting a candidate or promoting an older run; it derives its four inputs from the latest successful tagged release run. The desktop-production environment carries no protection rules in the development phase, so a release cycle is push, tag, Quality plus native build in parallel, candidate assembly and promotion with no human step after the push; adding required reviewers to that environment is the way to reintroduce an approval.¹⁵¹⁶

GitHub Actions runs the shared quality job before draft release, release or deploy. Deploy builds the backend, Admin, website and report-renderer images, applies prerequisites and runs the migration Job. That job applies migrations and executes the default-report seed twice, failing the deploy if either seed run fails, before the workflow rolls out and waits for the commit-addressed application and report-service deployments. The proxied office.monstaff.io A record was created through Cloudflare and is mirrored by the OpenTofu declaration; future DNS reconciliation remains separate because the deploy workflow has no Cloudflare credential. Signed platform packages and a live post-deploy canary remain release evidence, not locally self-certifiable results.¹⁴

The Admin image receives the numeric x.x.x MonStaff version from apps/desktop/package.json, the full commit SHA and UTC image-build time as non-secret Vite build arguments. The workflow rejects a non-numeric release version. The running sidebar displays the semantic version and exposes the commit and time in a tooltip; the short commit remains the image tag, so an operator can correlate the SPA with its commit-addressed container without confusing the tag with the release number.¹⁷¹⁸

Managed model artifacts follow the separate Managed AI Pack publication runbook. Compose and Kubernetes provision a versioned monstaff-ai-packs bucket; Kubernetes backup mirrors it alongside screenshots and desktop updates.

Selective Quality checks — 2026-09-09

Published and fully qualified after the daily-report deployment exposed unnecessary native builds: server/web-only changes skip the desktop renderer and native matrix; shared TypeScript or renderer changes retain desktop JavaScript checks without native compilation. Native, release/version, dependency/build tooling and unknown paths retain full qualification. Push scope includes all changes since the last successful trusted ancestor, so cancelled or failed earlier changes cannot be hidden by a later backend-only push; unavailable history/API falls back to all checks. A final always-running gate validates every selected job and each intentional skip. Deploy uses that exact-commit gate, while desktop release still requires every native lane and renderer check. Native caches now include Core/CLI’s root workspace as well as Tauri. All 72 delivery tests and the actual local web-only static wrapper pass. Remote run 34323687575 passed all ten jobs, including four native lanes and the final gate. The subsequent evidence-only push passed scoped run 34326563951 in 2m 46s, intentionally skipping every desktop job. The actual exact-SHA consumer accepted it for deployment and rejected it for a full desktop release.¹⁹²⁰ Existing CIR residual work is not closed by this change.²¹

CR2 storage compatibility pending deployment

The locally qualified storage change shares byte-verified immutable publication across desktop releases and AI packs. GCS signed update uploads require staging completion; legacy streaming verifies before replacing a download. Cleanup of abandoned staging uses the existing retention worker. CORS maintenance has dedicated storage-plan/storage-apply CI operations with a CORS-only plan guard on monstaff-screenshots-dev only, matching the browser report-template/bundle upload flows. The operator authorized a temporary bucket-scoped metadata grant and revocation; execution is narrowed to this one bucket. Remote Quality, grant/apply/browser checks and final rollout remain pending; GKE currently retains MinIO/PostgreSQL and the migration marker remains prepare.

The browser scope review supersedes the earlier three-bucket CORS proposal; update/AI-pack buckets remain excluded.

The Admin CSP correction, locally checked in the real container, permits the configured STORAGE_ORIGIN in connect-src as well as img-src. This is required for browser report uploads alongside bucket CORS; it adds no wildcard origin. Actual browser CORS qualification remains distinct.

The final local report-upload correction signs metadata headers explicitly for report template and bundle uploads; existing Admin clients already send x-amz-meta-sha256. Fourteen GCS checks in the actual image and ten MinIO checks pass. Actual browser acceptance was explicitly deferred by the operator; remote qualification and final rollout remain pending.

The publisher boundary correction preserves public GET verification for the desktop publisher rather than adding direct S3 read requirements. Fifteen GCS and eleven MinIO checks pass; main 88bda53 Quality succeeded, while rollout was held before apply for this correction. Browser acceptance remains deferred.

CORS applied 2026-10-30: CI apply and revocation evidence confirms shared-bucket-only CORS, expected origins allowed and untrusted/publisher-bucket preflights denied. The temporary bucket grant was removed and the full IAM binding hash matches baseline. Final application rollout remains pending the publisher correction; browser acceptance is deferred and the marker stays prepare.

Current CR2 result: 274499b — rollout e verifiche finali. Earlier pending-rollout notes above are historical; real Cloud Run application/data qualification and cutover remain CR3/CR4.

Qualificazione CR3

È disponibile localmente il workflow CR3, con applicazioni reali, copia isolata, profilo qualification, backend h2c opzionale e controller riprendibile. I default operativi restano normal/HTTP1. La qualificazione locale non attesta un deploy Cloud Run, DNS o OnlyOffice condiviso: le prove remote e il marker qualified sono ancora pendenti.

Per CR3 gli endpoint sono quelli HTTPS run.app già assegnati ai quattro servizi. Il runbook aggiornato non richiede domini di prova, credenziali Cloudflare, una nuova chiave KMS o una password fixture configurata in GitHub: riusa ENCRYPTION_KEY e il formato di backup esistenti, e genera la credenziale di prova in memoria. I callback OnlyOffice usano l’origine run.app del backend. Questa scelta non qualifica i futuri domini operativi CR4.

Footnotes

1. CR0 live baseline and local recovery qualification. ↩

2. Local interactive-terminal and Redis retirement. ↩

3. Remote Actions release trust configuration. ↩ ↩²

4. Desktop 0.3.43 four-platform publication and public verification. ↩ ↩²

5. Four-platform publication and CLI manifest version repair. ↩

6. Four-platform desktop release restoration. ↩ ↩²

7. Four-platform native gate repair for desktop 0.3.41. ↩

8. Windows CRLF version detection and release fail-fast repair. ↩

9. AT, TS, customer and report foundation implementation evidence. ↩

10. Active workstream quality, load and recovery evidence. ↩

11. Residual active-plan repository-local closure. ↩

12. Node 26 container package-manager bootstrap repair. ↩

13. Node 26 container deployment success. ↩

14. Report Center redesign, default-model seed and OnlyOffice deployment evidence. ↩ ↩²

15. Automatic desktop promotion after a successful tagged release. ↩

16. Release-dispatched desktop promotion. ↩

17. Admin deployment metadata surface. ↩

18. Admin semantic deployment version repair. ↩

19. Successful full qualification of selective Quality. ↩

20. Successful scoped remote Quality and consumer verification. ↩

21. Selective Quality checks and native workspace cache. ↩