Platform Catalog Upgrade

Audience: platform administrators. This runbook covers upgrading the shared SCF catalog to a new published version via Platform → Catalog in the web console. For what an upgrade means for a single organisation, see the Catalog Updates & Deprecated Controls guide.

An upgrade is a two-stage process by design:

Platform apply (this runbook) updates the shared catalog for everyone, additively. No organisation’s scoped data is touched.
Per-organisation reconciliation (run tenant-by-tenant from Platform → Tenants) brings each organisation’s scope in line with the new catalog, with a preview and rollback per organisation.

Until an organisation is reconciled, it keeps working against its previous catalog version and sees an upgrade-available banner in Org Settings.

The flow at a glance

upload workbook → staging (sanity gates) → diff preview → pair deprecations
→ typed-confirm apply → completion report        (revert available afterwards)

1. Upload

On Platform → Catalog, upload the new SCF workbook (.xlsx as published by the Secure Controls Framework). The run enters staging: the workbook is parsed in isolation and compared against the live catalog. Nothing is changed yet — staging, and every step up to Apply, is read-only.

The target version is detected from the workbook itself; the current version comes from the import-run ledger. Uploading the same or an older version is refused by default; re-staging a corrected same-version workbook (or deliberately downgrading) requires the explicit force option, and the run records that it was forced. A workbook whose version can’t be parsed is always refused — force won’t override that.

2. Staging and the sanity gates

Staging blocks the run (status blocked, with a report) if any gate trips:

the workbook version cannot be parsed;
the control count drops more than 5% against the live catalog;
any expected entity sheet yields zero rows;
framework-name extraction comes back empty.

The control-count gate is a straight comparison of workbook controls against live active controls — pairing the deprecations does not clear it. A release that legitimately retires more than 5% of the catalog will block, and that is the gate working as designed: confirm the workbook is complete before looking for a way past it.

A blocked run cannot be applied. Fix the workbook (or report the layout drift) and upload again; cancel the blocked run from its detail view.

3. Diff preview

Once staged, review the diff per entity (controls, domains, evidence, assessment objectives, capability themes, framework mappings), filtered by change class:

Change class	Meaning
Added	New in this version
Changed	Existing entries, with field-level old/new values shown
Deprecated	Active in the current catalog but absent from the workbook
Resurrected	Previously deprecated, present again
Unchanged	Counted, not listed

4. Pair deprecations

For each planned deprecation of a control, decide its successor (superseded by). The system suggests up to three candidates (same domain, similar name) — suggestions are display-only and never auto-applied; an administrator must pair (or explicitly record “no successor”) in the pairing editor.

Pairings drive the migrate suggestions organisations see during reconciliation, and the hint shown when someone tries to scope a retired control. Pairings can be corrected after apply via the audit-logged superseded-by correction endpoint (Platform → Catalog run detail).

5. Apply

Apply requires typing the exact target version as confirmation. The apply runs as a single transaction under an exclusive catalog lock:

changed entries are updated field-by-field (org-extracted artifact metadata and creation timestamps are preserved);
deprecations are a status flip — never a delete;
resurrections re-activate;
framework mappings are recomputed;
failure at any point rolls the whole transaction back, leaving the catalog untouched.

The completion report lists what changed, plus any controls whose description changes may warrant re-extraction of required artifact types. Re-extract those with backend/scripts/extract_artifact_types.py --force: the window assessor only extracts controls that have never been extracted (see ARTIFACT_TYPE_LAZY_EXTRACTION in Configuration), so it will not refresh an already-populated control on its own.

6. After the platform apply

The canonical catalog version (shown on version cards) is now the new one.
No organisation has been reconciled yet. Each org keeps its previous behaviour and sees the upgrade-available banner in Org Settings.
Reconcile organisations one at a time from Platform → Tenants: preview, confirm the per-row actions, apply. See the Catalog Updates & Deprecated Controls guide.

7. Revert (platform level)

Only the latest applied run can be reverted. Revert restores changed fields from the stored old values, marks rows added by the run as deprecated (never deletes), and re-activates rows the run deprecated.

Blocker rule: revert is refused while any organisation is reconciled forward to the run’s version — the error lists the blocking organisations. The ordered procedure is:

Roll back each blocking organisation’s reconciliation run (Platform → Tenants → organisation → Rollback).
Re-run the platform revert.

Rolling back an organisation restores its snapshotted rows verbatim; see the reconciliation guide for exact semantics.

Troubleshooting

Symptom	Cause / action
Run stuck in staging	Check the Celery `catalog` queue worker; the run detail shows the task error on failure.
Run blocked	A sanity gate tripped — read the report on the run detail; fix the workbook and re-upload.
Apply refused with a version mismatch	The staged run’s target no longer matches what you confirmed — refresh and re-review.
Revert returns a conflict	Organisations are reconciled forward; roll them back first (see above).
An organisation still “sees the old catalog”	Expected until that organisation is reconciled — this is the designed two-stage behaviour, not an error.

Catalog Updates & Deprecated Controls — what reconciliation means for an organisation
Monitoring — Celery queue health
Backup & Restore — platform backup procedures