Skip to content

Platform Catalog Upgrade

Audience: platform administrators. This runbook covers upgrading the shared SCF catalog to a new published version via Platform → Catalog in the web console. For what an upgrade means for a single organisation, see the Catalog Updates & Deprecated Controls guide.

An upgrade is a two-stage process by design:

  1. Platform apply (this runbook) updates the shared catalog for everyone, additively. No organisation’s scoped data is touched.
  2. Per-organisation reconciliation (run tenant-by-tenant from Platform → Tenants) brings each organisation’s scope in line with the new catalog, with a preview and rollback per organisation.

Until an organisation is reconciled, it keeps working against its previous catalog version and sees an upgrade-available banner in Org Settings.

upload workbook → staging (sanity gates) → diff preview → check declared successors
→ typed-confirm apply → completion report (revert available afterwards)

On Platform → Catalog, upload the new SCF workbook (.xlsx as published by the Secure Controls Framework). The run enters staging: the workbook is parsed in isolation and compared against the live catalog. Nothing is changed yet — staging, and every step up to Apply, is read-only.

The target version is detected from the workbook itself; the current version comes from the import-run ledger. Uploading the same or an older version is refused by default; re-staging a corrected same-version workbook (or deliberately downgrading) requires the explicit force option, and the run records that it was forced. A workbook whose version can’t be parsed is always refused — force won’t override that.

Staging blocks the run (status blocked, with a report) if any gate trips:

  • the workbook version cannot be parsed;
  • the control count drops more than 5% against the live catalog;
  • any expected entity sheet yields zero rows;
  • framework-name extraction comes back empty;
  • no framework registry exists for the live catalog version, and the platform cannot recover one (live_framework_registry);
  • the release would drop live frameworks that nothing accounts for (framework_churn).

The control-count gate is a straight comparison of workbook controls against live active controls — pairing the deprecations does not clear it. A release that legitimately retires more than 5% of the catalog will block, and that is the gate working as designed: confirm the workbook is complete before looking for a way past it.

A blocked run cannot be applied. Fix the workbook (or report the layout drift) and upload again; cancel the blocked run from its detail view.

Since v0.41.0 each catalog version’s framework registry — every framework’s display name and the publisher’s focal-document identifier — is stored with the catalog. When a new SCF release renames a framework’s column (the same document under a new label) or issues a new edition of the same document, the upgrade recognises a succession rather than a retirement plus an unrelated addition. Organisations’ framework selections follow the succession; nothing is deleted.

Removals are explained only by the publisher’s own focal-document identifiers (a rename, a new edition) or by the publisher’s declared removals in the workbook’s errata sheet — never by a guess. That is what framework_churn enforces. A block there is the gate doing its job: supply the workbook that explains the removals rather than bypassing it.

Staging therefore checks first that a framework registry exists for the live catalog version. Installs upgraded under older builds may not have one, and the platform recovers it automatically from the workbook of the run that applied the live version. Where that is impossible the run is blocked, and the blocked run’s page shows a Register your current catalog workbook card. Choose the SCF workbook matching the live catalog version (the 2026.2 workbook when 2026.2 is live) and press Register workbook, then discard the blocked run and upload the new workbook again. Headless installs can register the same way with the backfill-framework-registry admin command, documented in backend/cli/README.md.

Once staged, review the diff per entity (controls, domains, evidence, assessment objectives, capability themes, framework mappings), filtered by change class. The staged run keeps the staging checks under a collapsed Staging checks disclosure, so after registering a workbook you can confirm the registry and churn gates now pass without leaving the page. The change classes are:

Change classMeaning
AddedNew in this version
ChangedExisting entries, with field-level old/new values shown
DeprecatedActive in the current catalog but absent from the workbook
ResurrectedPreviously deprecated, present again
UnchangedCounted, not listed

From SCF 2026.3 onwards the workbook also carries the publisher’s own change sheets — frameworks added and removed, controls renumbered, merged, renamed, reworded, moved, and new — and the staged run shows a What the publisher changed panel, named for the version, before apply. That panel is the publisher’s account of the release; the diff is the platform’s. Read both.

The workbook is the authority for what replaced a retired control. For each control the release deprecates, the successor is the one the publisher declares: the Legacy SCF # column of the new workbook (the crosswalk from old identifiers to new), and from SCF 2026.3 the merge list on the publisher’s READ THIS sheet. The pairing editor lists every deprecation with its declared successor and where it came from (crosswalk or publisher merge list), and those successors are written as-is at apply. The platform does not generate successor suggestions of its own.

An administrator can override a declared successor with a different control, or record no successor explicitly. Only overrides are saved; clearing an override returns the row to the declared successor. A deprecation with no declaration and no decision is undecided and applies with no successor, which can be corrected afterwards. The apply report counts successors written from the declaration and successors overridden.

Successors drive the migrate defaults organisations see during reconciliation, and the hint shown when someone tries to scope a retired control. They can be corrected after apply via the audit-logged superseded-by correction endpoint (Platform → Catalog run detail).

Reused identifiers. A publisher merge can retire a control’s meaning while its identifier survives for a different control in the same release. In 2026.3, END-03 “Prohibit Installation Without Privileged Status” was merged into CHG-04.2, and END-03 now names “Endpoint Protection Mechanisms”. The diff classes such a row as Changed and marks it ID reused, naming what was merged where; the summary counts them. These controls are not deprecated, so no succession is written and nothing is re-scoped. Organisations that hold the identifier see the reuse in their reconciliation preview and should review whether the control now behind it still belongs in scope.

Apply requires typing the exact target version as confirmation. The apply runs as a single transaction under an exclusive catalog lock:

  • changed entries are updated field-by-field (org-extracted artifact metadata and creation timestamps are preserved);
  • deprecations are a status flip — never a delete;
  • resurrections re-activate;
  • framework mappings are recomputed;
  • failure at any point rolls the whole transaction back, leaving the catalog untouched.

The completion report lists what changed, plus any controls whose description changes may warrant re-extraction of required artifact types. Re-extract those with backend/scripts/extract_artifact_types.py --force: the window assessor only extracts controls that have never been extracted (see ARTIFACT_TYPE_LAZY_EXTRACTION in Configuration), so it will not refresh an already-populated control on its own.

  • The canonical catalog version (shown on version cards) is now the new one.
  • No organisation has been reconciled yet. Each org keeps its previous behaviour and sees the upgrade-available banner in Org Settings.
  • Reconcile organisations one at a time from Platform → Tenants: preview, confirm the per-row actions, apply. See the Catalog Updates & Deprecated Controls guide.

Only the latest applied run can be reverted. Revert restores changed fields from the stored old values, marks rows added by the run as deprecated (never deletes), and re-activates rows the run deprecated.

Blocker rule: revert is refused while any organisation is reconciled forward to the run’s version — the error lists the blocking organisations. The ordered procedure is:

  1. Roll back each blocking organisation’s reconciliation run (Platform → Tenants → organisation → Rollback).
  2. Re-run the platform revert.

Rolling back an organisation restores its snapshotted rows verbatim; see the reconciliation guide for exact semantics.

SymptomCause / action
Run stuck in stagingCheck the Celery catalog queue worker; the run detail shows the task error on failure.
Run blockedA sanity gate tripped — read the report on the run detail; fix the workbook and re-upload.
Run blocked on framework_churn or live_framework_registryNo framework registry for the live catalog version, so framework removals cannot be explained. Register the live-version workbook from the blocked run, discard that run, then upload the new workbook again. On headless installs use the backfill-framework-registry admin command (backend/cli/README.md).
Apply refused with a version mismatchThe staged run’s target no longer matches what you confirmed — refresh and re-review.
Revert returns a conflictOrganisations are reconciled forward; roll them back first (see above).
An organisation still “sees the old catalog”Expected until that organisation is reconciled — this is the designed two-stage behaviour, not an error.