Skip to content
SRE & DevOpsDeep Dive Published Updated 5 min readViews unavailable

Container Tags, Digests, SBOMs, and Provenance: Building a Verifiable Release Chain

Why latest is not an identity, how OCI digests anchor releases, and how signatures, attestations, SBOMs, and admission policy work together.

A container tag is a human-friendly pointer. A digest is a content-derived identifier for an OCI manifest. If app:latest points somewhere new tomorrow, yesterday’s deployment manifest no longer tells an investigator what ran. Pinning app@sha256:… makes the artifact reference stable, but digest pinning alone says nothing about who built it or whether the source and build process were trusted.

Evidence layers

An SBOM inventories components. A vulnerability scan compares that inventory and package metadata with known advisories. A signature links an identity or key to an artifact. A provenance attestation describes how an artifact was built. These records answer different questions and should be attached to the digest, retained with the release, and verified before deployment.

A defensible pipeline

Build once in an isolated CI identity; generate an SBOM and provenance; scan the artifact; sign or attest it using the organization’s supported keyless or key-managed workflow; then promote the same digest. Admission policy can require trusted issuer/subject claims and approved provenance rather than merely checking that some signature exists.

Registry controls

Use repository permissions, immutable release tags, retention exceptions, encryption, audit logs, and replication. Cleanup must preserve deployed and investigatory artifacts plus their attestations. Mirroring an image does not automatically mirror every signature format or policy record, so test verification at the destination.

Remaining limits

Signed malware is still signed. An SBOM may omit statically linked or vendored components. A clean scan is time-bound because new vulnerabilities are disclosed later. Supply-chain security therefore combines reproducible policy evidence, least privilege, patch/rebuild cadence, runtime controls, and incident response.

What a digest actually covers

An OCI digest is computed over a serialized object such as an image manifest or index. The manifest references a configuration object and layer blobs by their own digests. A multi-platform image usually exposes an index whose entries point to platform-specific manifests. Pinning only the index is stable, but an investigation should record which platform manifest and layers a runtime selected. Registry copying and garbage collection must preserve the complete referenced graph.

A digest provides integrity and identity for bytes, not origin. Anyone can calculate a digest for malicious content. The release record must connect digest to source revision, declared dependencies, builder identity, build parameters, and policy result. Store that connection in signed provenance or another tamper-evident system, and keep the verifier’s trust configuration under review.

SBOM quality and vulnerability decisions

An SBOM should declare its format and version, generator, generation time, subject digest, package identifiers, versions, and dependency relationships where available. Generate it from the final image as well as build metadata when possible: build-time and runtime views catch different material. Validate that the SBOM’s subject matches the deployed digest and retain it after registry cleanup.

Scanner output needs triage context. A package may be present but unreachable, or exploitable despite having no vendor fix. Policy should define severity thresholds, reachable-risk analysis where supported, exception owner, compensating controls, expiry, and rebuild deadline. Re-scan deployed digests or continuously reevaluate stored metadata because an artifact that was clean yesterday can match a newly published advisory tomorrow.

Provenance and verifier policy

Provenance should identify the builder, source, build invocation, and produced subject according to the adopted specification. SLSA defines provenance expectations and levels or tracks through its current specification; do not claim a level without satisfying the exact versioned requirements. Keyless signing shifts trust from a long-lived signing key toward an identity provider, certificate authority, and transparency evidence, but verifier policy must constrain issuer and subject rather than accept every valid certificate.

For key-managed signing, protect keys in an appropriate KMS or hardware boundary, separate build and verification permissions, rotate deliberately, and retain evidence needed to validate historical releases. Revocation or identity compromise needs a response procedure: identify affected digests, update admission policy, rebuild from trusted source, and preserve incident evidence.

Admission without creating an outage

Admission policy can require a trusted registry, digest, signature, provenance predicate, builder identity, and approved source. Roll it out in audit mode first, measure existing violations, and define availability behavior if the verifier, transparency service, or registry is unreachable. A fail-open policy preserves deployment availability but weakens enforcement; fail-closed protects the boundary but needs redundant verification infrastructure and an emergency process.

Policy exceptions must be narrow, time-limited, attributable, and visible in deployment records. Avoid a global “unsigned allowed” switch. Test rollback artifacts before enforcing new retention or signature rules, because an old known-good image that lacks current evidence can become undeployable during an incident.

A verifiable promotion record

For every environment, record source revision, image index and selected platform digest, SBOM digest, provenance and signature references, scanner database time, policy version, approver or automated decision, and deployment identity. Promote the same subject digest; if copying to another registry changes associated referrers or signature discovery, test verification at the destination.

Periodically select a deployed workload and reconstruct the chain from runtime digest back to source and builder. The audit fails if evidence exists but cannot be correlated, if retention deleted a required object, or if the verifier accepts an identity outside policy. This exercise tests the supply chain as a system rather than as a collection of security products.

Include negative fixtures in that exercise: an unsigned image, a valid signature from the wrong identity, altered provenance, a digest with an expired exception, and an artifact whose SBOM subject does not match. The verifier should reject each case with an actionable reason while the approved rollback image remains deployable. Record policy and trust-root versions so the result can be reproduced after rotation.

Related:

Sources:

Comments