In 2021, a malicious image uploaded to Docker Hub was pulled over 10 million times before it was detected. The SolarWinds attack demonstrated that supply chain compromise is a real and sophisticated threat vector. Container image security is no longer just "run a scanner" — it's a pipeline discipline that spans from base image selection through runtime enforcement.
Here's how to build a layered approach that catches problems at every stage.
The Container Supply Chain
Understanding what you're protecting starts with mapping where images come from:
Upstream OS vendor
→ Base image (ubuntu:22.04, node:20-alpine)
→ Your application layers
→ CI builds and pushes image
→ Registry stores image
→ Kubernetes pulls image
→ Runtime executes container
Vulnerabilities and tampering can enter at any stage. Your security controls need to cover the entire chain.
Base Image Selection
The base image is the foundation of your security posture. A bloated base image brings thousands of packages — each a potential vulnerability.
Prefer Minimal Base Images
# Avoid: Full OS base brings ~300 packages, many unnecessary
FROM ubuntu:22.04
# Better: Debian slim — smaller surface area
FROM debian:12-slim
# Best for most apps: Distroless — no shell, no package manager
FROM gcr.io/distroless/java17-debian12
# Smallest possible: Scratch (static binaries only)
FROM scratch
COPY myapp /myapp
CMD ["/myapp"]
Distroless images contain only the application and its runtime dependencies. No shell means no shell injection. No package manager means no way to download additional tools. Many CVEs in base images are irrelevant to distroless because the vulnerable component simply isn't there.
Multi-Stage Builds to Minimize Runtime Image
# Build stage: full SDK needed
FROM golang:1.22 AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -o server ./cmd/server
# Runtime stage: only the binary
FROM gcr.io/distroless/static-debian12
COPY --from=builder /app/server /server
USER nonroot:nonroot
EXPOSE 8080
ENTRYPOINT ["/server"]
The build stage has the full Go SDK, git, and build tools. The runtime image has just the compiled binary. The attack surface shrinks from ~1000MB to ~10MB.
Pin Base Image Digests
Tags are mutable. node:20-alpine today might not be the same image next week. For production, pin by digest:
# Mutable tag — can change without warning
FROM node:20-alpine
# Immutable digest — this exact image forever
FROM node:20-alpine@sha256:0a2dfbc3d5d99c1a40b0e5f11d0cb0aa6ef1e0be069c12f34f9e42d8a73de44d
Your CI pipeline should periodically update these digests as upstream images are patched.
Vulnerability Scanning in CI
Scan images before they can reach production. Multiple scanners are better than one because each has different vulnerability databases and detection capabilities.
Trivy: The Workhorse
Trivy is fast, free, and comprehensive. It scans OS packages, language dependencies, and IaC files:
# .github/workflows/security.yml
name: Container Security
on: [push, pull_request]
jobs:
scan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Build image
run: docker build -t myapp:${{ github.sha }} .
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@master
with:
image-ref: myapp:${{ github.sha }}
format: sarif
output: trivy-results.sarif
severity: CRITICAL,HIGH
exit-code: 1 # Fail pipeline on critical/high findings
- name: Upload Trivy results to GitHub Security
uses: github/codeql-action/upload-sarif@v3
if: always()
with:
sarif_file: trivy-results.sarif
SARIF upload integrates results directly into GitHub's Security tab, so developers see vulnerabilities in context.
Scanning Non-OS Dependencies
Trivy also scans language package managers, which is where many vulnerabilities live:
# Scan a directory for all vulnerability types
trivy fs --scanners vuln,secret,misconfig ./
# Output:
# package.json (npm)
# ├── lodash 4.17.15 - CVE-2021-23337 (HIGH)
# ├── axios 0.21.1 - CVE-2021-3749 (HIGH)
# └── ...
#
# Dockerfile
# ├── Running as root
# └── No HEALTHCHECK defined
Secrets Detection
Accidentally committed secrets are a critical risk. Scan for them before the image is pushed:
# Trivy secret scanning
trivy image --scanners secret myapp:latest
# Dedicated secret scanning with Gitleaks
docker run --rm -v $(pwd):/repo zricethezav/gitleaks:latest detect \
--source /repo \
--report-format sarif \
--report-path gitleaks-report.sarif
Common secrets found in container images: AWS credentials in build ARGs, database connection strings baked into images, API keys in environment variable defaults.
Image Signing with Cosign
Scanning tells you what's vulnerable. Signing tells you what's authentic. Cosign (from the Sigstore project) makes it easy to sign and verify images.
Signing Images in CI
# In GitHub Actions, using keyless signing (OIDC-based)
- name: Install Cosign
uses: sigstore/cosign-installer@v3
- name: Sign image
env:
COSIGN_EXPERIMENTAL: "true" # Enables keyless mode
run: |
cosign sign \
--yes \
myregistry.io/myapp:${{ github.sha }}
Keyless signing uses OIDC tokens from the CI provider (GitHub, GitLab, etc.) as the identity. No long-lived keys to manage. The signature is stored in Rekor, a public transparency log.
Verifying Images Before Deployment
# Verify an image was signed by your CI pipeline
cosign verify \
--certificate-identity-regexp="https://github.com/myorg/myapp" \
--certificate-oidc-issuer="https://token.actions.githubusercontent.com" \
myregistry.io/myapp:1.2.3
# Output includes:
# Verification for myregistry.io/myapp:1.2.3 --
# The following checks were performed on each of these signatures:
# - The cosign claims were validated
# - Existence of the claims in the transparency log was verified offline
# - The code-signing certificate claims were validated
If verification fails, the image was not built by your CI pipeline. Either it was tampered with, or someone pushed it manually.
Software Bill of Materials (SBOM)
An SBOM is a complete inventory of everything in your container image. Required by many compliance frameworks and increasingly by customers.
# Generate SBOM with Syft
synft packages myapp:latest -o spdx-json > sbom.spdx.json
# Attach SBOM to image in registry
cosign attach sbom \
--sbom sbom.spdx.json \
--type spdx \
myregistry.io/myapp:${{ github.sha }}
# Later: download and inspect SBOM for any image
cosign download sbom myregistry.io/myapp:1.2.3
With an SBOM attached, when a new zero-day vulnerability is announced (Log4j style), you can query your registry to find every image that contains the affected component — in minutes, not days.
Registry Security
Admission Control with OPA/Gatekeeper
Prevent unsigned or unscanned images from running in your cluster using Kubernetes admission controllers:
# Gatekeeper constraint: require signed images
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequireSignedImages
metadata:
name: require-signed-images
spec:
match:
kinds:
- apiGroups: ["apps"]
kinds: ["Deployment", "StatefulSet", "DaemonSet"]
excludedNamespaces:
- kube-system
parameters:
allowedIssuers:
- "https://token.actions.githubusercontent.com"
allowedSubjectPattern: "https://github.com/myorg/.*"
Registry Policies
Configure your container registry with policies that prevent insecure practices:
// AWS ECR registry policy example
{
"rules": [
{
"rulePriority": 1,
"description": "Keep last 30 tagged images",
"selection": {
"tagStatus": "tagged",
"tagPrefixList": ["release-"],
"countType": "imageCountMoreThan",
"countNumber": 30
},
"action": { "type": "expire" }
},
{
"rulePriority": 2,
"description": "Expire untagged images after 7 days",
"selection": {
"tagStatus": "untagged",
"countType": "sinceImagePushed",
"countUnit": "days",
"countNumber": 7
},
"action": { "type": "expire" }
}
]
}
Enable ECR scan-on-push to automatically scan every image when it arrives in the registry.
Runtime Security
Even with clean images, runtime security matters. Containers can be exploited through application vulnerabilities.
Security Contexts
Run containers with the minimum required privileges:
spec:
securityContext:
runAsNonRoot: true
runAsUser: 10001
runAsGroup: 10001
fsGroup: 10001
seccompProfile:
type: RuntimeDefault
containers:
- name: api
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop:
- ALL
add:
- NET_BIND_SERVICE # Only if binding to port < 1024
volumeMounts:
- name: tmp
mountPath: /tmp # Writable tmp for read-only root
volumes:
- name: tmp
emptyDir: {}
readOnlyRootFilesystem: true means attackers who compromise your container can't write malware or modify files on disk. They're limited to what's in memory.
Falco for Runtime Threat Detection
Falco watches system calls and alerts on suspicious behavior — even from otherwise clean images:
# Custom Falco rule
- rule: Unexpected Network Connection
desc: Detect unexpected outbound connections from api containers
condition: >
outbound and
container.name = "api" and
not fd.sip in (allowed_ip_ranges)
output: >
Unexpected connection from api container
(user=%user.name command=%proc.cmdline
connection=%fd.name container=%container.name)
priority: WARNING
A container that normally only accepts inbound connections suddenly making outbound connections to an unusual IP is a strong indicator of compromise.
Building the Full Pipeline
Here's the complete security gate sequence:
1. Code commit
→ Secret scanning (Gitleaks)
→ SAST scanning (CodeQL, Semgrep)
2. Image build
→ Multi-stage build (minimize attack surface)
→ Pin base image digests
3. Pre-push scanning
→ Vulnerability scan (Trivy)
→ Fail on CRITICAL/HIGH CVEs
→ Secrets in image scan
4. Push to registry
→ Registry scans on push (ECR, Artifact Registry)
→ SBOM generation and attachment
→ Image signing (Cosign)
5. Deploy to cluster
→ Admission controller verifies signature
→ Policy enforcement (no root, read-only FS)
→ Runtime monitoring (Falco)
6. Continuous
→ Periodic rescans of deployed images
→ Alert on newly discovered CVEs in running containers
Each layer catches different problems. The goal isn't perfect prevention — it's making the attacker's job as hard as possible at every stage.
Building something that needs to scale? We help teams architect systems that grow with their business. scopeforged.com
