Ephemeral Environments: Spinning Up a Full Stack Per Pull Request

Every pull request gets its own URL where you can see the running application, click through it, run integration tests against it, and merge with confidence. That is the promise of ephemeral environments. The reality is that they require real engineering to set up well, and they fail in ways that surprise teams who underestimate the cost.

This post is what an ephemeral environment actually involves, the patterns that hold up, and the gotchas that show up six months in.

What "Ephemeral Environment" Actually Means

A throwaway environment that spins up when a PR is opened, contains a deployed copy of the application, and is destroyed when the PR is closed (or after some idle period). Each PR has its own.

PR #1234 opened
  → CI builds the changes
  → Ephemeral env "pr-1234" spun up
  → URL: https://pr-1234.dev.example.com
  → QA reviews the env
  → Tests run against the env
PR merged
  → Env destroyed

The promise is fast feedback on every PR with real, running code — not just unit tests or screenshots.

Why Teams Want Them

• QA can review the actual change before merge, without bumping anyone off main staging.
• Designers can verify visual changes without a deploy queue.
• Integration tests against real services instead of mocks.
• Customer demos of in-progress features without contaminating production.
• Reproducible bug reports ("see PR #1234 environment for the broken state").

When ephemeral environments work, they raise development velocity meaningfully. When they break, they are an expensive distraction.

The Cost

Setting up ephemeral environments is significant work:

• The deployment pipeline has to support spinning up isolated copies
• Data has to come from somewhere (seeded, copied, mocked)
• External dependencies (payments, email, third-party APIs) need test modes
• DNS and TLS have to handle dynamic subdomains
• Costs scale with the number of open PRs
• Cleanup has to be reliable (orphaned environments are expensive)

A first-pass implementation is a week or two. A polished one is several months. Be honest about the investment.

Architecture Patterns

Three dominant patterns for hosting ephemeral environments:

Per-PR namespace in Kubernetes. Each PR gets a namespace (pr-1234). The same Helm chart deploys to it with overridden values. Cleanup is kubectl delete namespace.

# In CI on PR open
helm upgrade --install pr-${PR_NUMBER} ./chart \
  --namespace pr-${PR_NUMBER} \
  --create-namespace \
  --values ./chart/values/ephemeral.yaml \
  --set image.tag=${COMMIT_SHA} \
  --set ingress.host=pr-${PR_NUMBER}.dev.example.com

Per-PR Vercel/Netlify/Cloudflare Pages deployment. For frontend-heavy apps, the platform's built-in preview environments handle most of the work.

Per-PR ECS/Fargate/Cloud Run service. Each PR gets its own serverless deployment. Common for backends that already use the platform.

For most teams running on Kubernetes, the namespace-per-PR pattern is the canonical choice.

The Database Problem

This is where ephemeral environments get hard. Each PR needs data to be useful. Three patterns:

Shared database. All PRs read from one staging database. Cheap, fast to spin up, but PRs cannot make destructive changes without affecting each other.

Per-PR database. Each PR gets its own database, restored from a snapshot. Realistic, isolated, expensive. Spin-up is slow (restore time) and storage costs scale with PRs.

Per-PR database with seeded data. Each PR gets a fresh database with just enough seeded data for the test scenario. Cheap, fast, less realistic than copying production.

For most teams: shared database for the common case, per-PR database for PRs that need it (changes to migrations, schema-affecting changes). The PR's CI declares which option it needs.

Secrets and External Services

The ephemeral environment usually cannot reach production services. The pattern is to provide test-mode credentials:

• Payment APIs in test mode (Stripe test keys, etc.)
• Email through a tool like Mailtrap that captures rather than sends
• Third-party APIs that have sandbox environments
• Internal services that are pointed at staging versions

For services without test modes, the ephemeral environment uses mocks. The line between "real enough to be useful" and "so much mocking it lies" is the design challenge.

DNS and TLS

Each ephemeral environment needs a unique URL. The pattern is wildcard DNS plus wildcard TLS:

*.dev.example.com  →  ingress controller

A wildcard TLS certificate from Let's Encrypt or your CA covers *.dev.example.com. Each PR gets a unique subdomain (pr-1234.dev.example.com) without provisioning per-PR certificates.

This is one of the few places where wildcard certificates are clearly the right call.

Cleanup

The single most important operational concern: ephemeral environments must actually be ephemeral.

Failure modes that produce orphaned environments:

• PR is closed via merge but cleanup CI fails silently
• PR is closed via "close without merging" and no cleanup runs
• Branch is force-pushed and the closure trigger never fires
• Repository is renamed and webhooks break

Three layers of cleanup:

1. PR-close hook removes the environment immediately.
2. Idle timeout removes environments older than N days regardless of PR state.
3. Scheduled audit lists active environments and compares to open PRs, flagging or removing orphans.

Without all three, the environment count drifts up over months. Cloud bills follow.

Cost Control

A typical pattern: 10 active PRs × $5/day per environment = $1,500/month. Manageable.

A pathological pattern: 200 orphaned environments × $5/day × 365 days = $365,000/year. Real money.

Real-time cost monitoring per PR makes the cost visible. Hard limits on number of concurrent ephemeral environments prevent runaway scenarios.

Testing Against Ephemeral Environments

End-to-end tests against the PR's environment are the killer feature. The CI workflow:

1. Build and deploy the PR's environment
2. Wait for it to be healthy
3. Run end-to-end tests against the deployed URL
4. Report results back to the PR

The wait-for-healthy step is more important than it sounds. Tests fired before the deployment is fully up will fail intermittently. A readiness probe that the CI polls before starting tests prevents the flake.

When They Are Worth It

Ephemeral environments pay off when:

• The team is large enough that staging is a bottleneck
• Visual or UX changes need QA before merge
• Integration tests are the primary safety net
• Customer demos of in-progress work are common

They are less worth it when:

• The team is small and main can absorb post-merge bugs
• Unit and contract testing already give high confidence
• The application is heavily backend-only with little visual change
• Cost matters more than velocity (early-stage startups)

A Pragmatic Default

For mid-sized teams (10–50 engineers) on Kubernetes with good test coverage but real visual or QA review needs: invest in ephemeral environments. The week of setup pays back within a month.

For smaller teams or backend-heavy workloads: a shared staging environment is fine. The cost of ephemeral environments outweighs the marginal value over good staging.

For very large teams with platform engineers: ephemeral environments are table stakes. Skipping them is friction the platform should not allow.

The Tooling Landscape

Several products productize this:

• Vercel Preview Deployments / Netlify Deploy Previews / Cloudflare Pages. Built-in for frontend apps. Often the best choice if you are already on these platforms.
• Render Previews. Backend-aware variant of the above.
• Argo CD ApplicationSets with PR generator. For Kubernetes teams already using Argo CD.
• Custom scripts on top of Kubernetes. Most teams end up here. The flexibility justifies the maintenance.

Off-the-shelf products handle 80% of the use case. The remaining 20% — your specific seeding patterns, your secrets strategy, your test runner — is custom anyway.

What to Build vs Buy

Build:

• The seeding/data strategy
• The CI integration with your test runner
• The cleanup logic

Buy:

• DNS and TLS
• The hosting platform itself
• The CI runner
• Database snapshotting (RDS snapshots, etc.)

Most teams that try to build all of this from scratch underestimate the operational cost. The 80% you get from off-the-shelf tools is worth it.

Considering ephemeral environments for your delivery pipeline and not sure whether the cost justifies the velocity? We help teams scope the investment based on team size and the bottlenecks they actually have. scopeforged.com