Why E2E tests need an owner, not a orphan folder — and what 'eyes on production' really means: synthetic checks, alerts, post-deploy smoke, and closing the loop between CI green and users happy.
By codefunc
CI went green. You merged. Production looks fine — until Monday, when Search Console shows 400 new 404s and a tool page throws a client error nobody noticed because nobody was looking.
E2E tests in CI prove a journey worked once, in a controlled environment, before deploy. Eyes on production prove the same journeys — and the metrics around them — keep working after deploy, for real users, on real devices, with real CDN caches and ad scripts loaded.
These are different jobs. Confusing them is how teams end up with flaky Playwright specs nobody maintains and zero alerting when the homepage breaks at 2 a.m.
Bottom line: Own your E2E suite — named maintainers, tied to features, deleted when obsolete. Watch production — synthetic smoke, error tracking, traffic anomalies, and a five-minute post-deploy habit. Shipping without both is shipping blind.
| Question | Answered by |
|---|---|
| Did we break this flow before merge? | Unit, integration, E2E in CI |
| Is it still working for users right now? | Production observability + synthetic E2E |
| Who fixes it when it fails at night? | Ownership — not "the pipeline" |
The QA and test automation guide covers the pyramid — unit, integration, E2E in PR checks. This article covers who owns those E2E tests and what happens after they pass.
Ownership is not "we have a e2e/ folder." Ownership is an accountable answer to:
@skip forever?If the answer is "whoever has time" — you do not have ownership. You have orphaned automation.
flowchart TD
A[Consultant writes 40 Playwright specs] --> B[Team ships UI redesign]
B --> C[80% of specs fail]
C --> D[CI job disabled 'temporarily']
D --> E[Production breaks silently]
Temporary disable becomes permanent. The team loses confidence in E2E; confidence in production goes with it.
| Practice | Detail |
|---|---|
| Feature team owns feature specs | Checkout team owns checkout E2E — not a distant QA silo |
| Rotating maintainer | One person per sprint triages E2E failures — even on small teams |
| Spec ↔ user story link | PR template: "E2E updated? Y/N — which file?" |
| Flake budget | Zero tolerated flakes on main; quarantine or fix within 48h |
| Retirement policy | Delete specs when flows die — dead tests lie |
You build it, you run it (YBIYRI) usually means on-call for services. The same principle applies to journeys:
If you ship a flow users depend on, you ship the test that proves it works — and you respond when that test fails in CI or in production synthetics.
No handoff to "the automation team" after merge.
Owned E2E stays small enough to maintain:
| Tier | Count | Owner | Runs |
|---|---|---|---|
| P0 smoke | 3–8 journeys | Whole team | Every PR + every deploy |
| P1 regression | 10–20 | Feature owners | Nightly |
| P2 exploratory automation | Optional | Best-effort | On demand |
Example P0 list for a developer tools site:
When a test fails, the owner of last touch on that flow investigates first — not whoever is on-call for unrelated infrastructure.
Eyes on production means continuous awareness of user-visible health — not assuming green CI equals happy users.
It combines:
| Layer | What you see |
|---|---|
| Synthetic monitoring | Robots hit key URLs on a schedule |
| Real user monitoring (RUM) | Performance and errors in actual browsers |
| Error tracking | JS exceptions, stack traces, breadcrumbs |
| Traffic & business metrics | GA4, conversion funnels, 404 rate |
| Search & external signals | GSC crawl errors, uptime probes |
| Human smoke | Deliberate post-deploy check |
None of these alone is enough. Together they form defense in depth.
flowchart TB
subgraph pre [Before users notice]
CI[CI E2E on PR]
BUILD[Production build gate]
end
subgraph post [After deploy]
SYN[Synthetic E2E / HTTP checks]
ERR[Error tracking]
RUM[RUM / Core Web Vitals]
LOG[Analytics anomalies]
HUM[Human smoke checklist]
end
CI --> BUILD --> SYN
SYN --> ERR
ERR --> RUM
RUM --> LOG
LOG --> HUM
Synthetic monitoring runs scripted checks against production (or staging that mirrors prod) on a schedule — every 5 minutes, hourly, or after each deploy.
| Dimension | CI E2E | Synthetic production E2E |
|---|---|---|
| Target | Localhost / preview URL | Production domain |
| When | Pre-merge | Continuous / post-deploy |
| Data | Fixtures, test accounts | Real CDN, ads, analytics |
| Purpose | Block bad merges | Detect drift, outages, config errors |
| Owner | Dev writing PR | Team on-call + feature owners |
Production catches things CI cannot:
NEXT_PUBLIC_SITE_URL → broken canonicals, not broken localhostads.txt missing after bad FTP uploadindex.html without new JS chunksNot every check needs a browser. Schedule simple probes:
| Check | Method | Alert if |
|---|---|---|
| Homepage | GET / |
Not 200 |
| Critical tool | GET /tools/json-formatter |
Not 200 |
robots.txt |
GET /robots.txt |
Not 200 or wrong content |
sitemap.xml |
GET /sitemap.xml |
Not 200 or parse error |
ads.txt |
GET /ads.txt |
Missing line for AdSense |
Validate sitemap structure locally with the Sitemap XML Validator when debugging crawl issues — then fix the generator, not just the symptom in Search Console.
Use the HTTP Status Code Lookup when interpreting probe failures — know whether to retry (502) or page someone (500).
Express cron schedules clearly with the Cron Builder — document timezone in the same runbook as the job.
Run 3–5 Playwright specs against production from a scheduler (GitHub Actions cron, Better Stack, Checkly, Datadog Synthetics):
Never run destructive tests against prod — no form submits that email customers, no load tests on shared infra.
Tag synthetic runs in telemetry so you filter bot traffic from analytics.
Client-side tools need JS error tracking even when there is no server:
| Signal | Action |
|---|---|
Spike in TypeError on one tool page |
Reproduce + unit test regression |
| New error after deploy | Correlate with release tag |
| Error only on Safari | Browser-specific bug |
For static sites, errors often mean bad deploy artifacts (chunk 404) or browser extension interference — still worth tracking patterns.
GA4 is not just marketing. Watch:
| Metric | Might indicate |
|---|---|
| 404 page views spike | Broken links, bad deploy paths |
| Tool page views drop to zero | Routing or JS failure |
| Bounce rate jump on one URL | Layout broken, infinite loading |
| Geography-specific drop | CDN region issue |
Set weekly review — even 15 minutes — or automate anomaly alerts if your plan supports them.
Search Console and RUM report LCP, INP, CLS. Regressions often follow heavy third-party scripts (ads, analytics). Eyes on production includes before/after deploy comparison when enabling monetization.
CI green is necessary. Five minutes of human smoke after deploy catches what automation misses.
robots.txt, sitemap.xml, ads.txt reachable (if applicable)For JSON-driven config changes, run sample payloads through the JSON Validator before deploy — invalid registry JSON should fail CI, but human spot-check catches env-specific mistakes.
Document who runs smoke (release author, not "ops") and where (production URL, not staging only).
Full 24/7 paging is not mandatory for every side project. Accountability is.
| Team size | Minimum viable ownership |
|---|---|
| Solo | Weekly prod check + error email alerts + synthetic cron |
| 2–5 | Rotate "prod buddy" weekly; they triage alerts |
| 5+ | Formal on-call rota + runbooks + incident retros |
One doc per common failure:
| Incident | First steps |
|---|---|
| Site 502/503 | Check host status, disk, SSL |
| One tool blank | Reproduce, check browser console, rollback deploy |
| 404 spike | Diff sitemap, check FTP upload completeness |
| E2E synthetic red | Run spec locally against prod; compare with CI |
| GSC "Crawl anomaly" | Fetch robots/sitemap, check noindex |
Rollback must be rehearsed — redeploy previous out/ artifact or revert git and rebuild. If rollback takes an hour, you do not have eyes on production; you have hope.
Every production incident should leave two artifacts:
flowchart LR
A[Production incident] --> B[Fix + deploy]
B --> C[Add regression test]
C --> D[Add/update synthetic check]
D --> E[Runbook entry]
E --> F[Owner assigned]
Without step C, the bug returns. Without step D, the next failure waits for a user email. Without F, the same debate happens twice.
See also: Ship Fast Without Breaking Production for CI and deploy gates that sit upstream of this loop.
Each squad owns unit + integration + E2E for its surfaces and synthetics for its URLs. Platform squad owns shared infra monitors.
Everyone writes features; one E2E captain per sprint fixes flakes, updates selectors, reviews new specs. Captain rotates — knowledge spreads.
Keep P0 synthetics to HTTP + 2 browser journeys. Error tracking email to you. Weekly calendar block: "prod health."
Pick one model and write it down. Ambiguity is the enemy.
| Handoff mentality | Cost |
|---|---|
| "QA owns E2E" | Dev merges breaking UI; QA backlog grows |
| "DevOps owns prod" | App errors nobody understands |
| "Marketing owns analytics" | 404 spike unnoticed for weeks |
| "We'll add monitoring later" | Later is after the outage |
E2E ownership and production eyes are engineering responsibilities — even when specialists help implement tooling.
No servers does not mean no ops. Failure modes shift:
| Failure | Detection |
|---|---|
| Incomplete FTP upload | Synthetic GET on sample of URLs |
| Wrong base URL in build | Broken OG tags, bad sitemap URLs |
| Client JS exception | Sentry / browser errors |
| Ad script blocking render | RUM LCP spike |
| Stale cache | Synthetic checks after cache purge |
Privacy-first tools avoid server breaches — they still owe availability and correctness in the browser.
/, /robots.txt, /sitemap.xml| Anti-pattern | Fix |
|---|---|
| E2E in CI but nothing watches prod | Add synthetics |
| Synthetics green, users complaining | Add RUM + error tracking |
| Alerts with no runbook | Every alert links to first action |
| Alert fatigue (100 emails/day) | Tune thresholds; page only actionable |
| Ownership doc nobody read | Link from README and onboarding |
E2E ownership means the team that changes the UI maintains the tests that prove journeys work — in CI and in production synthetics. Eyes on production means scheduled checks, error and traffic signals, and a human smoke habit after deploy — not assuming merge equals mission accomplished.
Build the pyramid in CI. Then watch the real world. When something breaks, add a test and a monitor so it breaks for the robots next time — not for your users first.
Master SemVer for libraries and apps — pre-releases, caret vs tilde ranges, breaking-change discipline, and a release workflow that keeps dependents sane.
Understand Unix file permissions end to end — the user/group/other model, octal math, symbolic notation, common permission sets, and the security mistakes that cause 403s and leaked keys.
A practical guide to agile frameworks — Scrum sprints, Kanban flow, XP, and Lean — with comparison tables, ceremonies, and honest advice on what actually works for software teams.
Find a developer tool