The contact form on a company site I look after stopped producing leads. Not "fewer leads" — zero, for three days. No alert fired. The uptime monitor was green the entire time. The status page had a nice unbroken bar.
The monitor was checking the homepage. The homepage was fine. /contact had been returning a fatal error since a Tuesday afternoon deploy.
This is the most boring class of outage there is, and it is the one that costs the most, because nothing tells you it is happening. Below is what actually broke, the four ways a site can be broken while still answering 200, and the small set of checks I run now instead of pinging the root URL.
What broke
The site loads a list of helper files at boot. During an unrelated edit, that list was rewritten, and one entry did not survive the rewrite — a helper that supplied data to a few templates.
Most pages never touch that helper, so they kept rendering. One page called it in a template. That page fataled. In production, with display errors off, a fatal means an empty response body or a generic error page — depending on how the stack is configured, sometimes with a 500 status, sometimes with the web server's own branded page and a status you did not choose.
The error log had the answer on line one, at the exact timestamp of the deploy. Nobody looked, because nothing said to look.
Three days of a form that a campaign was actively driving traffic to. The traffic arrived. The page did not.
Four ways to be broken and still return 200
If your health check is curl -o /dev/null -w "%{http_code}" https://example.com/ you are testing one route, one time, for one property. Here are the failure modes that sail straight through it.
1. The route you don't check. A site is not a URL, it is a few dozen to a few hundred routes. Homepages are the least likely page to break, because everything touches them and everyone looks at them. The pages that break are the ones with one unusual dependency: the form page, the search page, the report that joins four tables.
2. The page renders, but a part of it silently didn't. Templates are forgiving by design. A partial that throws inside a try/catch, a widget whose data source returned an empty array, a block that renders only when a config key exists — none of these change the status code. The page looks right at a glance and is missing the thing that made it worth serving.
I found one of these in the same audit: a function that emitted structured data (Article, BreadcrumbList) existed, was correct, and was called from nowhere. Thirty news URLs had been building JSON-LD that never reached the HTML. Status: 200. Value delivered: none.
3. The page is fine and the form is not. This one is hard to see. A full-page HTML cache was turned on. HTML caches are dumb by nature: they store bytes. Those bytes included a CSRF token. Every visitor got the first visitor's token, every POST failed validation, and the failure looked to the user like a page reload with no message. Every page on the site: 200. Every form on the site: broken.
4. The error page returns 200. Custom error handlers that render a friendly "something went wrong" view and forget to set the status. Frameworks that catch late and fall back to a template. Reverse proxies serving a static maintenance page with the default status. Your monitor sees 200 and a body of some length, and reports health.
What to assert instead
The fix is not a bigger monitoring product. It is asserting the things you actually care about. Four checks, in increasing order of value.
Check every route, not one route
You already have the route list — it is in your router, your sitemap, or both. Walk it.
#!/usr/bin/env bash
# routes.txt: one path per line, from the sitemap or the router
FAIL=0
while read -r path; do
code=$(curl -s -o /tmp/body -w '%{http_code}' "https://example.com${path}")
size=$(wc -c < /tmp/body)
if [ "$code" != "200" ] || [ "$size" -lt 2000 ]; then
echo "FAIL ${path} status=${code} bytes=${size}"
FAIL=1
fi
done < routes.txt
exit $FAIL
The size floor is doing real work here. A fatal that returns 200 with an empty or near-empty body is caught by bytes, not by status. Pick the floor from your own smallest legitimate page, then subtract a little.
Assert content, not just bytes
One string per route, chosen to be the thing that page exists to do. The form page must contain a <form and a submit button. The article page must contain an <h1. The page with structured data must contain application/ld+json.
grep -q '<form' /tmp/body || echo "FAIL ${path}: no form in body"
grep -qi 'fatal error\|<b>Warning</b>\|stack trace\|Undefined variable' /tmp/body \
&& echo "FAIL ${path}: error signature in HTML"
That second grep catches the inverse case: the page that renders and prints a warning into the output where a visitor — or a crawler — can read it.
Diff the error log around every deploy
This is the cheapest high-value check in the list and almost nobody does it. Record the log size before the deploy, read the tail after.
BEFORE=$(stat -c%s /path/to/error.log)
# ... deploy ...
sleep 30
AFTER=$(stat -c%s /path/to/error.log)
if [ "$AFTER" -gt "$BEFORE" ]; then
tail -c $((AFTER - BEFORE)) /path/to/error.log
echo "New errors appeared during deploy — investigate before walking away."
fi
In our incident this would have printed the missing helper, by name, thirty seconds after the deploy that caused it. The information was there the whole time. Nothing surfaced it.
Round-trip the form
The only way to know a form works is to submit it. A synthetic submission is about twenty lines and can run hourly:
<?php
// 1. Fetch the page like a browser would, keeping cookies.
$ch = curl_init('https://example.com/contact');
curl_setopt_array($ch, [
CURLOPT_RETURNTRANSFER => true,
CURLOPT_COOKIEJAR => '/tmp/probe-cookies',
CURLOPT_COOKIEFILE => '/tmp/probe-cookies',
]);
$html = curl_exec($ch);
// 2. Pull the CSRF token out of the markup, exactly like a browser.
preg_match('/name="_token" value="([^"]+)"/', $html, $m);
$token = $m[1] ?? null;
if (!$token) { exit("FAIL: no CSRF token on the page\n"); }
// 3. Submit with a marker you can find and delete later.
$marker = 'probe-' . date('YmdHi');
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query([
'_token' => $token,
'name' => $marker,
'email' => 'probe@example.com',
'message' => 'automated probe, safe to delete',
]));
curl_exec($ch);
// 4. Assert the effect, not the response.
$found = (int) $pdo->query(
"SELECT COUNT(*) FROM leads WHERE name = " . $pdo->quote($marker)
)->fetchColumn();
echo $found === 1 ? "OK\n" : "FAIL: submitted, nothing stored\n";
Step 4 is the point of the whole thing. Not "did the POST return 200" — did a row appear. That single assertion catches the frozen-token case, the silently failing mail transport, the validation rule someone tightened, and the database column that quietly truncated the input.
Give the probe an obvious marker and delete its rows on a schedule. Yes, this writes to production. That is the trade: a handful of tagged test rows a day against not knowing your lead form is dead.
Why this keeps happening
Three reasons, and none of them are laziness.
Monitoring defaults are shaped by hosting, not by the product. A default check answers "is the machine up and serving". That was the right question when a machine going down was the common failure. The common failure now is code that is up and wrong.
Green is a feeling, and feelings are expensive to give up. An unbroken status bar makes people stop looking. Silence should mean "the assertions passed". In practice it means "nothing asserted anything".
Silent failure has no symptom by definition. A crash gets escalated in an hour. A form that swallows submissions gets noticed when someone asks why the pipeline is empty — which, in our case, took three days, and only because someone went looking for a different number.
The short version
- A status code is a transport fact. It says the request reached something. It does not say the page did its job.
- Check every route, not the homepage. Your sitemap already lists them.
- Assert one meaningful string per route, and grep the body for error signatures while you are there.
- Diff the error log across every deploy, automatically. Cheapest check here.
- Round-trip the forms that make you money, and assert the stored row, not the response.
- Anything that renders "sometimes" — structured data, widgets, conditional blocks — needs an output assertion, because code review will not catch a function that is never called.
None of this is sophisticated. The whole set is a cron job and about a hundred lines. The reason to write it is that the alternative is finding out from a person, and by then you have already paid for the traffic.
I build and maintain news and e-commerce platforms at alestaweb.com. Most of what I know about monitoring came from outages exactly this dumb.
