Every article about data validation ends in the same place. Mine does too: define the schema, validate at the boundary, and when the payload doesn't match, raise. That's an honest end to the topic and a dishonest end to the job.
raise ValidationError is punctuation. It states that this record is wrong. It says nothing about what the system owes the record next — and that part isn't a matter of taste. It's decided by one question:
Who is supposed to fix this, and can they?
The tutorial always assumes the same answer: a client sent bad JSON, you return 400, and the client fixes it. That's one boundary out of several, and it's the only one where the fixer is a person who will read your error message.
I run a science-and-tech digest that has three boundaries inside one process, with three different answers. The validation code is nearly identical in all three. What happens on the line after it isn't.
Data from strangers: nobody is going to fix it
Feeds are the classic hostile-but-not-malicious input. Publishers ship entries with no link, entries whose title is a chunk of HTML, entries with no date, and occasionally a document that isn't well-formed at all. None of these people are waiting for my error message.
The first decision isn't whether to reject. It's what to reject.
The naive version validates the feed: one malformed entry and the whole poll fails, so a publisher's typo silences a source for as long as the typo lives. What I do instead is validate per entry — an entry with no link or no title is skipped, the rest of the batch is kept, and nothing anywhere logs an incident, because this isn't one.
Reject the smallest thing you can name. The unit of rejection is a design decision and it's usually more consequential than the rule that triggered it. Get the unit wrong and a correct validator becomes an outage with good intentions.
The second decision is subtler, because sometimes I don't reject at all. Entries without a date get stamped with the time I saw them. That is, plainly, a fabricated value — the kind of thing every validation article tells you not to do. I do it anyway, because I can say exactly what it costs: the item is at most slightly too fresh, freshness decays over hours, and it leaves the ranking window on its own. What it buys is that the article exists at all, and a dropped article never comes back.
A default is a lie you've decided you can afford. Which is fine, as long as you can name the price. If you can't, you haven't chosen a fallback — you've chosen not to think about the case.
The third decision has nothing to do with validity. Some feeds — WordPress archives especially — hand you their entire history on a normal poll. Every entry perfectly valid, every field present, and accepting all of them would bury everything published this morning under a decade of old posts. So there's a ceiling on how many entries I take from one feed per poll.
Validity says nothing about quantity, and quantity is its own failure mode. A schema will happily certify ten thousand records that shouldn't have arrived together.
Data from a generator you invited: there's nobody to return the error to
This is the boundary the tutorials don't cover, and it's the one that made me write this. When a model returns a translation my checks reject, the 400 has no recipient. There's no client to notify, no caller to fix its request, no human on the other end of the connection.
Detecting the bad value is its own story, and I've told it separately: a schema-valid response can hold the right number of fields with the wrong language inside them, or a correct headline with a foreign clause welded onto its tail — which a share-based check waves through every time, because by proportion the correct text wins comfortably. The short version is that structured output guarantees shape and nothing about meaning, so the semantic validator is yours to write and it's about as much code as the schema.
Here I care about the line after that check. Not how the bad value is caught, but what the rejection becomes.
If a rejected translation is simply dropped, that's not validation, that's deletion. The next pass through the pipeline takes fresh work off the top; it never revisits what quietly disappeared. Rejecting a value and moving on feels rigorous and is indistinguishable, downstream, from never having tried.
What makes it real is that the rejection leaves a shape behind. The slot stays empty rather than being filled with the bad value, and empty is exactly what the regeneration path looks for. The record's own state is the queue. Nothing needs to be enqueued, retried, or remembered elsewhere, because absence is already the signal.
When there's nobody to return the error to, the rejection has to become a work item — or it's a deletion with better manners. And the corollary that decides your data model: never store a wrong value where an absent one would fit, because absence is a state the system can act on and wrongness looks exactly like success to every query you'll ever write.
Data from yourself: stop
The third boundary is configuration, and its policy is the opposite of the first two. Skipping is wrong. Substituting is wrong. Continuing is wrong.
The database URL in this project used to have a sensible-looking default pointing at a local file. With an unconfigured environment the application started perfectly, connected to an empty database, served an empty feed, and reported nothing unusual, because nothing unusual had happened by its own account. The symptom presented as "the feed is quiet today" rather than "the database was never configured", and those two look identical from the outside for as long as you're willing to keep looking.
The default is gone. The process now refuses to start and says which variable is missing.
The dangerous default is the one that works. A fallback that crashes on first use is a typo with a stack trace. A fallback that lets the process come up healthy is a phantom system: fully operational, structurally sound, connected to nothing. Config isn't the place for resilience — there is no degraded mode worth having when the thing that's misconfigured is what the service is for.
The line after the validator
Same library, same schema shape, three boundaries, three policies:
- from strangers — skip the smallest failing unit, substitute only where you can price the lie, and cap the volume separately from the validity;
- from a generator — turn the rejection into a state the pipeline reads as unfinished work;
- from yourself — refuse to start.
None of that is derivable from the schema, because the schema doesn't know who's on the other side of the boundary. It knows what the data should look like. Whether a wrong record is somebody else's problem, your system's own backlog, or a reason to stop the process is a question about ownership, and it's yours to answer for every boundary you draw.
Validation tells you the data is wrong. It never tells you what you owe it.
This is the third of three on the same seam, and they read in any order:
-
JSON and Data Validation in Python — the contract:
json,jsonschema, Pydantic, and where to put the boundary. - The Schema Was Valid. The Translation Was in Chinese. — what a schema doesn't buy you when the producer is a language model.
- This one — what the system does on the line after the rejection.
All three come out of the same science-and-tech digest.
