Most AI features fail quietly. A summary is a little off, a label is wrong, a suggestion gets ignored. Nobody files a ticket and the product keeps working.
AutoKon is not that kind of product. It is a financial verification pipeline for Indonesian property developers: a field worker photographs a housing unit, a vision-language model reads the photo, and that reading is part of what releases a contractor's payment. A wrong description does not degrade the experience. It moves money to the wrong place.
That one property changed how I built everything else.
The pipeline does not end at the model
The obvious architecture is photo in, VLM out, decision made. Claude Opus Vision is very good at describing a construction site. It will tell you the walls are rendered, the roof trusses are up, the floor screed is done. It will occasionally tell you all of that, with the same confidence, about a photo showing something else. Rarely, but often enough to matter.
So every AI-generated description passes through an LLM-as-judge evaluation stage before it goes anywhere near a disbursement decision. A second model grades the first one against the image and the claim, and the item proceeds only if it clears the bar.
The judge is not a quality dashboard bolted on after launch. It sits in the payment path. If the eval stage is down, disbursements wait. That is the right failure mode for this domain, and settling it early is what made the rest of the design tractable.
What the judge is actually measuring
Most eval work I see optimizes for whether the answer is good. For a payment gate that is the wrong question.
The question is whether the description is grounded: whether every claim in it is supported by what is in the photograph. A description can be well written, plausible and useful, and still assert a completed stage that is nowhere in the frame. That is the failure that must never reach the disbursement engine.
So the judge gets a narrow prompt. Is each claim here supported by this image? It is never asked whether the description reads well. Ungrounded claims fail the item even when the writing is excellent, and most of the value sits in that narrowing. A broad quality rubric produces a judge that agrees with the generator far too often, because both models share the same optimistic priors.
The number that makes a judge worth having
An automated judge is only worth having if it agrees with a human reviewer, and you have to be able to say by how much.
The pipeline targets Cohen's Îș â„ 0.80 against human labels, which is substantial agreement on any reading of the scale. That is an operating threshold, not a figure for a slide. Below it the judge is a source of false confidence rather than a control, and the honest response is to widen human review until the rubric or the labels improve.
Three things I learned getting there. Human labeling time is scarce, so spend it where the generator and the judge disagree rather than on the cases that were never in doubt. Ambiguous rubrics show up as low Îș well before they show up as bad outcomes; when agreement dropped, the cause was almost always a rubric that let two reasonable reviewers score the same photo differently, not a model that got worse. And the input distribution keeps moving. New site types, new phone cameras and the rainy season all change what the camera sees, so an eval suite frozen at launch decays.
Where the humans sit
The eval gate does not replace approval. AutoKon runs a five-role photo-evidence approval workflow, and field submissions arrive over WhatsApp through an n8n and Botpress flow rather than through a portal that every field worker would need an account for and training on.
The AI approves nothing. It clears the obviously complete, obviously grounded evidence out of a reviewer's queue and flags what needs a person. Reviewers get the ambiguous cases with their context attached, which is where human judgment earns its cost. That is what let one engineer run a platform across thousands of housing units and three timezones without the review burden growing in step with the number of units.
What transfers
Not every product moves money, but the questions carry. What is the worst thing a confident wrong answer can do here, and does your design start from that or from what the model can do? Is your metric measuring correctness against the source, or plausibility? Can you say what your agreement with a human actually is? Are the people still in the loop getting the hard cases or the easy ones?
The model was never the hard part of this build. The hard part was the structure around it, and being willing to let that structure stop the happy path when it has to.
