I had a system prompt that ended with: Reply with only a JSON object. Do not include code fences. Do not include explanation.
That should be enough. It is not.
Over a week of structured-output calls (12,400 of them, mostly Claude Sonnet 4.5 and 4.7), I logged how often the response was actually parseable as JSON on the first serde_json::from_str attempt. Result: 86.0%. The other 14% had at least one of these problems:
-
json ...wrappers (most common, about 9.3% of total) - Leading or trailing prose ("Here is the JSON you asked for:")
- Trailing comma in the last array or object element
- Smart quotes pulled in from training data
So I wrote llm-json-repair. Three passes, applied in order, each one cheap, none of them re-invoking the LLM. If a downstream model fix is needed, that is your problem to glue on. This crate is the local cleanup before you spend another API call.
The three passes
use llm_json_repair::repair;
let raw = r#"```
json
{
"intent": "book_flight",
"slots": {
"origin": "DAL",
"destination": "JFK",
},
}
```"#;
let cleaned = repair(raw)?;
let parsed: serde_json::Value = serde_json::from_str(&cleaned)?;
What repair actually does, in order:
Pass 1: strip fences
Look for the first ` and the last `. If both exist, take what is between them. The optional json language tag gets dropped along with the opening fence. If there is text after the closing fence, it is discarded.
This pass is a one-liner conceptually, but the edge case that bit me was a JSON string value that contained the literal substring ` (yes, a customer-support transcript got fed in once). So the matcher only strips fences at the outermost level and only if they are on their own line or at the start.
Pass 2: balanced extraction
Walk character by character. Find the first { or [. Count nesting. Stop when nesting hits zero. Return that substring.
This drops leading prose ("Here is the JSON:") and trailing prose ("Let me know if you need anything else.") without needing a model. It also handles the case where the model emitted two JSON objects in a row (rare but real); you get the first complete one.
`rust
let raw = "Sure thing! Here is the JSON:\n{\"intent\": \"refund\", \"reason\": \"late\"}\nLet me know if you have questions.";
let cleaned = repair(raw)?;
assert_eq!(cleaned, r#"{"intent": "refund", "reason": "late"}"#);
`
The cost of this pass is O(n) over the response. For a 4 KB response it runs in about 30 microseconds on my laptop.
Pass 3: trailing comma fix
Walk the candidate JSON. For each , followed only by whitespace and then } or ], delete the comma. Skip when inside a string literal (track quote state with escape handling).
This is the single most common syntactic failure I see from Claude when it is generating a long object: it puts a trailing comma after the last field. It is a small thing but serde_json rejects it. Fixing it locally saves you a round trip.
A real broken case from my logs
Raw response:
`jsonjson
`
{
"summary": "User wants to cancel order #4419",
"actions": [
{"name": "cancel_order", "args": {"id": 4419}},
{"name": "notify_user", "args": {"channel": "email",}},
],
}
`json
`
Three problems: fences, two trailing commas, valid JSON otherwise. After repair:
`json
{
"summary": "User wants to cancel order #4419",
"actions": [
{"name": "cancel_order", "args": {"id": 4419}},
{"name": "notify_user", "args": {"channel": "email"}}
]
}
`
Parses cleanly. No second LLM call.
When to bail out
If all three passes run and the result still does not parse, repair returns an error. You should not silently swallow that. The crate exposes try_repair which returns Result<String, RepairError> and gives you the failing pass in the error variant.
`rust
match try_repair(raw) {
Ok(s) => use_json(s),
Err(RepairError::Pass3(_)) => {
// bracket structure is fine but content is wrong
// worth one model-side retry with the error attached
}
Err(e) => log::warn!("could not repair: {e}"),
}
`
What this does not solve
A few honest limits.
- It does not invoke the LLM. If the model emitted a hallucinated field name, this crate cannot fix it. Use
agentcast-rsif you need a validate-and-retry loop. - It does not enforce a schema. It only restores parseability. You still need
serdederives orjsonschemaon top. - It does not handle every JSON5 extension. Single-quoted strings, unquoted keys, and comments are not supported. I have not seen Claude emit those, so I have not added the passes.
- It does nothing with NDJSON (newline-delimited JSON). One blob in, one blob out.
The whole crate is about 350 lines of safe Rust with no async. It works as a sync function or inside any runtime.
Repo: https://github.com/MukundaKatta/llm-json-repair
crates.io: llm-json-repair = "0.1"
Part of a small set of Rust crates I publish for the unglamorous LLM plumbing: parsing, cost, retry, budget, repair. Sibling crates: claude-cost, llm-retry, agentcast-rs.
