I was building a streaming chat interface against MonkeyCode's free model access and its free server option, and the first few test runs went so smoothly that I stopped thinking about the transport layer entirely. The server sent back clean SSE events, each one a complete JSON object carrying a token of text, and my parser happily decoded them one by one. Then I asked for a longer response, and the terminal lit up with json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) on the second event.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
The full response, captured to a file, was valid JSON. The model had not failed. My parser had failed because it assumed something the SSE protocol never guarantees: that each event contains a complete, self-contained JSON document.
What the Stream Actually Looked Like
Server-Sent Events is a simple framing protocol. Each event is a set of key: value lines terminated by a blank line, and the data: field carries the payload. When a model streams tokens, the server does not promise to flush one complete JSON object per event. It flushes whatever bytes are ready, which means a single logical object can be split across several events, or several objects can be packed into one event.
data: {"choices":[{"delta":{"content":"Hel
data: lo"}}]}
data: {"choices":[{"delta":{"content":" world"}}]}
data: [DONE]
My naive parser treated each data: line as an independent JSON document:
# Wrong: assumes every event is a complete JSON object
for event in sse_events:
payload = event.data
obj = json.loads(payload) # crashes on the first fragment
The first event in my trace contained {"choices":[{"delta":{"content":"Hel — a fragment with no closing brace. The second event contained lo"}}]} — a fragment with no opening brace. Both were meaningless on their own and perfectly meaningful together.
The Fix: Buffer, Then Decode Incrementally
The robust approach is to accumulate raw bytes in a buffer and attempt a decode only when the buffer might contain a complete object. Python's json.JSONDecoder.raw_decode is the right tool: it parses one JSON value from the start of a string and tells you where it ends, without requiring the whole string to be valid JSON.
import json
class IncrementalJSONParser:
def __init__(self):
self.buffer = ""
def feed(self, chunk: str) -> list:
self.buffer += chunk
decoder = json.JSONDecoder()
results = []
idx = 0
while idx < len(self.buffer):
try:
obj, end = decoder.raw_decode(self.buffer, idx)
except json.JSONDecodeError:
break # incomplete fragment; wait for more bytes
results.append(obj)
idx = end
while idx < len(self.buffer) and self.buffer[idx] in " \t\r\n":
idx += 1
self.buffer = self.buffer[idx:]
return results
The loop tries to decode at the current position. If it succeeds, it yields the object and advances past it, then tries again in case multiple objects arrived in the same chunk. If it fails, it stops and waits for more data. The leftover bytes stay in the buffer for the next call.
Using it with an SSE client is straightforward:
parser = IncrementalJSONParser()
for event in sse_events:
for obj in parser.feed(event.data):
handle_token(obj)
The parser keeps its own buffer across calls, so split fragments reassemble naturally and concatenated objects decode one after another.
Handling a Mid-Stream Disconnect
The incremental parser fixes the fragmentation problem, but free servers can also drop the connection mid-stream. At that point you have three choices:
- Discard the partial response and retry the whole request. Simple, but wasteful if the model was 90% done.
- Use the partial text if your use case tolerates truncation. A summary that ends mid-sentence is often better than no summary.
- Retry with a resume hint — send the partial text back to the model and ask it to continue. This is fragile and model-dependent; I would not rely on it.
For my batch summarizer, I chose option 1 with a retry cap of three attempts, because summaries are short and a truncated summary is worse than a delayed one. For a chat interface, option 2 is more humane: show what arrived, then let the user ask for more.
Limitations
The incremental parser handles JSON fragments and concatenated objects, but it has real limits:
- It assumes the stream is UTF-8 text. Binary-safe protocols need a different approach.
- It buffers everything in memory. For a multi-megabyte response, that defeats the purpose of streaming.
- It does not validate that the objects form a meaningful sequence. You still need to check for
[DONE]sentinels or other end-of-stream markers and filter them out before decoding. - If the provider wraps JSON in extra framing, like
data: [DONE]at the end, you need to handle that separately.
Who Should Not Use This
If your provider guarantees one complete JSON object per SSE event, the naive parser is fine and this whole article is overkill. If you are not actually streaming — if you just want the final response — call the non-streaming endpoint and json.loads the whole thing. Incremental parsing is a tool for the specific case where the stream is fragmented, the response is large, or you want to render tokens as they arrive.
The Takeaway
Streaming protocols and JSON documents have different ideas about where a message ends. SSE events are transport boundaries, not data boundaries, and confusing the two produces bugs that look like model failures but are actually client failures. Buffer, decode incrementally, and decide in advance what a mid-stream disconnect means for your use case.
If you are experimenting with free model access and a free server, expect the stream to be less polished than a paid tier: more fragmentation, more disconnects, more surprises. Build your parser to survive those surprises, and the model's output quality becomes the only thing you have to judge.
