This is a submission for DEV's Summer Bug Smash: Clear the Lineup powered by Sentry.
This is my second entry for the DEV x Sentry Bug Smash challenge. Entry #1 was a crash with a confusing error message. This one is the opposite and it is scarier: no crash, no message, no event. Just silence.
The bug that sends you nothing
smolagents runs LLM-generated Python in a sandboxed executor with a timeout. Issue #2473 claims that one line of model-generated math defeats it completely:
from smolagents.local_python_executor import LocalPythonExecutor
executor = LocalPythonExecutor(additional_authorized_imports=[], timeout_seconds=2)
executor.send_tools({})
executor("10 ** 10 ** 8") # 2 second timeout. Should be fine, right?
I ran this with a 2 second timeout and a faulthandler bomb set for 20 seconds. The timeout never fired. The process sat frozen until the external kill. An agent that generates this expression (and "compute this huge number" is exactly the kind of thing agents try) freezes its host process forever.
Zero comments on the issue, zero PRs. Mine now.
Why the timeout lies to you
smolagents' timeout is thread based: a worker thread runs the code, the main thread waits in future.result(timeout=2). That design is fine for almost everything, because CPython switches threads between bytecode instructions.
But 10 ** 10 ** 8 is not "almost everything". CPython computes arbitrary precision **, << and * inside a single C call that holds the GIL from start to finish. No bytecode boundary, no thread switch, no timeout. The result would have about 400 million bits. The computation takes somewhere between minutes and hours. Your watchdog needs the GIL to wake up, and it never gets it.
The faulthandler dump made it concrete, and it was worse than the issue described:
Thread 0x16d1f3000 (worker):
File "local_python_executor.py", line 753 in evaluate_binop <- computing the pow
Thread 0x1f00d5e80 (main):
File "threading.py", line 999 in start
File "concurrent/futures/thread.py", line 180 in submit <- never returned!
The main thread was still stuck inside ThreadPoolExecutor.submit. It never even reached future.result. The 2 second timer never armed at all.
The existing MAX_OPERATIONS guard (10 million AST operations) does not help either. This is a handful of AST nodes. The entire cost lives inside one of them.
The Sentry angle: monitoring for absence
Entry #1 was about a noisy failure. This bug is the opposite. I pointed a fresh Sentry project at a simulated agent run on the unpatched PyPI release (1.26.0) and got the most unsettling result possible: nothing. No event, no transaction, an empty project. The process was frozen mid-transaction and the SDK never got a chance to flush.
The lesson: for freeze-class bugs you need a supervisor. I added a small watchdog process that gives the worker 25 seconds, then kills it and reports what it saw, with the worker's faulthandler stack attached as evidence:
watchdog: starting worker (before) with 25s budget
worker: smolagents 1.26.0
worker: step 1 executing 'result = 10 ** 10 ** 8'...
watchdog: worker FROZE, killed it, reporting to Sentry
The resulting Sentry issue carries the whole story in one place: environment tagged before, handled by the supervisor, with the frozen frame (evaluate_binop, line 753 of local_python_executor.py) sitting in the attached worker_faulthandler_stack extra.
Sentry's Seer ran root cause analysis on that issue and independently landed on the same conclusion: signal and thread interruption need the GIL, and a single C-level big-int operation never releases it.
Seer's verdict, verbatim:
CodeAgent's 2s timeout uses Python signal-based interruption, which cannot fire during uninterruptible C-level big-int operations that hold the GIL. [...] A single large big-integer arithmetic operation runs entirely as one C-level call that holds the GIL continuously without yielding. CPython cannot deliver signals or switch threads during an uninterruptible C extension call, so no timeout callback fires for the duration of that operation.
The fix: you cannot interrupt it, so refuse to start it
Killing the computation mid-flight is impossible from Python. But predicting the damage is O(1). Before executing **, << or * on integers, the executor now estimates the result's bit length from the operands' bit lengths:
if op == "**":
estimated_bits = left.bit_length() * right # upper bound
elif op == "<<":
estimated_bits = left.bit_length() + right
elif op == "*":
estimated_bits = left.bit_length() + right.bit_length()
Above 1 million bits (about 300k digits, still generous) it raises an informative InterpreterError:
Operation '**' would produce an integer of around 400000000 bits, exceeding
the maximum of 1000000 bits allowed. Use smaller operands, or
pow(base, exp, mod) for modular exponentiation.
That message matters. The agent surfaces it to the model, and the model can actually act on it: use pow(base, exp, mod), which stays unrestricted because modular exponentiation is fast and legitimate. The agent recovers on the next step instead of hanging the host.
The after run on the patched build: the guard rejects the expression in 0.0 seconds, the error lands in Sentry as a normal actionable issue and step 2 executes fine.
watchdog: starting worker (after) with 25s budget
worker: smolagents 1.27.0.dev0
worker: step 1 error surfaced to the model: InterpreterError: ...
worker: step 2 executing '2 + 2'...
worker: step 2 ok
worker: DONE
watchdog: worker exited with code 0
A robot reviewed my robot fix
Minutes after I opened the PR, OpenAI's Codex reviewer flagged a real hole: my guard checked type(x) is int, which lets bool and int subclasses slip through. True << 10**9 and class BigInt(int) still reached the uninterruptible C calls. Fixed with isinstance, added both as regression tests. AI found the bug class, AI fixed it, AI reviewed the fix. I just steered.
Numbers
- Freeze reproduced at 20+ seconds (would have run for hours), external kill required
- Fix rejects the same expression in 0.0 seconds
- 9 explosive patterns blocked:
**,<<, chained*, all augmented forms,pow(a, b), bool and int subclass variants - 10 legitimate operations verified untouched: 100! via repeated
*=,pow(7, 2**64, 97), float pow,1 ** 10**9 - 19 new tests, full test file 416 passed, ruff clean
- The blocked-pattern tests hang forever on unpatched main. I verified that the honest way, with a stash and a kill switch.
Links
- Issue: https://github.com/huggingface/smolagents/issues/2473
- PR: https://github.com/huggingface/smolagents/pull/2551
- Entry #1: https://dev.to/himanshu_748/i-fixed-a-smolagents-bug-that-confused-everyone-who-hit-it-with-sentry-watching-the-whole-time-1im
The scariest bugs are not the ones that page you at 3am. They are the ones that make sure nothing ever pages you at all. Instrument for silence.
