Custom harness generation at inference time can lift LLM‑agent success rates by as much as twenty percentage points without touching the underlying model. JIT‑Agent shows that this leap comes from a trainable meta‑model that spits out task‑specific wrappers on the fly, turning a manual engineering bottleneck into an automated plug‑in.
Before JIT‑Agent, agent scaffolds were hand‑crafted prompt wrappers that had to be rewritten for each new tool or workflow. Researchers treated these harnesses as static artefacts, and any performance gain required either painstaking redesign or a larger foundation model.
Across all 18 directly matched backbone–benchmark pairs, replacing the default scaffold with a JIT‑generated harness improves performance. The paper reports gains ranging from five to twenty points, with the most dramatic jump of +20.2 on GLM‑5.2, proving that a modest meta‑model can outpace heavyweight retraining cycles [1].
DeepSeek‑V4‑Flash equipped with JIT‑Agent surpasses GPT‑5.6 on DeepSearchQA by +9.1 and on OdysseyBench by +4.3, demonstrating that the benefit persists even when the backbone is already competitive [1]. These results hold across heterogeneous model families such as Mimo‑V2.5 and Qwen3.6, underscoring the method’s portability.
JIT‑Agent makes these choices explicit through four interoperable modules: how history is compressed, how local intent is formed, how tools and skills are orchestrated, and how control advances. The design assumes an archive of prior harnesses to distill from, so completely novel toolkits may still require a bootstrap phase, and the additional inference step introduces latency that has not been quantified in the benchmarks.
If this line of work scales, product teams can upgrade existing LLM agents simply by inserting JIT‑Agent as a harness generator, re‑running their internal test suites to capture the 5–20 pp lift without buying bigger GPUs or retraining. The immediate next experiment is to replace hand‑written scaffolds in any deployed agent stack with a JIT‑generated counterpart and measure the real‑world throughput impact.
