Claude Opus 5

I think the most important thing here is not absolute performance. It's that organizations now have access to a Fable-ish model without Fable's 30-day data retention requirement[0].

> "Consistent with prior Opus models, Opus 5 does not have data retention requirements for general access."[1]

On the Opus model release page, the reason why Fable doesn't have an ARC-AGI score is because of that retention policy[2].

0: https://support.claude.com/en/articles/15425996-data-retenti...

1: https://www.anthropic.com/news/claude-opus-5

2: https://xcancel.com/arcprize/status/2064399134099153344

Doing testing with it now, specifically for image->html conversion.

Previously Fable was the best at this, followed by Gemini 3.1 pro (a surprising #2, but Google has great vision models).

Opus' results seem to be more accurate than Fable, following the design source of truth better.

Example results:

Design source of truth: https://image.non.io/73e239a3-880f-4793-b65f-4810be2d9378.we...

Opus 5 build: https://html.non.io/solaraOpus/

Fable 5 build: https://html.non.io/solara/

Note the buttons - for fable they're pill buttons, opus got the rounded rectangle nature of them. Opus' images are closer to the source of truth as well (both LLMs were provided with image gen capabilities for the assets).

Running more tests now, but preliminary results are saying this is indeed better than Fable in some areas. Crazy.

I compared the writing style of Opus 5 vs Fable 5, and Opus 5 continues many of the "Claude-isms" of its 4.8 predecessor in a way that Fable broke away from.

Opus 5 still uses "carry the argument", "worth stating plainly", ", and the trap", "The X matters more", the use of "move"

We need an "annoying English" benchmark.

- Fable 5 Max: https://gist.github.com/deet/3d97f854b48eac6658d642fa18bb24d...

- Opus 5 Max: https://gist.github.com/deet/1a43693a732dfccb4d0d914bfc42692...

Looking at all these releases it’s not a surprise that model routing is the fastest growing segment in AI right now.

There are 10+ LLM companies, each with dozens of models of different modalities, each model with multiple size variants, then different “thinking” levels, then agentic modes, “pro” modes, a “fast” option, standard vs flex vs batch execution. And of course each end combination has a different input/output/cache token price.

Companies that say “give me a prompt and I’ll route it to the most ideal and cost effective model and setting for you” are capturing a ton of value from a gap that model developers don’t seem to understand exists.

https://www.anthropic.com/news/claude-opus-5 - A blog post for those not wanting to go through a 190ish page pdf

Edit: It was pointed out to me that Opus 4.8 got "21%" for successfully fully completing ~1-in-5 tasks, but also got "55.7%" for obtaining significant partial credit on some of the ~4-in-5 tasks it could not fully complete.

---------------

Why does Anthropic say here that Opus 4.8 scored 55.7% on OSWorld 2.0 benchmark, but the paper published by the authors of OSWorld 2.0 say they achieved a benchmark of ~21% with Opus 4.8? [0]

That's a huge gap, considering that the paper was published just 2-4 weeks ago.

I understand that the benchmark authors have an incentive to publish lower numbers (to show that the benchmark has potential longevity) and that Anthropic has incentive to publish higher numbers, but the other models seem pretty inflated as well. The benchmark authors shows GPT-5.5 at 14%, and Anthropic shows GPT-5.6 Sol at 62.6%.

Is there any reasonable explanation for this? Do all the other benchmark numbers need to be sanity-checked as well? Are SOTA benchmarks really this difficult to get consistent, replicable results within a reasonable range of tolerance/variability? Can these benchmarks be compared from one paper to another, or are they only valid to compare intra-paper results?

0: https://arxiv.org/pdf/2606.29537

I had a moderately complex review in a large C/C++ codebase that Codex/GPT-5.6-sol already cleaned up so I threw it at Opus 5. 4 errors found. That seemed odd, so I handed it back to GPT. All were false. Opus doesn't seem to look at the wider context and understand which functions were called in certain contexts. I gave GPT's analysis back to Opus and it admitted its mistake. Maybe it's good for writing code, but as far as analysis it seems like it needs some work.

Isn’t it just hilarious that a model that seemed so superior to Fable but didn't get doomsay marketing from Anthropic got released without any issues? In theory, this was supposed to be AGI level according to Anthropic, yet here we are, just a normal Friday.