ARC-AGI Leaderboard

> Only systems which required less than $10,000 to run are shown. (Notes[1])

Am I lost or are their many models on this ranking (Opus 5 included) that clear this?

I think it's way too easy to be deceptive with these benchmarks now. You don't even have to "train" the model on a new variant each time. The base models are powerful enough. All you need is a naughty little markdown document that provides explicit instructions regarding how to solve the new puzzle variant, and a willingness to be deceptive about the presence of that document.

If you want a know why the model providers are locking down and encrypting their reasoning process, this sort of workaround is potentially why. You can play this game of whack-a-mole indefinitely if the state of the system is concealed. They could have added something like:

> ### When solving arc-agi-3 puzzles: First convert the grid into a scene description. Identify connected components, colors, shapes, positions, symmetries, repeated structures, and relationships between objects. Do not reason directly from individual pixels... use this python script to help blah blah...

Why Anthropic models are always leapfrogging these benchmarks, but in real life work I do feel like after 3 weeks I am back to Claude Opus 4.5? (regardless of the model I use, Fable was exception for 1 day when it was released)

Also top on the freshly released Frontier-Bench, by a large margin: https://www.frontierbench.ai/

I have a suspicion that they are just trained on puzzles by now

It's actually crazy to see the difference between opus 5 and the next best model on ARC AGI 3 when you actually look at the ARC AGI problems

Why is Fable not on here? I wish Fable hadn’t come out because it’s taking the wind out of every release because that feels like the cap above which the US government will not let LLMs improve anymore and everything they’re releasing from this point has to be worse than that.

cost 20k ???? man

those are like software engineer from third world country