r/LocalLLaMA · · 1 min read

Animated transition from AA Intelligence Index v4.1 to v4.3

Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.

Animated transition from AA Intelligence Index v4.1 to v4.3

I had all the data saved from AA's v4.1 index, so when they upgraded it in the wake of Astra's release, I could actually generate a before/after comparison.

  • All intelligence and price per task are sampled from AA on Sep 3rd and Sep 14th respectively.
  • Price per task of some open models were rescaled to reflect the cheapest available on OpenRouter as of Sep 3rd.
  • X axis is linear, because people's money is linear.

All models are the same. The only thing that changes is the weighted sum of the benchmarks that compose the Intelligence Index.

v4.1: https://github.com/crusaderky/llm-intelligence-cost-plot/blob/intelligence-index-v4.1/plots/high_intelligence.png

v4.3: https://github.com/crusaderky/llm-intelligence-cost-plot/blob/intelligence-index-v4.3/plots/high_intelligence.png

Highlights

  • GLM an Muse Spark remain more or less unaltered, in relative terms
  • GPT-5.6 Sol becomes a lot cheaper
  • GPT-6 Astra's intelligence flies up to the stars AND becomes cheaper
  • GPT-5.6 Luna gets a substantial uplift
  • Fable-5.1's price gap from Opus 5 shrinks, and becomes cheaper than Fable 5.0
  • Fable-5.1 at low, medium and high effort looks a lot more appealing
  • Sonnet 5 becomes even more expensive without any intelligence gains
  • Kimi-K3, Qwen3.8-Max, Gemini-3.8, and Grok 4.6 go down into the gutter
submitted by /u/crusaderky
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/LocalLLaMA