AI benchmarks: understanding the scores of Claude and other models
Benchmarks measure AI models' abilities — code, reasoning, knowledge. They're useful but tricky. Here's what they really measure, how to read them, and where to track Claude's scores as releases land (no invented numbers: exact values change with every version).
Latest benchmark & evaluation news
- What I have done with Claude Code in the last 60 days being a non tech person — Hacker News
- Show HN: We're inviting Anthropic to put the real Mythos 5 on our open benchmark — Hacker News
- Fiszki flashcards without an app: your AI quizzes you, FSRS keeps score — Hacker News
- Anthropic Opus 4.8 is new SOTA on ARC-AGI-3, Score: 1.5%, –$10K — Hacker News
The benchmarks that matter
A few references recur: SWE-bench (solving real software bugs, key for agentic coding), MMLU and MMLU-Pro (general knowledge), GPQA (expert-level scientific reasoning), MATH and GSM8K (math), HumanEval (code generation). Each lights up a different facet — none alone captures 'intelligence'.
How to read a benchmark without being fooled
An isolated score often lies. Beware data contamination (the test may have leaked into training), conditions (with or without tools, which prompt), and which versions are compared. A model can top a benchmark and disappoint on your task. The best test is your own.
Agentic benchmarks
The new generation measures tool use and autonomy: SWE-bench Verified, TAU-bench, agent benchmarks. This is where the future is decided, and where models built for action — like those behind Claude Code — are judged.
Tracking scores in real time
Numbers shift with every model release. Rather than freezing a quickly outdated ranking, follow official announcements and our feed, Models category, which relays results as they come.
Frequently asked questions
What's the best benchmark for code?
SWE-bench (and its Verified variant) is the reference for agentic coding, since it measures solving real software problems, not just isolated snippets.
Is Claude the best on benchmarks?
It depends on the benchmark and versions compared: Claude is regularly on top for agentic coding and reasoning, but no model dominates everywhere. Check up-to-date scores.
Where can I see Claude's scores?
In Anthropic's official announcements and in our feed (Models category), which relays benchmarks at each release.
Can benchmarks be trusted?
With caution: data contamination, test conditions and version choices can skew the reading. A score is a hint, not a truth.
Claude News is published by Héra SASU. Independent media, not affiliated with Anthropic.