A minimalist landscape map of LLM benchmark categories shown as labeled lanes on a near-black-on-white data dashboard, with the inference-speed lane highlighted in green

Blog · By Anton Gulin · Updated June 28, 2026

LLM Benchmarks: The Full 2026 Landscape

Last updated: 2026-06-28

The list of benchmarks that decided model rankings in 2023 is mostly retired. MMLU, HumanEval, and TruthfulQA have all saturated, and a new generation of ceiling tests, agentic suites, and contamination-resistant evaluations now does the real work of separating frontier models. This is a curated landscape of the LLM benchmarks that matter in 2026, organized by what each one reveals, with a one-line reason it matters and a current status: active, saturated, or contested. It closes with the category most lists omit, inference speed. For the underlying concept, see LLM Benchmark: A Complete Guide; for how these rankings have shifted this year, see our state of LLM benchmarks 2026 mid-year report.

What are the most important LLM benchmarks in 2026?

The most important LLM benchmarks in 2026 are Humanity’s Last Exam and GPQA Diamond for reasoning, FrontierMath for mathematics, SWE-bench Verified and Terminal-Bench for coding agents, tau2-bench and GAIA for agentic tasks, ARC-AGI-2 for fluid intelligence, LMArena for human preference, and tokens-per-second for speed. No single test ranks general capability.

The defining shift since 2023 is that benchmarks now expire fast. Once frontier models cluster in the low 90s on a test, that test can no longer separate them, so the field has moved to harder, fresher, and more agentic evaluations. The table below maps the full 2026 landscape by category and status; the sections that follow explain each entry. For a head-to-head of capability scores, see the LLM Benchmark: A Complete Guide.

BenchmarkCategory2026 statusSource
Humanity’s Last Exam (HLE)Reasoning (ceiling)ActiveCenter for AI Safety / Scale AI, 2025
GPQA DiamondReasoning (science)ActiveRein et al., NYU, 2023
MMLU-ProReasoning (knowledge)ContestedTIGER-Lab, 2024
MMLUReasoning (legacy)SaturatedHendrycks et al., 2020
FrontierMathMathematics (research)ActiveEpoch AI, 2024
AIME 2025Mathematics (olympiad)ContestedArtificial Analysis, 2026
SWE-bench VerifiedCoding (agentic)ContestedOpenAI, 2024
Terminal-BenchCoding (terminal)ActiveStanford / Laude, 2026
LiveCodeBenchCoding (contamination-free)ActiveJain et al., 2024
tau2-benchAgentic (tool use)ActiveSierra Research, 2025
GAIAAgentic (assistant)ContestedMialon et al. / Meta, 2023
ARC-AGI-2Fluid intelligenceContestedARC Prize Foundation, 2025
SimpleQAKnowledge / factualityActiveOpenAI, 2024
LMArenaHuman preferenceActiveUC Berkeley / Arena, 2023
AA Intelligence IndexCompositeActiveArtificial Analysis, 2026
MLPerf InferenceSpeed (hardware)ActiveMLCommons, 2025
Tokens/sec + TTFTSpeed (serving)ActiveArtificial Analysis, 2026

Which benchmarks reveal raw reasoning?

Reasoning benchmarks test multi-step problem solving across many subjects, and the 2026 versions are deliberately built to resist the saturation that retired their predecessors. The marquee ceiling test is Humanity’s Last Exam, where frontier models still score well under half; GPQA Diamond remains the standard for graduate-level science. Stanford’s AI Index names HLE among the benchmarks created specifically because older tests saturated (Source: Stanford HAI, AI Index 2026).

1. Humanity’s Last Exam (HLE)

Why it matters in 2026: It is the headline ceiling test that frontier labs now cite in every major launch. HLE is a 2,500-question multimodal exam written by domain experts, created jointly by the Center for AI Safety and Scale AI and released in January 2025 to counter MMLU saturation (Source: Stanford HAI, AI Index 2026). Status: active. At launch, frontier models scored below 25 percent; by mid-2026 the published range had climbed to roughly 24 to 41 percent on the standard leaderboard, so it still spreads the field (Source: Artificial Analysis, “Humanity’s Last Exam,” 2026).

2. GPQA Diamond

Why it matters in 2026: It is the cleanest measure of genuine graduate-level science reasoning still in wide use. GPQA Diamond is the highest-quality 198-question subset of the Graduate-Level Google-Proof Q&A benchmark from Rein et al. at NYU, written so non-expert PhD holders score around 34 percent even with web access (Source: Rein et al., arXiv:2311.12022, 2023). Status: active, though top models are beginning to crowd the upper range, which is why ceiling tests like HLE were added above it.

3. MMLU-Pro and MMLU

Why it matters in 2026: Together they show the saturation cycle in miniature. The original MMLU, 57 subjects from Hendrycks et al., is now saturated, with essentially every frontier model above 88 percent and the leader near 93 percent, a spread the AI Index calls statistical noise (Source: Stanford HAI, AI Index 2026). TIGER-Lab’s MMLU-Pro raised the choices from four to ten and added reasoning-heavy questions to spread the field again (Source: TIGER-Lab, arXiv:2406.01574, 2024), but by 2026 it is contested as the strongest models climb its upper range too.

Which benchmarks measure mathematics?

Math benchmarks grade objectively against numeric answers, which makes them ideal for tracking progress until they saturate. In 2026 the anchor is FrontierMath, a research-level test where even leading models stay well short of mastery, while older olympiad sets like AIME have become contested as scores cluster near the top and contamination concerns grow (Source: Epoch AI, FrontierMath, 2024).

4. FrontierMath

Why it matters in 2026: It is the hardest public math benchmark and the clearest signal that research-level reasoning is unsolved. FrontierMath was built by Epoch AI with over 70 mathematicians, using entirely novel, unpublished problems under data-protection protocols to prevent contamination (Source: Epoch AI, arXiv:2411.04872, 2024). Status: active. Early models scored under two percent; OpenAI’s o3 reached 25.2 percent in December 2024, and by 2026 leaderboard scores climbed toward the high 40s, with research-level Tier 4 still far lower (Source: Epoch AI, “FrontierMath Tier 4,” 2026).

5. AIME 2025

Why it matters in 2026: It is a useful contamination probe because a fresh exam ships each year. AIME 2025 uses all 30 problems from the 2025 American Invitational Mathematics Examination, with integer answers from 000 to 999, tracked as a live leaderboard by Artificial Analysis (Source: Artificial Analysis, AIME 2025, 2026). Status: contested, the top models have largely saturated it, so a one- or two-point gap is closer to noise than a real capability difference. Comparing AIME 2023, 2024, and 2025 scores helps detect memorized answers.

Which benchmarks test coding agents?

Coding benchmarks grade by executing tests, not matching text, which makes them strict, but the 2026 versions also expose a deeper problem: harness variance. The same model in different scaffolding can swing 10 to 20 points on SWE-bench, so standardized tooling now matters as much as the model (Source: digitalapplied.com, “SWE-Bench vs Terminal-Bench,” 2026). For a focused treatment, see the LLM Coding Benchmark guide.

6. SWE-bench Verified

Why it matters in 2026: It is still the most-cited coding-agent benchmark, but it has effectively topped out and splintered. SWE-bench Verified is a 500-issue, human-validated subset of real GitHub problems, curated by OpenAI for reliable scoring (Source: OpenAI, “Introducing SWE-bench Verified,” 2024). Status: contested, the SWE-bench family has fragmented into five incompatible variants, original, Verified, Pro, Multilingual, and Live, and vendors tune scaffolding to game specific tasks, which is why Scale AI’s SEAL lab built SWE-bench Pro with standardized tooling (Source: morphllm.com, SWE-bench Pro, 2026).

7. Terminal-Bench

Why it matters in 2026: It is the new standard for long-horizon agents working in a real command line. Terminal-Bench comprises 89 hand-crafted, human-verified tasks spanning scientific computing, software engineering, security, and system administration, each with a containerized Docker environment and a programmatic verification suite (Source: Terminal-Bench, arXiv:2601.11868, 2026). Status: active. Unlike function-writing tests, its tasks require end-to-end workflows, compiling code, training models, debugging systems, which is closer to how agents are actually deployed.

8. LiveCodeBench

Why it matters in 2026: It resists the contamination that inflated HumanEval to saturation. LiveCodeBench scrapes contest problems from LeetCode, Codeforces, and AtCoder only after each model’s training cutoff, so models cannot have memorized the answers (Source: Jain et al., arXiv:2403.07974, 2024). Status: active, its rolling, time-gated design keeps it fresh as a measure where the now-saturated HumanEval no longer separates frontier models.

Which benchmarks measure agentic behavior?

Agentic benchmarks test whether a model can use tools, follow rules, and finish multi-step tasks, not just answer a prompt. They are the fastest-growing category in 2026 and the most scaffold-dependent: the same Claude Opus 4 has scored 64.9 percent in one agent framework and 57.6 percent in another on GAIA, a seven-point gap from orchestration alone (Source: Holistic Agent Leaderboard, arXiv:2510.11977, 2025).

9. tau2-bench

Why it matters in 2026: It tests tool-using agents under realistic enterprise rules with a dual-control design. From Sierra Research, tau2-bench simulates customer-service interactions across retail, airline, and telecom domains, where both the agent and a simulated user modify a shared environment (Source: Sierra Research, tau2-bench GitHub, 2025). Status: active. The 2026 update added voice and knowledge-retrieval domains and made 75-plus task-quality fixes, growing to 38 model entries by April 2026, with its pass^k metric exposing reliability gaps single-shot tests miss.

10. GAIA

Why it matters in 2026: It is the clearest illustration of why you must ask “which leaderboard?” GAIA presents 466 questions chaining web browsing, file parsing, and multi-document reasoning, simple for humans but hard for AI (Source: Mialon et al., GAIA, arXiv:2311.12983, 2023). Status: contested. In May 2026 the same 466 tasks showed scores from 44.8 to 92.36 percent depending on whether you read the bare-model, vendor-scaffolded, or full-system leaderboard, a 30-to-50-point gap that is the entire point (Source: decodethefuture.org, “AI Agent Benchmarks 2026,” 2026).

11. ARC-AGI-2

Why it matters in 2026: It targets fluid, on-the-fly reasoning that pure scale does not solve cheaply. ARC-AGI-2, from the ARC Prize Foundation, tests abstraction and pattern induction on puzzles humans solve easily but models historically cannot (Source: ARC Prize, “Announcing ARC-AGI-2,” 2025). Status: contested. Frontier systems began 2025 near three percent; by early 2026 commercial refinement stacks pushed the public leaderboard from roughly 54 percent toward the high 80s, prompting ARC-AGI-3 to reset the frontier (Source: ARC Prize, “ARC Prize 2025 Results Analysis,” 2026).

Which benchmarks measure factuality and preference?

Beyond raw capability, two categories tell you whether a model is honest and whether people actually like using it. SimpleQA measures short-form factual accuracy and calibration; LMArena aggregates millions of blind human votes into a single rating. Both remain active and widely cited in 2026 (Source: LMArena, 2024).

12. SimpleQA

Why it matters in 2026: It exposes hallucination and calibration where older factuality tests saturated. SimpleQA, from OpenAI, contains 4,326 short, fact-seeking questions adversarially collected so each has one indisputable answer, graded correct, incorrect, or not attempted (Source: OpenAI, “Introducing SimpleQA,” 2024). Status: active. It proved hard even for strong models, OpenAI’s o1-preview reached only 42.7 percent at launch, making it a useful signal where TruthfulQA had leaked into training data and saturated.

13. LMArena

Why it matters in 2026: It is the closest public proxy for which model “feels” best to use. LMArena, formerly Chatbot Arena from UC Berkeley, shows two anonymous responses, collects a vote, and converts votes into an Elo-like rating using a Bradley-Terry model documented in an ICML 2024 paper (Source: Chiang et al., ICML 2024). Status: active, with millions of votes across hundreds of models. Read the confidence intervals before the rank, because top models routinely overlap, so their order is partly statistical noise.

Which benchmarks measure speed and efficiency?

This is the category most landscape lists omit. Speed benchmarks measure how fast a model serves output, not how smart it is, and the same model can differ by an order of magnitude across providers. The two reference points are MLPerf Inference, for standardized hardware throughput, and live tokens-per-second tracking, for production serving (Source: Artificial Analysis, 2026).

14. Artificial Analysis Intelligence Index

Why it matters in 2026: It is the leading composite, combining many tests so no single saturated benchmark dominates. The Index averages reasoning, knowledge, math, and coding evaluations run independently under identical conditions, and was rebuilt as version 4.0 in January 2026 after an earlier version capped out at 73 for the top model (Source: Artificial Analysis, Intelligence Benchmarking Methodology, 2026). Status: active. Treat it as a starting point, not a verdict, two models can reach the same total with very different capability profiles.

15. MLPerf Inference

Why it matters in 2026: It is the neutral, audited standard for hardware throughput under fixed latency gates. MLPerf Inference v5.1, released by MLCommons in September 2025, added a DeepSeek-R1 reasoning test averaging 3,880 output tokens, by far its longest, and an interactive scenario with strict time-to-first-token and per-token latency limits for agentic serving (Source: MLCommons, MLPerf Inference v5.1, 2025). Status: active. Its Closed division forces apples-to-apples comparison, which makes it the cleanest cross-hardware reference.

16. Tokens-per-second and TTFT

Why it matters in 2026: A model that wins on accuracy still loses in production if it serves text too slowly. Output speed in tokens per second, time-to-first-token, and inter-token latency determine whether an app feels instant or sluggish, and they vary enormously across providers serving the identical model (Source: Artificial Analysis, Performance Benchmarking, 2026). Status: active. Artificial Analysis measures these under controlled 1k and 10k-token workloads, and the same model served by different providers can differ by an order of magnitude, which is why TokenDyno tracks live tokens-per-second across providers. Pair any capability score with measured speed before you ship. To rank models by capability first, see AI Benchmark 2026 and Examples of LLM Benchmarks.

Frequently asked questions

What are the most important LLM benchmarks in 2026?

The most important benchmarks are Humanity’s Last Exam and GPQA Diamond for reasoning, FrontierMath for math, SWE-bench Verified and Terminal-Bench for coding agents, tau2-bench and GAIA for agentic tasks, LMArena for preference, and tokens-per-second for speed. Each measures a distinct skill, so teams weight two or three plus a private evaluation (Source: Stanford HAI, AI Index 2026).

Which benchmarks are saturated?

MMLU is fully saturated, with nearly every frontier model above 88 percent and differences near statistical noise (Source: Stanford HAI, AI Index 2026). HumanEval and TruthfulQA also saturated and were replaced by LiveCodeBench and SimpleQA. AIME 2025, MMLU-Pro, and SWE-bench Verified are now contested, clustering near the top or fragmenting into variants.

What new benchmarks emerged in 2025 to 2026?

Humanity’s Last Exam launched in January 2025 as a 2,500-question ceiling test (Source: Stanford HAI, AI Index 2026). ARC-AGI-2 arrived in 2025 for fluid reasoning, Terminal-Bench in 2026 for long-horizon command-line agents, and tau2-bench’s 2026 update added voice and knowledge domains. FrontierMath, from late 2024, became the anchor research-math test.

Why do LLM benchmarks keep getting replaced?

Benchmarks get replaced when they saturate or are contaminated. Once frontier models all score in the low 90s, a test cannot distinguish them, and leaked questions inflate scores. MMLU, HumanEval, and TruthfulQA all saturated, triggering harder successors like HLE, LiveCodeBench, and SimpleQA (Source: Stanford HAI, AI Index 2026).

Why do benchmark scores differ across leaderboards?

Because scaffolding changes results. The same model can swing 10 to 20 points on SWE-bench and 30 to 50 on GAIA depending on the agent framework, retry budget, and tooling around it (Source: digitalapplied.com, 2026). Always ask which leaderboard ran the test, and prefer standardized harnesses like SWE-bench Pro.

Are there benchmarks for inference speed?

Yes, and most lists omit them. Speed benchmarks track tokens per second, time-to-first-token, and inter-token latency rather than accuracy. MLPerf Inference standardizes hardware throughput, and independent measurers run live serving tests where the same model varies by an order of magnitude across providers (Source: Artificial Analysis, 2026).

Key takeaways

The 2026 LLM benchmark landscape splits into six lanes: reasoning (HLE, GPQA Diamond), math (FrontierMath, AIME), coding (SWE-bench Verified, Terminal-Bench, LiveCodeBench), agentic (tau2-bench, GAIA, ARC-AGI-2), factuality and preference (SimpleQA, LMArena), and speed (MLPerf Inference, tokens-per-second). Most classic tests, MMLU, HumanEval, TruthfulQA, have saturated, pushing the field toward harder, fresher, and more agentic evaluations.

When you read any benchmark, check three things: whether it has saturated, who ran it and with what scaffolding, and whether it tests tasks like yours. Then add the lane most lists skip, measured inference speed, because a model that scores well but serves slowly will still disappoint in production. Pair capability scores with live tokens-per-second before you decide.

Sources

← All posts