Live LLM leaderboard dashboard ranking AI models with Elo bars, capability scores, and a green tokens-per-second speed gauge

Blog · By Anton Gulin · Updated June 28, 2026

LLM Leaderboard 2026: The Live Tokens/sec Ranking

Last updated: 2026-06-28

An LLM leaderboard ranks large language models against each other so you can compare them at a glance instead of trusting vendor claims. But “the leaderboard” does not exist: there are many, each measuring something different, with different biases. This guide explains what LLM leaderboards are, walks through the major ones (LMArena, Artificial Analysis, HELM, the Hugging Face Open LLM Leaderboard, LiveBench, and SWE-bench), shows how to read them without being misled, and covers the dimension most of them ignore: live serving speed in tokens per second.

What is an LLM leaderboard?

An LLM leaderboard is a ranked table that scores large language models on one or more benchmarks and orders them from best to worst on a chosen metric. Some rank by human preference, some by aggregated test accuracy, and some by serving speed or cost. The metric defines what “top of the leaderboard” actually means.

How is a leaderboard different from a benchmark?

A benchmark is a single standardized test with its own dataset and scoring method; a leaderboard is the ranking that results when many models take one or more benchmarks. MMLU and SWE-bench are benchmarks. LMArena and Artificial Analysis are leaderboards. One leaderboard can aggregate dozens of benchmarks into a composite score, so a model’s rank depends entirely on which tests are included and how they are weighted. For a full breakdown of the underlying tests, see LLM Benchmark: A Complete Guide.

Why do LLM leaderboards matter in 2026?

Leaderboards matter because they are the only repeatable, public way to compare models before committing budget and engineering time. With new frontier models shipping monthly, buyers cannot test every option themselves. A leaderboard compresses thousands of evaluations into a shortlist, which is also why leaderboard pages increasingly surface in Google AI Overviews citing LLM leaderboards when people search for the best model. The catch is that a published rank only predicts your results when the leaderboard measures tasks like yours, on a clean test set, with the field not yet saturated. Treat a rank as evidence, not a verdict.

What are the major LLM leaderboards in 2026?

The major LLM leaderboards in 2026 are LMArena (human preference), Artificial Analysis (composite intelligence plus speed and price), Stanford HELM (holistic transparency), the now-archived Hugging Face Open LLM Leaderboard (open-weight models), LiveBench (contamination-limited capability), and SWE-bench (real-world coding). Each answers a different question, so they routinely crown different winners.

The table below summarizes what each ranks, how it scores, who maintains it, and where to find live standings. Because frontier rankings reshuffle within weeks, treat the table as a map of the landscape and click through to each source for current numbers (accessed 2026-06-28).

LeaderboardWhat it ranksMethodMaintainerSource
LMArena (formerly Chatbot Arena)Overall human preference across open-ended chatAnonymous A/B votes converted to a Bradley-Terry (Elo-style) ratingLMArena, originating from LMSYS / UC Berkeleylmarena.ai
Artificial AnalysisIntelligence, output speed, latency, and price togetherComposite Intelligence Index plus independently measured serving metricsArtificial Analysisartificialanalysis.ai
Stanford HELMHolistic capability and safety with full transparencyStandardized scenarios scored on multiple metrics, prompt-level reproducibilityStanford CRFMcrfm.stanford.edu/helm
Hugging Face Open LLM LeaderboardOpen-weight models on automated academic benchmarksFixed suite (IFEval, BBH, MATH, GPQA, MuSR, MMLU-Pro); archived March 2025Hugging Face / OpenEvalshuggingface.co (archived)
LiveBenchCapability without contaminationObjective ground-truth scoring on monthly-refreshed questions, no LLM judgeAbacus.AI with NYU, Nvidia, otherslivebench.ai
SWE-benchResolving real software-engineering issuesGenerated patches graded by the repository’s own unit testsPrinceton (Jimenez, Yang, et al.)swebench.com

What is the LMArena (Chatbot Arena) leaderboard?

LMArena, formerly Chatbot Arena, is the largest human-preference leaderboard. Users send a prompt, receive two anonymous model responses, and vote for the better one; those votes are converted into a Bradley-Terry rating, an Elo-style score where beating a strong model moves you more than beating a weak one (Source: LMArena, 2026). By 2026 it had aggregated more than 6.8 million votes across 360-plus models, the largest public preference dataset available (Source: LMArena, 2026). It is the closest proxy for “which model feels best to use,” but ratings carry bootstrap confidence intervals, and top models frequently overlap.

What is the Artificial Analysis leaderboard?

Artificial Analysis is the leaderboard closest to a buyer’s full decision, because it ranks intelligence, output speed, latency, and price side by side rather than in isolation. It runs every model on standardized conditions and publishes a composite Intelligence Index alongside independently measured serving metrics (Source: Artificial Analysis, 2026). Output speed is reported as output tokens per second after the first token, and for reasoning models it is measured after any thinking time (Source: Artificial Analysis, methodology, 2026). The platform takes roughly eight measurements per day and reports performance over a rolling 72-hour window, so its speed and price numbers reflect live API behavior rather than a one-time test.

What is the Stanford HELM leaderboard?

HELM (Holistic Evaluation of Language Models), maintained by Stanford’s Center for Research on Foundation Models, prioritizes transparency over a single headline number. The original 2022 framework evaluated models across 42 scenarios using seven metric categories, including accuracy, calibration, robustness, fairness, bias, toxicity, and efficiency (Source: Liang et al., arXiv:2211.09110, 2022). In March 2025 CRFM launched HELM Capabilities as its flagship leaderboard, benchmarking an initial 22 models across five capability-focused scenarios with full prompt-level reproducibility (Source: Stanford CRFM, 2025). HELM also spans specialized leaderboards such as MedHELM for medicine and AIR-Bench for safety. The project entered maintenance mode in mid-2026.

What was the Hugging Face Open LLM Leaderboard?

The Hugging Face Open LLM Leaderboard was, for two years, the default ranking for open-weight models, scoring them on a fixed suite of automated academic benchmarks. Its second version evaluated models on IFEval, BBH, MATH, GPQA, MuSR, and MMLU-Pro (Source: Hugging Face, 2024). It was archived in March 2025 and no longer accepts submissions, with the maintainers pointing users toward more active, contamination-resistant alternatives (Source: Hugging Face, 2025). It remains a useful historical record, but for current open-weight comparisons you should rely on live leaderboards rather than its frozen standings.

What is the LiveBench leaderboard?

LiveBench is a contamination-limited capability leaderboard built by Abacus.AI with collaborators from NYU, Nvidia, the University of Maryland, and others, and presented as an ICLR 2025 Spotlight; one contributor is Meta’s Yann LeCun (Source: LiveBench / Abacus.AI, 2025). It limits contamination by refreshing questions monthly from recent sources such as arXiv papers, news, and datasets, and it scores every question against objective ground truth with no LLM judge (Source: LiveBench, 2025). The team adopted automated scoring after finding that LLM-judge grading of hard reasoning and math problems had an error rate as high as 46% (Source: LiveBench, 2025). Its current suite spans roughly two dozen tasks across categories including reasoning, coding, math, data analysis, language, and instruction following.

What is the SWE-bench leaderboard?

SWE-bench, from Princeton, is the leading real-world coding leaderboard: models must generate patches that resolve actual GitHub issues, graded by running each repository’s own unit tests, so a patch either passes or fails (Source: Jimenez et al., arXiv:2310.06770, 2023). The widely cited SWE-bench Verified subset contains 500 human-validated tasks. Because grading mirrors real engineering work, it is among the most decision-relevant tests for development teams, though scores depend heavily on the agent scaffold used. For live standings, check the maintained leaderboard rather than any fixed figure (Source: SWE-bench, accessed 2026-06-28).

How do you read an LLM leaderboard correctly?

To read an LLM leaderboard correctly, check the confidence interval before the rank, confirm which benchmark or vote pool produced the score, and match the leaderboard’s metric to your actual workload. A model can top one leaderboard and trail on another because the two measure different skills. Rank order alone, without these checks, is the single most common way buyers are misled.

Why read the confidence interval before the rank?

On human-preference leaderboards, the top several models usually sit inside overlapping 95% confidence intervals, which means the exact rank order is partly statistical noise. LMArena derives those intervals from bootstrap resampling of the vote pool, and the confidence column is routinely ignored (Source: LMArena, 2026). When two models’ intervals overlap, you cannot call one the leader at procurement-grade confidence. The practical rule: read the interval first, and treat a one- or two-rank difference between tightly clustered models as a tie rather than a decisive gap.

Why match the leaderboard to your workload?

Different leaderboards reward different abilities, so the right one depends on what you are building. A customer-facing chat product is best served by human-preference rankings; a coding agent should be judged on SWE-bench; a latency-sensitive application should weight serving speed heavily. Knowledge and reasoning leaders like the top HELM or LiveBench models often differ from the SWE-bench coding leader and from the human-preference winner (Source: Artificial Analysis, 2026). Pick one or two axes that match your use case, then read those leaderboards rather than chasing a single overall rank. For a worked example, see Which LLM Performs Best on Benchmarks?.

Why prefer live leaderboards over static rankings?

A leaderboard is only trustworthy if its standings move when models do. Frontier releases land roughly monthly, and a “best model” written into a blog post or a frozen table is often stale within weeks. The archived Hugging Face Open LLM Leaderboard illustrates the risk: useful as history, but it stopped accepting submissions in March 2025, so it cannot reflect any model released since (Source: Hugging Face, 2025). Prefer continuously updated sources such as LMArena, Artificial Analysis, LiveBench, and the SWE-bench leaderboard, and always note the date you accessed them. A rank without an access date is a claim without a timestamp.

How are composite leaderboard scores weighted?

Composite leaderboards collapse many benchmarks into one number, and the weighting is a hidden editorial choice. Artificial Analysis builds its Intelligence Index from a basket of evaluations, then derives metrics like time-per-task by dividing output tokens per task by output speed and weighting each benchmark’s contribution (Source: Artificial Analysis, methodology, 2026). Two leaderboards can rank the same models differently simply because one weights coding more heavily and another weights reasoning. Before trusting a composite rank, check which benchmarks feed it and how much each counts, because the weighting, not the models, can decide the order.

What are the biases and limitations of LLM leaderboards?

The main limitations of LLM leaderboards are data contamination, access asymmetry, and saturation. Contamination inflates scores when test questions leak into training data. Access asymmetry lets large labs privately tune models against a leaderboard before public scoring. Saturation erases the gaps between top models. Each one makes a high rank less trustworthy than it looks.

How does contamination and overfitting distort leaderboards?

Contamination happens when benchmark questions appear in a model’s training data, so the model recalls answers instead of reasoning, and overfitting happens when models are tuned to the quirks of a specific test. The gap is measurable: Scale AI reported that a leading model resolving roughly 81% of SWE-bench Verified tasks resolved only about 46% on its contamination-resistant SWE-bench Pro under standardized scaffolding (Source: Scale AI, 2026). This is why contamination-limited designs like LiveBench, which refreshes questions monthly, exist, and why a frozen leaderboard such as the archived Hugging Face one ages quickly.

What is the “Leaderboard Illusion”?

“The Leaderboard Illusion” is a 2025 study from Cohere Labs and collaborators, published at NeurIPS 2025, that found systematic biases in Chatbot Arena. Analyzing roughly 2 million battles across 243 models over 16 months, the authors reported that a few large providers received disproportionate data access and could privately test many variants before public release, identifying up to 27 private Meta variants ahead of one launch (Source: Singh et al., arXiv:2504.20879, 2025). They estimated that extra Arena data could lift Arena-specific win rates substantially without improving general benchmarks. LMArena disputed several figures, arguing open models are better represented than the paper claimed (Source: LMArena, 2025). The lesson: even the largest leaderboard has structural biases worth understanding.

Why does saturation limit leaderboards?

Saturation occurs when frontier models all cluster near a benchmark’s ceiling, at which point the test can no longer separate them and small score gaps fall within noise. Classic benchmarks like MMLU, GSM8K, and HumanEval are now saturated, which is why the field migrated to harder tests such as GPQA Diamond, SWE-bench Verified, and Humanity’s Last Exam (Source: Liang et al., 2022; Source: Jimenez et al., 2023). When comparing models, weight leaderboards built on benchmarks that still have headroom over those where every frontier model looks identical.

Is there a leaderboard for LLM speed?

Yes. Most capability leaderboards ignore serving speed, but speed-focused leaderboards rank models and providers by output tokens per second, time to first token, and latency. These metrics decide whether an application feels instant or sluggish, and they vary enormously for the same model across providers and hardware, far more than capability scores do (Source: Artificial Analysis, 2026).

Why is tokens per second the missing dimension?

A model can top every capability leaderboard and still lose in production if it generates text too slowly to feel responsive or costs too much to serve at scale. Capability and speed answer different questions: one tells you whether a model can solve your task, the other whether it can do so fast and cheaply enough to ship. TokenDyno tracks live tokens per second across providers for exactly this reason, complementing the accuracy-focused leaderboards above. For current hardware and provider rankings, see Fastest LLM Inference in 2026.

How do speed leaderboards measure tokens per second?

Speed leaderboards measure output tokens per second as the sustained generation rate after the first token, time to first token as the initial delay, and end-to-end latency as the full response time. Artificial Analysis runs these measurements on a fixed cadence and reports them over a rolling window so the numbers track real API behavior rather than a single test (Source: Artificial Analysis, 2026). Crucially, the same model can run many times faster on one provider than another, so a speed leaderboard ranks model-and-provider combinations, not just models. For how the major speed and capability leaderboards line up, see LLM Leaderboards Compared.

Frequently asked questions

What is the best LLM leaderboard?

There is no single best LLM leaderboard, because each measures a different thing. For human-preference quality, LMArena is the largest. For a combined intelligence, speed, and price view, Artificial Analysis is the most complete. For coding, use SWE-bench; for transparency, use HELM. Match the leaderboard to your workload (Source: Artificial Analysis, 2026).

What is the LMArena leaderboard?

LMArena, formerly Chatbot Arena, is a human-preference leaderboard where users vote between two anonymous model responses to the same prompt. Votes are converted into a Bradley-Terry rating, an Elo-style score, across more than 6.8 million votes and 360-plus models. It is the leading proxy for which model people prefer in open-ended use (Source: LMArena, 2026).

How accurate are LLM leaderboards?

LLM leaderboards are useful but not absolute. Scores can be inflated by data contamination, distorted by overfitting and private testing, or rendered meaningless by saturation when top models cluster near the ceiling. Trust a rank most when the benchmark is fresh, the test set is clean, and it measures tasks like yours, and always read the confidence interval (Source: Singh et al., 2025).

Is there a leaderboard for LLM speed?

Yes. Capability leaderboards mostly ignore speed, but dedicated speed leaderboards rank models and providers by output tokens per second, time to first token, and latency. These metrics vary widely across providers for the same model, so speed should be evaluated separately from accuracy before deployment (Source: Artificial Analysis, 2026).

What is the difference between a benchmark and a leaderboard?

A benchmark is one standardized test with a fixed dataset and scoring method, such as MMLU or SWE-bench. A leaderboard is the ranking produced when models take one or more benchmarks, such as LMArena or Artificial Analysis. A single leaderboard can combine many benchmarks, so a model’s rank depends on which tests it includes (Source: LMArena, 2026).

Why do different LLM leaderboards show different winners?

Different leaderboards rank different skills, so they crown different leaders. Human-preference rankings, coding tests, and reasoning benchmarks rarely agree, because each rewards a distinct ability and uses a distinct grading method. A reasoning champion may write weaker code or serve slowly, which is why you should consult the leaderboard whose metric matches your use case (Source: Artificial Analysis, 2026).

Key takeaways

An LLM leaderboard ranks models on a chosen metric, but no single leaderboard is definitive: LMArena measures human preference, Artificial Analysis combines intelligence with speed and price, HELM prioritizes transparency, LiveBench limits contamination, and SWE-bench measures real coding. The archived Hugging Face Open LLM Leaderboard is now a historical reference. Read the confidence interval before the rank, match the metric to your workload, and discount saturated benchmarks.

Most importantly, capability leaderboards leave out serving speed, the dimension that often decides production success. Pair an accuracy ranking with a live tokens-per-second measurement before you commit, and revisit both regularly, because frontier standings reshuffle within weeks.

Sources

← All posts