LLM Coding Benchmark Leaderboard: Code Generation Rankings
There is no single “coding leaderboard.” Code generation is ranked across at least five live boards, each grading a different slice of the job: resolving real GitHub issues, editing files across languages, solving fresh competition problems, calling libraries correctly, or driving a terminal. This guide explains what each board measures, how its scores are computed, and where to find the current standings. It does not reproduce numbers, because every board updates continuously, leaderboards move weekly, and a score copied here would be stale before you read it. Open the live board, note the access date, and read the methodology behind the rank.
Which leaderboards rank LLM coding ability?
Five boards dominate code-generation evaluation in 2025–2026, each grading a distinct task: SWE-bench resolves real GitHub issues, Aider polyglot edits files across six languages, LiveCodeBench scores fresh competition problems, BigCodeBench tests library calls, and Terminal-Bench drives command-line agents (Source: SWE-bench, 2024; Source: Aider, 2024). The table maps each board to what it ranks and where current standings live.
| Leaderboard | What it ranks | Where to find it | Source |
|---|---|---|---|
| SWE-bench Verified | Resolving real GitHub issues (500 human-validated tasks) | swebench.com/verified.html | SWE-bench / OpenAI, 2024 |
| Aider polyglot | Editing code across 6 languages (225 exercises) | aider.chat/docs/leaderboards | Aider, 2024 |
| LiveCodeBench | Fresh, time-windowed competition problems | livecodebench.github.io | Jain et al., 2024 |
| BigCodeBench | Library and function-call usage (1,140 tasks) | bigcode-bench.github.io | Zhuo et al., 2024 |
| Terminal-Bench | Agent tasks in a real terminal (89 tasks, v2) | tbench.ai/leaderboard | Merrill et al., 2026 |
A model that tops one board can sit mid-pack on another, because each grades a different competence. Read the board whose task matches your use case, not the one with the most familiar name. For the underlying tests rather than the boards, see LLM Coding Benchmark.
What is the SWE-bench leaderboard?
The SWE-bench leaderboard ranks how often a system resolves real GitHub issues. The headline board, SWE-bench Verified, uses 500 human-validated tasks filtered from the original set in collaboration with OpenAI, each drawn from a resolved pull request with unit tests that fail before the fix and pass after it (Source: OpenAI, 2024; Source: SWE-bench, 2023).
How does SWE-bench grade a submission?
Each task pairs a GitHub issue with a hidden test patch. A submission is scored resolved only if the model’s code makes the FAIL_TO_PASS tests pass without breaking existing ones. The metric is the percentage of the 500 instances resolved. Because grading runs real test suites, the score reflects working patches, not plausible-looking diffs (Source: SWE-bench, 2023).
Why does SWE-bench Verified exist?
OpenAI and the SWE-bench authors built Verified after finding that some original tasks were under-specified or effectively unsolvable, which made the benchmark systematically underestimate models. Human annotators reviewed each instance for a clear problem statement and a correct test patch, producing a fairer 500-task subset (Source: OpenAI, 2024).
What to watch when reading SWE-bench standings
Two cautions. First, not every reported run uses all 500 tasks: Epoch AI notes that excluding a handful is common practice, with some vendor runs using 477 or 489 instances, so check the task count behind a number (Source: Epoch AI, 2026). Second, many SWE-bench rows are agent scaffolds, not bare models, so the rank reflects the harness plus the model, not the model alone. The official board is at swebench.com/verified.html; variants include Lite, Multimodal, and Bash-only.
What is the Aider polyglot leaderboard?
The Aider polyglot leaderboard ranks how well models write and edit code across six languages without human intervention. It uses 225 of the hardest Exercism exercises, drawn from C++, Go, Java, JavaScript, Python, and Rust, selected because they were solved by three or fewer models when the board launched in December 2024 (Source: Aider, 2024).
How does the polyglot benchmark score models?
Each model gets two attempts per problem. After a failed first attempt, it sees the failing unit-test output and tries again. The primary metric, pass_rate_2, is the share of the 225 exercises where all hidden tests pass after the second attempt. This rewards both raw problem-solving and the ability to correct mistakes from test feedback (Source: Aider, 2024).
Why does edit format matter on this board?
Aider also reports the percent of responses using a correct edit format, meaning edits Aider could apply without a retry. A model that solves a problem but emits malformed diffs loses effective accuracy, because failed edits never reach the test suite. Low edit-format compliance drags the headline score down, which is why this column is worth reading (Source: Aider, 2024).
How to read polyglot rows fairly
Configuration is encoded in the model label. Some rows fix a thinking-token budget or reasoning effort; architect-mode rows pair a planner model with a separate editor model, so they are system results, not single-model numbers. The board also tracks the dollar cost of running all 225 exercises. Live standings are at aider.chat/docs/leaderboards; the launch write-up is at aider.chat/2024/12/21/polyglot.html.
What is the LiveCodeBench leaderboard?
LiveCodeBench ranks code reasoning on competition problems collected continuously from LeetCode, AtCoder, and Codeforces, then time-stamped to fight contamination. Built by researchers at UC Berkeley, MIT, and Cornell, its v6 release spans problems published between May 2023 and April 2025, growing to 1,055 problems (Source: Jain et al., 2024).
How does LiveCodeBench control for contamination?
Every problem carries a release date, so you can filter to problems published after a model’s training cutoff and measure performance on genuinely unseen tasks. The authors show scores drop on post-cutoff problems, the signature of contamination on static benchmarks. This time-windowing is the board’s core methodological claim (Source: Jain et al., 2024).
What does LiveCodeBench measure beyond generation?
It is “holistic”: besides code generation, it scores self-repair, test-output prediction, and code execution, giving a wider view of coding competence than pass-rate alone. A default lite set prunes redundant test cases for faster evaluation, with a flag to run the full suite. The live leaderboard is at livecodebench.github.io; the paper is arXiv:2403.07974, published at ICLR 2025 (Source: Jain et al., 2024).
What is the BigCodeBench leaderboard?
BigCodeBench ranks practical programming that requires invoking multiple libraries and function calls, not toy algorithm puzzles. It contains 1,140 tasks spanning diverse domains, with two splits: Complete (from docstrings) and Instruct (from natural-language instructions). A 148-task Hard subset targets the most realistic cases (Source: Zhuo et al., 2024).
Why is BigCodeBench harder than HumanEval?
The authors report that even the best models reach roughly 60% on BigCodeBench-Complete and about 50% on Instruct, against 97% human performance, leaving a wide headroom that saturated benchmarks like HumanEval no longer offer. The gap reflects how hard it is to compose several tools correctly under realistic instructions (Source: Zhuo et al., 2024).
How does BigCodeBench keep evaluation reproducible?
Tasks are graded by rich test cases, averaging multiple assertions per problem, run in a controlled execution environment. The maintainers added a real-time code-execution session on the leaderboard and open-sourced the full stack, including tasks, tests, and harness. The board lives at bigcode-bench.github.io; the paper is arXiv:2406.15877, accepted at ICLR 2025 (Source: Zhuo et al., 2024).
What is the Terminal-Bench leaderboard?
Terminal-Bench ranks AI agents on hard, realistic tasks inside a real command-line environment: compiling code, training models, configuring servers, and debugging systems. Built by Stanford and the Laude Institute, the current Terminal-Bench 2.0 uses 89 curated tasks across software engineering, system administration, data processing, model training, and security (Source: Merrill et al., 2026).
How does Terminal-Bench score an agent?
Each task ships a pre-configured environment with startup and teardown scripts, and the agent is judged by whether its end-state passes the task’s verification tests. Because tasks run in sandboxed containers, the board measures end-to-end agent behavior, including tool use and recovery, rather than a single code completion (Source: Merrill et al., 2026).
What does Terminal-Bench rank, models or agents?
Mostly agents. The board lists many entries built on scaffolds such as OpenHands, Codex CLI, and Aider, so a row reflects the agent harness paired with a model, not the model in isolation. Read the scaffold column before comparing. The maintainers expect 2.0 to saturate within roughly a year and plan harder task sets. Standings are at tbench.ai/leaderboard.
How do speed and cost change a coding leaderboard ranking?
Accuracy boards ignore two production variables: throughput and price. A model that resolves 2% more issues but runs three times slower, or costs four times as much per run, can be the worse engineering choice for an interactive coding agent, where latency and dollars-per-task compound across thousands of calls (Source: Aider, 2024).
Aider already publishes the dollar cost of a full 225-exercise run, and Terminal-Bench results are sometimes paired with cost-per-task in third-party refreshes, but most accuracy boards omit speed entirely. To weigh quality against tokens-per-second and price, read an accuracy board alongside a live throughput tracker such as TokenDyno, then pick the point on the quality-speed-cost frontier your workload can afford. For a broader cross-board view, see LLM Leaderboard 2026.
How current are these coding leaderboards?
Currency varies by board. LiveCodeBench and Terminal-Bench add problems and tasks continuously and version their releases (LiveCodeBench v6, Terminal-Bench 2.0), so standings shift as new problems and models land. SWE-bench, Aider, and BigCodeBench update as new submissions arrive (Source: Jain et al., 2024; Source: Merrill et al., 2026).
Because of this, treat any score you see, here or elsewhere, as a snapshot. Open the official board, record the date you read it, and confirm the task count and configuration behind the number. A rank without a date and a methodology is a rumor. For a taxonomy of the tests feeding these boards, see Coding Benchmark LLM.
Frequently asked questions
What is the best coding LLM leaderboard?
There is no single best board; the right one depends on your task. SWE-bench Verified is strongest for real GitHub-issue resolution, Aider polyglot for multi-language file editing, LiveCodeBench for contamination-resistant problem solving, BigCodeBench for library usage, and Terminal-Bench for command-line agents (Source: SWE-bench, 2024; Source: Aider, 2024).
What is the SWE-bench leaderboard?
It ranks systems by the percentage of real GitHub issues they resolve. SWE-bench Verified uses 500 human-validated tasks, each with unit tests that must fail before a fix and pass after. A patch counts only if its tests pass. Current standings are at swebench.com/verified.html (Source: OpenAI, 2024).
How current are coding leaderboards?
They update continuously. LiveCodeBench and Terminal-Bench add fresh problems and version releases; SWE-bench, Aider, and BigCodeBench refresh as submissions arrive. Any score is a snapshot, so open the official board, note your access date, and check the task count and configuration behind each rank (Source: Jain et al., 2024).
Why do models rank differently across coding boards?
Because each board grades a different task and aggregates differently. A model strong at unit-tested issue resolution may be weaker at fresh competition problems or terminal agents. Many rows also report agent scaffolds rather than bare models, so the harness affects the rank as much as the model (Source: Merrill et al., 2026).
Does Aider polyglot test more than one language?
Yes. The polyglot benchmark spans six languages, C++, Go, Java, JavaScript, Python, and Rust, across 225 hard Exercism exercises. Models get two attempts per problem and are scored on pass_rate_2, the share solved after the second attempt, with edit-format compliance reported separately (Source: Aider, 2024).
Key takeaways
Coding ability is not one ranking but five complementary ones. SWE-bench measures issue resolution, Aider polyglot measures cross-language editing, LiveCodeBench measures contamination-resistant problem solving, BigCodeBench measures library composition, and Terminal-Bench measures terminal agents. Match the board to your task before you trust a rank.
Read every board live. Scores here are deliberately omitted because the boards move; the durable skill is opening the official source, recording the access date, and checking the task count, configuration, and whether a row is a model or an agent scaffold. Then layer speed and cost on top, because the fastest path to a passing test is rarely the cheapest, and the highest-accuracy model is rarely the fastest.
Sources
- SWE-bench Verified leaderboard (accessed 2026-06-28): https://www.swebench.com/verified.html
- OpenAI, Introducing SWE-bench Verified (2024): https://openai.com/index/introducing-swe-bench-verified/
- Jimenez et al., SWE-bench: Can Language Models Resolve Real-World GitHub Issues?, arXiv:2310.06770 (2023): https://arxiv.org/abs/2310.06770
- Epoch AI, SWE-bench Verified benchmark notes (2026): https://epoch.ai/benchmarks/swe-bench-verified
- Aider LLM Leaderboards (accessed 2026-06-28): https://aider.chat/docs/leaderboards/
- Aider, o1 tops aider’s new polyglot leaderboard (2024): https://aider.chat/2024/12/21/polyglot.html
- Jain et al., LiveCodeBench: Holistic and Contamination-Free Evaluation, arXiv:2403.07974 (2024), ICLR 2025; leaderboard: https://livecodebench.github.io/
- Zhuo et al., BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions, arXiv:2406.15877 (2024), ICLR 2025; leaderboard: https://bigcode-bench.github.io/
- Merrill et al., Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces, arXiv:2601.11868 (2026); leaderboard: https://www.tbench.ai/leaderboard