SWE-Bench 2026 Leaderboard: Claude vs GPT vs Gemini

SWE-bench Pro is designed around longer, more difficult software-engineering tasks across public repositories. Scale says it addresses contamination, narrow task coverage, oversimplified problems, and unreliable testing found in earlier benchmarks. Its lower scores leave more room to distinguish frontier systems.

The following is a focused comparison of the leading GPT, Claude, and Gemini entries visible on Scale’s public leaderboard. It is not a claim that these are the top entries across every provider.

On this evaluation, GPT-5.4’s 59.1% is 7.2 percentage points above Claude Opus 4.6 and 13 points above Gemini 3.1 Pro. That is evidence about these submitted systems on this public test—not proof that GPT will produce the best accepted change in every repository.

The labels matter. “Thinking,” “xHigh,” and the asterisks identify configuration or leaderboard qualifications that belong with the score. Removing them turns a system result into a misleading model claim.

SWE-bench Verified contains 500 human-reviewed issues from open-source Python repositories. The official SWE-bench site now provides a bash-only view that runs language models through a shared mini-SWE-agent environment, plus a broader leaderboard covering different agent systems. Those views answer different questions.

Anthropic’s Claude Opus 4.6 system card reports the following Verified results in one comparison table. Treat this as vendor-reported evaluation evidence, not a live reproduction of the official leaderboard.

A spread of only 4.7 points separates the five entries. More importantly, the benchmark is near saturation and its remaining cases do not cleanly measure model limits. In February 2026, OpenAI said it had stopped using Verified for frontier launches and recommended SWE-bench Pro.

OpenAI audited 138 Verified problems that one of its models did not consistently solve. It reported that at least 59.4% of that audited subset contained material issues in test design or problem description. Correct solutions could fail narrow tests, while broad tests could reward solutions that did not fully address the issue. OpenAI also identified contamination risk.

That finding does not make every Verified score worthless. Verified still records a meaningful period of coding-agent progress, and the official bash-only setup can support controlled comparisons. It does mean a one-decimal difference near 80% should not drive a purchase or support a headline declaring one model universally best.

SWE-bench evaluates a working system. The model receives an issue and a repository, but an agent loop decides how it inspects files, runs commands, edits code, uses context, retries, and spends tokens. A stronger scaffold can outperform a stronger raw model; a larger reasoning budget can improve the score while increasing cost and latency.

For a fair comparison, hold the dataset, agent, prompt, tool permissions, runtime, retry policy, reasoning setting, and pass criteria constant. If any of those change, label the result as a system comparison. The official SWE-bench bash-only view is valuable precisely because it attempts to standardize the environment for language-model comparisons.

A leaderboard position can change because a provider released a better model, submitted a stronger agent, increased its reasoning budget, corrected an evaluation, or because another entrant was added. The rank alone does not reveal which event occurred. Save the model identifier, score, configuration label, benchmark version, source URL, and access date together.

Also distinguish percentage points from percent improvement. Moving from 50% to 55% is a five-point increase and a 10% relative improvement. Neither number tells you whether the extra solved tasks resemble your work or justify additional latency and cost. Inspect per-task results when available, then reproduce the comparison on your own acceptance set.

The Pro ranking says GPT-5.4 deserves a place in any frontier coding evaluation. Claude Opus 4.6 and Gemini 3.1 Pro also solve substantial portions of a difficult public benchmark. It does not tell you which product will produce the most reviewable pull requests in your codebase.

Claude Code, Codex, Gemini’s coding tools, IDE agents, and custom harnesses expose different permissions, context management, review surfaces, integrations, and billing. A model that scores lower in one public scaffold can still win when its product workflow reduces reviewer effort or avoids regressions.

Two-week evaluation

For the current public SWE-bench Pro results, GPT-5.4 leads the named Claude and Gemini entries, followed by Claude Opus 4.6 and Gemini 3.1 Pro. For SWE-bench Verified, resist the temptation to crown a winner: the scores are compressed, setups vary, and OpenAI’s audit found serious limitations at the frontier.

Use Pro to create a shortlist. Use a controlled, repository-specific acceptance test to choose the system. Preserve the model version, agent, reasoning budget, date, and source beside every score so the comparison remains auditable when the leaderboard changes.

SWE-bench, “SWE-bench Verified” https://www.swebench.com/verified.html

Scale AI, “SWE-bench Pro (Public Dataset)” https://labs.scale.com/leaderboard/swe_bench_pro_public

OpenAI, “Why SWE-bench Verified no longer measures frontier coding capabilities” https://openai.com/index/why-we-no-longer-evaluate-swe-bench-verified/

Anthropic, “Claude Opus 4.6 System Card” https://www-cdn.anthropic.com/14e4fb01875d2a69f646fa5e574dea2b1c0ff7b5.pdf