SWE-bench Verified
GitHub issues from real Python repositories: can the model write a valid fix?
- Unit
- % resolved
- Results
- 35 results32 Tokscale models matched, 32 in the source
- Last update
- Jun 25, 2026Newest run date
- Retrieved
- Oct 21, 2026Source data through Oct 15, 2026
Conditions. Run by Epoch AI. Epoch upgraded its SWE-bench Verified scaffold in February 2026.
Attribution. Epoch AI, ‘Capabilities & benchmarking’, epoch.ai/benchmarks (CC BY 4.0)
History only. Newer agentic coding benchmarks are on the main board.
Results
One row per model: its best reported result. The rank is the position in this list, by score; the source's own ranking can differ.
Tokscale usage counts accounts that used the model in public Tokscale submissions, Oct 14 – Nov 12, 2026. It is shown next to the score and does not explain it.
| Rank | Model | Score (% resolved) | Conditions | Date | Tokscale usage (30 days) | Source link |
|---|---|---|---|---|---|---|
| 1 | Claude Opus 4.7Listed as claude-opus-4-7_max | 83.5%Standard error ±1.7 pts | Max effort | Apr 20, 2026Run date | 27active users | Source for Claude Opus 4.7 (opens in a new tab) |
| 2 | GPT-5.5Listed as gpt-5.5-pre-release_xhigh | 80.6%Standard error ±1.8 pts | XHigh effort | Apr 24, 2026Run date | 30active users | Source for GPT-5.5 (opens in a new tab) |
| 3 | Gemini 3.5 FlashListed as gemini-3.5-flash_high | 79.3%Standard error ±1.8 pts | High effort | Jun 1, 2026Run date | 3active users | Source for Gemini 3.5 Flash (opens in a new tab) |
| 4 | Claude Opus 4.6Listed as claude-opus-4-6 | 78.7%Standard error ±1.9 pts | Not reported | Feb 18, 2026Run date | 52active users | Source for Claude Opus 4.6 (opens in a new tab) |
| 5 | GLM-5.2Listed as glm-5.2_max | 78.7%Standard error ±1.9 pts | Max effort | Jun 25, 2026Run date | 13active users | Source for GLM-5.2 (opens in a new tab) |
| 6 | DeepSeek V4 ProListed as deepseek-v4-pro_max | 77.6%Standard error ±1.9 pts | Max effort | Jun 18, 2026Run date | 51active users | Source for DeepSeek V4 Pro (opens in a new tab) |
| 7 | Qwen3.7 MaxListed as qwen3.7-max | 77.3%Standard error ±1.9 pts | Not reported | Jun 18, 2026Run date | 3active users | Source for Qwen3.7 Max (opens in a new tab) |
| 8 | GPT-5.4Listed as gpt-5.4-2026-03-05_high | 76.9%Standard error ±1.9 pts | High effort | Mar 6, 2026Run date | 1active users | Source for GPT-5.4 (opens in a new tab) |
| 9 | Qwen3.6 Max PreviewListed as qwen3.6-max-preview | 76.7%Standard error ±1.9 pts | Not reported | May 28, 2026Run date | No Tokscale usage in the last 30 days | Source for Qwen3.6 Max Preview (opens in a new tab) |
| 9 | Kimi K2.6Listed as kimi-k2.6 | 76.7%Standard error ±1.9 pts | Not reported | May 8, 2026Run date | 4active users | Source for Kimi K2.6 (opens in a new tab) |
| 9 | Claude Opus 4.5Listed as claude-opus-4-5-20251101 | 76.7%Standard error ±1.9 pts | Not reported | Feb 5, 2026Run date | 4active users | Source for Claude Opus 4.5 (opens in a new tab) |
| 12 | Gemini 3.1 Pro Preview CustomtoolsListed as gemini-3.1-pro-preview-customtools | 75.6%Standard error ±2.0 pts | Not reported | Feb 24, 2026Run date | 1active users | Source for Gemini 3.1 Pro Preview Customtools (opens in a new tab) |
| 13 | Gemini 3 Flash PreviewListed as gemini-3-flash-preview | 75.4%Standard error ±2.0 pts | Not reported | Feb 18, 2026Run date | 4active users | Source for Gemini 3 Flash Preview (opens in a new tab) |
| 14 | Claude Sonnet 4.6Listed as claude-sonnet-4-6 | 75.2%Standard error ±2.0 pts | Not reported | Feb 21, 2026Run date | 55active users | Source for Claude Sonnet 4.6 (opens in a new tab) |
| 15 | GPT-5.3 CodexListed as gpt-5.3-codex_high | 74.8%Standard error ±2.0 pts | High effort | Feb 25, 2026Run date | 5active users | Source for GPT-5.3 Codex (opens in a new tab) |
| 16 | GLM-5.1Listed as glm-5.1 | 74.2%Standard error ±2.0 pts | Not reported | May 15, 2026Run date | 5active users | Source for GLM-5.1 (opens in a new tab) |
| 17 | Kimi K2.5Listed as kimi-k2.5 | 73.8%Standard error ±2.0 pts | Not reported | Feb 17, 2026Run date | No Tokscale usage in the last 30 days | Source for Kimi K2.5 (opens in a new tab) |
| 17 | GPT-5.2Listed as gpt-5.2-2025-12-11_high | 73.8%Standard error ±2.0 pts | High effort | Feb 12, 2026Run date | No Tokscale usage in the last 30 days | Source for GPT-5.2 (opens in a new tab) |
| 19 | GPT-5 2025 08 07Listed as gpt-5-2025-08-07_high | 73.6%Standard error ±2.0 pts | High effort | Feb 6, 2026Run date | No Tokscale usage in the last 30 days | Source for GPT-5 2025 08 07 (opens in a new tab) |
| 20 | Claude Opus 4.1Listed as claude-opus-4-1-20250805 | 73.3%Standard error ±2.0 pts | Not reported | Feb 11, 2026Run date | No Tokscale usage in the last 30 days | Source for Claude Opus 4.1 (opens in a new tab) |
| 21 | Gemini 3 Pro PreviewListed as gemini-3-pro-preview | 72.9%Standard error ±2.0 pts | Not reported | Feb 13, 2026Run date | 1active users | Source for Gemini 3 Pro Preview (opens in a new tab) |
| 22 | GLM-5Listed as glm-5 | 72.1%Standard error ±2.1 pts | Not reported | Feb 15, 2026Run date | 2active users | Source for GLM-5 (opens in a new tab) |
| 23 | Claude Sonnet 4.5Listed as claude-sonnet-4-5-20250929 | 71.3%Standard error ±2.1 pts | Not reported | Feb 5, 2026Run date | 23active users | Source for Claude Sonnet 4.5 (opens in a new tab) |
| 24 | Claude Opus 4Listed as claude-opus-4-20250514 | 70.7%Standard error ±2.1 pts | Not reported | Feb 6, 2026Run date | No Tokscale usage in the last 30 days | Source for Claude Opus 4 (opens in a new tab) |
| 25 | GPT-5.1Listed as gpt-5.1-2025-11-13_high | 68.0%Standard error ±2.1 pts | High effort | Feb 18, 2026Run date | No Tokscale usage in the last 30 days | Source for GPT-5.1 (opens in a new tab) |
| 26 | GPT-5 MiniListed as gpt-5-mini-2025-08-07_medium | 64.7%Standard error ±2.2 pts | Medium effort | Feb 1, 2026Run date | 1active users | Source for GPT-5 Mini (opens in a new tab) |
| 27 | o3Listed as o3-2025-04-16_medium | 62.3%Standard error ±2.2 pts | Medium effort | Feb 12, 2026Run date | No Tokscale usage in the last 30 days | Source for o3 (opens in a new tab) |
| 28 | Claude 3.7 SonnetListed as claude-3-7-sonnet-20250219 | 61.0%Standard error ±2.2 pts | Not reported | Feb 4, 2026Run date | No Tokscale usage in the last 30 days | Source for Claude 3.7 Sonnet (opens in a new tab) |
| 29 | Qwen3.6 PlusListed as qwen3.6-plus | 57.9%Standard error ±2.2 pts | Not reported | May 14, 2026Run date | 1active users | Source for Qwen3.6 Plus (opens in a new tab) |
| 30 | Gemini 2.5 ProListed as gemini-2.5-pro | 57.6%Standard error ±2.3 pts | Not reported | Feb 13, 2026Run date | 1active users | Source for Gemini 2.5 Pro (opens in a new tab) |
| 31 | GPT-4.1 2025 04 14Listed as gpt-4.1-2025-04-14 | 48.5%Standard error ±2.3 pts | Not reported | Feb 8, 2026Run date | 5active users | Source for GPT-4.1 2025 04 14 (opens in a new tab) |
| 32 | GPT-4o 2024 11 20Listed as gpt-4o-2024-11-20 | 31.0%Standard error ±2.1 pts | Not reported | Feb 11, 2026Run date | 1active users | Source for GPT-4o 2024 11 20 (opens in a new tab) |
Showing 32 of 32 models
Score by release date
Each dot is one model's best result on SWE-bench Verified, placed at the model's release date. The dashed line is the best score among models released up to that date.
- Used on Tokscale in the last 30 days
- No Tokscale usage in the last 30 days
- Best score so far
Claude Opus 4.7: 83.5% · released Apr 16, 2026 · Max effort · 27 active users on Tokscale in the last 30 days
32 models, released Nov 20, 2024 to Jun 16, 2026. Highest: Claude Opus 4.7, 83.5%, released Apr 16, 2026. Use the left and right arrow keys to read each model in release order.20%40%60%80%100%
Show data
| Model | Released | Score | Tokscale users, 30 days |
|---|---|---|---|
| GLM-5.2 | Jun 16, 2026 | 78.7%Max effort | 13 |
| Qwen3.7 Max | May 19, 2026 | 77.3% | 3 |
| Gemini 3.5 Flash | May 19, 2026 | 79.3%High effort | 3 |
| DeepSeek V4 Pro | Apr 24, 2026 | 77.6%Max effort | 51 |
| GPT-5.5 | Apr 23, 2026 | 80.6%XHigh effort | 30 |
| Qwen3.6 Max Preview | Apr 20, 2026 | 76.7% | No Tokscale usage in the last 30 days |
| Kimi K2.6 | Apr 20, 2026 | 76.7% | 4 |
| Claude Opus 4.7 | Apr 16, 2026 | 83.5%Max effort | 27 |
| GLM-5.1 | Apr 7, 2026 | 74.2% | 5 |
| Qwen3.6 Plus | Mar 31, 2026 | 57.9% | 1 |
| GPT-5.4 | Mar 5, 2026 | 76.9%High effort | 1 |
| Gemini 3.1 Pro Preview Customtools | Feb 19, 2026 | 75.6% | 1 |
| Claude Sonnet 4.6 | Feb 17, 2026 | 75.2% | 55 |
| GLM-5 | Feb 11, 2026 | 72.1% | 2 |
| GPT-5.3 Codex | Feb 5, 2026 | 74.8%High effort | 5 |
| Claude Opus 4.6 | Feb 5, 2026 | 78.7% | 52 |
| Kimi K2.5 | Jan 27, 2026 | 73.8% | No Tokscale usage in the last 30 days |
| Gemini 3 Flash Preview | Dec 17, 2025 | 75.4% | 4 |
| GPT-5.2 | Dec 11, 2025 | 73.8%High effort | No Tokscale usage in the last 30 days |
| Claude Opus 4.5 | Nov 24, 2025 | 76.7% | 4 |
| Gemini 3 Pro Preview | Nov 18, 2025 | 72.9% | 1 |
| GPT-5.1 | Nov 13, 2025 | 68.0%High effort | No Tokscale usage in the last 30 days |
| Claude Sonnet 4.5 | Sep 29, 2025 | 71.3% | 23 |
| GPT-5 Mini | Aug 7, 2025 | 64.7%Medium effort | 1 |
| GPT-5 2025 08 07 | Aug 7, 2025 | 73.6%High effort | No Tokscale usage in the last 30 days |
| Claude Opus 4.1 | Aug 5, 2025 | 73.3% | No Tokscale usage in the last 30 days |
| Gemini 2.5 Pro | Jun 17, 2025 | 57.6% | 1 |
| Claude Opus 4 | May 22, 2025 | 70.7% | No Tokscale usage in the last 30 days |
| o3 | Apr 16, 2025 | 62.3%Medium effort | No Tokscale usage in the last 30 days |
| GPT-4.1 2025 04 14 | Apr 14, 2025 | 48.5% | 5 |
| Claude 3.7 Sonnet | Feb 24, 2025 | 61.0% | No Tokscale usage in the last 30 days |
| GPT-4o 2024 11 20 | Nov 20, 2024 | 31.0% | 1 |
Method and matching
How results are run
- Run by Epoch AI. Epoch upgraded its SWE-bench Verified scaffold in February 2026.
- Kept as history: the official SWE-bench leaderboard's newest entry is from February 2026.
Matching to Tokscale models
- 35 of 35 results are matched to 32 Tokscale models: 23 by exact id, 12 by normalized name.
- No match is marked as reviewed by hand yet.
- Reasoning effort, thinking budget and agent are conditions of a result. They never change which model a result belongs to.
License and attribution
- CC BY 4.0 (opens in a new tab), retrieved Oct 21, 2026.
- Epoch AI, ‘Capabilities & benchmarking’, epoch.ai/benchmarks (CC BY 4.0)
Sources and attribution
Each score keeps its own source, unit and run conditions. Tokscale does not combine benchmarks into one score, and a missing score means the model was not evaluated, not that it scored zero.