# DeepSWE leaderboard

Coding benchmark

> Every figure below is reproduced as its upstream source published it: nothing is modelled, estimated, interpolated or converted. `Unavailable` means no source published the value — it is never a zero. Each value carries its evidence state and the date it was observed.

Page: https://modelchoice.dev/leaderboards/benchmarks/coding-deep-swe

Every model the catalog carries a published DeepSWE value for, ranked by that value.

## Benchmark record

| Field | Value |
| --- | --- |
| Category | Coding |
| Measure | Pass@1 with confidence interval, cost, time, and token metadata |
| Tasks | 113 software engineering tasks across 91 repositories and 5 languages |
| Difficulty | Long-horizon software engineering |
| Published by | [DeepSWE benchmark blog](https://deepswe.datacurve.ai/blog) |

A long-horizon software engineering benchmark from Datacurve for measuring frontier coding agents on original tasks drawn from active open-source repositories.

## DeepSWE ranking

65 models with a published DeepSWE value, ordered by that value, highest first. Models the source has not scored on this benchmark are not listed — they are unmeasured, not last.

| Rank | Model | Provider | Pass@1 with confidence interval, cost, time, and token metadata | Evidence |
| --- | --- | --- | --- | --- |
| 1 | [Muse Wonder Full](https://modelchoice.dev/models/muse-wonder-full) | Meta | 79.0 | Observed 2026-11-21 · source benchlm:models |
| 2 | [Muse Wonder](https://modelchoice.dev/models/muse-wonder) | Meta | 78.8 | Observed 2026-11-21 · source benchlm:models |
| 3 | [Gemini 4 Argon](https://modelchoice.dev/models/gemini-4-argon) | Google | 77.9 | Observed 2026-11-21 · source benchlm:models |
| 4 | [DeepSeek V4.1 Flash 1017](https://modelchoice.dev/models/deepseek-v4-1-flash) | DeepSeek | 77.0 | Observed 2026-11-21 · source benchlm:models |
| 5 | [Qwen4 Max](https://modelchoice.dev/models/qwen4-max) | Alibaba | 75.7 | Observed 2026-11-21 · source benchlm:models |
| 6 | [Kimi K3.1 Pro](https://modelchoice.dev/models/kimi-3-1-pro) | Moonshot AI | 75.5 | Observed 2026-11-21 · source benchlm:models |
| 7 | [Muse Spark 1.3](https://modelchoice.dev/models/muse-spark-1-3) | Meta | 75.4 | Observed 2026-11-21 · source benchlm:models |
| 8 | [Ember-1](https://modelchoice.dev/models/ember-1) | Fireworks | 75.2 | Observed 2026-11-21 · source benchlm:models |
| 9 | [Hy4](https://modelchoice.dev/models/hy4) | Tencent | 74.9 | Observed 2026-11-21 · source benchlm:models |
| 10 | [Claude Fable 5.2](https://modelchoice.dev/models/claude-fable-5-2) | Anthropic | 74.3 | Observed 2026-11-21 · source benchlm:models |
| 11 | [Claude Opus 5.5](https://modelchoice.dev/models/claude-opus-5-5) | Anthropic | 74.2 | Observed 2026-11-21 · source benchlm:models |
| 12 | [GPT-6 Astra](https://modelchoice.dev/models/gpt-6-astra) | OpenAI | 74.1 | Observed 2026-11-21 · source benchlm:models |
| 13 | [Pareto 26.9](https://modelchoice.dev/models/pareto-26-9) | Unbiased | 74.0 | Observed 2026-11-21 · source benchlm:models |
| 14 | [Gemini 3.8 Flash](https://modelchoice.dev/models/gemini-3-8-flash) | Google | 73.8 | Observed 2026-11-21 · source benchlm:models |
| 15 | [MiniMax M3.1 Flash](https://modelchoice.dev/models/minimax-m3-1-flash) | MiniMax | 73.7 | Observed 2026-11-21 · source benchlm:models |
| 16 | [MAI Code 2](https://modelchoice.dev/models/mai-code-2) | Microsoft | 73.2 | Observed 2026-11-21 · source benchlm:models |
| 17 | [SWE-2](https://modelchoice.dev/models/swe-2) | Cognition | 73.0 | Observed 2026-11-21 · source benchlm:models |
| 18 | [GPT-5.6 Sol](https://modelchoice.dev/models/gpt-5-6-sol) | OpenAI | 72.7 | Observed 2026-11-21 · source benchlm:models |
| 19 | [Particle-1](https://modelchoice.dev/models/particle-1) | Particle AI | 72.6 | Observed 2026-11-21 · source benchlm:models |
| 20 | [Inkling 2 Small](https://modelchoice.dev/models/inkling-2-small) | Thinking Machines Lab | 72.1 | Observed 2026-11-21 · source benchlm:models |
| 21 | [GPT-6.1 Sol](https://modelchoice.dev/models/gpt-6-1-sol) | OpenAI | 71.9 | Observed 2026-11-21 · source benchlm:models |
| 21 | [MiMo-V2.6-Pro](https://modelchoice.dev/models/mimo-v2-6-pro) | Xiaomi | 71.9 | Observed 2026-11-21 · source benchlm:models |
| 23 | [ERNIE 5.5 Flash Preview](https://modelchoice.dev/models/ernie-5-5-flash-preview) | Baidu | 71.2 | Observed 2026-11-21 · source benchlm:models |
| 24 | [Claude Sonnet 5.5](https://modelchoice.dev/models/claude-sonnet-5-5) | Anthropic | 71.0 | Observed 2026-11-21 · source benchlm:models |
| 24 | [Grok 4.7](https://modelchoice.dev/models/grok-4-7) | xAI | 71.0 | Observed 2026-11-21 · source benchlm:models |
| 26 | [Pareto 26.10 Preview](https://modelchoice.dev/models/pareto-26-10-preview) | Unbiased | 69.9 | Observed 2026-11-21 · source benchlm:models |
| 27 | [GPT-5.6 Terra](https://modelchoice.dev/models/gpt-5-6-terra) | OpenAI | 69.6 | Observed 2026-11-21 · source benchlm:models |
| 28 | [Claude Opus 5](https://modelchoice.dev/models/claude-opus-5) | Anthropic | 68.8 | Observed 2026-11-21 · source benchlm:models |
| 28 | [GPT-6 Sol](https://modelchoice.dev/models/gpt-6-sol) | OpenAI | 68.8 | Observed 2026-11-21 · source benchlm:models |
| 30 | [MiMo-V2.6-Flash](https://modelchoice.dev/models/mimo-v2-6-flash) | Xiaomi | 67.9 | Observed 2026-11-21 · source benchlm:models |
| 31 | [Step 5 Preview](https://modelchoice.dev/models/step-5-preview) | StepFun | 67.7 | Observed 2026-11-21 · source benchlm:models |
| 32 | [Kimi K3](https://modelchoice.dev/models/kimi-3) | Moonshot AI | 67.5 | Observed 2026-11-21 · source benchlm:models |
| 33 | [Claude Fable 5.1](https://modelchoice.dev/models/claude-fable-5-1) | Anthropic | 67.4 | Observed 2026-11-21 · source benchlm:models |
| 34 | [GPT-5.6 Luna](https://modelchoice.dev/models/gpt-5-6-luna) | OpenAI | 67.2 | Observed 2026-11-21 · source benchlm:models |
| 35 | [GLM-5.3](https://modelchoice.dev/models/glm-5-3) | Z.AI | 66.9 | Observed 2026-11-21 · source benchlm:models |
| 36 | [GPT-6 Luna](https://modelchoice.dev/models/gpt-6-luna) | OpenAI | 66.6 | Observed 2026-11-21 · source benchlm:models |
| 37 | [Grok 4.6](https://modelchoice.dev/models/grok-4-6) | xAI | 65.9 | Observed 2026-11-21 · source benchlm:models |
| 38 | [Gemini 3.7 Flash](https://modelchoice.dev/models/gemini-3-7-flash) | Google | 65.3 | Observed 2026-11-21 · source benchlm:models |
| 39 | [Hy4 flash](https://modelchoice.dev/models/hy4-flash) | Tencent | 64.6 | Observed 2026-11-21 · source benchlm:models |
| 39 | [LongCat-2.5](https://modelchoice.dev/models/longcat-2-5) | Meituan | 64.6 | Observed 2026-11-21 · source benchlm:models |
| 41 | [Hy4 preview](https://modelchoice.dev/models/hy4-preview) | Tencent | 64.3 | Observed 2026-11-21 · source benchlm:models |
| 42 | [GLM-5.3-Flash](https://modelchoice.dev/models/glm-5-3-flash) | Z.AI | 63.4 | Observed 2026-11-21 · source benchlm:models |
| 43 | [DeepSeek V4 Pro](https://modelchoice.dev/models/deepseek-v4-pro) | DeepSeek | 62.7 | Observed 2026-11-21 · source benchlm:models |
| 43 | [DeepSeek V4 Pro 0813](https://modelchoice.dev/models/deepseek-v4-pro-max) | DeepSeek | 62.7 | Last good 2026-11-18 · source benchlm:models |
| 45 | [Ultramarine 1 Flash](https://modelchoice.dev/models/ultramarine-1-flash) | Paper Instruments | 61.5 | Observed 2026-11-21 · source benchlm:models |
| 46 | [SWE-2 Flash](https://modelchoice.dev/models/swe-2-flash) | Cognition | 60.1 | Observed 2026-11-21 · source benchlm:models |
| 47 | [Muse Spark Open](https://modelchoice.dev/models/muse-spark-open) | Meta | 60.0 | Observed 2026-11-21 · source benchlm:models |
| 48 | [MAI Code 2 Flash](https://modelchoice.dev/models/mai-code-2-flash) | Microsoft | 59.9 | Observed 2026-11-21 · source benchlm:models |
| 49 | [Ling 3.1 Flash](https://modelchoice.dev/models/ling-3-1-flash) | InclusionAI | 59.7 | Observed 2026-11-21 · source benchlm:models |
| 50 | [Muse Spark 1.2](https://modelchoice.dev/models/muse-spark-1-2) | Meta | 59.3 | Observed 2026-11-21 · source benchlm:models |
| 51 | [Qwen3.8-Flash-Next](https://modelchoice.dev/models/qwen3-8-flash-next) | Alibaba | 58.7 | Observed 2026-11-21 · source benchlm:models |
| 52 | [Qwen3.8-Omni-Flash](https://modelchoice.dev/models/qwen3-8-omni-flash) | Alibaba | 57.8 | Observed 2026-11-21 · source benchlm:models |
| 53 | [Qwen3.8 Max](https://modelchoice.dev/models/qwen3-8-max) | Alibaba | 56.6 | Observed 2026-11-21 · source benchlm:models |
| 54 | [Ornith-1.5-397B](https://modelchoice.dev/models/ornith-1-5-397b) | Ornith AI | 56.0 | Observed 2026-11-21 · source benchlm:models |
| 55 | [DeepSeek V4 Flash 0731](https://modelchoice.dev/models/deepseek-v4-flash-max) | DeepSeek | 54.4 | Observed 2026-11-21 · source benchlm:models |
| 56 | [Gemini 3.6 Flash](https://modelchoice.dev/models/gemini-3-6-flash) | Google | 49.0 | Observed 2026-11-21 · source benchlm:models |
| 57 | [Inkling 2 Mini](https://modelchoice.dev/models/inkling-2-mini) | Thinking Machines Lab | 48.2 | Observed 2026-11-21 · source benchlm:models |
| 58 | [Mercury 3](https://modelchoice.dev/models/mercury-3) | Inception | 46.3 | Observed 2026-11-21 · source benchlm:models |
| 59 | [Ripple-65B Preview](https://modelchoice.dev/models/ripple-65b-preview) | Ricursive Intelligence | 44.3 | Observed 2026-11-21 · source benchlm:models |
| 60 | [Beam 1.0 501B-A23B](https://modelchoice.dev/models/reflection-beam) | Reflection AI | 43.3 | Observed 2026-11-21 · source benchlm:models |
| 61 | [Beam 1.0 501B-A23B FP8](https://modelchoice.dev/models/reflection-beam-fp8) | Reflection AI | 43.0 | Observed 2026-11-21 · source benchlm:models |
| 62 | [Beam 1.0 501B-A23B NVFP4](https://modelchoice.dev/models/reflection-beam-nvfp4) | Reflection AI | 42.7 | Observed 2026-11-21 · source benchlm:models |
| 63 | [Qwen3.8-27B](https://modelchoice.dev/models/qwen3-8-27b) | Alibaba | 42.2 | Observed 2026-11-21 · source benchlm:models |
| 64 | [Laguna S 2.1](https://modelchoice.dev/models/laguna-s-2-1) | Poolside | 40.4 | Observed 2026-11-21 · source benchlm:models |
| 65 | [Ornith-1.5-35B-A3B](https://modelchoice.dev/models/ornith-1-5-35b-a3b) | Ornith AI | 22.0 | Observed 2026-11-21 · source benchlm:models |

Rows are ordered by the value the source published, highest first. The source does not state whether a higher value is the better result, so this document does not either: for a benchmark that measures a rate of failure, read the table from the bottom.
