# xdev-bench-hard leaderboard

external benchmark

> Every figure below is reproduced as its upstream source published it: nothing is modelled, estimated, interpolated or converted. `Unavailable` means no source published the value — it is never a zero. Each value carries its evidence state and the date it was observed.

Page: https://modelchoice.dev/leaderboards/benchmarks/external-xdev-bench-hard

xdev-bench-hard (Internal, pass@3). Every model the catalog carries a published xdev-bench-hard value for, ranked by that value.

## Benchmark record

| Field | Value |
| --- | --- |
| Category | external |
| Measure | Pass@3 |
| Tasks | Vulnerability exploitation tasks |
| Difficulty | Offensive security research |
| Published by | [Kimi K3.1 Pro Tech Blog: Advancing Open Intelligence](https://www.kimi.ai/blog/kimi-k3-1-pro) |

Moonshot AI's internal benchmark of expert-curated, source-code-based vulnerability discovery and exploitation tasks, most reproducing recently disclosed vulnerabilities with mitigations enabled.

## xdev-bench-hard ranking

1 model with a published xdev-bench-hard value, ordered by that value, highest first. Models the source has not scored on this benchmark are not listed — they are unmeasured, not last.

| Rank | Model | Provider | Pass@3 | Evidence |
| --- | --- | --- | --- | --- |
| 1 | [Kimi K3.1 Pro](https://modelchoice.dev/models/kimi-3-1-pro) | Moonshot AI | 81.0 | Observed 2026-11-21 · source benchlm:models |

Rows are ordered by the value the source published, highest first. The source does not state whether a higher value is the better result, so this document does not either: for a benchmark that measures a rate of failure, read the table from the bottom.
