# Claude Opus 4.5

Anthropic · Proprietary · rank 78 · bench-align-v5

> Every figure below is reproduced as its upstream source published it: nothing is modelled, estimated, interpolated or converted. `Unavailable` means no source published the value — it is never a zero. Each value carries its evidence state and the date it was observed.

Page: https://modelchoice.dev/models/claude-opus-4-5  
JSON: https://modelchoice.dev/api/model/claude-opus-4-5

## Facts

| Field | Value | Evidence |
| --- | --- | --- |
| Canonical id | `claude-opus-4-5` | — |
| Overall score | 54.4 | Observed 2026-11-21 · source benchlm:models |
| Verified score (BenchLM): a separate BenchLM figure, not the overall score | 58 | Observed 2026-11-21 · source benchlm:models |
| Context window | 200K tokens | Observed 2026-11-21 · source benchlm:models |
| Max output (models.dev) | 64,000 tokens | Observed 2026-11-21 · source modelsdev:catalog |
| Modalities (models.dev) | Input: text, image, PDF · Output: text | Observed 2026-11-21 · source modelsdev:catalog |
| Release date | 2025-11-24 | Observed 2026-11-21 · source benchlm:models |
| Access type | Proprietary | — |
| Blended $/1M (75% input / 25% output) | $10.00 | Derived from the input and output rates below |

## Capability evidence

Seven axes from the ranking source. An axis the source did not score is unavailable, not zero. Beside each score is BenchLM's separate verified score for that axis; the two are published side by side and are not interchangeable.

| Axis | Score | Evidence | Verified (BenchLM) | Verified evidence |
| --- | --- | --- | --- | --- |
| Agentic | 52.4 | Observed 2026-11-21 · source benchlm:models | 50.7 | Observed 2026-11-21 · source benchlm:models |
| Coding | 55.9 | Observed 2026-11-21 · source benchlm:models | 55 | Observed 2026-11-21 · source benchlm:models |
| Knowledge | 64 | Observed 2026-11-21 · source benchlm:models | 63 | Observed 2026-11-21 · source benchlm:models |
| Reasoning | 70.5 | Observed 2026-11-21 · source benchlm:models | 70.5 | Observed 2026-11-21 · source benchlm:models |
| Multimodal & Grounded | 47.5 | Observed 2026-11-21 · source benchlm:models | 47.5 | Observed 2026-11-21 · source benchlm:models |
| Instruction Following | 31.9 | Observed 2026-11-21 · source benchlm:models | 31.9 | Observed 2026-11-21 · source benchlm:models |
| Math | 52.5 | Observed 2026-11-21 · source benchlm:models | 52.5 | Observed 2026-11-21 · source benchlm:models |

## Runtime service evidence

Measured values with the date they were observed. Nothing is inferred from a sibling model or a provider claim. Regional or per-endpoint measurements appear only when the API supplies them; none are modelled.

| Measurement | Value | Observed | Last good | Evidence |
| --- | --- | --- | --- | --- |
| Time to first token | 1.18 s | 2026-11-21 | 2026-11-21 | Observed 2026-11-21 · source benchlm:speed |
| Throughput | 52 tok/s | 2026-11-21 | 2026-11-21 | Observed 2026-11-21 · source benchlm:speed |

## Endpoint and price matrix

Every published price component, including cache reads and writes.

| Component | USD | Evidence |
| --- | --- | --- |
| Input / 1M tokens | $5.00 | Observed 2026-11-21 · source benchlm:pricing |
| Output / 1M tokens | $25.00 | Observed 2026-11-21 · source benchlm:pricing |
| Cache read / 1M tokens | Unavailable | Unavailable · source benchlm:pricing |
| Cache write / 1M tokens | $6.25 | Observed 2026-11-21 · source modelsdev:catalog |
| Blended / 1M (75% input / 25% output) | $10.00 | Derived — from the input and output rates above; it has no source record of its own |
| Cost per successful task (LiveBench) | Unavailable | Unavailable · source livebench:table |

**Pricing note.** Anthropic's Claude Opus 4.5 launch page lists Opus 4.5 at $5 input / $25 output per million tokens.

## Workload-aware monthly cost example

10 conversations per day × 8 messages × 22 active days, 1200 input and 400 output tokens per message, no cache. Derived here from the published rates above by this site's own calculator — not a figure any source published.

| Field | Value |
| --- | --- |
| Modelled monthly cost | $28.16 |
| Modelled tokens | 2.82M |

## Benchmark record

87 matched benchmark rows with their published value, unit, and provenance.

### Knowledge

| Benchmark | Value | Tasks | Difficulty | Provenance |
| --- | --- | --- | --- | --- |
| AA MMLU-Pro (Artificial Analysis MMLU-Pro) | 88.9 | Professional multi-subject questions | Professional knowledge and reasoning | [Artificial Analysis MMLU-Pro Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/mmlu-pro) |
| GPQA (Graduate-Level Google-Proof Q&A) | 87 | 448 questions | Graduate level | [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://arxiv.org/abs/2311.12022) |
| SuperGPQA (SuperGPQA: Scaling LLM Evaluation Across 285 Graduate Disciplines) | 70.6 | 285 disciplines | Graduate level | [SuperGPQA: Scaling LLM Evaluation Across 285 Graduate Disciplines](https://arxiv.org/abs/2502.14739) |
| MMLU-Pro (Massive Multitask Language Understanding Professional) | 89.5 | Multiple subjects | Professional level | [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://arxiv.org/abs/2406.01574) |
| HLE (Humanity's Last Exam) | 30.8 | Expert-level questions | Frontier expert level | [Humanity's Last Exam](https://lastexam.ai/) |
| Artificial Analysis Intelligence Index | 23.7 | Cross-benchmark intelligence index | Display-only external reference | [Artificial Analysis](https://artificialanalysis.ai/) |
| AA-GPQA Diamond (Artificial Analysis GPQA Diamond) | 81.0 | Graduate-level science questions | Graduate-level science reasoning | [Artificial Analysis GPQA Diamond Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/gpqa-diamond) |
| AA-HLE (Artificial Analysis Humanity's Last Exam) | 13.2 | Expert-level questions | Frontier expert reasoning | [Artificial Analysis Humanity's Last Exam Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/hle) |
| AA-Omniscience Index (Artificial Analysis Omniscience Index) | -4.1 | Knowledge questions | Broad factual knowledge | [AA-Omniscience: Knowledge and Hallucination Benchmark](https://artificialanalysis.ai/evaluations/omniscience) |
| AA-Omniscience Accuracy (Artificial Analysis Omniscience Accuracy) | 40.9 | Knowledge questions | Broad knowledge | [Artificial Analysis model benchmarks](https://artificialanalysis.ai/models/grok-4-3) |
| AA-Omniscience Hallucination Rate (Artificial Analysis Omniscience Hallucination Rate) | 76.2 | Knowledge questions | Factuality | [Artificial Analysis model benchmarks](https://artificialanalysis.ai/models/grok-4-3) |
| MMLU-Redux | 96.6 | Broad academic QA | Advanced general knowledge | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |
| C-Eval | 92.2 | Chinese academic and professional exams | High school to professional level | [C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Models](https://arxiv.org/abs/2305.08322) |
| GPQA Diamond (Vals) (GPQA Diamond, Vals AI run) | 79.5 | Graduate-level science questions | Expert reasoning | [Vals AI GPQA Diamond, Vals AI run leaderboard](https://www.vals.ai/benchmarks/gpqa) |
| MMLU-Pro (Vals) (MMLU-Pro, Vals AI run) | 85.6 | Academic multiple-choice questions | Broad academic knowledge | [Vals AI MMLU-Pro, Vals AI run leaderboard](https://www.vals.ai/benchmarks/mmlu_pro) |

### Coding

| Benchmark | Value | Tasks | Difficulty | Provenance |
| --- | --- | --- | --- | --- |
| Terminal-Bench 2.0 | 59.3 | Terminal-based software tasks | Professional software engineering | [Terminal-Bench 2.0](https://www.tbench.ai/) |
| SWE-bench Verified (Software Engineering Benchmark Verified) | 80.9 | 500 verified issues | Professional software engineering | [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://arxiv.org/abs/2310.06770) |
| LiveCodeBench v6 | 84.8 | Fresh programming problems | Competitive programming level | [LiveCodeBench official repository and release documentation](https://github.com/LiveCodeBench/LiveCodeBench) |
| SWE-bench Pro | 57.1 | 1,865 repository problems | Long-horizon professional engineering | [SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?](https://arxiv.org/abs/2509.16941) |
| SWE Multilingual | 77.5 | Multilingual software-engineering tasks | Professional software engineering | [MiniMax M2.7: Early Echoes of Self-Evolution](https://www.minimax.io/news/minimax-m27-en) |
| NL2Repo | 43.2 | Natural language to repository tasks | System-level software comprehension | [MiniMax M2.7: Early Echoes of Self-Evolution](https://www.minimax.io/news/minimax-m27-en) |
| LiveCodeBench (Vals) (LiveCodeBench, Vals AI run) | 75.0 | Competitive programming problems (easy, medium, hard) | Frontier coding | [Vals AI LiveCodeBench, Vals AI run leaderboard](https://www.vals.ai/benchmarks/lcb) |
| PostTrainBench v1 (PostTrainBench v1 (archived)) | 17.1 | Post-training four base models on seven benchmarks | Autonomous ML engineering | [PostTrainBench v1 leaderboard (archived)](https://posttrainbench.com/?version=v1) |

### Mathematics

| Benchmark | Value | Tasks | Difficulty | Provenance |
| --- | --- | --- | --- | --- |
| AIME26 (AIME 2026) | 95.1 | Competition math problems | Olympiad-style mathematics | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |
| HMMT Feb 2025 (Harvard-MIT Mathematics Tournament February 2025) | 92.9 | Competition math problems | Olympiad-style mathematics | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |
| HMMT Nov 2025 (Harvard-MIT Mathematics Tournament November 2025) | 93.3 | Competition math problems | Olympiad-style mathematics | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |
| HMMT Feb 2026 (Harvard-MIT Mathematics Tournament February 2026) | 85.3 | Competition math problems | Olympiad-style mathematics | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |
| MMAnswerBench | 84.0 | Multimodal math questions | Advanced mathematical reasoning | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |
| FrontierMath v2 (Tiers 1-3) (FrontierMath v2 Tiers 1-3) | 34.386 | 285 private advanced mathematics problems | From olympiad-plus to early research mathematics | [FrontierMath v2 benchmark hub](https://epoch.ai/benchmarks/frontiermath-tiers-1-3-v2) |
| FrontierMath v2 (Tier 4) (FrontierMath v2 Tier 4) | 4.878 | 41 private extreme-difficulty mathematics problems | Research-level mathematics requiring hours or days of expert work | [FrontierMath Tier 4 v2 leaderboard](https://epoch.ai/benchmarks/frontiermath-tier-4-v2?view=graph&tab=leaderboard) |

### Reasoning

| Benchmark | Value | Tasks | Difficulty | Provenance |
| --- | --- | --- | --- | --- |
| LongBench v2 | 64.4 | Long-context tasks | Hard long-context | [LongBench v2](https://arxiv.org/abs/2412.15204) |
| AI-Needle | 74 | Long-context retrieval | Long-context memory | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |
| AA-LCR (Artificial Analysis Long Context Reasoning) | 70.7 | Long-context reasoning tasks | Long-context reasoning | [Artificial Analysis model benchmarks](https://artificialanalysis.ai/models/grok-4-3) |
| CritPt (Critical Physics Tasks) | 0.3 | Research-level physics questions | Research-level physics reasoning | [CritPt Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/critpt) |

### Instruction Following

| Benchmark | Value | Tasks | Difficulty | Provenance |
| --- | --- | --- | --- | --- |
| IFEval (Instruction-Following Eval) | 90.9 | 541 prompts across 25 instruction types | Instruction precision | [Instruction-Following Evaluation for Large Language Models](https://arxiv.org/abs/2311.07911) |
| IFBench (Instruction Following Benchmark) | 58 | — | — | [BenchLM](https://benchlm.ai/benchmarks/ifbench) |
| AA-IFBench (Artificial Analysis IFBench) | 43.0 | Verifiable instruction constraints | Instruction precision | [Artificial Analysis IFBench Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/ifbench) |

### Multilingual

| Benchmark | Value | Tasks | Difficulty | Provenance |
| --- | --- | --- | --- | --- |
| MMLU-ProX | 85.7 | Multilingual professional QA | Professional multilingual | [MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation](https://arxiv.org/abs/2503.10497) |
| NOVA-63 | 56.7 | Broad multilingual evaluation | Broad multilingual capability | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |
| INCLUDE | 86.2 | Cross-lingual understanding | Broad multilingual capability | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |
| PolyMath | 79.0 | Multilingual math problems | Advanced multilingual reasoning | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |
| MAXIFE | 79.2 | Multilingual instruction following | Advanced multilingual instruction following | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |

### Agentic

| Benchmark | Value | Tasks | Difficulty | Provenance |
| --- | --- | --- | --- | --- |
| Terminal-Bench 2.0 | 59.3 | Terminal-based software tasks | Professional software engineering | [Terminal-Bench 2.0](https://www.tbench.ai/) |
| Gert Labs (Gert Labs Composite Game Benchmark) | 64.23 | Novel game environments | Agentic coding and decision-making | [Gert Labs rankings](https://gertlabs.com/rankings) |
| OSWorld-Verified | 66.3 | 369 real-world computer tasks (361 when eight Google Drive tasks are excluded) | Multi-step desktop and cross-application workflows | [OSWorld](https://os-world.github.io/) |
| CyberGym | 50.6 | 1,507 vulnerability analysis instances | Real-world cybersecurity | [CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale](https://www.cybergym.io/) |
| JobBench | 32.3 | 130 tasks across 35 occupations | Professional multi-source workflows | [JobBench: Aligning Agent Work With Human Will](https://arxiv.org/abs/2605.26329) |
| OSWorld | 66.3 | Computer-use tasks | Broad computer-use suite | [GLM-5V-Turbo](https://docs.z.ai/guides/vlm/glm-5v-turbo) |
| MCP Atlas | 42.3 | Tool-integrated agent tasks | Advanced tool use | [Introducing GPT-5.4 mini and nano](https://openai.com/index/introducing-gpt-5-4-mini-and-nano/) |
| Toolathlon | 43.5 | Multi-tool workflows | Advanced tool use | [Introducing GPT-5.4 mini and nano](https://openai.com/index/introducing-gpt-5-4-mini-and-nano/) |
| τ²-bench results (τ²-Bench Tool-Agent-User Evaluation) | 86.3 | Airline, retail, and telecom customer-service task sets | Dual-control customer-service workflows | [τ²-Bench: Evaluating Conversational Agents in a Dual-Control Environment](https://arxiv.org/abs/2506.07982) |
| Claw-Eval | 59.6 | 300 tasks, 2,159 rubrics | Real-world general, multi-turn, and native multimodal agent execution | [Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agents](https://arxiv.org/abs/2604.06132) |
| QwenClawBench | 52.3 | Real-world agent workflows | Broad real-world agentic execution | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |
| τ³-bench results (τ³-Bench Tool-Agent-User Evaluation) | 70.2 | Corrected customer-service tasks plus knowledge and voice evaluation modes | Long-horizon, multimodal, and knowledge-aware tool use | [Official τ³-bench repository and release notes](https://github.com/sierra-research/tau2-bench) |
| VITA-Bench | 23.3 | Interactive consumer-service agent tasks | Long-horizon real-world workflows | [VitaBench: Benchmarking LLM Agents with Versatile Interactive Tasks in Real-world Applications](https://vitabench.github.io/) |
| DeepPlanning | 26.4 | Travel planning and constrained shopping | Constrained agent planning | [DeepPlanning: Benchmarking Long-Horizon Agentic Planning with Verifiable Constraints](https://arxiv.org/abs/2601.18137) |
| MCP-Tasks | 71.8 | MCP-integrated tool tasks | Advanced MCP workflows | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |
| WideResearch | 76.4 | Open-ended research tasks | Broad research-agent workflows | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |

### Multimodal & Grounded

| Benchmark | Value | Tasks | Difficulty | Provenance |
| --- | --- | --- | --- | --- |
| MMMU (Massive Multi-discipline Multimodal Understanding) | 80.7 | Multimodal academic reasoning | Frontier multimodal | [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://arxiv.org/abs/2401.05508) |
| MMMU-Pro (Massive Multi-discipline Multimodal Understanding Pro) | 70.6 | Multimodal academic reasoning | Frontier multimodal | [MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark](https://arxiv.org/abs/2409.02813) |
| AA-MMMU-Pro (Artificial Analysis MMMU-Pro) | 71.2 | Multimodal academic reasoning | Frontier multimodal | [Artificial Analysis MMMU-Pro Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/mmmu-pro) |
| Design Arena Website (Design Arena Website Elo) | 1234 | Website generation comparisons | Design and website generation | [OpenRouter Grok 4.3 benchmarks](https://openrouter.ai/x-ai/grok-4.3/benchmarks) |
| OmniDocBench 1.5 | 87.7 | Document understanding tasks | Grounded document reasoning | [Introducing GPT-5.4 mini and nano](https://openai.com/index/introducing-gpt-5-4-mini-and-nano/) |
| RealWorldQA | 77.0 | Real-world visual question answering | General visual reasoning | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |
| Video-MME (with subtitle) (Video-MME with subtitle) | 77.6 | Video understanding | Multimodal video reasoning | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |
| Video-MME (w/o subtitle) (Video-MME without subtitle) | 81.4 | Video understanding | Multimodal video reasoning | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |
| MathVision | 74.3 | Visually grounded math problems | Advanced multimodal mathematics | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |
| We-Math | 70.0 | Visually grounded math problems | Advanced multimodal mathematics | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |
| DynaMath | 79.7 | Dynamic visual math problems | Advanced multimodal mathematics | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |
| AI2D_TEST (AI2D test split) | 87.7 | Diagram understanding | Structured visual reasoning | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |
| CountBench | 90.6 | Visual counting tasks | Fine-grained visual perception | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |
| ERQA | 46.8 | Evidence-based visual QA | Grounded multimodal reasoning | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |
| VideoMMMU | 84.4 | Video-grounded expert reasoning | Frontier multimodal video reasoning | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |
| MLVU (M-Avg) (MLVU mean average) | 81.7 | General video understanding | Broad multimodal video reasoning | [Qwen3.6 launch benchmarks](https://qwen.ai/blog?id=qwen3.6) |
| ScreenSpot Pro | 45.7 | 1,581 grounding instructions | Professional GUI grounding | [ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use](https://arxiv.org/abs/2504.07981) |
| SimpleVQA | 65.7 | Visual QA tasks | General visual understanding | [GLM-5V-Turbo](https://docs.z.ai/guides/vlm/glm-5v-turbo) |
| V* | 67.0 | Frontier multimodal reasoning tasks | Frontier multimodal | [GLM-5V-Turbo](https://docs.z.ai/guides/vlm/glm-5v-turbo) |
| CharXiv (CharXiv Reasoning) | 68.5 | Scientific chart reasoning | Scientific visualization reasoning | [CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs](https://charxiv.github.io/) |

### external

| Benchmark | Value | Tasks | Difficulty | Provenance |
| --- | --- | --- | --- | --- |
| Chess Puzzles (Chess Puzzles (Epoch AI)) | 14.0 | 100 programmatically generated chess positions with a single best move | Novel chess positions, single best move | [Chess Puzzles (Epoch AI)](https://epoch.ai/benchmarks/chess-puzzles) |
| Mystery Game Puzzles (Mystery Game Puzzles (Epoch AI)) | 22.0 | 100 single-best-move positions from an undisclosed game | Unfamiliar game positions, single best move | [Mystery Game Puzzles (Epoch AI)](https://epoch.ai/benchmarks/mystery-game-puzzles) |
| Furniture Assembly (Furniture Assembly Benchmark (Epoch AI)) | 28.3 | 60 photos of three IKEA builds, some with deliberate assembly mistakes | Spatial reasoning from technical diagrams to real photos | [Furniture Assembly Benchmark (Epoch AI)](https://epoch.ai/benchmarks/furniture-assembly) |
| EBR-bench (Earthborne Rangers (EBR-bench)) | 14.3 | Ten playthroughs of an Earthborne Rangers campaign segment per sample (eight learning, two scored) | Learning an unfamiliar game over multi-hour playthroughs | [Earthborne Rangers (EBR-bench), Epoch AI](https://epoch.ai/benchmarks/ebr-bench) |
| FailureSensorIQ | 66.7 | Multiple-choice questions on sensors and failure modes for 10 industrial assets | Industrial maintenance and reliability knowledge | [BenchLM](https://benchlm.ai/benchmarks/failuresensoriq) |
| BinaryAudit (BinaryAudit (backdoor detection in compiled binaries)) | 36.8 | 19 backdoor and timebomb detection tasks on stripped binaries, three runs each (57 runs); 6 tooling and 8 clean-binary tasks reported alongside | Reverse engineering without source code or symbols | [We hid backdoors in ~40 MB binaries and asked AI + Ghidra to find them (Quesma)](https://quesma.com/blog/introducing-binaryaudit/) |
| OTelBench (OTelBench (OpenTelemetry instrumentation tasks)) | 29.0 | 23 OpenTelemetry instrumentation tasks in 11 languages, three attempts each (69 runs) | Distributed tracing and context propagation across services | [Benchmarking OpenTelemetry: can AI trace your failed login? (Quesma)](https://quesma.com/blog/introducing-otel-bench/) |
| NYT Connections (NYT Connections LLM Benchmark (extended version)) | 49.4 | 940 NYT Connections puzzles, each with up to four added trick words | Wordplay, word knowledge and decoys that fit almost-but-not-quite | [NYT Connections LLM Benchmark (Lech Mazur, GitHub)](https://github.com/lechmazur/nyt-connections) |
| SkillsBench official leaderboard (BenchFlow) (SkillsBench v1.1 official leaderboard (BenchFlow)) | 49.0 | 87 tasks in 8 domains (SkillsBench v1.1) | Multi-step agent tasks | [SkillsBench leaderboard](https://www.skillsbench.ai/leaderboard) |

## Lifecycle and limitations log

Lifecycle events the source associates with this model.

No lifecycle event references this model. That is not evidence the model has no lifecycle plan — only that this source published none.

## What this profile does not claim

Values are reproduced exactly as their sources published them, in the units those sources declared; none are converted, interpolated, or averaged across providers. Any field marked unavailable was attempted and not returned. Last attempted fetch for this model's score: 2026-11-21 05:17 UTC.
