# Gemini 3.8 Flash

Google · Proprietary · rank 29 · bench-align-v5

> Every figure below is reproduced as its upstream source published it: nothing is modelled, estimated, interpolated or converted. `Unavailable` means no source published the value — it is never a zero. Each value carries its evidence state and the date it was observed.

Page: https://modelchoice.dev/models/gemini-3-8-flash  
JSON: https://modelchoice.dev/api/model/gemini-3-8-flash

## Facts

| Field | Value | Evidence |
| --- | --- | --- |
| Canonical id | `gemini-3-8-flash` | — |
| Overall score | 67.7 | Observed 2026-11-21 · source benchlm:models |
| Verified score (BenchLM): a separate BenchLM figure, not the overall score | 75 | Observed 2026-11-21 · source benchlm:models |
| Context window | 1M tokens | Observed 2026-11-21 · source benchlm:models |
| Max output (models.dev) | 65,536 tokens | Observed 2026-11-21 · source modelsdev:catalog |
| Modalities (models.dev) | Input: text, audio, image, PDF, video · Output: text | Observed 2026-11-21 · source modelsdev:catalog |
| Release date | 2026-09-02 | Observed 2026-11-21 · source benchlm:models |
| Access type | Proprietary | — |
| Blended $/1M (75% input / 25% output) | $1.50 | Derived from the input and output rates below |

## Capability evidence

Seven axes from the ranking source. An axis the source did not score is unavailable, not zero. Beside each score is BenchLM's separate verified score for that axis; the two are published side by side and are not interchangeable.

| Axis | Score | Evidence | Verified (BenchLM) | Verified evidence |
| --- | --- | --- | --- | --- |
| Agentic | 72.8 | Observed 2026-11-21 · source benchlm:models | 66.3 | Observed 2026-11-21 · source benchlm:models |
| Coding | 77.2 | Observed 2026-11-21 · source benchlm:models | 77.2 | Observed 2026-11-21 · source benchlm:models |
| Knowledge | 78.4 | Observed 2026-11-21 · source benchlm:models | 78.3 | Observed 2026-11-21 · source benchlm:models |
| Reasoning | 71.3 | Observed 2026-11-21 · source benchlm:models | 71.3 | Observed 2026-11-21 · source benchlm:models |
| Multimodal & Grounded | 87.8 | Observed 2026-11-21 · source benchlm:models | 87.8 | Observed 2026-11-21 · source benchlm:models |
| Instruction Following | Unavailable | Unavailable · source benchlm:models | Unavailable | Unavailable · source benchlm:models |
| Math | 64.1 | Observed 2026-11-21 · source benchlm:models | 64.1 | Observed 2026-11-21 · source benchlm:models |

## Runtime service evidence

Measured values with the date they were observed. Nothing is inferred from a sibling model or a provider claim. Regional or per-endpoint measurements appear only when the API supplies them; none are modelled.

| Measurement | Value | Observed | Last good | Evidence |
| --- | --- | --- | --- | --- |
| Time to first token | 19.76 s | 2026-11-21 | 2026-11-21 | Observed 2026-11-21 · source benchlm:speed |
| Throughput | 115 tok/s | 2026-11-21 | 2026-11-21 | Observed 2026-11-21 · source benchlm:speed |

## Endpoint and price matrix

Every published price component, including cache reads and writes.

| Component | USD | Evidence |
| --- | --- | --- |
| Input / 1M tokens | $0.75 | Observed 2026-11-21 · source benchlm:pricing |
| Output / 1M tokens | $3.75 | Observed 2026-11-21 · source benchlm:pricing |
| Cache read / 1M tokens | $0.075 | Observed 2026-11-21 · source benchlm:pricing |
| Cache write / 1M tokens | Unavailable | Unavailable · source openrouter:pricing |
| Blended / 1M (75% input / 25% output) | $1.50 | Derived — from the input and output rates above; it has no source record of its own |
| Cost per successful task (LiveBench) | Unavailable | Unavailable · source livebench:table |

**Pricing note.** Google lists introductory Gemini 3.8 Flash rates through December 31, 2026: $0.75 input, $0.075 cached input, and $3.75 output per million tokens. Flex and Batch input and output rates are half those amounts. On January 1, 2027, standard rates rise to $1.50 input, $0.15 cached input, and $7.50 output; Flex and Batch rates rise to $0.75 input, $0.075 cached input, and $3.75 output. Cache storage is $0.50 per million tokens per hour through 2026 and $1.00 afterward.

## Workload-aware monthly cost example

10 conversations per day × 8 messages × 22 active days, 1200 input and 400 output tokens per message, no cache. Derived here from the published rates above by this site's own calculator — not a figure any source published.

| Field | Value |
| --- | --- |
| Modelled monthly cost | $4.22 |
| Modelled tokens | 2.82M |

## Benchmark record

86 matched benchmark rows with their published value, unit, and provenance.

### Knowledge

| Benchmark | Value | Tasks | Difficulty | Provenance |
| --- | --- | --- | --- | --- |
| BioMysteryBench (human-solvable) (BioMysteryBench Human Solvable) | 88.8 | Human-solvable computational biology investigations | Expert computational biology | [Claude Opus 5 System Card](https://www-cdn.anthropic.com/c5fbac3f0b1280a933ebd26d3cb8bb9f5bdeaf48/Claude%20Opus%205%20System%20Card.pdf) |
| BioMysteryBench (human-difficult) (BioMysteryBench Human Difficult) | 56.5 | Human-difficult computational biology investigations | Frontier computational biology | [Claude Opus 5 System Card](https://www-cdn.anthropic.com/c5fbac3f0b1280a933ebd26d3cb8bb9f5bdeaf48/Claude%20Opus%205%20System%20Card.pdf) |
| HLE-Verified | 54.9 | 1,811 verified or revised expert questions | Frontier multidisciplinary expert reasoning | [HLE-Verified: A Systematic Verification and Structured Revision of Humanity's Last Exam](https://arxiv.org/abs/2602.13964) |
| LABBench2 (LABBench2: An Improved Benchmark for AI Systems Performing Biology Research) | 86.2 | Nearly 1,900 biology-research tasks | Real-world biology research | [LABBench2: An Improved Benchmark for AI Systems Performing Biology Research](https://arxiv.org/abs/2604.09554) |
| Artificial Analysis Intelligence Index | 30.9 | Cross-benchmark intelligence index | Display-only external reference | [Artificial Analysis](https://artificialanalysis.ai/) |
| AA-GPQA Diamond (Artificial Analysis GPQA Diamond) | 95.3 | Graduate-level science questions | Graduate-level science reasoning | [Artificial Analysis GPQA Diamond Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/gpqa-diamond) |
| AA-HLE (Artificial Analysis Humanity's Last Exam) | 47.8 | Expert-level questions | Frontier expert reasoning | [Artificial Analysis Humanity's Last Exam Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/hle) |
| AA-Omniscience Index (Artificial Analysis Omniscience Index) | 23.3 | Knowledge questions | Broad factual knowledge | [AA-Omniscience: Knowledge and Hallucination Benchmark](https://artificialanalysis.ai/evaluations/omniscience) |
| AA-Omniscience Accuracy (Artificial Analysis Omniscience Accuracy) | 50.5 | Knowledge questions | Broad knowledge | [Artificial Analysis model benchmarks](https://artificialanalysis.ai/models/grok-4-3) |
| AA-Omniscience Hallucination Rate (Artificial Analysis Omniscience Hallucination Rate) | 55.1 | Knowledge questions | Factuality | [Artificial Analysis model benchmarks](https://artificialanalysis.ai/models/grok-4-3) |
| GPQA Diamond (Vals) (GPQA Diamond, Vals AI run) | 94.4 | Graduate-level science questions | Expert reasoning | [Vals AI GPQA Diamond, Vals AI run leaderboard](https://www.vals.ai/benchmarks/gpqa) |
| MMLU-Pro (Vals) (MMLU-Pro, Vals AI run) | 90.2 | Academic multiple-choice questions | Broad academic knowledge | [Vals AI MMLU-Pro, Vals AI run leaderboard](https://www.vals.ai/benchmarks/mmlu_pro) |

### Coding

| Benchmark | Value | Tasks | Difficulty | Provenance |
| --- | --- | --- | --- | --- |
| Terminal-Bench 2.1 | 89.4 | Terminal-based software-agent tasks | Professional software engineering | [Terminal-Bench](https://www.tbench.ai/) |
| FrontierSWE v2 | 19.6 | 34 ultra-long-horizon engineering and research tasks | Ultra-long-horizon frontier software engineering | [FrontierSWE v2](https://www.frontierswe.com/blog/v2) |
| Bug Hunt Bench | 18.0 | 105 planted bugs across two production TypeScript repositories | Blind production-repository bug finding and repair | [Bug Hunt Bench method, caveats, and definitions](https://bughunt.productcompass.pm/method) |
| AA Coding Index (Artificial Analysis Coding Index) | 76.3 | Cross-benchmark coding index | Display-only external reference | [Artificial Analysis model leaderboards](https://artificialanalysis.ai/leaderboards/models) |
| AA-SciCode (Artificial Analysis SciCode) | 56.6 | Scientific coding subproblems | Scientific programming | [Artificial Analysis SciCode Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/scicode) |
| LiveCodeBench (Vals) (LiveCodeBench, Vals AI run) | 89.5 | Competitive programming problems (easy, medium, hard) | Frontier coding | [Vals AI LiveCodeBench, Vals AI run leaderboard](https://www.vals.ai/benchmarks/lcb) |
| SWE-bench (Vals) (SWE-bench, Vals AI run) | 80.0 | Real repository issues by human time bucket | Frontier coding agents | [Vals AI SWE-bench, Vals AI run leaderboard](https://www.vals.ai/benchmarks/swebench) |
| DeepSWE | 73.8 | 113 software engineering tasks across 91 repositories and 5 languages | Long-horizon software engineering | [DeepSWE benchmark blog](https://deepswe.datacurve.ai/blog) |
| ScarfBench (ScarfBench: Cross-Framework Application Migration in Enterprise Java) | 66.0 | 204 framework migration tasks across 34 enterprise Java applications | Whole-application framework migration | [ScarfBench: A Benchmark for Cross-Framework Application Migration in Enterprise Java](https://arxiv.org/abs/2605.06754) |

### Mathematics

| Benchmark | Value | Tasks | Difficulty | Provenance |
| --- | --- | --- | --- | --- |
| FrontierMath v2 (Tiers 1-3) (FrontierMath v2 Tiers 1-3) | 68.421 | 285 private advanced mathematics problems | From olympiad-plus to early research mathematics | [FrontierMath v2 benchmark hub](https://epoch.ai/benchmarks/frontiermath-tiers-1-3-v2) |
| FrontierMath v2 (Tier 4) (FrontierMath v2 Tier 4) | 21.951 | 41 private extreme-difficulty mathematics problems | Research-level mathematics requiring hours or days of expert work | [FrontierMath Tier 4 v2 leaderboard](https://epoch.ai/benchmarks/frontiermath-tier-4-v2?view=graph&tab=leaderboard) |

### Reasoning

| Benchmark | Value | Tasks | Difficulty | Provenance |
| --- | --- | --- | --- | --- |
| ARC-AGI-1 (ARC-AGI-1 Semi-Private Evaluation) | 98.50 | Semi-private ARC-AGI-1 evaluation set | Abstract visual reasoning | [ARC Prize leaderboard](https://arcprize.org/) |
| ARC-AGI-2 (Abstraction and Reasoning Corpus for AGI v2) | 89.166667 | Visual pattern completion and abstract reasoning | Expert-level — hardest public reasoning benchmark | [ARC-AGI-2: A Harder General Intelligence Benchmark](https://arcprize.org/arc-agi/2/) |
| ARC-AGI-3 (Abstraction and Reasoning Corpus for AGI v3) | 10.366279 | Interactive game-like tasks with hidden rules | Frontier agentic reasoning | [ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence](https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf) |
| AA-LCR (Artificial Analysis Long Context Reasoning) | 81.3 | Long-context reasoning tasks | Long-context reasoning | [Artificial Analysis model benchmarks](https://artificialanalysis.ai/models/grok-4-3) |
| MLCR-AA (Medical Long Context Reasoning (MLCR-AA)) | 21.7 | Long, fragmented medical-record reasoning | Long-context medical reasoning | [Medical Long Context Reasoning (MLCR-AA)](https://artificialanalysis.ai/evaluations/mlcr-aa) |
| CritPt (Critical Physics Tasks) | 18.3 | Research-level physics questions | Research-level physics reasoning | [CritPt Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/critpt) |
| ARC-AGI-3 Provider Adapter (ARC-AGI-3 Semi-Private: Provider Adapter harness) | 35.00 | — | — | [ARC Prize verified ARC-AGI-3 results](https://arcprize.org/leaderboard) |

### Agentic

| Benchmark | Value | Tasks | Difficulty | Provenance |
| --- | --- | --- | --- | --- |
| Terminal-Bench 4.0 | 19.10 | 66 professional computer-work tasks | Frontier autonomous knowledge work | [Terminal-Bench 4.0](https://www.tbench.ai/news/terminal-bench-4-0) |
| AA Briefcase (Artificial Analysis Briefcase) | 1203 | Professional knowledge-work tasks | Professional work | [Artificial Analysis Briefcase Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/aa-briefcase) |
| AA AutomationBench (Artificial Analysis AutomationBench) | 59.9 | Business-process automation tasks | Agentic automation | [Artificial Analysis AutomationBench Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/automationbench-aa) |
| AA ITBench (Artificial Analysis ITBench-AA) | 52.5 | IT incident-response tasks | Enterprise IT operations | [Artificial Analysis ITBench-AA Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/itbench-aa) |
| AA Tau3 Banking (Artificial Analysis Tau3-Banking) | 44.9 | Banking tool-use workflows | Agentic banking workflows | [Artificial Analysis Tau3-Banking Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/tau3-banking) |
| Terminal-Bench 2.1 | 89.4 | Terminal-based software-agent tasks | Professional software engineering | [Terminal-Bench](https://www.tbench.ai/) |
| GDPval-AA | 1545 | Agentic real-world work tasks | Professional agentic workflows | [DeepSeek-V4 Technical Report](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/blob/main/DeepSeek_V4.pdf) |
| GDPval-AA (GDPval-AA normalized) | 46.8 | Economically valuable tasks | Professional agentic workflows | [Artificial Analysis model benchmarks](https://artificialanalysis.ai/models/grok-4-3) |
| AA Agentic Index (Artificial Analysis Agentic Index) | 41.1 | Cross-benchmark agentic index | Display-only external reference | [Artificial Analysis model leaderboards](https://artificialanalysis.ai/leaderboards/models) |
| AA Terminal-Bench 4.0 (Artificial Analysis Terminal-Bench v4.0) | 19.7 | Terminal-based agent tasks | Agentic software engineering | [Artificial Analysis Terminal-Bench v4.0 Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/terminalbench-v4-0) |
| GDP.pdf (Artificial Analysis GDP.pdf) | 21.0 | Professional document-production tasks | Professional knowledge work | [GDP.pdf Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/gdp-pdf) |
| AA-AnalystAgent (Artificial Analysis AnalystAgent) | 46.3 | Spreadsheet and document analysis questions | Business and data analysis | [AA-AnalystAgent Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/aa-analyst-agent) |
| OSWorld 2.0 | 59.0 | 108 long-horizon computer-use workflows | Long-horizon professional workflows | [OSWorld2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks](https://arxiv.org/abs/2606.29537) |
| Finance Agent v2 | 61.4 | Financial analyst task categories | Professional expert-task agent workflow | [Finance Agent v2](https://www.vals.ai/benchmarks/fabv2) |
| Terminal-Bench 2.1 (Vals) (Terminal-Bench 2.1, Vals AI run) | 81.3 | Difficult terminal tasks | Frontier agentic | [Vals AI Terminal-Bench 2.1, Vals AI run leaderboard](https://www.vals.ai/benchmarks/terminal-bench-2-1) |
| ApprenticeBench (ApprenticeBench: end-to-end computer use, continual learning, and long-horizon agency on a real accounts-payable job) | 24 | 100 vendor bills processed in sequence inside a simulated construction company | Long-horizon computer use with offline and online continual learning | [ApprenticeBench: a step change in AI's job readiness](https://neocognition.io/blog/apprentice-bench/) |
| Hermes Index (Hermes Index: agentic capability and cost per task inside Hermes Agent) | 38.1 | Four suites: Hermes Bench (150 tasks), TerminalBench 4 without its 4 GPU tasks, TerminalBench Science and SkillsBench | Frontier agentic | [Hermes Index: an agentic model leaderboard](https://portal.nousresearch.com/bench) |
| Hermes Bench (Hermes Bench: 150 agent tasks graded on the files and state the agent leaves behind) | 62.2 | 150 tasks across 25 categories | Frontier agentic | [Hermes Bench (Hermes Index)](https://portal.nousresearch.com/bench) |
| MazeBench (MazeBench: visual spatial reasoning in a 3D open world) | 4 | 100 hidden gems across 256 rooms in one open world | Long-horizon spatial planning, with single puzzles that can take over one hundred moves | [MazeBench Results](https://mazebench.com/blog?post=maze-bench-results) |
| Terminal-Bench 4.1 | 18.20 | 66 professional computer-work tasks | Frontier autonomous knowledge work | [Terminal-Bench 4.1](https://www.tbench.ai/news/terminal-bench-4-1) |
| AA Cyber Index (Artificial Analysis Cyber Index) | 25.1 | Three defensive cybersecurity evaluations | Real-world vulnerability discovery and patching | [Artificial Analysis Cyber Index](https://artificialanalysis.ai/evaluations/artificial-analysis-cyber-index) |
| CWE-Bench-AA (Artificial Analysis CWE-Bench-AA) | 52.5 | Real-world vulnerability patching tasks | Automated vulnerability remediation | [CWE-Bench-AA Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/cwe-bench-aa) |
| DeepsecBench-AA (Artificial Analysis DeepsecBench-AA) | 14.5 | Vulnerability discovery in real codebases | Security code review | [DeepsecBench-AA Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/deepsecbench-aa) |
| CyberGym-E2E-AA (Artificial Analysis CyberGym-E2E-AA) | 8.4 | 131 end-to-end vulnerability reproduction and patching tasks | Real-world cybersecurity | [CyberGym-E2E-AA Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/cybergym-e2e-aa) |
| MKA (Multimodal Knowledge Agent Benchmark) | 35.3 | 200 multimodal professional tasks across 10 industry verticals | Expert multimodal knowledge work | [MKA Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/mka) |

### Multimodal & Grounded

| Benchmark | Value | Tasks | Difficulty | Provenance |
| --- | --- | --- | --- | --- |
| AA-MMMU-Pro (Artificial Analysis MMMU-Pro) | 85.6 | Multimodal academic reasoning | Frontier multimodal | [Artificial Analysis MMMU-Pro Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/mmmu-pro) |
| Design Arena Website (Design Arena Website Elo) | 1287 | Website generation comparisons | Design and website generation | [OpenRouter Grok 4.3 benchmarks](https://openrouter.ai/x-ai/grok-4.3/benchmarks) |
| LVBench | 87.1 | Long-form video question answering | Extended temporal reasoning | [Qwen3.8-Max: A New Bar for Coding and Cowork](https://qwen.ai/blog?id=qwen3.8) |
| CharXiv w/o tools (CharXiv Reasoning without tools) | 86.2 | Scientific chart reasoning (tool-free) | Scientific visualization reasoning | [CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs](https://charxiv.github.io/) |
| ParseBench-1 (ParseBench-1: document parsing for AI agents (LlamaIndex leaderboard, first release)) | 71.20 | 2,078 human-verified enterprise pages from insurance, finance and government documents | No method is strong on all five dimensions; charts and visual grounding separate the field | [ParseBench: A Document Parsing Benchmark for AI Agents](https://arxiv.org/abs/2604.08538) |
| ExtractBench (LlamaIndex) (ExtractBench: schema-guided enterprise document extraction (LlamaIndex leaderboard)) | 80.71 | 370 documents, 4,869 pages, 8 business domains, 67 document types, each with a JSON Schema | Short documents are near solved; long record lists and grounding separate systems | [ExtractBench: A Benchmark for Schema-Guided Enterprise Document Extraction](https://arxiv.org/abs/2607.29677) |
| ParseBench-2 (ParseBench-2: severity-aware document parsing (LlamaIndex leaderboard)) | 76.67 | 4,317 page-level test cases from 2,645 documents in 13 datasets | Charts and visual grounding separate the field; several systems that lead on text and tables score near zero on charts | [ParseBench-2: Severity-Aware Evaluation of Document Parsing](https://www.parsebench.ai/parsebench-2.pdf) |

### Instruction Following

| Benchmark | Value | Tasks | Difficulty | Provenance |
| --- | --- | --- | --- | --- |
| Gray Swan IPI (15 attempts) (Gray Swan Indirect Prompt Injection, attack success at 15 attempts) | 5.5 | Indirect prompt-injection attacks | Adversarial prompt injection | [Gemini 4 Argon evaluation methodology](https://deepmind.google/models/evals-methodology/gemini-4-argon) |

### external

| Benchmark | Value | Tasks | Difficulty | Provenance |
| --- | --- | --- | --- | --- |
| MaxxBench (MaxxBench (mutated DeepSWE tasks)) | 53.4 | 89 mutated DeepSWE feature-implementation tasks, two attempts per model | Long-horizon software engineering under changed requirements | [Why MaxxBench (unLAP research post)](https://maxxbench.unlap.ai/research/why-maxxbench) |
| Chess Puzzles (Chess Puzzles (Epoch AI)) | 58.0 | 100 programmatically generated chess positions with a single best move | Novel chess positions, single best move | [Chess Puzzles (Epoch AI)](https://epoch.ai/benchmarks/chess-puzzles) |
| Mystery Game Puzzles (Mystery Game Puzzles (Epoch AI)) | 47.0 | 100 single-best-move positions from an undisclosed game | Unfamiliar game positions, single best move | [Mystery Game Puzzles (Epoch AI)](https://epoch.ai/benchmarks/mystery-game-puzzles) |
| Furniture Assembly (Furniture Assembly Benchmark (Epoch AI)) | 31.7 | 60 photos of three IKEA builds, some with deliberate assembly mistakes | Spatial reasoning from technical diagrams to real photos | [Furniture Assembly Benchmark (Epoch AI)](https://epoch.ai/benchmarks/furniture-assembly) |
| MirrorCode (MirrorCode (Epoch AI and METR)) | 38.9 | 30 reimplementation tasks (15 programs in two languages), three attempts each | Days-long software engineering without source code | [MirrorCode (Epoch AI)](https://epoch.ai/benchmarks/mirrorcode) |
| Epoch Automation Reports | 42.3 | 11 open-ended work tasks in five categories from Epoch AI's own work | Open-ended professional research and design work | [Epoch Automation Reports](https://epoch.ai/benchmarks/epoch-automation-reports) |
| FACTS Multimodal | 57.6 | About 1,500 image questions: 711 public and 811 held out | Image-grounded factual answering | [BenchLM](https://benchlm.ai/benchmarks/facts-multimodal) |
| DAYJOB | 2.6 | 130 long-horizon tasks: 80 finance and 50 healthcare | Long-horizon professional work | [BenchLM](https://benchlm.ai/benchmarks/dayjob) |
| DAYJOB Healthcare | 1.2 | 50 long-horizon healthcare tasks across five subfields | Long-horizon clinical and healthcare operations work | [BenchLM](https://benchlm.ai/benchmarks/dayjob-healthcare) |
| FailureSensorIQ | 69.0 | Multiple-choice questions on sensors and failure modes for 10 industrial assets | Industrial maintenance and reliability knowledge | [BenchLM](https://benchlm.ai/benchmarks/failuresensoriq) |
| Code2Video | 1440 | Creative briefs rendered as motion-graphics video | Code-driven motion design | [BenchLM](https://benchlm.ai/benchmarks/code2video) |
| Landing Page Arena | 1360 | 12 client briefs for small-business landing pages, rated by human votes | Marketing page design and copy | [BenchLM](https://benchlm.ai/benchmarks/landing-page-arena) |
| Chart Arena | 1473 | 13 chart briefs with small datasets, rated by human votes | Data visualisation | [BenchLM](https://benchlm.ai/benchmarks/chart-arena) |
| Joke Arena | 1539 | 100 joke prompts in 10 categories, rated by human votes | Humour writing | [BenchLM](https://benchlm.ai/benchmarks/joke-arena) |
| Pelican Arena | 1563 | One prompt: an animated SVG of a pelican riding a bicycle, rated by human votes | Vector drawing and animation in code | [BenchLM](https://benchlm.ai/benchmarks/pelican-arena) |
| Landmark Arena | 1346 | 20 famous landmarks built as three.js web pages, rated by human votes | 3D scenes in browser code | [BenchLM](https://benchlm.ai/benchmarks/landmark-arena) |
| ASCII Arena | 1655 | ASCII drawings of subjects from Google's Quick, Draw! dataset, rated by human votes | Text-character drawing | [BenchLM](https://benchlm.ai/benchmarks/ascii-arena) |
| Game Arena Go | 1170 | Head-to-head Go games on a 13 x 13 board | Strategic planning and rule-following | [BenchLM](https://benchlm.ai/benchmarks/game-arena-go) |
| BabaIsBench (BabaIsBench (Baba Is You intro levels, pass@1)) | 100 | 8 intro levels of Baba Is You, three attempts per level (Stage 0); 15 further levels run once (Stage 1) | Rule-rewriting puzzles that need planning over many moves | [Baba Is Solved by Fable 5 and GPT-5.6 Sol, but at what cost? (Quesma)](https://quesma.com/blog/baba-is-bench/) |
| NYT Connections (NYT Connections LLM Benchmark (extended version)) | 97.4 | 940 NYT Connections puzzles, each with up to four added trick words | Wordplay, word knowledge and decoys that fit almost-but-not-quite | [NYT Connections LLM Benchmark (Lech Mazur, GitHub)](https://github.com/lechmazur/nyt-connections) |
| StealthMark (StealthMark Intelligence Benchmarks (overall score)) | 47.9 | 8 original tasks: Newton's Mirror, Ground Truth, Latch, Foresight, Mona Lisa, Selfie, Beach and Raccoon | Chained physical reasoning, planning without tools and precise visual construction | [StealthMark: The AI Benchmark Explained (Stealth Models)](https://stealthmodels.com/stealthmark) |
| FrontierPhysics (End-to-End Physics Research Benchmark for AI Agents) | 0.0 | 56 research projects from 42 physicists across nine fields, three trials each (168 rollouts per setup) | Open-ended research projects that took their authors about 270 days on average | [FrontierPhysics: Evaluating Agents for End-to-End Physics Research (arXiv, November 4, 2026)](https://arxiv.org/abs/2611.06910) |
| HoneyBench (HoneyBench v0.2 (Goodhart Labs)) | 80.5 | 20 private honeypot tasks across 7 domains with exploitable evaluators | Agentic alignment and specification gaming | [HoneyBench v0.2](https://goodhartlabs.com/honeybench) |

## Independent scores (Epoch AI)

A second, independent score source: Epoch's own figures in Epoch's units, one line per model version Epoch evaluated (effort setting as Epoch labels it); higher is better. Not part of this site's capability scores or rank.

| Benchmark | Variant | Score | Unit | Epoch date | Evidence |
| --- | --- | --- | --- | --- | --- |
| GPQA Diamond | high | 0.9539 | fraction (0–1) | 2026-09-02 | Observed 2026-11-21 · source epoch:benchmarks |
| FrontierMath Tiers 1-3 (v2) | high | 0.6842 | fraction (0–1) | 2026-09-02 | Observed 2026-11-21 · source epoch:benchmarks |
| FrontierMath Tier 4 (v2) | high | 0.2195 | fraction (0–1) | 2026-09-02 | Observed 2026-11-21 · source epoch:benchmarks |
| SimpleQA Verified | high | 0.697 | fraction (0–1) | 2026-09-02 | Observed 2026-11-21 · source epoch:benchmarks |

Source: [Epoch AI, CC BY 4.0](https://epoch.ai/benchmarks) ([licence](https://creativecommons.org/licenses/by/4.0/)). Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from https://epoch.ai/benchmarks.

## Lifecycle and limitations log

Lifecycle events the source associates with this model.

No lifecycle event references this model. That is not evidence the model has no lifecycle plan — only that this source published none.

## What this profile does not claim

Values are reproduced exactly as their sources published them, in the units those sources declared; none are converted, interpolated, or averaged across providers. Any field marked unavailable was attempted and not returned. Last attempted fetch for this model's score: 2026-11-21 05:17 UTC.
