# AA AutomationBench (strict) leaderboard

Agentic benchmark

> Every figure below is reproduced as its upstream source published it: nothing is modelled, estimated, interpolated or converted. `Unavailable` means no source published the value — it is never a zero. Each value carries its evidence state and the date it was observed.

Page: https://modelchoice.dev/leaderboards/benchmarks/agentic-aa-automation-bench-strict

Artificial Analysis AutomationBench, strict pass@1. Every model the catalog carries a published AA AutomationBench (strict) value for, ranked by that value.

## Benchmark record

| Field | Value |
| --- | --- |
| Category | Agentic |
| Measure | Strict pass@1 |
| Tasks | End-to-end business-workflow automation tasks |
| Difficulty | Agentic automation |
| Published by | [Artificial Analysis AutomationBench Benchmark Leaderboard](https://artificialanalysis.ai/evaluations/automationbench-aa) |

The strict pass@1 score on Artificial Analysis's AutomationBench-AA, where a business-workflow task counts only when it is fully completed.

## AA AutomationBench (strict) ranking

3 models with a published AA AutomationBench (strict) value, ordered by that value, highest first. Models the source has not scored on this benchmark are not listed — they are unmeasured, not last.

| Rank | Model | Provider | Strict pass@1 | Evidence |
| --- | --- | --- | --- | --- |
| 1 | [Muse Ember](https://modelchoice.dev/models/muse-ember) | Meta | 37.8 | Observed 2026-11-21 · source benchlm:models |
| 2 | [GPT-6 Luna](https://modelchoice.dev/models/gpt-6-luna) | OpenAI | 20.2 | Observed 2026-11-21 · source benchlm:models |
| 3 | [Claude Haiku 5.5](https://modelchoice.dev/models/claude-haiku-5-5) | Anthropic | 13.9 | Observed 2026-11-21 · source benchlm:models |

Rows are ordered by the value the source published, highest first. The source does not state whether a higher value is the better result, so this document does not either: for a benchmark that measures a rate of failure, read the table from the bottom.
