# OpenBookQA leaderboard

Knowledge benchmark

> Every figure below is reproduced as its upstream source published it: nothing is modelled, estimated, interpolated or converted. `Unavailable` means no source published the value — it is never a zero. Each value carries its evidence state and the date it was observed.

Page: https://modelchoice.dev/leaderboards/benchmarks/knowledge-open-book-qa

Every model the catalog carries a published OpenBookQA value for, ranked by that value.

## Benchmark record

| Field | Value |
| --- | --- |
| Category | Knowledge |
| Measure | 4-way multiple choice |
| Tasks | Elementary science questions |
| Difficulty | Elementary science reasoning |
| Published by | [Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering](https://arxiv.org/abs/1809.02789) |

A science question-answering benchmark that tests whether models can apply a small open-book set of elementary science facts to multi-step reasoning questions.

## OpenBookQA ranking

No model in the catalog has a published OpenBookQA score. The benchmark is defined by Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering, but the catalog carries no value for it yet. An absent value is shown as absent here rather than as a zero.

Rows are ordered by the value the source published, highest first. The source does not state whether a higher value is the better result, so this document does not either: for a benchmark that measures a rate of failure, read the table from the bottom.
