ModelChoiceby MonoMind AI Labexternal benchmark
WeirdML v3 leaderboard
Every model the catalog carries a published WeirdML v3 value for, ranked by that value.
- Printed
- Page
- https://modelchoice.dev
About this benchmark
Håvard Tveit Ihle's WeirdML v3 gives coding agents 11 hand-made machine-learning and data-analysis tasks with unfamiliar data, unspecified goals or very limited feedback, and scores how well and how efficiently they improve as they spend tokens. 0 models in the ModelChoice catalog have a published value.
When it's a good referenceUse it as an independent view from outside the main ranking sources. Read the source's method before relying on it for a decision.
Categoryexternal
MeasureAgent harness runs up to 50M tokens, token-curve scoring
Tasks11 hand-made ML and data-analysis tasks (15 configurations)
DifficultyOpen-ended ML engineering with unfamiliar data and little feedback
No model in the catalog has a published WeirdML v3 score.The benchmark is defined by WeirdML v3, but the catalog carries no value for it yet. An absent value is shown as absent here rather than as a zero.
About WeirdML v3
What the benchmark measures and how its value is published, from the source’s own record.
What is WeirdML v3?
WeirdML v3 is an external benchmark published by WeirdML v3. Håvard Tveit Ihle's WeirdML v3 gives coding agents 11 hand-made machine-learning and data-analysis tasks with unfamiliar data, unspecified goals or very limited feedback, and scores how well and how efficiently they improve as they spend tokens. Its evaluation set is described by the source as 11 hand-made ML and data-analysis tasks (15 configurations). The source rates its difficulty as open-ended ML engineering with unfamiliar data and little feedback.
How is WeirdML v3 scored?
Each model's result is published by WeirdML v3 as agent harness runs up to 50M tokens, token-curve scoring. ModelChoice prints that value to 1 decimal place, exactly as published, and never converts or rescales it. This page ranks the 0 models that carry a published value, highest value first; a model the source never scored is left out rather than ranked at zero. The source does not state whether a higher value is the better result, so neither does this page.
About MonoMindmonomind.oneMonoMind is an AI transformation partner for businesses.
We are a digital agency and AI consulting studio. We help companies move from “we should be using AI” to AI that works inside their business and produces results.
What we do
- AI Strategy
- We start with your business, not the technology. We find where AI can reduce cost, save effort, or create new revenue, then build a clear roadmap you can act on.
- Build & Deploy
- We design, build, and launch the websites, platforms, and AI-powered tools in that roadmap. Everything we build is meant to go live and be used.
- Marketing & Growth
- A good product still needs to reach people. We turn your digital presence into a growth channel through content, search, and campaigns built for your audience.
How we work
- One team, start to finish. The people who shape your strategy also deliver it. Nothing gets lost between planning and launch.
- Priced by value. We charge based on the outcome we deliver, not the effort it takes. Our incentive is your result.
- Local understanding. We know the languages, platforms, and business culture of the markets we serve, so our work fits how your customers actually behave.
Our belief
AI should make a business simpler and stronger, not more complicated. Our name reflects that: Mono for focus, Mind for intelligence. We focus on the few things that matter most, and we use AI to get them done well.
Let’s talk about what AI can do for your business.
daren@monomind.one · https://monomind.one
ModelChoiceby MonoMind AI LabEvidence for practical AI runtime and cost decisions.
Every figure in this document comes from a cited upstream source and carries the date it was read. A value no source published is shown as unavailable, never as zero. The live page is updated as the sources change.
- Live page
- https://modelchoice.dev
- Sources
- https://modelchoice.dev/sources
- Printed
© 2026 MonoMind AI Lab · modelchoice.dev