Knows, but can’t do
Beats human experts on protocol quizzes. A quiz isn’t an experiment.
Anthropic, Oct 202532 tasks in the library today, more each quarter
We train AI scientists on real lab work.
RL environments graded by real experiments, not opinions.
Graded task
CytometryA real bone-marrow run where two stains failed. Find the cells.
- Answer key
- Independent evidence
- Wet-lab experts
- Found the cells
- Frontier model
- Found under half, twice
Design partners
- fleet
- micro1.
The problemWhat we see in real runs
AI knows biology. It can’t judge experiments.
And AI labs can’t train what they can’t grade.
When a stain fails, AI gets lost
Two stains failed in a real bone-marrow run. Experts still found the cells. Opus 5.5 found under half, twice.
Complexity AI test, 2026Sees it, won’t call it
In our blind runs, models found a subtle real effect, then declined to call it.
Complexity AI test, 2026The judgment isn’t written down
Papers report the result, not why an expert trusted a run or threw it out.
Wet-lab experts we work with
Root cause
Models learn what they’re graded on. Nobody grades them on real experiments.
What we buildAvailable now
RL environments now. A task engine that keeps them fresh.
Real lab tasks for frontier labs and RL environment vendors, graded against what actually happened in the lab.
RL environments
Now- For
- Frontier labs and RL environment vendors
- What
- Real lab tasks, inside the software scientists use
- Data
- Raw instrument files plus the lab’s measured result
- Graders
- Tested on real outcomes, backed by a library of real failures
Task engine
Next- Discover
- Failures seen in paid analysis jobs and expert sessions
- Rebuild
- On public data, with a planted change where needed
- Key
- Independent evidence, or known by design
- Top-up
- Licenses or runs, only when a buyer orders
Failures pick the next task. Real work shows new failures.
How the loop worksSix steps, every turn
AI does the thinking. Real experiments keep the score.
Each turn adds real failures and harder tasks. Better models create demand for the next tier.
Real failures found.
We find where AI breaks on real lab work, in paid analysis jobs and expert sessions with AI.
Paid jobs and expert sessionsWhere it happens
The loop
- Real failures found01
- Built on real data02
- Models practice03
- Failures show up04
- We target the gap05
- The frontier moves06
Customers & partnersEnvironment onboarded
Customers and partners
Complexity AI is part of our benchmarking product. We’re working with them to run evaluations and investigate model failures on their environment.
Working with wet-lab experts at
Wet-lab experts help us find where models fail and why an expert would trust a run.
OT-2 twin · prototypeOpentrons simulator 9.1.1
An action is only half the story.
The other half is what changed. This prototype replays a simulator run and links each OT-2 command to pipette state and tracked well volumes. Twins built from real robot runs are on the roadmap.
OT-2 sample dilution and transfer
13 recorded operations · Opentrons simulator 9.1.1
Recorded run
Completed
00:000 / 13
Schematic replay of an official-simulator run. No live hardware is attached.
Roadmap · next 24 monthsTargets, not commitments
From environments to the task engine.
Data stays public by default, with planted changes where needed. Licenses only on order.
Frontier packs
- RL environments rebuilt after every model release
- Expert sessions that find where models fail
- 32 tasks today, more each quarter
Failure library
- New environment families and private test sets
- A failure library across instruments
- A paid analysis service, confidential like any CRO
- Small error-check models
Instrument twins
- OT-2 first, built from real robot runs
- Run-QC models trained on our failure map
Bring the experiment your model gets wrong.
Training AI scientists, building RL environments, or running lab work you’d like analyzed? Tell us what you’re working on.
contact@complexity-ai.com