We study how AI holds up in real work.
Our research comes from running our own agents, models and benchmarks every day. We test, measure and keep what holds up, then build it into our products and client systems.
Model evaluation
What each model does well, what it costs and where it fails.
We benchmark models on real tasks from our own work: coding, research, writing and long agent runs, at different settings and prices.
The results decide which model takes which job.
Routing and cost
Which model should take which task, decided by measurement.
Routing research turns evaluation results into rules: the least expensive model that qualifies for a task gets it.
This work powers MUSCLE Flex.
Multi-agent systems
How agents plan, hand off and review each other's work.
We run teams of agents across several machines with a shared task board, review gates and scheduling.
We study where they fail and build the fixes back in.
People in charge
Where a person should decide, and how to make that easy.
Shared context, clear hand-offs and approval steps let people steer agents without watching every step.
MUSCLE Cistern is our testbed.
Context and memory
How agents keep what they learned across long projects.
Agents lose the thread on long work. We study how shared context, task history and written lessons keep them on course.
What works goes into MUSCLE Cistern.
How we run a study.
Start with a decision.
Every study starts with a choice someone has to make: which model takes a task, whether an agent can run unattended, what a change will cost.
Use real work.
We build task sets from real work: code changes, research, writing and long agent runs. Public benchmarks are a starting point, not the answer.
Measure quality, cost and failure.
Each model or agent runs the same tasks at different settings. We record what it got right, what it cost and how it failed.
Change the system.
Results become routing rules, review gates and defaults. Then we measure again, because models keep changing.
Where the research goes
- Routing and cost
- MUSCLE Flex
- Shared context
- MUSCLE Cistern
- Review gates
- Client systems
- Evaluation suites
- Client systems
The least expensive model that qualifies
6 min readEvaluate on your own tasks
7 min readMeasure before you claim.