Notes from the lab.
Research notes, field guides and essays from our work. We publish when something has held up in practice.
The least expensive model that qualifies
Routing is two questions asked in order: which models can do this task, and which of those costs least. Most routing mistakes come from asking them the other way round.
6 min readEvaluate on your own tasks
Public benchmarks tell you how a model does on someone else's work. A small suite built from your own tasks tells you what will happen when you change models.
7 min readAgents need a home base
An agent with no shared memory starts every task as a stranger. A home base gives agents and people the same board, the same notes and the same record of what was decided.
6 min readKeep people in charge of the machine
Agents can run unattended. That is different from running unsupervised. The difference is a handful of deliberate places where a person decides.
5 min read