We have tried a lot of ways to wire tools to language models. These four are the ones still in production.
1. The narrow specialist
One model, one tool, one job. No planner, no router.
- Best for: well-defined transformations (extract, classify, rewrite).
- Failure mode: almost none. This is the boring, reliable pattern.
2. Plan-then-execute
The model writes a plan first, a human or a cheap validator approves it, then a second pass executes.
- Best for: multi-step workflows where mistakes are expensive.
- Failure mode: plans drift from execution. Mitigate by passing the plan back in at every step.
3. ReAct with a budget
Classic reason-act-observe loop, but with a hard cap.
const MAX_STEPS = 6;
const MAX_USD = 0.25;
- Best for: open-ended research and lookup tasks.
- Failure mode: infinite tool loops. The budget is not optional.
4. Router + specialists
A small, fast model classifies the request and hands it to one of N specialist agents.
| Layer | Model | Job |
|---|---|---|
| Router | small/cheap | Pick one of N |
| Specialist | larger | Do the actual work |
- Best for: customer-facing surfaces with mixed traffic.
- Failure mode: router confidence is silently low. Log routing decisions and review weekly.
What we stopped doing
- Giving one model 20 tools and hoping. Pick 3.
- Free-form JSON without a schema. Always validate.
- Letting the model retry its own tool errors more than twice in a row.