The dumb orchestrator: keep the control loop out of the model
Most multi-agent systems fail because the control flow itself is generated by a language model. Invert it: a deterministic orchestrator that only routes, retries and times out, with all judgement pushed to the leaves where it can be observed and replaced.
There is a design that shows up in almost every multi-agent project, and it goes like this: a "manager" model receives the goal, decides which specialist agents to call, decides in what order, decides when the work is done, and hands back a result.
It demos beautifully. It is also the single most common reason these systems become impossible to operate.
What actually goes wrong
Failures compound instead of surfacing. If each routing decision is 95% reliable, a six-step plan is a coin flip. Worse, a bad decision does not raise an error — it produces a plausible next step, so the system fails by drifting rather than by stopping.
Debugging becomes archaeology. When the control flow lives in a model's output, there is no execution graph to inspect. You get a transcript. Reproducing a bug means reproducing a sampling outcome, and the answer to "why did it do that" is frequently "it felt like it".
Cost and latency go quadratic. Every routing decision is a call, and each one carries the accumulated conversation. You pay tokens for the privilege of deciding what to do next, repeatedly, at the most expensive layer of the stack.
Retries stop being safe. Deterministic orchestrators can retry a failed step because the step is identified. When the plan is regenerated on each attempt, a retry is a new plan — sometimes a completely different one, sometimes one that repeats a side effect you already committed.
Invert it
The pattern I keep coming back to is dumb orchestrator, smart agent.
The orchestrator is ordinary software. It has no model in it. Its entire job is:
- dispatch a task to a worker
- enforce a timeout
- retry with backoff on a known-transient failure
- record state transitions
- fan out and join when the graph says so
- stop when a terminal state is reached
That is a queue and a state machine. It can be read, tested and reasoned about. It behaves the same way on Tuesday as it did on Monday.
The intelligence goes to the leaves. Inside a single task, an agent can be as clever as it likes: choose tools, iterate, revise, give up. That is where model judgement is genuinely irreplaceable — and it is bounded, observable and cheap to replace when a better model shows up.
The line between them
The useful test is: would a wrong answer here corrupt state, or just produce a bad output?
Control-flow decisions corrupt state. Sending the same invoice twice, deleting the wrong branch, marking a task complete that is not — these are orchestration mistakes, and they are exactly the ones a language model should not be making unsupervised.
Content decisions produce output. Which library to use, how to phrase a summary, whether a test failure is a real bug — these benefit from judgement, and a wrong answer is visible and correctable.
Put the model on the second list, keep it off the first.
But some plans really are dynamic
They are, and this is where the pattern needs nuance rather than dogma.
When the task graph genuinely cannot be known in advance, let a model propose a plan — then hand that plan to the deterministic orchestrator to execute. Planning becomes a bounded, inspectable step that produces data, instead of an invisible loop that produces side effects. You can log the plan, diff it against previous ones, validate it against a schema, and refuse to execute a step that is not in the allowed set.
The rule is not "no model in planning". It is no model inside the control loop.
What you get back
Systems built this way are boring in the best sense. Failures land on a specific step instead of dissolving into a transcript. Retries are safe because the step is identified rather than regenerated. Cost drops because you stop paying a model to decide what a for loop already knows. Swapping the model behind one worker becomes a local change instead of a system-wide regression risk.
None of this is novel — it is the same lesson distributed systems learned twenty years ago, arriving late to a new field wearing new vocabulary. The industry currently rewards putting the model everywhere. Production rewards the opposite: determinism where it is cheap, intelligence where it is irreplaceable.