# The dumb orchestrator: keep the control loop out of the model Source: https://bszczyglo-landing.vercel.app/blog/dumb-orchestrator Author: Bartłomiej Szczygło Published: 2026-07-22 Format: Deep Dive Topics: agent-infrastructure, context-and-tooling Term introduced: dumb orchestrator, smart agent > Most multi-agent systems fail because the control flow itself is generated by a language model. Invert it: a deterministic orchestrator that only routes, retries and times out, with all judgement pushed to the leaves where it can be observed and replaced. There is a design that shows up in almost every multi-agent project, and it goes like this: a "manager" model receives the goal, decides which specialist agents to call, decides in what order, decides when the work is done, and hands back a result. It demos beautifully. It is also the single most common reason these systems become impossible to operate. ## What actually goes wrong **Failures compound instead of surfacing.** If each routing decision is 95% reliable, a six-step plan is a coin flip. Worse, a bad decision does not raise an error — it produces a plausible next step, so the system fails by drifting rather than by stopping. **Debugging becomes archaeology.** When the control flow lives in a model's output, there is no execution graph to inspect. You get a transcript. Reproducing a bug means reproducing a sampling outcome, and the answer to "why did it do that" is frequently "it felt like it". **Cost and latency go quadratic.** Every routing decision is a call, and each one carries the accumulated conversation. You pay tokens for the privilege of deciding what to do next, repeatedly, at the most expensive layer of the stack. **Retries stop being safe.** Deterministic orchestrators can retry a failed step because the step is identified. When the plan is regenerated on each attempt, a retry is a new plan — sometimes a completely different one, sometimes one that repeats a side effect you already committed. ## Invert it The pattern I keep coming back to is **dumb orchestrator, smart agent**. The orchestrator is ordinary software. It has no model in it. Its entire job is: - dispatch a task to a worker - enforce a timeout - retry with backoff on a known-transient failure - record state transitions - fan out and join when the graph says so - stop when a terminal state is reached That is a queue and a state machine. It can be read, tested and reasoned about. It behaves the same way on Tuesday as it did on Monday. The intelligence goes to the leaves. Inside a single task, an agent can be as clever as it likes: choose tools, iterate, revise, give up. That is where model judgement is genuinely irreplaceable — and it is bounded, observable and cheap to replace when a better model shows up. ## The line between them The useful test is: **would a wrong answer here corrupt state, or just produce a bad output?** Control-flow decisions corrupt state. Sending the same invoice twice, deleting the wrong branch, marking a task complete that is not — these are orchestration mistakes, and they are exactly the ones a language model should not be making unsupervised. Content decisions produce output. Which library to use, how to phrase a summary, whether a test failure is a real bug — these benefit from judgement, and a wrong answer is visible and correctable. Put the model on the second list, keep it off the first. ## But some plans really are dynamic They are, and this is where the pattern needs nuance rather than dogma. When the task graph genuinely cannot be known in advance, let a model **propose** a plan — then hand that plan to the deterministic orchestrator to execute. Planning becomes a bounded, inspectable step that produces data, instead of an invisible loop that produces side effects. You can log the plan, diff it against previous ones, validate it against a schema, and refuse to execute a step that is not in the allowed set. The rule is not "no model in planning". It is **no model inside the control loop**. ## What you get back Systems built this way are boring in the best sense. Failures land on a specific step instead of dissolving into a transcript. Retries are safe because the step is identified rather than regenerated. Cost drops because you stop paying a model to decide what a `for` loop already knows. Swapping the model behind one worker becomes a local change instead of a system-wide regression risk. None of this is novel — it is the same lesson distributed systems learned twenty years ago, arriving late to a new field wearing new vocabulary. The industry currently rewards putting the model everywhere. Production rewards the opposite: determinism where it is cheap, intelligence where it is irreplaceable.