Workflow vs Agent: Deterministic Control and Model-Directed Decisions

The demo worked. Production didn't. Somewhere between the two, the system took a path nobody wrote down, and when it failed there was no artifact explaining why. That gap is the real subject of the workflow-vs-agent question.

Most teams argue about labels. They ask whether their system "is an agent" or "is a workflow," then reach for a taxonomy that settles nothing. The label doesn't matter. The mechanism does. The only question that changes your architecture is this: who owns the next action at runtime?

Answer that, and the rest of the design follows. This article gives you a decision procedure built on four axes — uncertainty, policy, recovery, and testability — and a boundary that lets you stop choosing between the two extremes.

The Question Is Who Owns the Next Action

Strip away the marketing and there are exactly two ways control can flow through a multi-step system.

Deterministic orchestration. The code path is fixed at design time. A model may fill a step — classify this ticket, extract these fields, draft this reply — but it does not choose the successor step. The orchestrating code does, regardless of what the model returns.

Model-directed control. The model selects the next action from an available action space, and the runtime executes what it selects. The model's job is transition selection: given current state and observations, pick the next move.

That's the discriminator. Not loop count. Not tool use. Not whether an LLM is involved.

Both architectures can loop. Both can call tools. Both can run for twenty steps or one. A ReAct-style agent with a retry cap is still model-directed; a fixed pipeline that calls an LLM five times is still deterministic. If you find yourself arguing about whether something "counts" as an agent because it loops, you're arguing about the wrong property.

Treat this as a spectrum with named hybrid points rather than a binary. Most systems that survive production sit in the middle, and the interesting engineering is in choosing where.

Four Axes That Actually Decide It

Before comparing architectures, fix the basis for comparison. Feature checklists mislead because they reward capability over fit. These four axes reward fit.

Uncertainty. Is the correct next step knowable at design time, or does it depend on runtime evidence the designer cannot enumerate? If you can list the branches, you can write them. If the right move depends on what a log says, what a user meant, or what a tool returned in a shape you didn't anticipate, you cannot.

Policy. Which actions are permitted, who authorizes them, and can a permission be bypassed by a model output? A permission that lives in a prompt is a suggestion. A permission that lives in code is a constraint.

Recovery. When a step fails, is the correct response a fixed retry or compensation path, or a judgment call about what went wrong? Retry-with-backoff is deterministic. "Figure out why the payment failed and decide whether to refund or retry" is not.

Testability. Can you assert on the path taken, or only on the final output? Path-level assertions catch misroutes. Output-only assertions catch them after the damage.

Here is the rule I use, stated up front: autonomy is justified only where uncertainty is real and the cost of a wrong branch is bounded and recoverable. Everywhere else, determinism is cheaper, faster, and easier to defend.

Deterministic Orchestration: What You Buy and What You Pay

A fixed control path is not a limitation. It's a set of guarantees you get for free.

Path-level reproducibility. Same input class, same sequence. When something breaks, you can reproduce it, bisect it, and replay it. This is the property that makes debugging tractable at all.

Enforceable policy. Approval gates, idempotency keys, and retry semantics live in code. A model cannot talk its way past a guard it never gets to evaluate. If a step requires human sign-off, the transition simply doesn't fire until the sign-off exists.

Cheap evaluation. You can assert on transitions, not just outputs. A trace becomes a deterministic artifact you can diff against a known-good run.

You pay in brittleness. Inputs that fall outside the enumerated branches either fail loudly or — worse — get forced down a wrong path.

The failure mode to name: a workflow that silently misroutes an out-of-distribution input because no branch matched and the default was permissive. The system didn't crash. It did the wrong thing confidently, and nothing in the trace flagged it.

That default branch is where deterministic systems quietly rot. A permissive fallback turns "I don't know this input" into "I'll treat it like the common case," and the misroute compounds downstream.

Model-Directed Control: Where It Earns Its Cost

The model's real job in a model-directed system is narrow and specific: transition selection from a bounded action space. Everything else — memory, tool execution, state persistence — is harness work, not model work.

It earns its cost when the correct path genuinely depends on runtime evidence. Ambiguous inputs. Open-ended investigation. Variable-length tasks where the number of steps isn't knowable until you're inside them. A support agent diagnosing an unfamiliar failure, a research loop following a thread wherever it leads — these have no enumerable branch set, and forcing them into one produces the misroute failure above.

The failure modes appear the moment the model owns the transition:

• Unbounded loops. The model keeps selecting actions without converging.
• Premature termination. It declares success before the work is done.
• Action-space drift. It selects actions whose preconditions were never met.
• Confident precondition violations. It calls a tool that requires state the system isn't in, and the call fails or, worse, half-succeeds.

There's a subtler cost. When the procedure lives in a prompt, compliance is soft. Probabilistic selection does not guarantee a required step was performed. A model can skip a mandatory verification and still produce a fluent, plausible final answer. The output looks right; the process was violated.

And the economics are different. Cost and latency scale with decisions, not with work. Every transition is another inference you pay for and must observe. A deterministic pipeline that makes three model calls costs three calls. A model-directed loop that makes three calls on the easy path makes thirty on the hard one — and you won't know which until it runs.

The Boundary: Deterministic Spine, Bounded Reasoning Leaves

You don't have to pick an extreme. The architecture that resolves the tradeoff splits the system in two.

The spine owns process state, allowed transitions, approval boundaries, retry and timeout behavior, idempotency, and final action authority. It's a state machine, and it's deterministic.

The leaves do bounded reasoning: classify, extract, summarize, recommend, draft, compare, validate. Each leaf has a typed contract and a narrow tool set. The model reasons inside the leaf; it does not choose what happens after the leaf returns.

Three hybrids worth naming, because they ship:

Hybrid	Who controls the transition	When it fits
Agent-in-workflow	The spine; a node delegates to a bounded agent	Open-ended reasoning inside a known process
Workflow-as-tool	The model invokes a fixed sub-procedure	The model decides whether, code decides how
Plan-then-execute	The model drafts a plan; the runtime executes it	The path is knowable once, at plan time

Plan-then-execute has a sharp edge: if execution can revise the plan mid-flight, it crosses into model-directed territory. If it can't, you've got an AI-written workflow — deterministic at runtime, generated at design time.

The boundary test is simple. If a wrong decision here is unrecoverable or unauditable, the decision belongs in the spine, not the leaf. And keep the boundary absolute in code. A guard the model can argue past is not a boundary; it's a suggestion with extra steps.

One Transition, Traced End to End

Abstract rules are easy to nod at and hard to apply. So let's walk a single transition through the whole machine: a support system handling a refund request.

State. The spine holds order_id, amount, refund_attempts, and policy_limit. None of this lives in the prompt. The model never owns it.

Observation. The leaf receives a typed payload: the customer's message, the order record, and the prior attempt count. That's the entire visible surface. The model does not see the policy limit — it doesn't need to, because it isn't authorized to enforce it.

Candidate actions. The leaf's action space is bounded and typed: {approve_refund, request_more_info, escalate_to_human}. Three options, not an open tool registry. The model selects one and returns a structured object with a confidence field.

Policy gate. The spine intercepts the selection before anything executes. If the model chose approve_refund and amount > policy_limit, the gate rejects the transition and routes to escalate_to_human. The model's choice was a proposal, not a decision. This is the seam where most systems get it wrong — they let the model's output be the action.

Side effect. Only after the gate passes does the tool layer execute, with an idempotency key derived from order_id and refund_attempts. If the model retries, the key prevents a double refund.

Recovery. If the tool call fails, the spine increments refund_attempts. Under the retry budget, it re-enters the leaf with the failure appended to the observation. Over budget, it escalates. The model never decides how many times to retry — that's a spine invariant.

Trace. Every hop writes a durable record: the state the model saw, the action space it chose from, the selection it returned, the gate's verdict, and the tool outcome. Four fields, and a bad branch becomes a diff instead of archaeology.

Now map the four axes onto this one transition:

Axis	What it decided here
Uncertainty	The customer's intent is ambiguous, so the selection among three actions is a leaf job
Policy	The refund limit is a spine guard the model cannot see or bypass
Recovery	Retry budget and escalation are spine invariants, not model choices
Testability	The trace asserts on the gate verdict and the idempotency key, not just the final message

Notice what stayed deterministic: the limit, the retry budget, the escalation path, the idempotency key. Notice what went to the leaf: the judgment call about which of three actions fits an ambiguous message. That split is the whole architecture, and it's per-decision — not per-system.

The candidate action is structured and reaches the policy gate before any refund tool executes.

Recovery, Authorization, and Observability at the Seam

Most designs fail at the seam, not in either half. The spine and the leaves each work fine in isolation; the machinery that connects them is where production breaks.

Recovery differs by owner. The spine gets deterministic retries and compensations — fixed paths for known failures. Model-directed steps get bounded retry budgets, escalation rules, and a human handoff when the budget exhausts. A model-directed step with no escalation path is a step that fails silently at 3 a.m.

Authorization must be enforced outside the model. The model proposes an action. A policy layer decides whether it's permitted. The tool layer executes only what policy approved. Three separate stages, and the model only touches the first. This is the difference between "the agent decided to issue a refund" and "the agent requested a refund, policy checked the amount against the limit, and the tool executed it."

Idempotency and side-effect ordering matter more, not less, when the model chooses the sequence. A deterministic pipeline has one ordering to reason about. A model-directed loop has as many orderings as the model can produce. Every side-effecting tool call needs an idempotency key, because the model may retry it.

Observability is non-negotiable. Record the decision, the state the model saw, the action space it chose from, and the outcome. Without all four, a bad branch is archaeology — you're reconstructing intent from a final answer, and you'll guess wrong. Durable artifacts — traces, state snapshots, approval records — are what make a model-directed path reviewable after the fact.

A Decision Procedure You Can Run on a Real System

Here's the procedure, in order. Run it on one system this week.

Step 1: Enumerate the transitions. If you can list them, the spine owns them. Write them down. The act of writing exposes the ones you can't list.

Step 2: For each transition, ask whether the correct choice depends on runtime evidence you cannot enumerate. That — and only that — is where autonomy is justified. Everything else is a branch you can write.

Step 3: For each candidate autonomous step, check recoverability and auditability. Unrecoverable plus unauditable means it stays deterministic, no matter how well the model performs in testing.

Step 4: Define the action space and the typed contract for each leaf before writing prompts. The contract is the interface. If you can't type it, you can't test it, and you can't bound it.

Step 5: Instrument the seam first, then widen autonomy only where traces show the model's choices are consistently correct. Widen on evidence, not on optimism.

And the case for not using model-directed control: high-volume, low-variance tasks. If the input distribution is stable and the correct path is known, a fixed path is cheaper, faster, and easier to test. Autonomy there buys you nothing and costs you reproducibility.

The Decision Rule

Put the process in code. Put the reasoning in the model. Put the permission in a layer the model cannot reach.

That's the whole architecture in one sentence, and it resolves the workflow-vs-agent question by refusing to answer it at the system level. The answer is per-decision. Some decisions belong to the spine; some belong to a bounded leaf; the boundary between them is the engineering.

Your next action: pick a single transition in a system you already run. Write down its allowed successors and its recovery path. If you can write both, move it to the spine. If you can't, you've found a real candidate for model-directed control — and you now know exactly what to instrument before you trust it.

The adjacent question is how the harness enforces that boundary at runtime — how it gates transitions, records decisions, and keeps a model's proposal from becoming an executed action without passing through policy. That's where the boundary stops being a diagram and starts being a constraint.