Agent Workflow Topologies: Routers, Conditional Gates, Parallelism, Fan-Out, and Map-Reduce

A fan-out graph looks fast on the whiteboard. In production it stalls because one branch is slow, one branch fails silently, and the synthesis step merges the wreckage into confident prose. The shape was never the problem. The missing dependency contract was.

I have watched this failure repeat across enough agent systems that I now treat it as the default outcome of topology-by-imitation. Someone copies a diamond from a framework sample, wires four agents together, ships it, and then spends a week debugging a graph that was never matched to the task. The drawing was fine. The contract underneath it was absent.

Topology is a dependency contract. It decides where latency accumulates, where failure stops or spreads, and where aggregation error gets baked into the final answer. Get the contract right and the shape almost picks itself. Get it wrong and no amount of prompt tuning will save the run.

This article assumes you already know what an agent harness is: the runtime layer that supplies state, tools, execution, and observability around a model. It assumes you can reason about typed state and checkpoints. What it does not assume is that you have a deliberate method for choosing how stages connect. That is the gap we close here.

Topology Is a Dependency Contract, Not a Diagram

Two questions get conflated constantly, and separating them is the first move.

The first is task structure: what actually depends on what. Does the budget analysis need the destination research before it can start? Does the summary need every branch, or just a quorum? These are facts about the problem, not preferences about the architecture.

The second is control authority: who decides what runs next. A static scheduler, a rule, or a model. These are facts about your system's design.

A topology is the mapping between the two. When the mapping is faithful, the graph is boring and predictable. When it is not, you get the production symptom from the opening: a graph that looks parallel but behaves like a race, or a pipeline that looks sequential but hides an undeclared dependency.

I judge every topology on four axes:

• Dependency fidelity. Does the graph's edge structure match the task's real dependency structure? Extra edges waste latency. Missing edges produce garbage downstream.
• Latency profile. Where does wall-clock time actually go? The critical path, not the average branch, sets the clock.
• Failure containment. When a node fails, does the failure stop at an edge or propagate to the final answer?
• Aggregation determinism. Given the same branch outputs, does the merge step produce the same result? If not, what varies?

One more distinction matters before we compare shapes: static graph shape is not runtime scheduling. A diamond drawn on paper is still a scheduler decision about when branches actually start. Two systems with identical diagrams can have completely different latency and failure behavior depending on whether the scheduler launches branches eagerly, waits for a shared input, or serializes on a lock. The diagram is the contract's sketch. The scheduler is the contract's enforcement.

The shape of the graph must match the dependency structure of the task. Every edge is a place where a failure can either be contained or propagated. Choose which, deliberately, before you write orchestration code.

The failure mode of topology-by-imitation is that it copies the sketch and ignores the enforcement. A framework sample that fans out cleanly in a demo may share mutable state under load, or may assume branches are independent when your task's branches both write to the same artifact. The sample is not wrong. It is answering a different dependency question than yours.

Routers and Conditional Gates: Different Jobs, Similar Syntax

Routers and gates get treated as the same thing because both evaluate a condition and pick a path. They are not the same. The distinction determines what you log, what you can replay, and where a wrong decision becomes invisible.

A router selects among alternative capabilities or subgraphs based on request classification. It answers: which specialist should handle this? The router reads the request, picks a label, and dispatches. Everything downstream is a specialist that never sees the other branches. The router's job ends at the dispatch.

A gate evaluates run state at a control point and decides whether an already-defined path proceeds, branches, retries, or stops. It answers: given where we are, what should happen next? The gate operates inside a path, not at the entry. It reads typed state — iteration count, validation status, a quality score — and chooses among continue, branch, retry, or abort.

The same conditional expression can be implemented as either. Consider a check on whether a retrieved document set is sufficient. As a router, you classify the request as "needs-more-retrieval" or "proceed-to-answer" and dispatch to different subgraphs. As a gate, you evaluate the retrieval result inside a single path and decide whether to loop back for another retrieval pass or continue to synthesis. The router version is easier to evaluate in isolation because the decision is a discrete classification with a labeled set. The gate version is easier to bound because the loop budget lives in typed state next to the decision. Choose based on whether the condition is a request property (router) or a run-state property (gate).

Router Failure Modes

A router is cheap because it usually runs one model call or one rule evaluation. It is failure-prone because a single wrong label sends the entire request down the wrong path, and the wrong path does not know it is wrong.

Distinguish two kinds of routing:

Static routing uses rules, schema, keywords, or a classifier you trained. It is inspectable, cheap, and deterministic. Use it when request types are genuinely separable by surface features and you can enumerate them.

Model-directed routing asks a model to pick the branch. It handles fuzzier boundaries and novel request types. It is also non-deterministic, harder to evaluate, and prone to confident misclassification on inputs near a boundary.

The cost of a misroute is not a crash. It is plausible output from the wrong specialist. A request that should have gone to the refund path lands in the technical-support path, and the support agent produces a fluent, well-structured answer to the wrong question. That is harder to detect than an error, because nothing in the output signals the mistake. The signal lives in the routing decision, which means you have to log it.

Use a router when request types are genuinely separable and the cost of a wrong branch is bounded and detectable. If a misroute produces output you cannot distinguish from a correct one, you do not have a router. You have a coin flip with good grammar.

Gate Failure Modes

The most important split for gates is between deterministic and model-judged predicates.

Deterministic predicates read typed state: schema validation passed, a tool returned a specific status code, a counter exceeded a threshold, a required field is present. These are inspectable after the run. You can replay the decision from the trace.

Model-judged predicates ask a model whether some condition holds: is this answer good enough, is the task complete, does this output contradict the input. These are necessary when the condition is genuinely semantic. They are also where loops go to die.

Loop termination is the classic bug. A model-judged gate that asks "is this good enough?" will, under the right conditions, answer "no" forever. The model has no intrinsic sense of budget. It has no memory of the last four attempts unless you put them in state. It will happily request another revision until your token spend hits a wall you did not build.

The fix is not a better prompt. The fix is a guard condition evaluated against typed state: a maximum iteration count, a budget ceiling, a no-progress detector that compares the current output to the previous one. The model judges quality. The harness judges whether to keep going.

# Illustrative gate predicate. RunState and GateAction are your types.
def should_continue(state: RunState) -> GateAction:
    if state.iteration >= state.max_iterations:
        return GateAction.ABORT          # budget guard, deterministic
    if state.last_score is None:
        return GateAction.CONTINUE       # no signal yet
    if state.last_score >= state.threshold:
        return GateAction.CONTINUE       # quality met
    if state.last_score <= state.prev_score:
        return GateAction.ABORT          # no progress, stop burning budget
    return GateAction.RETRY

The second failure mode is subtler: gates that swallow errors. A gate that routes every exception to a generic handle_failure branch feels robust. In practice it hides the real fault. The failure branch produces a plausible fallback, the run completes, and the actual bug — a malformed tool response, a schema drift, a timeout — never surfaces in the trace. Route failures to specific branches that name the fault, or let them propagate. A generic failure branch is where debugging goes to disappear.

Parallelism, Fan-Out, and Fan-In

Fan-out is only valid when branches are genuinely independent. This is the constraint people skip, and it is the one that breaks systems.

If two branches share mutable state, they are not parallel. They are sequential with extra steps and a race condition. Shared state includes the obvious (a database row both branches update) and the subtle (a context object both branches append to, a cache both branches populate, a file both branches write). Before you fan out, ask what each branch reads and what each branch writes. If the write sets overlap, you have a dependency you did not draw.

Fan-in semantics matter more than fan-out, and they are usually left implicit. Four options, four different failure contracts:

The slowest branch sets the wall clock. This is the anti-pattern that eats the latency win: one branch takes 30 seconds while the others take 5, and the join waits for the slow one. Parallelism bought you nothing on the critical path. Timeouts and per-branch budgets are part of the topology, not an afterthought bolted on after the first slow run.

Partial failure policy must be declared before you write the join. Three options:

• Fail the whole graph. Correct when every branch is required for a valid answer.
• Degrade with missing inputs. Correct when the aggregation can reason about absence explicitly.
• Retry the branch in isolation. Correct when the failure is transient and the branch is idempotent.

The cost profile is the part that surprises people. Parallel branches multiply token and tool spend. Fan-out is a latency-for-cost trade, not a free speedup. If your task is not latency-sensitive, a sequential pipeline may be cheaper and easier to debug for the same result.

Map-Reduce: Scaling Breadth Without Losing the Thread

Map-reduce is the specific case of fan-out where the aggregation step is the hard part. The map phase is the easy half. The reduce phase is where the design lives or dies.

The map phase partitions the input space so each unit is independently processable. The partition boundary has to be meaningful to the task. Splitting a document by character count cuts sentences and destroys local context. Splitting by section preserves it. The partition is a modeling decision, not a mechanical one.

The reduce phase is conflict resolution and compression, not concatenation. An aggregator told to "combine everything" produces averaged mush: fluent, confident, and wrong in ways no single branch would have been. The aggregator needs explicit rules:

• Priority. When two branches disagree, which wins? By source authority, by recency, by confidence score?
• Deduplication. When three branches report the same finding, it appears once.
• Tie-breaking. When priority is equal, what decides? A deterministic rule, not a model's mood.

The real constraint is context budget. The reduce step has to fit many branch outputs into a bounded window. This forces summarization or hierarchical reduction. Single-shot reduce works when branch outputs are small and few. Hierarchical reduce — reduce in layers, then reduce the layers — earns its extra latency when the branch count exceeds what one window can hold. The extra layer is a cost, and it is worth paying only when the alternative is truncation.

Synthesis without criteria produces confident, averaged mush. It reads well. It is wrong in ways no single branch would have been. Specify priority, deduplication, and tie-breaking before the reduce step runs, not after you read its output.

Choosing a Topology: A Decision Matrix

The four axes from the opening — dependency fidelity, latency profile, failure containment, and aggregation determinism — are only useful if you apply them across topologies. Here is the compact comparison.

Run this against a real task. It takes ten minutes and saves a week.

Step one: write the dependency graph of the task itself. Before choosing any agent shape, draw the real dependencies. What must happen before what? What can happen in any order? What must happen together? This graph is the ground truth. Every topology you consider is a candidate approximation of it.

Step two: check independence. For every pair of nodes you want to parallelize, ask what they read and what they write. If the write sets overlap, they are not parallel. Draw the edge you missed.

Step three: declare the failure contract per edge. For each edge, decide: propagate, contain, retry, or degrade. Write it down. This is the contract. If you cannot decide, you do not yet understand the dependency.

Step four: declare the aggregation contract. Who merges? By what criteria? With what budget? If the answer is "the model figures it out," you have not declared a contract. You have deferred a decision to the least inspectable component in the system.

When not to use a complex topology. A single well-tooled agent or a linear pipeline often beats a graph. If the task has no real branching and no genuine independence, a graph adds coordination cost, failure surface, and debugging difficulty for nothing. The simplest topology that honors the dependency structure is the right one. Complexity is a cost you pay for a capability you need, not a default.

Hybrids are normal. Router at the entry, fan-out in the middle, evaluator loop at the end. This is a reasonable shape for a research-and-synthesize task. But each addition must be justified by a dependency, not by symmetry. A graph that looks balanced is not thereby correct.

Observability and Recovery Across Topologies

Topology choice determines what you can observe and how you can recover. Each shape needs a different trace.

Routers need the decision and its confidence. Log the label, the input features that drove it, and the confidence score. Without this, a misroute is invisible.

Fan-out needs per-branch status and timing. Which branches started, when, how long each took, which returned, which failed. The critical path is only visible if you record per-branch timing.

Reduce needs the inputs and the merge rule applied. Log what each branch contributed and which rule resolved each conflict. When the final answer is wrong, you need to know whether the branches were wrong or the merge was.

Checkpoint granularity follows topology. A fan-out graph needs per-branch checkpoints so a single failed branch can resume without rerunning the whole map. A pipeline needs per-stage checkpoints. A router needs the decision checkpointed so a retry does not re-classify.

Idempotency requirements rise with parallelism. Retried branches must not double-apply side effects. If a branch sends an email, charges a card, or writes a row, the retry policy has to account for the first attempt having partially succeeded. This is a tool-design problem, but topology determines how often you hit it.

A Minimal Verification Trace

Metrics are inventory until you connect them to a decision. Here is one fan-out/map-reduce run recorded so that a failure can be attributed to routing, execution, or aggregation.

{
  "topology_version": "fanout-v3",
  "run_id": "r-8842",
  "branches": [
    {"id": "b1", "input_hash": "a91f", "status": "ok", "start_ms": 0, "end_ms": 4200},
    {"id": "b2", "input_hash": "c33d", "status": "ok", "start_ms": 0, "end_ms": 5100},
    {"id": "b3", "input_hash": "e07a", "status": "timeout", "start_ms": 0, "end_ms": 30000}
  ],
  "join_policy": "best_effort",
  "reduce_rule": "priority_by_source_then_recency",
  "reduce_inputs": ["b1", "b2"],
  "final_status": "degraded",
  "attribution": "execution"
}

Read the trace against the four axes. Dependency fidelity: did every branch that should have run actually run? Latency profile: which branch set the critical path? Failure containment: did the join policy absorb b3's timeout or propagate it? Aggregation determinism: given b1 and b2, does the reduce rule produce the same output on replay?

The attribution field is the point. A wrong final answer could come from a misroute (routing), a branch that returned bad output (execution), or a reduce rule that resolved a conflict incorrectly (aggregation). Without the trace, you debug the final answer. With it, you debug the specific stage.

Evaluation signal per topology:

• Routing accuracy. Fraction of requests dispatched to the correct branch, measured against a labeled set. Informs whether the router needs retraining or a rule change.
• Branch success rate. Fraction of branches that complete without error, per branch type. Informs whether a specific branch needs a timeout, a retry, or a redesign.
• Aggregation fidelity. Does the reduce output match what a careful human would produce from the same branch outputs? Informs whether the reduce rule is adequate or needs another criterion.
• End-to-end latency distribution. Not the mean. The tail. The p95 is where the slow branch lives. Informs whether the join policy needs a timeout or the slow branch needs to be split.

Contain failure at the narrowest edge that still lets the graph produce a useful result. A branch failure that degrades one section is cheaper than a graph failure that produces nothing.

The Contract Comes First

Draw the task's dependency graph first. Then pick the smallest topology that honors it. Then declare the failure and aggregation contracts before writing orchestration code.

That order matters because it inverts the usual workflow. Most teams pick a shape, wire it up, and discover the dependency structure by debugging production. The dependency structure was knowable from the start. It was just never written down.

Take one existing multi-stage agent workflow you own. Write down its real dependency edges — what actually depends on what, not what the diagram says. Then compare that list to the implemented shape. Every mismatch is either wasted latency (an edge that does not exist in the task but exists in the graph) or a hidden failure path (an edge that exists in the task but not in the graph). Fix the mismatches before you add another agent. The graph you have is probably already more complex than the task requires.