What Is an Agent Harness? The Software Layer Around an LLM

Two teams ship the same model, the same prompt, the same tool list. One agent finishes a multi-step task. The other stalls, loops, and quietly corrupts its own state. The model is not the variable. The harness is.

For chat and single-tool calls, "the agent is the model plus a while loop" holds up fine. That loop tracks messages, appends the user turn, calls the model, and stops when the model stops asking for tools. It breaks the moment a task needs durable state, enforced policy, deterministic verification, or recovery from partial failure. The model proposes judgment: decomposition, action selection, interpretation of feedback, and when to stop. The harness enforces what actually happens. Everything that must be true for judgment to become work lives in that layer.

The Boundary: Model, Harness, Framework, Application

The working definition I use: the harness is every piece of code, configuration, and execution logic that is not the model. If you are not the model, you are the harness. A raw model is not an agent; it becomes one when a runtime gives it state, tool execution, feedback loops, and enforceable constraints.

That definition is deliberately broad, so it needs three boundaries.

Harness vs. framework. A framework supplies building blocks you assemble: message types, tool abstractions, graph nodes, provider clients. A harness is the assembled, opinionated runtime that already drives model and tool calls, manages context, applies approvals, and keeps a multi-step task progressing. You can build a harness on a framework, or hand-roll one directly on a provider SDK. The distinction is assembly and opinion, not dependency count.

Harness vs. application. The application owns UX: streaming, progress display, human input collection, the approval dialog itself. The harness owns execution semantics underneath it. When a tool call needs a human decision, the harness decides what runs and the application collects the decision.

Agent vs. harness. The word "agent" gets used for both the whole working system and the model doing the reasoning. In this article, "agent" means the whole system and "model" means the reasoning component. The harness is the layer between them.

The invariant worth defending: the model proposes the next action and interprets results; the harness validates, limits, records, and can override the transition. Behavior you cannot express as a harness mechanism will not reliably happen.

What the Model Cannot Do on Its Own

Rather than memorize a component taxonomy, derive it. Start from the model's structural gaps and work backward to the harness feature that closes each one.

The model has no memory between calls. It has no ability to execute the code it writes. It keeps no record of what it tried five steps ago. It has no mechanism to check whether it succeeded. Each gap maps to a harness responsibility.

Without weight updates, the only way to add knowledge is context injection. That single fact makes context assembly, compaction, and retrieval harness concerns, not model concerns. Knowledge cutoffs mean current facts — library versions, API changes, live system state — must arrive through search or tool calls that the harness provides and routes.

The derivation rule: work backwards from the desired agent behavior to the harness feature that makes it possible. If you want the agent to remember a project convention across sessions, you need a memory file that gets injected at start and reloaded when the agent edits it. If you want it to stop after three failed attempts, you need an iteration limit in the loop. Teams that skip this step add tools and prompts reactively, then cannot explain why the agent behaves differently across runs.

The Core Components of a Harness

Here is the component model, mapped to the gap it closes and the symptom you see when it is missing.

Component	Model gap it closes	Symptom when absent
Capabilities: tools, skills, MCP integrations, and their descriptions	No ability to act on the world	Agent describes actions instead of taking them
Bundled infrastructure: filesystem, sandbox, shell, browser	No execution environment	Agent cannot inspect or modify real artifacts
Orchestration: driving loop, iteration limits, subagent spawning, handoffs, model routing	No control flow across steps	Infinite loops, or one context window doing everything
State and persistence: session instructions, per-call history, todo state, memory files	No memory between calls	Agent repeats work, forgets decisions
Context management: compaction, token budgets, progressive disclosure	Fixed context window	Sessions degrade or hard-fail near the limit
Deterministic middleware: hooks for compaction, continuation, lint, formatting	No guaranteed side effects	Correctness depends on the model choosing to comply

Two of these deserve emphasis.

Tool descriptions are harness surface. They shape action selection as much as the tool implementation does. A tool with a vague description is a harness bug, not a model bug.

Deterministic middleware runs outside the model. Compaction, continuation, lint checks, and formatting should not depend on the model deciding to do them. If a step must happen every iteration, put it in a hook.

Parallel subtasks need isolated context. That is an orchestration constraint, not a prompting trick. If two subagents share a context window, they will contaminate each other's working state.

One Run, End to End

The component table tells you what exists. A trace tells you how it moves. Here is a framework-neutral walkthrough of a single task: "fix the failing test in auth/session.py."

state = { goal, history: [], artifacts: {}, attempts: 0 }

loop:
  proposal = model(context=assemble(state))
  # proposal: { action: "run_tests", args: { path: "auth/session.py" } }

  decision = policy(proposal)          # allow | require_approval | deny
  if decision == deny: halt(reason)
  if decision == require_approval: await human; record(decision)

  result = execute(proposal)           # sandboxed, bounded, timed
  persist(result)                      # append to history + artifacts
  signal = validate(result)            # tests, linter, type checker

  if signal == pass: halt(success)
  if signal == validation_failure and attempts < 3:
      attempts += 1; continue          # bounded repair
  if signal == policy_violation: halt(reason)
  if signal == timeout and idempotent(proposal): retry
  if signal == stale_state: refresh(); continue
  halt(reason)

Three things are load-bearing here. First, the model proposes; the harness disposes. Second, every side effect is persisted and associated with the action that caused it — the invariant is that no side effect is considered complete until its result is recorded and linked to its cause. Third, the loop terminates on a named condition, not on the model's mood.

signal == validation_failure and attempts < 3

Execution, Authorization, and the Permission Boundary

Shell and filesystem access is where a harness stops being a wrapper and becomes an operating concern. Model the capability explicitly rather than granting ambient access.

Approval flows come in three shapes: standing approvals for a class of commands, auto-approval rules for known-safe patterns, and per-call approval for everything else. The harness decides what runs; the application collects the human decision.

For local shell execution, run the logic in an isolated environment and keep explicit approval in place before commands are allowed to run. This is the recommendation from teams shipping these patterns in production, and it is the right default.

Permission tiers should be harness-level policy: what the agent can read, write, execute, and reach over the network. Least privilege is a code constraint, not a prompt request.

Prompt-level guardrails are advisory. If a constraint must hold, it belongs in code that can refuse the action. A model that is told not to delete files will eventually delete a file.

Feedback Loops and Verification

The harness acts as a control layer. It observes the effects of agent actions and regulates subsequent state transitions rather than merely forwarding error text to the model.

Deterministic sensors do the observing: linters, parsers, compilers, type checkers, unit and integration tests, static analyzers, runtime monitors, CI pipelines. These turn a trajectory into inspectable signals — pass/fail outcomes, diagnostics, failing traces, coverage gaps, policy violations.

The harness then chooses among real branches. The signal type determines the response, and each response persists different evidence:

Signal	Harness response	Evidence to persist
Validation failure	Bounded repair (retry with feedback)	Failing test, diff, attempt count
Policy violation	Halt or escalate to approval	Rule triggered, proposed action
Timeout	Retry only if the action is idempotent	Duration, retry count, idempotency key
Stale state	Refresh context, replan	External change detected, new snapshot
Partial side effect	Compensate or roll back	Written artifacts, rollback result
Silent wrong objective	Halt; requires human review	Goal drift signal, last known-good state

Name these as explicit branches in code, not vibes in a prompt. A validation failure and a policy violation should not trigger the same recovery path.

This matters more than model choice for long-horizon work. The bottleneck of autonomy is often the reliability of the system connecting model output to long-horizon actions and persistent state, not the reasoning ability of the base model. That claim comes from vendor and independent research on long-horizon tasks, including work showing that harness design can dominate model choice for both task completion and cost. Treat it as a strong signal, not settled consensus — the research is recent and the benchmarks are still contested.

Observability, Recovery, and Durable Artifacts

Per-service-call history persistence is what makes a run reconstructable. Without it, debugging a multi-step failure becomes archaeology.

Capture model calls, tool invocations and their outputs, approval decisions, compaction events, state transitions, and the sensor results that drove each branch. Recovery patterns follow: resumable sessions, bounded retry with backoff, checkpointing durable state, and idempotency for side-effecting tools so a retry does not double-charge or double-write.

Durable artifacts — files, patches, reports, memory files — outlive the session and become the agent's continuity across runs.

Plan for this failure taxonomy:

• Context overflow
• Tool-call loops
• Stale state after external change
• Partial writes
• Approval deadlock
• Silent success on a wrong objective

If you cannot replay the run from persisted artifacts, you do not have observability — you have logs.

When a Harness Is Overkill

Single-turn generation, classification, extraction, and summarization need a call, not a runtime. Adding a loop adds latency, cost, and failure surface with no benefit.

Fixed, known-step workflows are better served by an explicit pipeline or state machine than by an agent that decides its own next step. Determinism you can write down should not be delegated to a model.

The threshold for a harness: the task requires multiple dependent steps, the next step depends on observed results, the work must survive process restarts, or the agent needs capabilities the model cannot have natively.

Cost and latency are harness-level levers. Iteration limits, compaction strategy, model routing, and tool granularity change the bill more than prompt wording does.

Where Reliability Is Won

The harness is where reliability is won or lost. Not the model, not the prompt. The runtime layer around the model.

Pick one failed or expensive task your system already handles. Replay its last execution and mark every transition: model proposal, policy gate, side effect, persisted state update, validator signal, recovery branch. Then find the earliest transition that is unbounded or unobservable — no iteration limit, no persisted result, no named branch, no recorded decision. That gap is your next engineering task, and it is almost always cheaper to fix than swapping models.

The adjacent concept to study next is harness engineering as a discipline — designing these components deliberately instead of accreting them one debugging session at a time.