advanced
14 min read
A2A Explained: How Agent-to-Agent Communication Differs from MCP
MCP answers "what can I call?" A2A answers "who can I hand this to?" Different questions, different failure modes.
Read tutorialpractice
Reliability and Recovery
Engineering mechanisms that make software and AI workflows handle failures, retries, resumption, cleanup, duplicate work, and error propagation predictably.
Tagged articles
Reliability and Recovery articles
45 articles in this tag.
advanced
17 min read
Agent Execution Environments: Filesystems, Shells, Browsers, Code, Databases, and Sandboxes
The demo worked because the agent was running on your laptop. It had your credentials, your network, your home directory, and your shell history. Move it…
Read tutorial
advanced
9 min read
The Agent Harness as an Operating System: Capabilities, State, Execution, and Lifecycle
The demo agent works. It reads the file, runs the test, patches the bug. Then it ships to a second tenant, the session runs for six hours, the process dies…
Read tutorial
advanced
11 min read
Agent Loop Pathologies: Repetition, Oscillation, Goal Drift, Plateaus, and Regression
An agent that has run forty steps and produced an answer no better than step three is not stuck. It is broken in a specific, nameable way.
Read tutorial
advanced
15 min read
Agent Observability and Artifacts: Traces, State Snapshots, Lineage, Cost, and Debugging
A user reports a wrong answer. You pull the trace. Every tool call succeeded, every span is green, latency is normal, and no error was logged anywhere. The…
Read tutorial
advanced
16 min read
Agent State and Persistence: Typed State, Checkpoints, Stores, Threads, and Artifacts
Kill a long agent mid-task and you learn which of those two things your system actually believes. If the process dies and the transcript survives but the…
Read tutorial
advanced
18 min read
Agent Workflow Topologies: Routers, Conditional Gates, Parallelism, Fan-Out, and Map-Reduce
A fan-out graph looks fast on the whiteboard. In production it stalls because one branch is slow, one branch fails silently, and the synthesis step merges…
Read tutorial
advanced
15 min read
The Anatomy of an Agent Loop: State, Action, Observation, Feedback, and Termination
The demo works. The agent books the flight, summarizes the repo, files the ticket. Then you point it at a task with one extra branch, and it calls the same…
Read tutorial
advanced
17 min read
Chroma Context-1: What Is a Self-Editing Search Agent?
Hop three. The context window is full. Half of it is plausible garbage — passages that matched the query, ranked well, and turned out to be useless. The…
Read tutorial
advanced
17 min read
Coding Agent Harnesses: Workspaces, Shells, Artifacts, Tests, and Long-Horizon State
The agent writes the function, re-reads it, decides it looks correct, and stops. The code is wrong. This is the most common failure in autonomous coding,…
Read tutorial
advanced
16 min read
Decomposition and Planning Prompts: Least-to-Most, Plan-and-Solve, and Self-Discover
The first sub-answer is wrong, and every later step inherits it with full confidence.
Read tutorial
advanced
12 min read
DeepSeek Harness Explained: What Does “Everything Is a Plugin” Mean?
“Everything is a plugin” is a claim about architecture, not a feature list. It says the agent loop itself is replaceable — and that is expensive to mean.
Read tutorial
advanced
13 min read
Designing a Workspace Operations Agent: Tools, State, Persistence, and Human Approval
The demo works. The agent reads the queue, drafts the update, and posts it. Then someone asks who approved the change, which run produced it, and how to…
Read tutorial
advanced
14 min read
Designing an Adaptive Knowledge Agent: Routing, Retrieval, Memory, and Context Assembly
The demo works on turn one. By turn five, the agent is answering a question you already answered, citing a document it retrieved three turns ago, and…
Read tutorial
advanced
9 min read
Designing Python Exception Boundaries: Custom Exceptions, Chaining, Re-Raising, and Exception Groups
A boundary that leaks sqlite3.OperationalError into your domain code has not failed loudly. It has failed quietly, by making every caller learn your…
Read tutorial
advanced
15 min read
Evaluator Design and Feedback Contracts: Graders, Judges, Disagreement, and Targeted Retry
An evaluator is only as useful as the set of distinct repair actions its output can select.
Read tutorial
advanced
14 min read
Fault Tolerance and Durable Execution: Retries, Resumption, Rollback, and Duplicate Prevention
A research agent runs for twenty minutes. It pays for dozens of model calls, executes a dozen tool calls, and commits three external side effects. Then the…
Read tutorial
intermediate
12 min read
Functions as Values in Python: Higher-Order Functions, Callbacks, and Callable Objects
Six months after the pipeline shipped, nobody can tell what the handler actually takes.
Read tutorial
advanced
13 min read
Human Feedback Loops: Approval, Grading, Editing, Escalation, and Resumption
The demo works. The agent reaches a consequential step, pauses, and asks a human to approve. The human clicks approve. Then the resumed run re-executes the…
Read tutorial
advanced
16 min read
Long-Running Agent Loops: Checkpoints, Budgets, Progress Signals, and Recovery
An agent processes 200 units of work over four hours, dies on unit 201, and restarts from zero. The crash is not the failure. The failure is that nothing…
Read tutorial
advanced
10 min read
MCP’s 2026 Stateless Redesign: What Changed and Why It Matters
The redesign did not delete state. It relocated it — and everything downstream follows from where it landed.
Read tutorial
advanced
14 min read
MCP for Agent Capabilities: Resources, Tools, Prompts, Discovery, and Runtime Context
A team wires up an MCP server, watches the tools appear in the client, and concludes the integration works. Then a tool gets renamed upstream, a resource…
Read tutorial
advanced
11 min read
OpenAI’s Evolving Agents SDK: Why Agent Workspaces, Sandboxes, and Durable State Matter
An agent that only works until the container restarts is not an agent. It is a demo with a heartbeat.
Read tutorial
advanced
13 min read
Operational Loop Patterns: Turn-Based, Goal-Based, Time-Based, and Proactive Agents
A goal-based agent that never terminates is not a model failure. It is a contract failure.
Read tutorial
advanced
18 min read
Planning and Replanning Loops: Plan-and-Execute, Progress State, and Plan Repair
An agent that plans beautifully and then executes a stale plan into a wall is not a planning failure. It is a state failure.
Read tutorial
advanced
15 min read
Planning and Scheduling Harnesses: Task Graphs, Dependencies, ReWOO, and LLMCompiler
A plan that reads like a clean numbered list can still deadlock at runtime, because the list never told the scheduler which steps actually depend on which…
Read tutorial
advanced
13 min read
Prompt Chaining: Designing Multi-Stage Cognitive Workflows
The demo works. You paste a gnarly task into one prompt—extract the entities, reconcile them against the schema, flag the contradictions, write the…
Read tutorial
intermediate
13 min read
Prompt Engineering as Interface Design: From Vague Requests to Explicit Behavior
The prompt worked on your laptop. In production, the same prompt returns a paragraph on Monday, a bulleted list on Tuesday, and a confident fabrication on…
Read tutorial
advanced
13 min read
Python `async`/`await` and the Event Loop: Coroutines, Tasks, Cancellation, and I/O
A coroutine that never runs is not a syntax error. It is an object you built and never drove.
Read tutorial
advanced
13 min read
Concurrency vs Parallelism in Python: Asyncio, Threads, Processes, and Interpreters
Someone adds concurrency to a slow service and the service gets slower. Throughput drops, latency climbs, and the incident channel fills with timeouts. The…
Read tutorial
intermediate
14 min read
Python Context Managers and the `with` Statement: Deterministic Resource Management
A leaked connection, a lock that never released, a transaction that committed on the wrong path. None of these are exotic failures. They are ordinary…
Read tutorial
intermediate
13 min read
Python Exceptions and Error Handling: `try`, `except`, `else`, `finally`, and `raise`
A try statement is not a safety net. It is a four-clause control-flow machine, and most production bugs come from putting logic in the wrong slot.
Read tutorial
intermediate
19 min read
Python Iterators, Generators, and Lazy Evaluation: How Iteration Really Works
A generator is not a lazy list. It is a suspended function with a position, and once you can see that position, most "why did my pipeline eat 8 GB" bugs…
Read tutorial
advanced
13 min read
Python Multiprocessing and Process Pools: When Separate Processes Are Worth It
One core is pinned at 100%. Fifteen sit idle. You wrap the loop in a pool, run it again, and the wall clock barely moves — or gets worse. Nothing about the…
Read tutorial
advanced
16 min read
ReAct Agent Loops: Tool Use Through Thought, Action, and Observation
An agent that acts without reasoning is not reasoning. It is guessing with a tool schema.
Read tutorial
advanced
15 min read
Role Isolation, Approvals, and Independent Validation in Agent Runtimes
That is the shape of the failure I keep seeing in agent systems that look governed and are not. A team writes a planner prompt, a coder prompt, a reviewer…
Read tutorial
advanced
13 min read
Search and Tree Loops: Branching, Backtracking, Beam Search, MCTS, and LATS
A linear agent loop is a commitment device. It commits to one trajectory, and every later step inherits the consequences of the earliest mistake.
Read tutorial
advanced
15 min read
Self-Harness: Can AI Agents Improve Their Own Harnesses?
The loop that improves the harness can also edit the evaluator, the permissions, and the rollback path. That is the whole problem in one sentence.
Read tutorial
advanced
16 min read
Structured Outputs as Data Contracts: JSON Schema, Validation, Repair, and Retry
The 2 a.m. page is almost always the same story. A pipeline that extracted clean JSON in the demo starts throwing parse errors in production, and the logs…
Read tutorial
advanced
13 min read
Thin vs Thick Agent Harnesses: How Much Scaffolding Does a Model Need?
Two teams run the same model on the same task. One ships reliably. The other drowns in tool-call latency, context bloat, and incidents nobody can explain…
Read tutorial
advanced
16 min read
Time-Based and Proactive Agent Loops: Scheduling, Events, Deduplication, and Idempotency
A scheduled agent that opens two pull requests for the same dependency bump is not a model problem. It is a delivery-semantics problem wearing a model…
Read tutorial
advanced
16 min read
Tool Engineering for AI Agents: Schemas, Descriptions, Contracts, Errors, and Idempotency
A failed agent run rarely looks like an interface problem. It looks like a model problem: wrong tool, malformed arguments, an error swallowed and retried…
Read tutorial
advanced
10 min read
What Is an Agent Harness? The Software Layer Around an LLM
Two teams ship the same model, the same prompt, the same tool list. One agent finishes a multi-step task. The other stalls, loops, and quietly corrupts its…
Read tutorial
advanced
14 min read
What Is Graph Engineering for AI Agents? From Prompt, Context, Harness, and Loop Engineering to System Intelligence
The demo works. Then you add a second worker, a shared artifact store, and a verifier, and the system starts producing results nobody can attribute,…
Read tutorial
advanced
12 min read
Workflow vs Agent: Deterministic Control and Model-Directed Decisions
The demo worked. Production didn't. Somewhere between the two, the system took a path nobody wrote down, and when it failed there was no artifact…
Read tutorial