Reflection vs Reflexion: Improving the Current Attempt vs Learning from Prior Attempts

A team adds a "reflect" step to their agent. Scores tick up a few points. Encouraged, they bolt on a memory buffer so the agent can "remember its mistakes." Sometimes the numbers climb. Sometimes the agent gets worse with every trial, repeating a lesson it wrote for itself three attempts ago. Same word on the whiteboard — reflection — two completely different machines underneath.

That conflation is the expensive part. Reflection and Reflexion are not two flavors of the same technique. They operate on different objects, at different loop scopes, with different memory contracts. Pick the wrong one and you will spend a week tuning a mechanism that cannot produce the gain you are chasing.

Here is the invariant I want you to hold for the rest of this article:

Reflection mutates the current artifact. Reflexion mutates the next attempt's context. If nothing persists after the attempt ends, no amount of critique quality produces cross-trial learning.

Everything else follows from that.

Two Independent Axes, Not One

Most comparisons start with a feature list. That is backwards. Start with two questions that vary independently, because treating them as one axis is what produces the naming collision in the first place:

1. What does the critique change — the artifact in front of you, or the context of the next attempt?
2. What grounds the critique — the model's own judgment, or an external signal?

The first question is loop scope. Within-episode revision keeps everything inside one attempt: generate, critique, revise, ship. Across-episode verbal reinforcement ends the attempt, writes a lesson, and re-injects that lesson when the next attempt begins. The second question is grounding. A critique anchored to a test result, an environment reward, or a scorer is a different animal from a critique anchored to the model's opinion of itself.

Cross these two axes and the naming collision becomes obvious. A team ships a memory buffer with no evaluator and wonders why the stored lessons are confident nonsense. Another team builds a tight evaluator but never persists anything, then wonders why trial two looks exactly like trial one. They each expected the other mechanism's gains because both were labeled "reflection."

The canonical patterns live in two cells. The other two are real designs, not strawmen. A within-attempt loop can be gated by a deterministic test or a scorer, and a cross-trial system can persist model-generated lessons that were never checked against an external signal. The mechanism is not the label. It is where state lives and what is allowed to change it.

Reflection: Revising the Artifact in Front of You

Reflection is a within-trajectory loop. The shape is generator → critic → reviser, and the revised artifact replaces the previous one inside the same attempt. The loop closes on the artifact.

The reason a separate critic pass helps at all is that a model continuing its own draft tends to continue its own framing. Ask it to critique against explicit criteria — in a different role, sometimes a different prompt — and it can surface issues its own continuation would have glossed over. This is the Self-Refine family: same model wearing three hats, loop closed inside one episode.

The grounding problem is the whole story. When the critique has no external signal, the loop optimizes for plausibility. It converges on confident rewording. The prose gets tighter, the structure gets cleaner, and the answer stays wrong — because nothing in the loop can tell right from wrong, only fluent from less fluent.

That asymmetry explains where reflection pays off. It works when verification is easier than generation: rewriting, formatting, constrained edits, summarization against a rubric. It flattens on tasks where the model genuinely cannot distinguish a correct answer from a well-argued incorrect one.

Bounded iteration is not a nicety here. It is a hard requirement. Without a stopping rule and a measurable criterion, a revision loop burns tokens and drifts — each pass slightly more confident, no closer to correct. My rule: if you cannot state the criterion the reviser is checking against, you do not have a reflection loop. You have a paraphrase loop with a budget.

Reflexion: Turning Failure Into Context for the Next Trial

Reflexion is a cross-trial loop, and it is best understood as a state and control-flow contract rather than a slogan. Three roles:

• Actor produces a trajectory by interacting with the environment.
• Evaluator produces a scalar or binary signal — a score, a pass/fail.
• Self-Reflection converts that signal plus the trajectory into a short verbal summary.

That summary is appended to a memory buffer and re-injected as context on the next episode. The improvement is carried by text, not by weight updates. This is the core claim of the Reflexion paper: convert binary or scalar feedback from the environment into verbal feedback, and let that text act as a semantic gradient — a concrete direction to move in, derived from a reward that would otherwise be unusable by an LLM. Keep the metaphor honest: it is an analogy for directional feedback, not a literal gradient and not a parameter update.

The memory buffer is the entire point. Strip it out and you have reflection with extra steps. Keep it and you have something closer to trial-and-error learning: attempt, fail, write down why, try again with the note in hand.

The termination contract has two conditions, and both must exist: loop until the Evaluator passes the trajectory, or until a max-trial budget is hit. Missing the first means you never stop on success. Missing the second means you never stop at all.

The Evaluator is load-bearing. A noisy evaluator, or one that grades its own homework, turns the memory buffer into a store of confident wrong lessons — and because those lessons get re-injected every trial, the error compounds instead of washing out. If you take one thing from this section: Reflexion is only as good as the signal feeding it. The reflection text is downstream of the score.

The State Contract: Produced, Stored, Consumed

The cleanest way to keep these mechanisms straight is to stop asking "what persists" and start asking three separate questions:

• Produced: what did this attempt generate? (artifact, trajectory, evaluator output, lesson text)
• Stored: what did the system write down? (nothing, a log, an episodic buffer, a database row)
• Consumed: what does the next decision actually read?

Only the third one changes behavior. An implementation can persist artifacts, evaluator traces, and telemetry without any of it feeding the next attempt — that is storage, not learning. And a Reflexion-style buffer can be scoped to a single task, a session, or a longer-lived store depending on the design; the invariant is that prior state is read by the next decision, not that bytes survive somewhere.

Here is the transition trace for each loop, with the boundaries labeled:

Within-attempt reflection
  task state ──▶ generate ──▶ artifact
       ▲                         │
       │                    critique(criteria)
       │                         ▼
       └──── revise ◀──── critique text
  terminate: criterion met OR max_iters
  consumed by next decision: revised artifact (same attempt)

Cross-trial Reflexion
  task state ──▶ attempt(memory) ──▶ trajectory
                                          │
                                     evaluator
                                          ▼
                                    score / pass
                                          │
                                   self-reflection
                                          ▼
                                       lesson ──▶ memory
  terminate: evaluator passes OR max_trials
  consumed by next decision: memory (next attempt)

Read the two traces side by side and the difference is mechanical. The reflection loop feeds the critique back into the artifact. The Reflexion loop feeds the lesson forward into memory. Different edge, different object, different scope.

Side-by-Side: State, Feedback, and Cost

The cost shapes differ in a way that matters for budgeting. Reflection multiplies calls inside one attempt: generate, critique, revise, maybe repeat. Reflexion multiplies whole attempts, so cost scales with trials × trajectory length. A Reflexion run with five trials and long trajectories is not five times a single attempt — it is five attempts plus the reflection calls plus the growing context each one carries.

That growth is the second cost. The memory buffer is re-injected every trial, so it competes for context budget with the actual task state. A buffer that grows unbounded will eventually crowd out the observations the agent needs to act. Cap it, summarize it, or drop the oldest entries — but do not let it grow silently.

Both mechanisms are inference-time only. Neither updates weights. That is the shared constraint and the shared ceiling: you are buying improvement with context and compute, not with learning that survives the session. Weight updates and fine-tuning on reflection traces are a separate design choice, not part of either loop.

One caution on evidence. Published lifts — Reflexion's reported pass@1 gains on coding benchmarks, Self-Refine's reported average improvement across tasks — come from specific benchmarks, models, and setups. Treat those numbers as directional evidence that the mechanism can work, not as a guarantee for your task. Your evaluator, your task distribution, and your model version are the variables that decide.

Choosing by Memory and Feedback Contract

The decision reduces to two questions about your system, not about the techniques:

1. Can the agent attempt the task more than once?
2. Is there a signal that can score an attempt?

The logic is mechanical. Cross-trial memory only pays off if there is a next trial to read it. A memory buffer with no retries is a write-only log. And a scorer with no retries is still useful — as an in-attempt gate that decides whether to revise or ship — but it does not need persistence.

The "Yes / Yes" cell deserves a caveat rather than a slogan. Cross-trial memory is the higher-leverage option only when three conditions hold: the failure information is likely to transfer to the next attempt, the evaluator can distinguish progress from noise, and the retry budget and context cost are acceptable. If lessons are task-specific trivia that will not generalize, or if the buffer is already crowding out task state, a within-attempt evaluator-gated revision is the cheaper and more honest choice. Reflexion does not "usually win" by default; it wins when the lesson is reusable and the signal is trustworthy.

Failure Modes and When Not to Use Either

Memory poisoning. Reflections written from a bad evaluator get re-injected and compound the error across trials. The agent learns the wrong lesson and applies it faithfully. This is the most damaging failure because it looks like progress — the buffer is filling up, the agent is "learning."

Reflection theater. Critique with no criteria produces fluent restatement, not correction. The output reads better and means the same thing. If your reviser cannot name what it is checking, it is not checking anything.

Unbounded loops. Missing max-trials or max-iterations turns a bounded feedback system into a token furnace. Both mechanisms need a ceiling, and the ceiling needs to be enforced in code, not in a prompt.

Context dilution. Long reflection buffers push the task state out of the window. Later trials get worse, not better, because the agent is reading its own notes instead of the problem. Watch the ratio of memory tokens to task tokens.

And the cases where you should skip both: a single deterministic pass with no retry, a task where the cost of an extra attempt exceeds the value of the improvement, or a task where a cheap deterministic check already catches the failure. If a regex or a type checker catches it, you do not need a critique loop.

Smallest Useful Implementation

The fastest way to feel the difference is to build both, narrowly, on one task. Here is the skeleton for each.

def reflect_once(task, criteria, max_iters=3):
    artifact = generate(task)
    for _ in range(max_iters):
        critique = critique_against(artifact, criteria)
        if criterion_met(artifact, criteria):
            break
        artifact = revise(artifact, critique)
    return artifact  # nothing persists

def reflexion(task, evaluator, max_trials=5):
    memory = []
    for trial in range(max_trials):
        trajectory = attempt(task, memory)      # memory injected into prompt
        score, passed = evaluator(trajectory)
        if passed:
            return trajectory
        lesson = write_reflection(trajectory, score)
        memory.append(lesson)                   # persists to next trial
    return trajectory

Two things to notice. In the reflection skeleton, the loop closes on artifact — the revised version replaces the old one, and nothing survives the function. In the Reflexion skeleton, the loop closes on memory — the trajectory is discarded, the lesson is kept, and the next attempt reads it.

Now instrument it so you can prove the mechanism works. Log five things per trial: the score, the reflection text, a lesson identifier or version, the memory token count, and whether the next attempt's behavior actually changed. That last one is the one people skip, and it is the one that matters. A reflection that does not change the next attempt is decoration.

The cheap experiment: run the same task three ways — reflection only, memory only, both — and compare trial-over-trial scores, not final outputs. Final outputs hide the mechanism. Trial curves show it. And because a single three-trial run cannot separate learning from noise, repeat the run or use a small task set and compare against a no-loop baseline. The goal is to attribute improvement to a component, not to claim a benchmark result.

Keep the first version narrow: one task, one evaluator, one memory slot. If it fails, you want the failure to be attributable to one component, not to an interaction you cannot isolate.

The Decision Rule

Identify whether your system can retry and whether it can score an attempt. If both hold, build the memory buffer — but only after you have confirmed that the lessons transfer and the context cost is tolerable. If only one holds, use the mechanism that matches the constraint you actually have. If neither holds, stop adding loops and go fix the feedback signal — both mechanisms are downstream of it, and neither will rescue a system that cannot tell whether it succeeded.

Then do the concrete thing: take one existing agent task, add per-trial score logging and a single memory slot, run it for three trials, and inspect whether the reflection text actually changed the next attempt's behavior. If it did not, you have found your bottleneck — and it is not the prompt.