Reflexion Agents: Advanced Self-Improvement in LangGraph
Modern LLM applications have moved far beyond the simple pattern:
User → Prompt → LLM → Answer
That architecture works well for straightforward tasks, but it starts breaking down when an AI system must produce accurate, evidence-backed, context-aware, and production-quality results.
An LLM can generate a plausible answer while:
- missing important requirements,
- making unsupported claims,
- ignoring part of the user's question,
- using outdated knowledge,
- selecting poor reasoning strategies,
- or producing an answer that sounds correct but is incomplete.
A powerful approach to address this problem is the Reflexion Agent.
Instead of accepting the first generation, the system creates an answer, evaluates its weaknesses, gathers additional information when necessary, and then produces a better version.
The core idea is:
User Query
↓
Responder
↓
Self-Critique
↓
Generate Search Queries
↓
External Tools / Knowledge
↓
Revisor
↓
Improved Response
↓
Repeat
↓
Final Answer
Reflexion extends the basic "reflection agent" idea: the system combines iterative self-critique with external tools and citations so that later responses can be more current, supported, and useful.
This article walks through how to take that idea from a learning example to an industry-level AI/ML engineering architecture using LangGraph.
1. What Problem Are Reflexion Agents Actually Solving?
Before understanding Reflexion, we need to understand the limitation of a normal LLM application.
Suppose we build an AI assistant for a software engineering team.
A developer asks:
"Our FastAPI service started returning 500 errors after the latest deployment. Analyze the problem and recommend a fix."
A traditional LLM application might respond:
The issue may be caused by a database connection problem.
Check your DATABASE_URL and restart the service.
The answer sounds reasonable.
But a production engineer would immediately ask:
- Which endpoint is failing?
- What is the actual stack trace?
- Did the database schema change?
- Was there a recent migration?
- Is the error happening only in production?
- What changed in the deployment?
- Are connection pool limits exhausted?
- Is the database reachable?
- Is the environment variable configured correctly?
- Are there similar incidents in previous deployments?
The first LLM response may not ask any of these questions.
This is where Reflexion becomes useful. Instead of treating the first response as the final answer, we treat it as a draft.
The system asks:
"What is wrong with my answer?"
Then:
"What information do I need to verify or improve it?"
Then it calls tools.
Then:
"Now that I have additional evidence, how should I revise my answer?"
That is the fundamental engineering idea behind Reflexion.
2. Reflection vs Reflexion
These terms are closely related, but useful to distinguish.
Basic Reflection typically follows:
Generate
↓
Critique
↓
Generate Again
Example:
User:
Write a LinkedIn post about becoming a software engineer.
Generator:
"Becoming a software engineer requires learning..."
Reflector:
"The post is generic. Add a stronger hook and practical experience."
Generator:
"After years of debugging..."
This generator → reflector pattern is the simplest self-improvement loop, and LangGraph can maintain the evolving message state across iterations.
3. What Makes Reflexion Different?
Reflexion extends this idea. The important addition is external feedback and evidence.
The workflow becomes:
Generate
↓
Critique
↓
What information is missing?
↓
Generate search/tool queries
↓
Use external tools
↓
Evaluate evidence
↓
Revise
↓
Repeat
Reflexion agents typically combine:
- self-critique,
- external tools,
- real-time information,
- citations,
- structured outputs,
- responder/revisor roles,
- iterative state maintained by the graph.
The distinction, in short:
Reflection
= "Is my answer good?"
Reflexion
= "Is my answer good?
What is missing?
What evidence do I need?
Can I verify it?
How should I revise it?"
4. The Most Important Mental Model
Think of Reflexion as a quality-control loop around an LLM.
Do not think:
LLM = intelligence
Instead think:
LLM
↓
Draft
↓
Evaluator
↓
Evidence Retrieval
↓
Revision
↓
Quality Check
↓
Final Result
The LLM is one component inside the system. LangGraph becomes the orchestration layer responsible for controlling this process.
5. Reflexion Agent Architecture
┌──────────────────────┐
│ User Query │
└──────────┬───────────┘
↓
┌──────────────────────┐
│ Responder │
│ Generate Draft │
└──────────┬───────────┘
↓
┌──────────────────────┐
│ Reflection │
│ What is missing? │
│ What is wrong? │
│ What is irrelevant? │
└──────────┬───────────┘
↓
┌──────────────────────┐
│ Search Queries │
└──────────┬───────────┘
↓
┌──────────────────────┐
│ External Tools │
│ Web / DB / API / RAG │
└──────────┬───────────┘
↓
┌──────────────────────┐
│ Revisor │
│ Improve + Verify │
└──────────┬───────────┘
↓
Continue?
↙ ↘
YES NO
↓ ↓
More Tools Final
↓ Answer
Revisor
Responder → tool execution → revisor nodes, connected through LangGraph edges, with an iteration limit controlling the loop.
6. Why LangGraph Is Important
You could implement a small reflection loop with plain Python:
for i in range(3):
answer = generate(answer)
critique = reflect(answer)
But production agents quickly get more complicated — multiple tool branches, conditional retries, and decision points:
┌── Web Search
│
Responder ───┼── Vector DB
│
└── Internal API
↓
Revisor
↓
Quality Check
This is exactly where graph-based orchestration becomes valuable. LangGraph lets us represent nodes, edges, state, conditional routing, iteration, termination, and accumulated messages.
Use StateGraph with MessagesState (not MessageGraph****) to maintain a growing message list, where Human, AI, and tool messages accumulate across iterations:
from langgraph.graph import StateGraph, MessagesState, START, END
builder = StateGraph(MessagesState)
This gives you the same "append-only message history" behavior MessageGraph used to provide, but stays on the supported, current API — and lets you add extra fields (iteration count, critique, citations) to the same state object, which MessageGraph couldn't do.
7. The Three Most Important Components
7.1 Responder
The responder creates the first structured answer:
Understand Query
↓
Generate Answer
↓
Identify Weaknesses
↓
Generate Search Queries
Example structured output:
{
"answer": "The API failure is probably related to database connectivity.",
"reflection": {
"missing": [
"database connection logs",
"recent migration status",
"deployment changes"
],
"superfluous": [
"generic server restart advice"
]
},
"search_queries": [
"FastAPI PostgreSQL connection pool 500 errors",
"SQLAlchemy connection pool production errors"
]
}
A common pattern is to define a schema like AnswerQuestion, with reflection information and search queries, and bind the LLM to that schema so the output is structured rather than free text.
8. Why Structured Output Matters
Do not make your downstream agent parse this:
I think the answer is probably X.
You should investigate Y.
Maybe search for Z.
That creates fragile parsing. Instead:
from pydantic import BaseModel
class Reflection(BaseModel):
missing: list[str]
superfluous: list[str]
class AnswerQuestion(BaseModel):
answer: str
reflection: Reflection
search_queries: list[str]
Now the workflow knows exactly what each field means:
answer → current answer
reflection.missing → information that needs investigation
reflection.superfluous → unnecessary information
search_queries → external knowledge required
Schema-based output separates response, critique, missing information, irrelevant information, and search queries cleanly — this is much closer to software engineering than writing a clever prompt.
9. The Revisor
The revisor takes:
Original Question + Previous Answer + Self-Critique + Tool Results
and produces:
Improved Answer + Updated Critique + References + New Search Queries
revised = revisor(
question,
previous_answer,
critique,
tool_results
)
The revisor schema is usually derived from the answer schema, with citation/reference fields added so the revision can incorporate evidence explicitly.
10. Real Industry Project: Production Incident Analysis Agent
Imagine your company runs:
FastAPI · PostgreSQL · Redis · Docker · Kubernetes
GitHub · Prometheus · Grafana · Sentry
A production engineer asks:
"Why did the payment API start returning 500 errors after today's deployment?"
Instead of a simple chatbot, we build a Production Incident Reflexion Agent whose job is to investigate the incident and continuously improve its diagnosis.
11. Step 1 — Initial Query
Payment API started returning 500 errors after today's deployment.
Find the probable root cause and recommend a safe fix.
Responder's initial hypothesis:
The payment service may be failing because of a
database connectivity issue introduced during deployment.
Missing information it should flag:
- application logs
- database errors
- deployment diff
- migration status
- recent configuration changes
Search/tool queries generated:
- payment service deployment database errors
- PostgreSQL connection pool errors
- latest migration failures
12. Step 2 — External Tools
In a real engineering system, tools could be:
search_docs()
get_deployment_diff()
get_application_logs()
query_metrics()
get_recent_incidents()
search_runbook()
query_database()
@tool
def get_deployment_diff(service: str):
...
@tool
def get_recent_logs(service: str):
...
@tool
def search_runbook(query: str):
...
Reflexion is no longer just an LLM. It becomes:
LLM + Tools + State + Feedback Loop
13. Step 3 — Evidence Changes the Diagnosis
Suppose the tools return:
Deployment diff:
SQLAlchemy upgraded from 1.4 → 2.x
Logs:
sqlalchemy.exc.OperationalError: connection pool exhausted
Metrics:
DB connections: 95 / 100
Previous incident:
Same error occurred when pool_size was configured incorrectly.
The original hypothesis ("maybe database connectivity") becomes much stronger:
Root cause candidate: the deployment changed database connection
behavior, causing the application pool to exhaust available
PostgreSQL connections.
The important part: this second answer isn't "another LLM generation" — it's a revision grounded in evidence.
14. Step 4 — Revisor
{
"answer": "...",
"reflection": {
"missing": [
"confirm exact configuration change",
"verify database connection limit"
]
},
"citations": [
"...deployment diff...",
"...production logs..."
],
"search_queries": [
"SQLAlchemy pool_size deployment configuration"
]
}
Resulting recommendation:
1. Verify the connection pool configuration.
2. Confirm PostgreSQL max_connections.
3. Compare the previous production configuration.
4. Roll back the configuration if the change caused exhaustion.
5. Do not immediately increase max_connections without checking
application pool behavior.
Much more useful than "restart the service."
15. The LangGraph State
The state is the heart of the system.
from typing import TypedDict
class AgentState(TypedDict):
messages: list
user_query: str
draft: str
critique: dict
search_queries: list[str]
evidence: list[dict]
citations: list[str]
iteration: int
Build it on StateGraph, not the deprecated MessageGraph:
from langgraph.graph import StateGraph, START, END
builder = StateGraph(AgentState)
Accumulated message state — Human, AI, and tool messages — carries previous responder and tool outputs into the revisor and subsequent iterations.
16. The Graph
graph = StateGraph(AgentState)
graph.add_node("responder", responder)
graph.add_node("tools", execute_tools)
graph.add_node("revisor", revisor)
graph.add_edge(START, "responder")
graph.add_edge("responder", "tools")
graph.add_edge("tools", "revisor")
graph.add_conditional_edges(
"revisor",
should_continue,
{"tools": "tools", "end": END}
)
app = graph.compile()
Workflow:
START
↓
Responder
↓
Tools
↓
Revisor
↓
Should Continue?
├── YES → Tools → Revisor
└── NO → END
This topology — responder → tool execution → revisor, followed by conditional routing based on an iteration counter — is the standard Reflexion shape.
17. Why We Need an Iteration Limit
Without a limit:
Responder → Tool → Revisor → Tool → Revisor → Tool → Revisor → ∞
Your application can consume unbounded tokens, API calls, latency, money, and infrastructure resources.
In production, use multiple stopping conditions:
iteration >= MAX_ITERATIONS
confidence >= threshold
no_missing_information
no_new_evidence
quality_score >= threshold
18. Better Production Routing
A naive implementation:
if iteration >= 4:
return "end"
A smarter one:
def should_continue(state):
if state["iteration"] >= 4:
return "end"
if not state["search_queries"]:
return "end"
if not state["critique"]["missing"]:
return "end"
return "tools"
Now the agent doesn't blindly run a fixed number of iterations — it stops when it has enough information.
19. A More Advanced Quality Gate
For high-value enterprise applications, add a dedicated evaluator:
Responder → Tool Retrieval → Revisor → Evaluator
↓
Quality > 0.85?
YES → END
NO → Revisor
scores = {
"accuracy": 0.91,
"completeness": 0.84,
"evidence_quality": 0.93,
"instruction_following": 0.96,
}
overall_score = (
scores["accuracy"] * 0.40 +
scores["completeness"] * 0.25 +
scores["evidence_quality"] * 0.25 +
scores["instruction_following"] * 0.10
)
This creates a measurable quality-control mechanism.
20. Reflexion Is Not Magic Self-Learning
When we say "the agent improves itself," don't read that as "the neural network is updating its weights."
The practical architecture improves current task execution by generating a draft, reflecting on it, retrieving new information, feeding previous outputs back into context, and generating a revised answer.
Think:
Runtime improvement
not:
Model-weight learning
That distinction matters when explaining Reflexion to an ML engineering team.
21. Reflexion vs RAG
RAG:
Query → Retrieve → Generate
RAG answers: "What information should I provide to the model?"
Reflexion:
Query → Generate → Critique → Identify missing information → Retrieve → Revise
Reflexion asks: "What is wrong or missing in my current answer, and what information do I need to improve it?"
They can be combined.
22. Reflexion + RAG
User → Responder → Reflection → Retrieval Strategy
↙ ↘
Vector DB Web Search
↘ ↙
Evidence
↓
Revisor
↓
Quality Gate
↓
Final
For internal enterprise knowledge, sources can include PostgreSQL, Confluence, GitHub, Jira, S3, documentation, runbooks, and a vector DB.
23. Reflexion + ReAct
ReAct pattern:
Thought → Action → Action Input → Observation → Final Answer
Tools provide observations that influence the next reasoning step. ReAct focuses on Reasoning + Acting; Reflexion focuses on evaluating and improving previous attempts. They work well together:
Reflexion Controller
↓
ReAct Agent
↓
Tool Execution
↓
Observations
↓
Reflection
↓
Revisor
24. The Difference in One Diagram
Simple LLM User → LLM → Answer
RAG User → Retriever → LLM → Answer
ReAct User → LLM → Tool → Observation → LLM → Answer
Reflection User → Generator → Critic → Generator → Answer
Reflexion User → Responder → Critique → Tool/Knowledge → Revisor
→ Quality Check → More iteration? → Final Answer
25. Prompt Engineering for Reflexion
A weak reflection prompt ("Review this answer") is usually insufficient. A stronger one defines explicit responsibilities:
You are a senior software engineering reviewer.
Review the proposed incident diagnosis.
Identify:
1. Unsupported assumptions
2. Missing evidence
3. Contradictions
4. Irrelevant information
5. Important questions that remain unanswered
6. External information that should be retrieved
7. Evidence required before making a production recommendation
Do not rewrite the answer yet. Return structured output.
This separates critique from revision — an extremely valuable split.
26. Why Separate Responder and Revisor Roles?
You could ask the same LLM call to improve its own answer, but separate roles give clearer responsibilities:
Responder = Generate useful answer → "How can I answer this?"
Revisor = Attack weaknesses → "Why might this answer be wrong?"
That mindset difference tends to produce better results, which is why responder and revisor are usually modeled as distinct LangGraph nodes.
27. Memory and Context
A Reflexion agent needs to remember previous attempts:
Iteration 1: Answer → Database problem. | Critique → Need logs.
Iteration 2: Answer → Connection pool exhaustion. | Critique → Need deployment diff.
Iteration 3: Answer → Deployment changed SQLAlchemy config. | Critique → Need metrics confirmation.
Without state, iteration 3 wouldn't know what happened in iteration 1. With LangGraph state, the accumulated HumanMessage → AIMessage → ToolMessage → ... history becomes part of the workflow context automatically.
28. But Don't Store Everything Forever
A naive system might keep 100 iterations × 10 tool responses × large documents inside the context. That causes token growth, latency, higher cost, and context-window pressure.
A production system should summarize:
state["evidence_summary"]
state["latest_critique"]
state["open_questions"]
state["citations"]
instead of carrying every raw tool response forever.
29. Citation-Aware Revision
If the first answer says:
PostgreSQL connections are probably exhausted.
the revisor should ideally produce:
The logs show connection pool exhaustion, while the deployment
configuration shows a change to database connection handling.
Evidence:
- Production logs
- Deployment configuration diff
- Database metrics
This gives the user a clear claim → evidence → source chain, increasing response depth and transparency.
30. A Production-Grade Reflexion State
class ReflexionState(TypedDict):
user_query: str
draft: str
critique: dict
missing_information: list[str]
search_queries: list[str]
evidence: list[dict]
citations: list[dict]
iteration: int
quality_score: float
final_answer: str
31. Node Responsibilities
responder_node → reflection_node → tool_planner_node
→ tool_executor_node → revisor_node → evaluator_node → router_node
- Responder — generate the initial answer.
- Reflection — identify weaknesses.
- Tool Planner — determine what information is needed.
- Tool Executor — call APIs, search, databases, RAG, etc.
- Revisor — produce a stronger answer.
- Evaluator — measure quality.
- Router — decide whether to continue or stop.
This separation makes the system easier to test and maintain.
32. Observability Is Critical
You should be able to answer: why did the agent make this decision, which tools were called, how many iterations happened, which evidence influenced the final answer, what did the revisor change, and why did the graph stop.
Run ID: incident-8392
Iteration 1
Responder → 1.2s
Reflection → 0.8s
Tools → 2.1s
Iteration 2
Revisor → 1.5s
Tools → 1.9s
Iteration 3
Evaluator → 0.7s
Final Score: 0.91
Essential when debugging production agent behavior.
34. Where Reflexion Is Most Valuable
Reflexion pays off when mistakes are expensive:
- Software engineering — code review, bug analysis, architecture recommendations, migration planning, security analysis, incident investigation.
- Research — literature analysis, competitive research, technical reports, evidence-backed summaries.
- Enterprise knowledge — policy interpretation, internal documentation, compliance workflows, operational support.
- Customer support — complex troubleshooting, multi-step diagnosis, policy-aware responses.
- Data science — SQL generation, data analysis, hypothesis generation, model diagnostics.
35. Where Reflexion May Be Overkill
Don't build a six-node Reflexion graph for "What is 2 + 2?" or "Convert 5 km to meters" — a deterministic tool is better. A simple User → LLM → answer may be perfectly adequate for low-risk conversational tasks. The architecture should match the problem.
36. Common Engineering Mistakes
Mistake 1 — Infinite Reflection. Fix with max_iterations plus quality-based termination.
Mistake 2 — Reflection Without Evidence. "Make the answer more detailed" is just another opinion. Better: critique → missing information → tool retrieval → evidence → revision.
Mistake 3 — Unstructured Output. Avoid if "missing:" in response: string parsing — use structured schemas.
Mistake 4 — Blindly Trusting Search Results. Search results are evidence, not truth. The revisor should evaluate source quality, relevance, freshness, and claim support.
Mistake 5 — Carrying Too Much Context. Don't send every historical message and raw document into every iteration — use summaries, relevant evidence, and open questions.
Mistake 6 — Assuming Self-Critique Means Correctness. An LLM can produce a confident critique and still be wrong. Self-critique + external evidence + evaluation is stronger than self-critique alone.
37. Reflexion as a Software Design Pattern
GENERATE → EVALUATE → GATHER EVIDENCE → REVISE → EVALUATE → STOP / REPEAT
This pattern applies independently of domain:
- Coding agent: Generate Code → Run Tests → Analyze Failures → Fix Code → Run Tests Again
- Research agent: Generate Report → Identify Unsupported Claims → Search Sources → Revise Report → Verify Citations
- SQL: Generate SQL → Run SQL → Analyze Error → Fix SQL → Run Again
- Customer support: Draft Response → Check Policy → Identify Missing Context → Retrieve Account Info → Revise Response
Reflexion is more valuable as an engineering pattern than as a single prompting technique.
38. The SQL Example
User: "Find customers who spent more than $10,000 in the last 90 days."
Initial response:
SELECT customer_id
FROM orders
WHERE amount > 10000;
Reflection: this checks individual orders, not total spending — needs aggregation by customer and date filtering.
Revised:
SELECT customer_id
FROM orders
WHERE order_date >= CURRENT_DATE - INTERVAL '90 days'
GROUP BY customer_id
HAVING SUM(amount) > 10000;
If executing this fails with column "order_date" does not exist, the agent can inspect the schema and revise again — Reflexion combined with tools.
39. Reflexion + Code Execution
LLM → Generate Code → Execute Code → Observation → Reflection
→ Fix Code → Execute Again → Tests Pass? → Final
Environment feedback (compiler, unit tests, linter, static analyzer, security scanner, runtime, database) is a much stronger signal than another LLM's opinion.
40. A Useful Architecture for Your AI/ML Projects
Frontend → Agent API (FastAPI) → LangGraph Orchestration
↓
┌───────────────────┼───────────────────┐
Responder Tools Revisor
↓ ↓ ↓
LLM RAG/API LLM + Schema
└───────────────────┼───────────────────┘
↓
Evaluator
↓
Final Answer
Supporting infrastructure: PostgreSQL, Redis, vector database, object storage, tracing, evaluation, monitoring.
41. What an Experienced AI Engineer Should Take Away
Principle 1 — LLM generation is probabilistic. Don't automatically trust the first output.
Principle 2 — Critique should be structured. Use schemas instead of free-form text.
Principle 3 — Critique should drive action. If something is missing, retrieve it.
Principle 4 — External evidence improves groundedness. Use search, APIs, databases, RAG, or code execution.
Principle 5 — State matters. The agent needs to remember what it previously attempted.
Principle 6 — Iteration must be controlled. Always define stopping conditions.
Principle 7 — Quality must be measurable. Don't assume another iteration helped — evaluate it.
42. Reflexion Interview Question
"What is a Reflexion Agent?"
A strong answer: A Reflexion Agent is an agent architecture that improves an initial LLM response through iterative feedback. A responder generates a structured draft, a reflection step identifies weaknesses and missing information, external tools can be invoked to obtain additional evidence, and a revisor generates an improved response. LangGraph orchestrates this loop using state (typically StateGraph, not the deprecated MessageGraph), nodes, edges, conditional routing, and iteration limits. Unlike a simple reflection loop, a practical Reflexion workflow incorporates external information and citations so revisions are grounded in additional evidence.
43. Final Architecture to Remember
USER
│
▼
┌─────────────┐
│ RESPONDER │
└──────┬──────┘
▼
┌─────────────┐
│ REFLECTION │
│ Missing? │
│ Wrong? │
│ Irrelevant? │
└──────┬──────┘
▼
┌─────────────┐
│ TOOL PLANNER│
└──────┬──────┘
▼
┌─────────────────────────┐
│ TOOLS │
│ Web Search · RAG │
│ Database · APIs │
│ Code Execution · Tests │
└───────────┬─────────────┘
▼
┌─────────────┐
│ REVISOR │
└──────┬──────┘
▼
┌─────────────┐
│ EVALUATOR │
└──────┬──────┘
│
┌──────┴──────┐
Good Enough? No → back to Reflection
│YES
▼
┌──────────────┐
│ FINAL ANSWER │
└──────────────┘
44. Conclusion
Reflexion Agents represent an important transition from single-shot LLM applications to iterative AI systems. The fundamental idea is simple:
Don't trust the first generation.
Generate. Critique. Investigate. Revise. Evaluate. Repeat when necessary.
LangGraph provides the orchestration layer that turns that concept into a controllable workflow: state + nodes + edges + tools + conditional routing + iteration limits.
The real value for an experienced AI/ML engineer isn't memorizing one Reflexion implementation — it's recognizing when a problem requires iterative verification instead of one-shot generation.
For a production system, the mature architecture is therefore not LLM → Answer but potentially:
LLM → Reason → Critique → Retrieve → Verify → Revise → Evaluate → Answer
Combined with RAG, ReAct, tool calling, structured outputs, automated evaluation, observability, and human-in-the-loop controls, Reflexion becomes a powerful building block for production-grade agentic AI systems.
A strong AI agent is not necessarily the agent that generates the best answer on the first attempt. It is the system that knows how to recognize weaknesses, gather the right evidence, revise its work, and stop when the result is good enough.
That is the engineering mindset behind Reflexion Agents.
0 Comments