Why does our agent sometimes hallucinate a tool result or mis-handle a tool error, and how do we trace the root cause?

Most teams don’t discover tool hallucinations and mis-handled errors because the agent “crashes.” They discover them when a user says, “That’s not what the tool returned,” or “Why did it ignore the error and keep going?” These are silent failures: the LLM produced a fluent answer that doesn’t match reality, and there’s no stack trace pointing to a single line of broken code.

Quick Answer: Tool hallucinations and mis-handled tool errors happen when the LLM’s reasoning diverges from actual tool behavior—because of missing context, ambiguous prompts, or noisy intermediate state. You fix them by tracing the full agent trajectory: every tool call, argument, result, and follow‑up decision, then tightening prompts, schemas, and error-handling logic based on what the trace shows.

---

The Quick Overview

• What It Is: A trace-first workflow for diagnosing and fixing agents that hallucinate tool outputs or mishandle tool failures, using LangSmith to capture and replay the exact decision path.
• Who It Is For: Teams running production agents with tools (RAG, coding, internal APIs, MCP tools, A2A workflows) who need to understand why an agent “lied” about a tool result or ignored an exception.
• Core Problem Solved: You can’t debug non-deterministic agent behavior with traditional logs. You need reasoning traces that show what the agent thought the tool did, step-by-step, and where that diverged from reality.

---

How It Works

At a high level, you instrument your agent so each tool call, LLM step, and nested sub-task becomes a “run” inside a trace. When the agent hallucinates a tool result or mis-handles an error, you:

• Pull up the exact trace for that execution.
• Inspect the tool call(s), arguments, and raw results.
• Compare them to the LLM messages before and after the tool.
• Identify where the reasoning went off the rails (schema mismatch, prompt ambiguity, unhandled error, memory corruption, etc.).
• Turn that trace into a test case and fix prompts, tools, or error handlers—then re-run and compare.

LangSmith is built around this loop: observe → explain → evaluate → deploy with guardrails.

1. Capture the full agent trajectory

Instead of a flat log line like tool=fetch_order status=200, LangSmith records a structured trace that links:

• Inputs: the user query, system prompts, and tool descriptions available at each step.
• LLM runs: the full prompts (or redacted versions) and model outputs.
• Tool runs: function name, arguments, raw results, latency, and any exceptions.
• Nested structure: which tool calls were triggered by which LLM decisions, including loops and branches.
• Threads: how context evolved over multiple turns (e.g., a bad assumption stored at turn 6 causing issues at turn 11).

This is the only reliable way to answer: “Did the agent hallucinate the tool result, or did the tool actually return that data?”

2. Identify hallucinated tool results vs. real tool bugs

Once you have traces, you can differentiate failure modes:

• Pure hallucination: The tool call never happened, but the LLM fabricated a result.
• Partial hallucination: The tool returned something, but the LLM embellished or misinterpreted it.
• Tool bug or API issue: The tool returned incorrect data, and the agent faithfully passed it through.
• Error-handling bug: The tool raised an error, but the agent ignored it, mis-parsed it, or kept going with stale context.

LangSmith’s run timelines and tool-call detail view make this explicit: you can see exactly which tools were invoked, with what arguments, and what they returned.

3. Close the loop with tests and evals

A single fix is fragile unless you prevent regressions. With LangSmith, you:

1. Convert failing traces into datasets (examples).
2. Define evaluators that check:
   • Did the agent call the expected tool?
   • Did it describe the tool result correctly?
   • Did it handle errors by surfacing them or trying a fallback?
3. Run regressions via:
   • Offline evals: on candidate changes before deployment.
   • Online evals: on production traffic, sampling traces for continuous checking.

Align Evals lets you calibrate LLM-as-judge evaluators using human corrections so your automated scoring matches what your team considers “good behavior.”

---

Why agents hallucinate tool results or mis-handle errors

Most of these failures fall into a few categories. Traces help you distinguish them and choose the right fix.

1. Ambiguous or weak tool prompts

If the model isn’t crystal-clear on when and how to call a tool, it will often:

• Skip calling the tool and “guess” from its prior knowledge.
• Call the tool but then overwrite or embellish the returned result.
• Invent status or error messages that sound plausible.

Typical causes:

• Tool description doesn’t specify that the agent must never fabricate results.
• The system prompt doesn’t say “always prefer tool output over prior knowledge.”
• The model isn’t instructed to surface uncertainty or escalate human review for unexpected tool responses.

Trace signal: You see an answer that references a tool result, but no tool run exists in the trace, or the tool run’s result clearly contradicts the final answer.

2. Schema and parsing mismatches

If the tool returns data in a structure that doesn’t match the prompt’s expectations, the LLM will improvise:

• Misreading fields (“status” vs. “state”).
• Ignoring nested objects or arrays.
• Treating error payloads as successful results.

Trace signal: Tool run shows a well-formed response, but the subsequent LLM step uses the wrong fields or treats an error-like payload as normal data.

3. Poorly handled tool errors

Tools fail in more ways than HTTP 500s:

• Timeouts.
• Partial results.
• Validation errors.
• “Soft errors” embedded in the payload (e.g., "success": false with a message).

If your agent’s runtime or prompt doesn’t teach the model how to react, you’ll see:

• The agent pretending the operation succeeded.
• The agent ignoring the error and continuing with stale or default values.
• The agent retrying indefinitely or in tight loops.

Trace signal: Tool run has an error or error-like payload, but the following LLM step either:

• Proceeds as though it succeeded, or
• Doesn’t mention the error to the user or calling system.

4. State and memory drift

In long-running agents or multi-turn threads, the problem often starts earlier:

• The agent stores a wrong assumption in memory (“order is shipped”).
• Later, even if a tool returns a corrected value, the agent still trusts the stale memory.
• Over multiple turns, that bad context compounds into obviously wrong behavior.

Trace signal: When you view the full thread, you see a wrong fact written to memory at turn N; tool calls at turn N+K are correct, but the agent’s responses still reflect the outdated fact.

5. Non-determinism and edge prompts

LLMs are stochastic. Without clear constraints and evals, they:

• Behave correctly for common patterns.
• Fail on rare structured inputs, long tail tools, or unusual error strings.

Trace signal: Same prompt and input schema behave differently run-to-run. In one trace, the model calls the tool and uses the result. In another, it hallucinated without calling the tool.

---

A trace-first workflow to find the root cause

Here’s a concrete debugging loop using LangSmith.

1. Find the bad run

   • Search traces by:
     • User ID / session.
     • Tool name.
     • Error status or custom tags.
   • Or let the Insights Agent flag anomalies like sudden spikes in failed tools or mismatched tool usage patterns.

2. Inspect the run timeline

   • Open the trace; look for:
     • The user query where the hallucination or mis-handling occurred.
     • All tool runs in that time window.
     • The LLM messages immediately before and after each tool call.

3. Check tool call correctness

   • Did the agent call the right tool?
   • Were the arguments correct and complete?
   • Did the tool return what your backend or API docs say it should?

4. Compare tool outputs to the final answer

   • Highlight where the agent:
     • Added fields that don’t exist in the tool output.
     • Changed values (e.g., price, status, date).
     • Ignored error codes or messages.

5. Trace the root cause backward

   • If the hallucination happened with no tool call:
     • Check the system + tool prompts—do they enforce tool use?
   • If the hallucination followed a real tool call:
     • Check for schema mismatches and ambiguous field naming.
   • If the error was ignored:
     • Check whether your tool integration surfaced the error clearly, or if the model saw it as normal data.
   • If memory drift is involved:
     • Use thread view to find the first point where the wrong assumption entered the context.

6. Turn the trace into an eval

   • Save the trace as an example in a dataset.
   • Add a human label describing:
     • Expected tool behavior.
     • Expected agent response (or at least constraints: “must not fabricate fields,” “must surface error”).
   • Attach evaluators:
     • LLM-as-judge comparing final output to tool result.
     • Custom checks over tool runs (e.g., “tool X must be called when Y appears in the query”).

7. Fix and re-run

   • Update prompts:
     • Strengthen instructions around tool usage and error handling.
     • Clarify schemas and field names.
   • Update tools:
     • Normalize error formats.
     • Return structured status fields instead of freeform strings.
   • Update runtime:
     • Add guardrails that validate tool outputs before passing them back to the model.
     • Enforce retries, circuit breakers, or fallback flows for specific error types.
   • Re-run the dataset with offline evals. Compare old vs new traces side-by-side in LangSmith before deploying.

---

Features & Benefits Breakdown

Core Feature	What It Does	Primary Benefit
Reasoning traces	Capture every LLM step, tool call, argument, result, and nested run structure.	See exactly where tool hallucination or error mis-handling began.
Thread and timeline views	Visualize multi-turn conversations and long-running agents over time.	Understand how context and memory drift drive downstream failures.
Evals and datasets (Align Evals)	Turn real traces into test cases and run LLM-as-judge and custom checks.	Prevent regressions and catch tool-related issues before production.
Insights Agent & analytics	Analyze thousands of traces to find patterns in tool usage, loops, and failure modes.	Automatically surface inefficient or error-prone tool behavior.
Durable runtime & deployment	Provide exactly-once execution, durable checkpointing, and tool-level approvals.	Reduce mis-handled errors in production with robust agent execution.
Framework-agnostic instrumentation	Integrate via SDKs (Python, TS, Go, Java) or OpenTelemetry with any agent stack or tooling.	Debug tool hallucinations without rewriting your existing framework.

---

Ideal Use Cases

• Best for production agents with critical tools: Because you can’t tolerate hallucinated tool outputs when agents touch payments, logistics, support systems, or internal CRMs. LangSmith lets you trace every call, validate behavior, and roll back safely.
• Best for teams iterating on complex toolchains: Because when your agent chains multiple tools or uses multi-agent workflows (A2A, MCP), traditional logs are useless. Traces and evals make it possible to understand why one branch handled an error correctly and another didn’t.

---

Limitations & Considerations

• Traces don’t fix bad tools or models on their own: LangSmith shows you exactly what went wrong, but you still need to update prompts, schemas, or tool implementations. The platform is a microscope, not an auto-repair.
• You need a clear definition of “correct” behavior: To get value from evals, your team must decide what “good” tool usage looks like—e.g., when to fail closed, when to retry, when to ask for human approval. Align Evals helps encode this, but it still requires domain guidance.

---

Pricing & Plans

LangSmith is designed for teams of any size, from single-agent prototypes to fleets of production agents processing billions of events.

• Usage is pay-as-you-go based on traces and storage, so you can start small and scale as your agent traffic grows.
• Base plans include shorter trace retention (e.g., 14 days) suitable for iteration and smaller workloads.
• Higher tiers and enterprise plans add:
  • Extended retention (e.g., 400 days) for deep forensic analysis and long-term eval datasets.
  • Seat-based access for engineering, data science, and operations teams.
  • Enterprise controls: SSO/SAML, SCIM, audit logs, RBAC/ABAC, and encryption.
  • Deployment flexibility: US/EU data residency, hybrid and self-hosted options to keep data in your VPC.

Typical plan fit:

• Team / Growth: Best for product teams building serious agents that rely on a handful of critical tools and need robust traces, evals, and basic governance.
• Enterprise: Best for large organizations (including 35% of the Fortune 500) needing long retention, advanced security controls, custom SLAs, and the ability to instrument many agents across multiple business units.

For exact pricing and retention options, talk to our team.

---

Frequently Asked Questions

How do I tell if a bad answer came from tool hallucination or a tool bug?

Short Answer: Check the trace: if there’s no tool call (or the tool result contradicts the final answer), it’s hallucination; if the tool returned bad data and the agent simply passed it through, it’s a tool-side bug.

Details:
Open the failing run in LangSmith and:

1. Look for tool runs:
   • If the agent describes a tool result but no corresponding tool run exists, it hallucinated.
   • If the tool run exists, compare its raw output to the final answer.
2. If the raw tool output is correct but the final answer differs:
   • This is a reasoning issue (schema mismatch, misinterpretation, or embellishment).
3. If the raw tool output is wrong:
   • The agent is behaving correctly; your fix is in the backend or external API, but you can still adjust prompts to detect and surface inconsistent tool behavior.

Turn this specific trace into a dataset example and add an evaluator that checks consistency between tool outputs and final answers so you catch similar issues automatically in the future.

---

How can I reduce tool error mis-handling without hand-inspecting every trace?

Short Answer: Define structured error-handling expectations, instrument them as evals and runtime checks, and let LangSmith’s evals and Insights Agent surface patterns in tool failures.

Details:

1. Normalize tool errors:

   • Make tools return structured error shapes ({ "status": "error", "code": "...", "message": "..." }) instead of free-text.
   • Update prompts to instruct the LLM to treat "status": "error" as a hard failure requiring either:
     • A retry.
     • A fallback tool.
     • A clear surfaced error to the user or upstream caller.

2. Instrument evals:

   • Write a simple evaluator that:
     • Scans traces for tool runs with status="error".
     • Checks that the following LLM run:
       • Acknowledged the error, and
       • Did not treat error payloads as normal data.
   • Use Align Evals to calibrate these checks with human-reviewed examples.

3. Use analytics and Insights Agent:

   • Look at dashboards for:
     • Tools with unusually high error rates.
     • Sessions with repeated retries or loops on the same tool.
   • Let Insights Agent summarize “why” an agent keeps hitting failed tools (e.g., wrong arguments, missing parameters, bad routing logic).

4. Add runtime guardrails:

   • Use LangSmith Deployment or your own runtime to:
     • Block downstream steps if a critical tool fails.
     • Trigger human-in-the-loop approval in high-risk flows.
     • Attach tool-level approvals via Fleet/Agent Builder when tools trigger sensitive actions.

This shifts you from manual inspection of individual failures to systematic, eval-driven detection of mis-handled errors.

---

Summary

Tool hallucinations and mis-handled errors aren’t edge cases; they’re what happens when non-deterministic reasoning meets imperfect tools and ambiguous prompts. You won’t fix them with more logging alone—you need traces that capture the full agent trajectory and a workflow that turns those traces into datasets, evals, and safer deployments.

LangSmith gives you that trace-first loop:

• Instrument every tool call and LLM step.
• Inspect reasoning around tool outputs and errors.
• Convert real failures into evals and regression tests.
• Deploy agents on a runtime with memory, checkpointing, and approvals so the same bug doesn’t bite you twice.

When you can replay what your agent actually did—step by step—it stops being a mystery why it hallucinated a tool result or ignored an error, and starts being an engineering problem you can systematically solve.

---

Next Step

Get Started