How can we see exactly which tool calls an agent made (inputs/outputs) and where it decided to branch or stop?

Most teams only find out an agent made the wrong tool call after a user complains. By then, the original prompt, tool inputs, intermediate steps, and branching decisions are gone—or buried in logs that don’t tell you what actually happened. To debug and improve agents in production, you need a replayable view of the full execution: every tool call, its inputs and outputs, and exactly where the agent decided to branch, loop, or stop.

Quick Answer: In LangSmith, you see every tool call and branching decision as a high-fidelity trace. Each agent run is captured as a tree of steps (runs) showing prompts, tool inputs/outputs, and nested calls, so you can replay what happened, understand why, and turn those traces into datasets and evals.

The Quick Overview

• What It Is: A trace-first way to inspect agent behavior in LangSmith so you can see each tool call, its arguments and result, and how those steps fit into a branching execution tree.
• Who It Is For: Teams building serious agents—support agents, workflow copilots, retrieval-heavy apps—who need to debug failures, compare versions, and enforce reliability and policy at scale.
• Core Problem Solved: Agents are non-deterministic and multi-step. Traditional logs only show input and final output; they don’t reveal which tools were called, in what order, with what arguments, and where the agent chose to branch or stop.

How It Works

LangSmith instruments your agent so every step becomes a run. Runs capture inputs, outputs, errors, timings, and metadata across models, tools, retrievers, and custom logic. LangSmith then links these runs into a trace: a tree-structured execution timeline that shows exactly what happened, in what order, and why.

1. Capture runs at every step:
   Your agent is instrumented via LangChain, LangGraph, the LangSmith SDKs, or OpenTelemetry. Each model call, tool call, and nested chain is recorded as a run with full inputs and outputs.

2. Assemble a trace (execution tree):
   LangSmith stitches all related runs into a single trace. Parent–child relationships show where the agent branched (e.g., called multiple tools), looped (repeated a step), or stopped (returned a final answer or error).

3. Inspect and iterate:
   In the LangSmith UI, you replay the trace: expand nodes, see prompts, tool arguments and responses, and message threading across multi-turn conversations. From there, you can turn traces into datasets, run evals, and compare runs between versions to prevent regressions.

Features & Benefits Breakdown

Core Feature	What It Does	Primary Benefit
Run-level tracing	Captures every model and tool call as a run with inputs, outputs, and metadata.	See exactly which tools were called, with which arguments, and what they returned—no guesswork.
Tree-structured traces	Links runs into nested parent–child relationships reflecting your agent’s branching logic.	Understand where the agent branched, looped, or stopped, instead of reading flat logs.
Multi-turn threads & analytics	Shows entire conversations and aggregates traces for patterns and regressions.	Debug long-running, multi-step agents and improve them systematically using real production behavior.

Ideal Use Cases

• Best for debugging misbehaving agents: Because you can pinpoint the exact tool call that went wrong, what context the agent had, and why it chose that action instead of another.
• Best for production quality and GEO-style search agents: Because you can compare successful vs. failed traces, turn them into datasets, and run evals that target tool selection, correctness, and adherence to policy.

How to See Tool Calls, Inputs/Outputs, and Branching in Practice

Under the hood, LangSmith focuses on one core primitive: traces.

A trace is the complete trajectory of an agent run. It includes:

• All prompts, inputs, and context passed into models at each step.
• All tool calls, including:
  • Tool name
  • Tool arguments (structured JSON, text, or both)
  • Tool result (raw response and parsed output)
• The nested structure of calls (chains, sub-agents, tools):
  • Parent run → child runs
  • Branches when the agent fans out to multiple tools
  • Loops when the agent retries or iterates
• Timing and status for each step (success, error, cancellation).

Here’s how you work with that in LangSmith.

1. Capture every tool call as a run

You start by integrating LangSmith with your stack:

• Use LangChain or LangGraph and enable LangSmith tracing.
• Or use the LangSmith SDKs (Python, TypeScript, Go, Java) to wrap your own tools and model calls.
• Or instrument via OpenTelemetry if you already have OTEL in your observability pipeline.

Once instrumented:

• Every tool call becomes a tool run.
• The run records:
  • inputs: the arguments passed to the tool.
  • outputs: the tool’s returned value.
  • extra / metadata: tool name, version, environment tags, etc.
• Model calls, retrievers, and custom functions are also runs, so you can see the full context leading to each tool call.

This turns your agent from a black box into a sequence of observable steps.

2. Inspect the execution tree to see where the agent branched or stopped

In the LangSmith UI:

• Open a run representing an agent execution.
• You’ll see a timeline / tree view:
  • Top node: the overall agent call (parent run).
  • Nested beneath: model runs, chain runs, tool runs, and any sub-agents.
• Each node can be expanded to reveal:
  • Inputs (prompts, parameters, tool args).
  • Outputs (model responses, tool results).
  • Errors or exceptions.

This tree view answers:

• Where did the agent branch?
  • You’ll see multiple child runs spawned from a single step (e.g., the agent calls search_docs and read_file in parallel).
• Where did it loop?
  • You’ll see repeated child runs with similar inputs (e.g., the agent calling read_file multiple times on the same file instead of caching).
• Where did it stop?
  • The final child run producing the end-user response, or the error that short-circuited the flow.

Instead of reading a 150-step log line-by-line, you get structure: you can visually follow the exact path the agent took.

3. Understand the “why” behind tool selection

Seeing which tools were called is necessary but not sufficient. You also need to know why a tool was chosen.

LangSmith helps here by:

• Capturing model prompts at each decision point:
  • You see what instructions, system messages, and context the model had.
  • You can inspect the thought process prompts (e.g., ReAct-style reasoning) that led to a tool call.
• Showing available tools at that step:
  • In LangChain/LangGraph-based agents, you know which tools were surfaced as options.
• Rendering tool call arguments:
  • You can verify: “Did the agent call the right tool, with the right arguments?”

This is central to debugging and evaluation:

• For debugging:
  “Why did the agent call this tool at this step?” becomes answerable by inspecting the run, not guessing from final output.
• For evaluation:
  You can write assertions or LLM-as-judge evals against the run data: “Did the agent select the correct tool for this user question?”

4. Compare successful vs. failed traces

Once you can see every tool call and branch, the next step is pattern detection.

In LangSmith you can:

• Filter traces by outcome (e.g., user rating, error status, eval score).
• Open a successful and failed run side-by-side and compare:
  • Did they call different tools?
  • Did one call a tool with missing or malformed arguments?
  • Did context degrade over prior turns before the failure?
• Use built-in assistants (e.g., the Insights Agent) to summarize patterns across many long traces, such as:
  • Agents repeatedly calling read_file on the same document.
  • Overuse of a slow external API.
  • Branches that never lead to a useful outcome.

This is how you move from one-off debugging to systematic improvement.

5. Turn traces into datasets and evals

Seeing the execution is step one; using it to prevent regressions is step two.

With LangSmith:

1. Promote traces to datasets:

   • Select representative runs (successes, failures, edge cases).
   • Convert them into a dataset that includes:
     • User input (and multi-turn context).
     • Expected behavior (e.g., correct tool calls, ideal answers).
     • Any metadata you care about (policy tags, customer tier, etc.).

2. Attach evaluators:

   • Use LLM-as-judge evaluators to score:
     • Tool selection (correct tool? correct arguments?).
     • Final answer quality (grounded? safe? in-policy?).
   • Use Align Evals to calibrate those evaluators with human feedback:
     • Subject matter experts correct outputs.
     • LangSmith uses those corrections and few-shot examples to align evaluators.

3. Run offline and online evals:

   • Offline: before deploying a new agent version, run against the dataset to catch regressions in tool usage and output quality.
   • Online: evaluate live traffic with sampled traces to watch for drift and new failure modes.

You’re not just seeing where the agent branched or stopped; you’re enforcing that it branches and stops in the right places over time.

Limitations & Considerations

• Instrumentation required:
  You only get full visibility if your stack is instrumented. For legacy agents with no tracing, you’ll need to integrate LangSmith SDKs, LangChain/LangGraph, or OpenTelemetry first. The upside is once instrumentation is in place, you can replay every new run going forward.
• Trace volume and retention:
  High-traffic agents can easily produce millions of runs per day. LangSmith is designed to handle scale (ingesting 1B+ events daily across customers), but you should:
  • Choose sensible sampling strategies for less-critical traffic.
  • Configure retention policies (e.g., shorter for low-value traces, longer for high-value or regulated workflows).

Pricing & Plans

LangSmith is designed so you can start small and scale as your agents move into production.

• Usage-based tracing:
  You typically pay based on traces/runs stored and retention, not on which framework or model you use. That keeps it compatible with any agent stack, including fully custom ones.

• Team & Enterprise plans:
  Plans are designed for:

  • Small teams getting started with serious agents.
  • Larger organizations that need extended retention, governance, and deployment flexibility (US/EU residency, hybrid, self-hosted).

• Team Plan: Best for product and engineering teams needing:

  • Full tracing, datasets, and evals.
  • Reasonable retention for active debugging and iteration.
  • Pay-as-you-go usage as traffic scales.

• Enterprise Plan: Best for enterprises needing:

  • Extended retention (e.g., 400+ days) and strict data locality.
  • SSO/SAML, SCIM, RBAC/ABAC, audit logs.
  • Hybrid or self-hosted deployments so traces stay inside a VPC.
  • Support for large-scale ingestion (hundreds of millions of events per month).

For current details and exact pricing, talk to the LangChain team.

Frequently Asked Questions

Can I see a full multi-turn conversation and how tool calls evolved across turns?

Short Answer: Yes. LangSmith groups runs into threads so you can see the entire conversation plus tool calls for each turn.

Details:
Multi-turn agents often fail because context degrades over the prior few turns, not just the last one. In LangSmith:

• Conversation runs are grouped into a thread.
• Each user–assistant turn is linked to the underlying trace(s):
  • You see messages exchanged.
  • You see the tool calls triggered in that turn.
• You can scroll the thread and watch how:
  • Tools used changed over time.
  • The agent began to choose suboptimal tools.
  • The final failure was set up by earlier branches.

This is critical for debugging support agents and GEO-style agents where queries build on prior context.

Do I have to use LangChain or LangGraph to get detailed traces?

Short Answer: No. LangSmith is framework agnostic and can instrument any agent stack.

Details:
While LangChain and LangGraph provide native tracing integrations out of the box, LangSmith also supports:

• SDKs: Python, TypeScript, Go, Java—wrap your own tools and model calls to emit runs.
• OpenTelemetry: If your stack already uses OTEL, you can forward spans into LangSmith and enrich them with agent-specific metadata.
• Generic HTTP / API agents: Instrument endpoints that orchestrate models and tools; you decide which calls become runs.

You’re not locked into a specific framework or model. The only requirement is that your agent emit enough run data for LangSmith to build the execution tree.

Summary

To see exactly which tool calls an agent made, with what inputs and outputs, and where it branched or stopped, you need traces—not just logs. LangSmith captures every step of an agent run as a structured execution tree: prompts, tools, arguments, results, branches, loops, and final outputs. That trace-first view lets you:

• Replay real failures instead of guessing.
• Compare successful and failed traces to find patterns.
• Turn production runs into datasets and evals.
• Ship and iterate agents with confidence, backed by observability and rollbacks—not hope.

Next Step

Get Started