Why does our multi-step agent choose the wrong tool or take the wrong branch, and how do we debug it?

Most teams discover tool selection bugs the hard way—after a multi-step agent confidently calls the wrong API, follows the wrong branch, and ships a bad action to production. The language looks fine. The trace passes unit tests. And yet, the agent routed a refund through the “VIP” workflow, or hit web search when it should have called your internal account service. That’s not a copywriting problem; it’s a systems problem in how you evaluate and debug tool behavior.

This explainer walks through why multi-step agents choose the wrong tool or branch, how to instrument them so you can actually see what’s going wrong, and how to turn that visibility into reliable guardrails—not just nicer logs.

---

Quick Answer: Multi-step agents choose the wrong tool or branch because you’re optimizing for response quality instead of decision quality, and you lack evaluators that understand sessions, traces, and tool actions. Galileo fixes this by giving you end-to-end evaluation, Signals to detect failure patterns, and Protect to turn those evaluations into real-time guardrails on tool selection and branch decisions.

The Quick Overview

• What It Is: A Galileo-powered workflow to detect, debug, and prevent wrong tool choices and incorrect branches in multi-step agents.
• Who It Is For: Teams running production agents or RAG systems that orchestrate multiple tools, branches, and workflows—and need them to behave reliably under real user load.
• Core Problem Solved: You stop “flying blind” on tool behavior. Instead of guessing why an agent picked the wrong tool or path, you get trace-level visibility, specialized evaluators, and guardrails that intercept bad decisions before they execute.

How It Works

At a high level, you solve wrong-tool and wrong-branch behavior by treating tool selection as a first-class evaluation target—not an afterthought hidden behind a final answer.

Galileo’s approach:

1. Instrument the full agent journey (sessions → traces → spans):
   Capture every decision step: the user input, reasoning, tools considered, tool selected, parameters passed, and resulting outputs. This is the raw material for debugging wrong branches.

2. Evaluate decision quality, not just text quality:
   Use Galileo’s Evaluation Engine to score tool selection and usage: Was the right tool chosen? Were parameters correct and well-formed? Did the agent follow the correct branch given policy and context?

3. Convert evals into guardrails on live traffic:
   Distill evaluators into Luna / Luna‑2 models, run them at low latency across 100% of traffic via Protect, and attach concrete actions—block, redact, override, or trigger a webhook—when an agent tries to take a bad action or follow the wrong branch.

1. Instrument: See the agent’s decision path, not just the output

Multi-step agents are defined by their traces: multi-hop reasoning, tool calls, and branches between tools and workflows. If you only see the final answer, you’ll never understand why they chose the wrong tool.

With Galileo, you:

• Capture sessions (user journeys), broken into:
• Traces (end-to-end tasks or conversations), composed of:
• Spans (individual steps like model calls, tool calls, and branch decisions).

For each span, you log:

• Model prompt and response
• Candidate tools and tool selected
• Tool parameters (including malformed or missing fields)
• Route/branch taken (e.g., “billing_flow” vs “support_flow”)
• Latency and cost per span

This lets you answer questions like:

• “Why did the agent call web search instead of get_account_details?”
• “Why did it go down the escalation branch when the user was clearly self-serve?”
• “When it did pick the right tool, what parameters did it pass?”

2. Evaluate: Measure tool selection and branch quality

Most evaluation setups stop at “Did the final answer look correct?” That’s how you end up with an agent that writes beautiful explanations and still hits the wrong APIs.

Galileo’s Evaluation Engine focuses on the decision layer:

• Tool selection quality:
  • Did the agent choose the appropriate tool given the user goal and available context?
  • Did it avoid unnecessary or dangerous tools (e.g., mutation APIs when only read is needed)?
• Tool usage quality:
  • Were parameters correct, complete, and well-formatted?
  • Did the agent sequence tools correctly (e.g., authenticate → fetch → update)?
• Branch decision quality:
  • Did the agent follow the correct business or policy branch?
  • Did it respect guardrail policies (e.g., “no refunds > $500 without human review”)?

Galileo gives you 20+ out-of-the-box evaluators for agents, RAG, safety, and security, plus the ability to define custom evaluators in plain language—then automatically generate LLM-as-judge evaluators from those descriptions. You can refine them with few-shot examples and SME annotations so they reflect your actual workflows, not some generic benchmark.

3. Guardrail: Enforce correct tool and branch behavior in production

Debugging in dev is necessary, but it doesn’t stop failures in production. The real shift is turning your evaluations into guardrails that actively control what the agent can do.

With Galileo Protect:

• Evaluators are distilled into Luna / Luna‑2 small language models.
• These run on a purpose-built inference stack so you can:
  • Score every tool call and branch decision with sub‑200ms latency.
  • Cover 100% of production traffic at up to 10,000+ requests/min.
  • Do it at 97% lower cost than heavyweight LLM judges.

When a decision violates a guardrail metric—wrong tool, unsafe tool, wrong branch, parameter risk—Protect:

• Intercepts the action before it hits your systems.
• Triggers explicit actions:
  • Block the tool call entirely.
  • Redact or sanitize parameters.
  • Override with a safer alternative (e.g., switch to read-only tool).
  • Fire a webhook to your own orchestration or incident system.
• Allows versioning and rollbacks of policies without redeploying code.

This closes the loop: offline evals of tool behavior become production guardrails that prevent wrong tools and wrong branches from ever reaching your backend or users.

Features & Benefits Breakdown

Core Feature	What It Does	Primary Benefit
Evaluation Engine for Tool & Branch Quality	Scores tool selection, parameter usage, and branch decisions across traces.	Reveals why agents choose the wrong tools or branches instead of just telling you the final answer was “bad.”
Signals for Unknown Failure Patterns	Analyzes 100% of production traces to surface emerging patterns like recurring wrong-tool usage or branch drift.	You catch new tool and routing issues after the first signal, not after thousands of bad actions.
Protect Guardrails with Luna‑2	Distills evaluators into compact models that run sub‑200ms and act as a real-time firewall on tool calls and branches.	Turns your best evaluation logic into always-on production guardrails with low latency and low cost.

Ideal Use Cases

• Best for multi-tool customer support and ops agents:
  Because they often have to choose between similar tools (e.g., “refund,” “credit,” “discount”) and follow strict escalation branches. Galileo enforces the correct path and catches wrong-tool calls before they hit billing or CRM systems.

• Best for complex internal agents with high-risk tools:
  Because they have access to mutation APIs (e.g., provisioning, access control, database writes). Galileo adds a safety layer so tool selection and branch decisions are evaluated and gated against your policies in real time.

Why multi-step agents choose the wrong tool or branch

Let’s name the common failure modes you’re likely seeing in agents tied to the slug why-does-our-multi-step-agent-choose-the-wrong-tool-or-take-the-wrong-branch-and:

1. Prompt-only steering with no explicit tool policy
   You rely on vague instructions like “Use the best tool to answer the question” and hope the model infers your business rules. It won’t. Without explicit tooling constraints and feedback, the agent learns to optimize for helpful-sounding language, not safe routing.

2. Generic evaluators that don’t understand your domain
   Off-the-shelf “Correctness” or “Helpfulness” scores don’t know the difference between get_transactions and get_statements, or when a refund should trigger an approval branch. So you think the agent is “good” because the answer sounds right, but it’s hitting the wrong systems.

3. No ground truth on tool and branch decisions
   You might have labeled data for “good answers,” but not for “correct sequence of tools and branches.” Without ground truth on the decision path, you can’t train or calibrate evaluators specific to your workflows.

4. Lack of visibility into parameter-level failures
   Even when the agent picks the right tool, it might pass malformed or incomplete parameters:

   • Missing required IDs
   • Wrong data types (string vs numeric)
   • Overly broad filters that leak extra data
     These don’t show up in text-only evals; they live inside spans.

5. Policy drift and environment changes
   Business rules change (“no refunds on this SKU”, “new VIP tier”), but your prompts and evaluators don’t keep up. The agent keeps following an outdated branch, and nobody notices until a compliance review.

6. You can’t afford to run heavyweight judges on live traffic
   You may have a powerful LLM-as-judge in offline experiments, but can’t run it on every production decision. That’s how you end up evaluating 1% of traffic and missing the other 99% where the wrong tool gets called.

How to debug wrong tools and wrong branches with Galileo

Here’s a concrete workflow that matches the question in the slug—why your multi-step agent chooses the wrong tool or branch, and how to debug it end-to-end.

Step 1: Capture traces with tool and branch spans

• Integrate Galileo’s SDK or tracing into your agent framework.
• Ensure every tool call and branch decision is a span with:
  • The tool name and category.
  • Input parameters.
  • Any reasoning from the model (chain-of-thought stored privately).
  • Outputs and downstream effects.

Now you can walk a single user journey and see exactly where the wrong tool or branch was selected.

Step 2: Build ground truth for tool behavior

You don’t need a million labels; you need representative ones:

• Take synthetic scenarios, dev runs, and real production traces.
• Ask SMEs to annotate:
  • What tool should have been used at each decision point.
  • Whether the branch taken was correct (e.g., “self‑serve”, “human escalation”).
  • Whether parameters were valid/safe.

These annotations become your evaluation asset—a living reference for how the agent should behave.

Step 3: Configure evaluators for tool and branch quality

Using Galileo’s Evaluation Engine:

• Start with out-of-the-box agent evaluators (tool selection, parameter formatting, safety).
• Add custom evaluators your domain requires, e.g.:
  • “Was refund_tool used only when refund conditions are met?”
  • “Did the agent ever bypass the ‘fraud_check’ branch when it should not?”
• Let Galileo generate LLM-as-judge evaluators from natural language descriptions.
• Tighten them with CLHF: add few-shot examples pulled from real failures, then retrain so the evaluator matches your production reality.

Now you have quantifiable metrics like:

• tool_selection_appropriateness
• branch_policy_compliance
• parameter_completeness

Step 4: Use Signals to find recurrent patterns

Once evaluators are running on traces:

• Signals surfaces patterns you didn’t know to look for:
  • A spike in wrong-tool usage after a prompt update.
  • A specific route (e.g., “password_reset_flow”) frequently taking the wrong branch.
  • Consistent parameter formatting issues tied to one tool or model version.

For each signal, you can:

• Drill into representative traces.
• Understand the failure mode in context.
• Promote that pattern into a dedicated evaluator so it becomes a reusable check.

This is how you move from “Why did it do that once?” to “How do we prevent this pattern across all traffic?”

Step 5: Turn evaluators into production guardrails with Protect

Finally, you ship reliability:

• Distill your most important evaluators into Luna / Luna‑2 guardrail models.
• Deploy them with Protect in front of your agent’s tool/router layer.
• Define concrete actions when a guardrail is tripped:
  • If tool_selection_appropriateness fails → block call and return an error-safe response.
  • If branch_policy_compliance fails → override branch to a safe route or escalate to a human.
  • If parameters are unsafe (e.g., possible PII or over-broad query) → redact or downgrade the tool to read-only.

Because Luna‑2 is optimized for evaluation, you can run this on every single decision with sub‑200ms latency and at a fraction of the cost of an LLM judge. That gives you something most teams don’t have: continuous enforcement, not occasional sampling.

Limitations & Considerations

• You still need domain expertise:
  Galileo accelerates evaluation, but it can’t invent your business logic. You need SMEs to define what “correct tool” and “correct branch” mean in your domain and to review edge cases.

• Bad orchestration can still limit results:
  If your underlying agent framework doesn’t expose tool and branch decisions as spans, you’ll have gaps in visibility. Plan for traces from day one—sessions → traces → spans is the backbone of meaningful debugging.

Pricing & Plans

Galileo is built for teams who need production reliability, not just a lab demo. While specific pricing depends on volume, deployment model, and support level, the structure is straightforward:

• Evaluation Engine and Signals scale with traces/month (e.g., 5,000 traces/month to millions).
• Protect guardrails and Luna‑2 usage scale with requests/min and latency/cost constraints you define.
• Enterprise deployments can run SaaS, VPC, or on‑prem and include SOC 2 Type II posture and HIPAA-capable infrastructure with BAAs.

Example packaging:

• Growth Plan: Best for product teams and startups needing to get one or two multi-step agents safely into production, with full Evaluate + Signals coverage and an initial set of Protect guardrails.
• Enterprise Plan: Best for larger organizations needing multi-environment deployment (SaaS/VPC/on‑prem), 100% traffic coverage with Luna‑2, advanced governance, SSO, and tight integration with existing security/compliance workflows.

For details tailored to your traffic, latency budget, and infra requirements, contact the Galileo team.

Frequently Asked Questions

How do I tell if the wrong tool or branch is the agent’s fault or the orchestrator’s?

Short Answer: Instrument both the model’s reasoning and the orchestrator’s routing decisions, then evaluate them separately.

Details:
You need to split responsibility across spans:

• Model span: Shows what the LLM recommended—what tool or branch it “wanted” to use.
• Orchestrator span: Shows which tool or branch actually ran, based on routing logic.

Using Galileo, you can:

• Evaluate the model span for “tool recommendation correctness.”
• Evaluate the orchestrator span for “routing correctness.”
• Compare the two: if the model recommended the right tool but the orchestrator overrode it incorrectly, you fix routing; if the model recommendation was wrong, you fix prompts, models, or tool descriptions.

This avoids the common trap of blaming the model for bugs that actually live in glue code.

Can I use my own evaluators or models for checking tool usage?

Short Answer: Yes. You can bring custom evaluators and have Galileo distill them into Luna‑based guardrails.

Details:
If you already have internal evaluators or LLM-as-judge prompts:

• Define them as custom evaluators in Galileo’s Evaluation Engine.
• Use your own models or let Galileo host and orchestrate them.
• As you validate their performance with live data and SME feedback, Galileo can:
  • Distill their behavior into compact Luna / Luna‑2 models.
  • Serve those models in Protect so they run across all production traffic with low latency and cost.
• You retain your domain logic while offloading infrastructure, optimization, and scaling.

That way, you’re not forced into generic evaluators; you upgrade your own into production-grade guardrails.

Summary

When your multi-step agent chooses the wrong tool or takes the wrong branch, the root cause is nearly always the same: you aren’t measuring decision quality along the trace, and you can’t afford to run your best evaluators on live traffic. You’re flying blind on the exact layer—tools, parameters, branches—where the most expensive failures occur.

Galileo fixes this by turning agent behavior into a measurable system:

• Instrument sessions → traces → spans so tool calls and branches are first-class citizens.
• Use Evaluation Engine and Signals to score tool selection, parameter quality, and branch decisions, and to detect emerging failure patterns.
• Distill those evaluators into Luna‑powered guardrails with Protect, so bad decisions are intercepted with explicit actions before they hit your systems.

If you can run your best evaluators continuously in production, you don’t just have observability. You have reliable agents.

Next Step

Get Started