How do we set Galileo guardrails to block prompt injection or data exfiltration before a tool executes?

Quick Answer: Use Galileo Protect guardrails to score every input and intermediate agent step for prompt injection and data exfiltration, then attach deterministic actions (block, redact, override, or webhook) that fire before any tool call is executed.

The Quick Overview

• What It Is: A runtime guardrail flow in Galileo Protect that intercepts and scores prompts, tool arguments, and model outputs for injection and exfiltration risk, and blocks bad actions before they hit your tools or data.
• Who It Is For: Teams shipping agents or RAG systems into production who can’t afford “execute first, review later” behavior—especially in regulated or sensitive data environments.
• Core Problem Solved: Preventing attackers or misaligned prompts from hijacking an agent, exfiltrating internal data, or bypassing system policies before a tool action runs.

How It Works

In most agent stacks, tool calls are the blast radius. If a prompt injection slips through at the wrong time, your agent might run a dangerous query, leak confidential data, or overwrite something critical—long before you see a log.

Galileo flips that pattern by inserting guardrails directly in the tool execution path. Protect, powered by Luna-2 evaluation models, scores each input and intermediate step against prompt injection and data exfiltration metrics in under ~200ms, then applies deterministic policies to stop risky actions before tools fire.

At a high level:

1. Instrument traces and tool calls: Your agent sends sessions → traces → spans (including tool invocations and arguments) to Galileo.
2. Evaluate for injection/exfil risk with Protect: Each prompt, tool call, and output is scored by specialized evaluators (e.g., prompt injection risk, data exfiltration risk, PII leak risk).
3. Enforce guardrail policies pre-execution: Based on scores and conditions, Protect intercepts risky calls and applies actions: block, redact, override with a safe response, or trigger a webhook/escalation.

1. Connect your agent and expose tool spans

First, make the agent’s decision-making visible so Protect can act in time:

• Send sessions → traces → spans to Galileo, where:
  • A session = one end-to-end user interaction.
  • A trace = the agent’s multi-step reasoning for that session.
  • A span = individual steps like “model completion,” “tool selection,” or “tool execution.”
• Ensure your instrumentation includes:
  • The user prompt and system/instruction prompts.
  • The model output that chooses a tool.
  • The tool name and arguments before the tool runs.
• For RAG, also capture:
  • User query.
  • Retrieved documents or chunks.
  • Final answer.

This trace structure lets Galileo see not just what the agent said, but which tool it’s about to call, with what parameters, and in what context.

2. Turn evals into injection/exfiltration guardrails

Galileo’s core pattern is eval-to-guardrail. You don’t just “monitor” injection/exfiltration—you define how to detect it, prove the evaluator works offline, then promote it into a live guardrail.

The workflow:

1. Define or select evaluators in Evaluate:

   • Use Galileo’s Evaluation Engine with:
     • Out-of-the-box safety/security evaluators (e.g., prompt injection, sensitive information exposure).
     • Custom evaluators, including LLM-as-judge from a written description (e.g., “Flag any attempt to override system instructions, request raw database access, or export entire datasets.”).
   • Evaluate against:
     • Synthetic test sets (e.g., known injection patterns).
     • Your dev/staging traces.
     • Early production traffic samples.

2. Refine with SMEs and CLHF:

   • Add subject matter expert annotations on ambiguous prompts (e.g., “Is this an allowed admin query or an exfil attempt?”).
   • Use these examples to calibrate or few-shot improve the evaluator, increasing precision for your domain.

3. Distill to Luna-2 for production:

   • Galileo distills your evaluators into compact Luna / Luna-2 models for:
     • Low-latency (< ~200ms).
     • Low-cost, 97% lower than heavyweight LLM-judges at 100% traffic coverage.
   • Now the same evaluator that passed your offline tests runs on every live trace.

3. Configure Protect policies to intercept tools before execution

Once evaluators are in place, Protect becomes your firewall. You define explicit rules that operate on evaluator scores, trace fields, and tool metadata.

Common pattern:

1. Create a Protect policy for injection/exfiltration:

   • Target: “All tool calls” or a subset of critical tools (e.g., run_sql_query, export_s3_object, send_email).
   • Scope: specific environments or projects (e.g., prod).

2. Attach evaluators and thresholds:

   • Example evaluators:
     • prompt_injection_risk
     • data_exfiltration_risk
     • sensitive_data_leak
   • Example thresholds:
     • If prompt_injection_risk >= 0.8
     • OR data_exfiltration_risk >= 0.7
     • THEN apply guardrail.

3. Define deterministic actions on violation:

   • Block the tool call:
     • Prevents the tool from executing at all.
   • Override the model response:
     • Return a safe, policy-aligned message (e.g., “I can’t run that operation due to security policies.”).
   • Redact sensitive fields:
     • Strip identifiers or data from tool arguments before allowed calls execute.
   • Webhook / escalate:
     • Notify security/ops or send to a human reviewer.
     • Optionally, log a higher-severity incident.

4. Return structured decisions to your agent:

   • Protect responds with:
     • Allow/block decision.
     • Optionally, a replacement response.
   • Your agent’s runtime treats Protect as a decision oracle in the tool path:
     • If blocked → skip tool execution, show override to user.
     • If allowed → proceed with the tool call as normal.

Features & Benefits Breakdown

Core Feature	What It Does	Primary Benefit
Luna-2–powered injection/exfil evaluators	Scores prompts, tool calls, and outputs for prompt injection and data exfiltration risk at low latency.	Enables always-on protection at 100% traffic coverage without paying LLM-as-judge costs.
Protect guardrail policies	Lets you define rules that combine evaluator scores, tool names, and context, then execute actions (block, redact, override, webhook).	Turns offline evals into live guardrails that stop risky tool actions before they execute.
Trace-level observability (sessions → traces → spans)	Captures the full agent workflow, including tool selection, arguments, and final responses.	Makes it possible to detect both obvious attacks and subtle policy drift in multi-step agent flows.

Ideal Use Cases

• Best for agentic systems with powerful tools: Because you can intercept risky tool calls like run_arbitrary_code, run_sql_query, or export_data before they execute, and block or route them safely.
• Best for RAG systems over sensitive corpora: Because Protect can detect attempts to exfiltrate large swaths of internal documents or bypass document-level access controls, and redact or override those responses in real time.

Limitations & Considerations

• Guardrail coverage depends on instrumentation: If tool calls and arguments aren’t exposed as spans in your traces, Protect can’t intercept them. Make sure you send the pre-execution tool metadata to Galileo.
• Evaluator fit is domain-specific: Out-of-box evaluators are strong baselines, but high-stakes environments should calibrate and improve them with SME annotations and CLHF to avoid over-blocking or missing domain-specific attacks.

Pricing & Plans

Galileo is priced to support teams from early-stage pilots to enterprise-scale deployment, with options across SaaS, VPC, and on-prem.

• Growth / Team plans: Best for product teams and startups needing eval-to-guardrail workflows, core Protect guardrails, and coverage for thousands of traces per month as they harden their first RAG/agent apps.
• Enterprise plans: Best for larger organizations needing custom deployment (VPC/on-prem), SOC 2 Type II and HIPAA/BAA support, dedicated Luna-2 inference capacity, and high-throughput coverage (e.g., 10,000+ requests/min) across multiple products.

(For exact pricing and throughput tiers, talk to Galileo’s team.)

Frequently Asked Questions

Can Galileo block a prompt injection before my agent calls a tool?

Short Answer: Yes. Protect can intercept the agent’s chosen tool call, score it for injection risk, and block or override it before the tool runs.

Details:
Once you instrument your agent’s tool spans, you configure a Protect policy that:

1. Evaluates the prompt and tool call with a prompt_injection evaluator.
2. Checks the score against your threshold.
3. If it’s above threshold:
   • Returns a “block” decision to your runtime.
   • Optionally includes a safe override message to show the user.
4. Your agent runtime respects this decision and skips the tool execution.

Because these evaluators are served via Luna-2, they run at low latency and can be applied to 100% of traffic, not just sampled logs.

How does Galileo detect data exfiltration attempts versus normal queries?

Short Answer: By combining specialized evaluators with your domain context and tools, then calibrating on real examples until the model reliably distinguishes legitimate access from exfil patterns.

Details:
Out of the box, Galileo’s Evaluation Engine includes evaluators for sensitive data exposure and risky outputs. You can extend these with custom criteria like:

• “Requests to export entire tables or document sets.”
• “Repeated attempts to enumerate internal services or S3 buckets.”
• “Queries that combine user-level data with broad aggregation in unusual ways.”

You:

1. Label examples from your own traffic as “legitimate” vs “exfil-like.”
2. Use these to tune the evaluator via CLHF and few-shot patterns.
3. Distill to Luna-2 and attach to Protect policies.
4. Set actions (block/redact/override) when exfiltration risk crosses your threshold.

Over time, Signals can surface new suspicious patterns; you can then promote those patterns into new evaluators and guardrail rules, tightening protection without adding glue-code.

Summary

If you let tools execute first and review logs later, you don’t have security—you have a backlog of incidents waiting to happen. Galileo closes that gap by turning your best injection and exfiltration evaluators into real-time guardrails via Protect. Every prompt, tool call, and output is scored by Luna-2, and policy-driven actions block, redact, or override risky behavior before any tool executes. The result is agents and RAG systems that can use powerful tools without giving attackers the keys to your data.

Next Step

Get Started