Security says we need auditability for AI actions (who approved what, what the agent did, when)—what should we implement?

Security and risk teams are right to demand auditability for AI actions. Once agents can call tools, move money, touch customer data, or update systems, you need the same—or higher—level of traceability you expect from any production system: who did what, when, with whose approval, and based on which inputs.

Quick Answer: You should implement trace-first observability for all AI actions, plus explicit approval workflows and audit logs tied to identities, tools, and timestamps. With LangSmith and Fleet/Agent Builder, you get end‑to‑end traces, human‑in‑the‑loop approvals, and enterprise auditability for every agent decision.

---

The Quick Overview

• What It Is: A stack of tracing, approvals, and audit logging that records every AI agent action—from model prompts to tool calls to human approvals—in a structured, queryable way.
• Who It Is For: Security, compliance, and engineering teams running AI agents in production who need to prove control over agent behavior and meet internal or external audit requirements.
• Core Problem Solved: AI agents act autonomously and non‑deterministically; without trace‑level observability and approvals, you can’t explain or reconstruct what happened when something goes wrong.

---

How It Works

The core idea is simple: treat traces as the source of truth for agent behavior, then layer approvals and audit logs on top. Instead of only logging final responses, you log the full run timeline: prompts, tool calls, intermediate decisions, and human interventions—linked to identities and timestamps.

In LangSmith and Fleet/Agent Builder, that looks like:

1. Instrument and trace every run:

   • Use LangSmith SDKs (Python, TypeScript, Go, Java) or OpenTelemetry to send traces for all agent runs.
   • Each trace captures prompts, responses, tool calls, parameters, and errors in order, with timestamps.
   • Message threads capture multi‑turn conversations so you can replay what led to any action.

2. Require approvals for sensitive actions:

   • Mark certain tools (e.g., “wire_funds”, “delete_user”, “send_email_to_customers”) as approval‑required.
   • Fleet/Agent Builder routes these actions to humans with clear context: what the agent wants to do, why, and what data it used.
   • Approvers can accept, modify, or reject actions; the decision is saved back into the trace.

3. Log and surface audit events:

   • Every action, tool call, and approval decision generates structured events: who requested, who approved/rejected, when, and what changed.
   • Security and compliance can search, filter, and export logs (by user, tool, time range, outcome) to satisfy audits or investigations.
   • Admin features like SSO/SAML, SCIM, RBAC/ABAC, and audit logs ensure changes to configurations, tools, and policies are also tracked.

This gives you a replayable, end‑to‑end history of your AI agents: exactly what happened, in what order, who approved it, and how the model arrived at that behavior.

---

Features & Benefits Breakdown

Core Feature	What It Does	Primary Benefit
Trace‑First Observability	Captures complete run timelines: prompts, tool calls, intermediate outputs, errors, and final responses.	Lets security, risk, and engineering teams replay any AI action and see exactly what the agent did and why.
Human‑in‑the‑Loop Approvals	Requires explicit approval on sensitive tools and actions, recording who approved/denied and when.	Prevents uncontrolled autonomy and gives auditors a clear record of who authorized what.
Enterprise Audit Logging	Logs every agent action, configuration change, and approval with identity, timestamp, and context.	Satisfies auditability requirements and accelerates investigations, post‑mortems, and compliance reviews.

---

Ideal Use Cases

• Best for regulated workflows (finance, healthcare, legal, insurance): Because you can prove that sensitive actions (payments, PHI access, contract edits) were either approved by a human or blocked—with full traces and identity‑linked logs for every step.
• Best for internal operations and support automation: Because when an agent misroutes a case, updates a record, or sends a message, you can reconstruct the decision path, compare successful vs failed runs, and feed corrections back into evaluation and improvement loops.

---

Limitations & Considerations

• You still need clear policies and tool design: Auditability doesn’t replace governance. You should define which actions require approvals, which tools are read‑only vs write‑capable, and what “good behavior” looks like in your domain. LangSmith will show you what happened; your policies must define what is allowed.
• Tracing adds observability overhead: You’re collecting and retaining a lot of behavioral data. For most teams this is the point—you can’t debug or audit what you don’t capture—but you’ll want to configure retention (e.g., 14‑day vs 400‑day), sampling, and data residency (US/EU, hybrid, or self‑hosted) to match your risk posture and cost constraints.

---

Pricing & Plans

LangChain is built for teams that are serious about agents but still want flexible costs: seat‑based plans for builders and pay‑as‑you‑go usage for traces.

At a high level:

• You pay for seats (how many people build, observe, and manage agents).
• You pay for usage (traces/events, storage, evaluations).
• Enterprise plans add longer retention (e.g., 400‑day), dedicated regions, hybrid/self‑hosted options, and advanced security controls.

Examples:

• Team / Growth Plan: Best for product and platform teams needing trace‑first observability, evaluation, and approvals to ship their first production agents. Includes LangSmith tracing, dashboards, basic retention, and Fleet/Agent Builder so non‑technical users can run agents under admin control.
• Enterprise Plan: Best for organizations with strict security and compliance requirements, needing long‑term auditability, data residency, and private networking. Adds extended retention, VPC/hybrid or self‑hosted deployment, SSO/SAML, SCIM, RBAC/ABAC, audit logs, and support for routing traces at scale (1B+ events/day).

For exact pricing, volume discounts, and deployment options, talk to the LangChain team.

---

Frequently Asked Questions

How do we prove “who approved what” for AI actions?

Short Answer: Use human‑in‑the‑loop approvals tied to your identity provider, and log every decision directly into the agent trace and audit log.

Details:
With Fleet/Agent Builder and LangSmith:

• Sensitive tools are marked as approval‑required.
• When an agent attempts one of these actions, it triggers an approval workflow instead of executing immediately.
• The approval UI shows:
  • The conversation or request that triggered the action.
  • The agent’s reasoning (where available).
  • The exact tool, parameters, and proposed changes.
• Approvers authenticate via SSO/SAML, so decisions are linked to real corporate identities.
• Approve/deny decisions—and any edits—are written into:
  • The trace timeline (so you can replay the full sequence).
  • The audit log (so you can filter by approver, action, or time).

Auditors can then ask and answer questions like “Who approved outbound payment #12345 on March 7?” directly from the data, without reverse‑engineering logs.

---

How do we capture “what the agent did and when” in a way security will accept?

Short Answer: Instrument every agent with LangSmith tracing so you record complete run timelines and convert them into structured audit logs.

Details:
LangSmith assumes that you can’t secure what you can’t replay. To give security teams acceptable visibility, you:

1. Instrument your stack:

   • Use LangSmith SDKs (Python, TypeScript, Go, Java) or OpenTelemetry to capture all runs.
   • Integrate with any agent framework—LangChain, LangGraph, custom code—without lock‑in.

2. Capture rich traces:

   • For each run, you log:
     • Caller identity (user, system, service).
     • Input messages and context.
     • Model prompts and responses.
     • Tool calls with parameters and results.
     • Branching logic: retries, loops, fallbacks.
     • Errors, timeouts, and cancellations.
   • Multi‑turn threads are preserved as message sequences, so you can see exactly what led to each action.

3. Expose to security and compliance:

   • Dashboards and search let teams filter by user, tool, status, or time range.
   • Annotation queues let subject‑matter experts review risky or random samples of traces and flag policy issues.
   • Align Evals lets you calibrate automated evaluators based on human feedback, so you can measure policy adherence systematically.

For security, this looks like traditional auditability: every action has a timestamp, an identity, a context, and a replayable record of how the agent got there.

---

Summary

If security is asking for auditability around AI actions—who approved what, what the agent did, and when—the answer is not just “better logs.” You need trace‑first observability plus explicit approval and audit workflows:

• Traces as the record of behavior: Capture full run timelines so you can replay and explain any agent decision.
• Human‑in‑the‑loop for sensitive tools: Require approval before high‑impact actions and log every decision with identity and timestamps.
• Enterprise‑grade auditability: Keep structured audit logs, RBAC/ABAC, and deployment options (US/EU, hybrid, self‑hosted) that match your risk profile.

LangSmith and Fleet/Agent Builder give you this end‑to‑end: from tracing and evaluation to approval and deployment, so you can ship agents that work—and satisfy security—without giving up autonomy or velocity.

---

Next Step

Get Started