What production metrics should we track for an LLM app (hallucinations, escalation rate, latency, cost per conversation)?

LLMs are probabilistic. That’s the root problem in production: the same prompt can behave differently from one request to the next. If you’re not instrumenting and tracking the right metrics, you don’t have a product—you have a demo with logs.

In this guide, I’ll walk through the production metrics that actually matter for an LLM app—hallucinations, escalation rate, latency, cost per conversation, and a few others that teams often miss—plus how to wire them into a real evaluation and monitoring loop.

---

The Quick Overview

• What It Is: A practical metric framework for running LLM apps in production with predictable quality and cost.
• Who It Is For: Teams shipping RAG chatbots, voice agents, summarizers, and tool-using agents who need to move beyond “it seems good” into measurable reliability.
• Core Problem Solved: Turning fuzzy LLM behaviors (hallucinations, bad escalations, slow responses) into deterministic, trackable signals you can improve over time.

---

The Core Production Metrics for LLM Apps

You can group production metrics for LLM apps into five buckets:

1. Quality & correctness
2. Safety & policy adherence
3. User & escalation outcomes
4. Latency & reliability
5. Cost & efficiency

You need coverage across all five to keep an app both accurate and viable at scale.

---

1. Quality & Correctness Metrics

1.1 Hallucination Rate

What it is:
How often the model produces factually incorrect, fabricated, or unsupported content.

Why it matters:
For RAG systems, support copilots, finance, healthcare, or legal use cases, hallucinations are the fastest path to user mistrust and real-world risk.

How to define it:

• Binary metric:
  • hallucination = 1 if the answer includes claims not supported by ground truth (docs, APIs, or domain rules).
  • hallucination = 0 otherwise.
• Hallucination rate:
  • hallucination_rate = (# of responses with hallucination) / (total responses)

How to measure it in practice:

• Use synthetic datasets and scenarios where you know the expected answer or can at least bound it.
• Run LLM-as-judge evaluations with prompts that compare the answer to:
  • RAG context passages
  • API responses
  • Known reference answers
• Have an explicit rubric like:
  • 0 = fully grounded
  • 1 = minor unsupported detail, not harmful
  • 2 = major unsupported or conflicting claim

In Future AGI, this is a standard eval metric you’d attach in the Evaluate stage—on both pre-production datasets and live traces sampled from production.

---

1.2 Task Success Rate (or Goal Completion Rate)

What it is:
Percentage of conversations or tasks where the user’s goal is achieved without human intervention.

Why it matters:
Hallucination rate tells you “how wrong.” Success rate tells you “how useful.”

Examples:

• For a support bot: issue resolved without ticket escalation.
• For a sales copilot: correct quote generated and sent.
• For a research assistant: correct summary and citations produced.

Formula:

task_success_rate = (# successful_sessions) / (total_sessions)

Success can be labeled by:

• Explicit user feedback (“Did this answer your question?”)
• Downstream events (refund processed, ticket closed)
• Offline evaluations on traces (agents or humans validating correctness)

---

1.3 Response Quality (Relevance, Completeness, Structure)

For many apps, you’ll want more nuance than “success/fail”:

• Relevance: Did the answer actually address the user’s query?
• Completeness: Did it cover all required parts?
• Structure & format: Did it follow the schema, tone, or template you need (e.g., JSON, step-by-step instructions)?

These are typically scored via:

• Deterministic evals on synthetic datasets
• LLM-as-judge with structured rubrics (1–5 or 0–1)
• Custom metrics for your domain (e.g., “percentage of required fields present”)

You can aggregate to:

• avg_relevance_score
• avg_completeness_score
• format_compliance_rate

---

2. Safety & Policy Metrics

Safety isn’t a vibe. It’s a set of measurable categories and enforcement points.

2.1 Safety Violation Rate

What it is:
Percentage of inputs/outputs that trigger a safety policy violation.

Common categories:

• Toxicity / hate / harassment
• Sexism / protected characteristics
• Self-harm / violence content
• Privacy violations (PII exposure)
• Financial or health advice policy breaches
• Prompt injection or jailbreak attempts

Metrics:

• safety_violation_rate = (# of unsafe_events) / (total_events)
• Category breakdown (e.g., % toxicity, % privacy)

With something like Future AGI’s Monitor & Protect, you:

• Screen both inputs and outputs in real time
• Attach multimodal safety metrics (text, image, audio, video)
• Configure blocking thresholds and log every block for audit

---

2.2 Input Attack Rate (Prompt Injection / Jailbreak Attempts)

What it is:
The rate at which users (or adversaries) attempt prompt injection, data exfiltration, or jailbreaks.

Why it matters:
It’s a leading indicator of risk and a key signal for tightening prompts, tools, and safety policies.

Metrics:

• attack_attempt_rate = (# inputs flagged as injection/jailbreak) / (total_inputs)
• blocked_attack_rate = (# blocked attacks) / (# detected attacks)

---

3. User & Escalation Metrics

You mentioned escalation rate explicitly. That’s central for any support or workflow automation use case.

3.1 Escalation Rate

What it is:
The proportion of sessions that require handoff to a human or another system.

Types of escalations:

• Failure-based: model can’t answer or is low-confidence
• Policy-based: request is out-of-scope or restricted
• User-requested: user explicitly asks for a human

Formula:

escalation_rate = (# sessions escalated) / (total_sessions)

You should segment by:

• Intent or topic
• Model version / workflow variant
• User segment or channel (web, voice, chat)

Why it matters:

• Too low may mean the model is overconfident and not escalating when it should.
• Too high may kill your ROI and signal poor grounding or prompt design.

The real value comes when you connect escalation events to traces and your evaluation stack, so you can see why escalations happen and fix root causes.

---

3.2 Deflection Rate

For support bots and agents:

deflection_rate = 1 - escalation_rate

This is your direct cost and capacity lever. Track:

• Deflected tickets per day/week
• Deflection by topic to see where the bot is actually helping

---

3.3 User Satisfaction (CSAT/NPS/Thumbs Up–Down)

These are classic UX metrics but crucial in LLM apps:

• Per-message or per-session thumbs up/down
• Short in-flow CSAT (“Was this helpful?”)
• Task-specific ratings (e.g., “Rate this summary 1–5”)

The key is to:

• Log feedback on the same trace as the LLM steps, tools, and context
• Use it downstream in datasets and Improve loops to train better prompts or workflows

---

4. Latency & Reliability Metrics

LLM performance is not just about quality; it’s about speed and stability.

4.1 Time to First Token (TTFT) & Total Latency

What it is:

• TTFT: time from request to the first byte/token of model output
• Total latency: time from request to final answer (including tool calls, RAG, safety checks)

Metrics:

• p50, p90, p95 latency (overall and per step)
• Latency breakdown:
  • Model inference
  • Retrieval / database
  • Tool/API calls
  • Guardrail/safety checks
  • Orchestration overhead

Why it matters:

• Voice agents and chat UIs are extremely sensitive to TTFT.
• Guardrails and monitoring must run with minimal latency or they become the bottleneck.

Future AGI’s production focus is on tracing every step so that you can see which part of the pipeline is slowing you down and optimize accordingly.

---

4.2 Error Rate & Timeouts

Track how often the system fails to respond properly:

• Model errors: provider failures, rate limits, malformed responses
• Tool failures: timeouts, invalid outputs, API errors
• Orchestrator errors: exceptions in your agent framework

Metrics:

• error_rate = (# failed_requests) / (total_requests)
• timeout_rate
• retry_rate and success-after-retry
• Common error codes

---

5. Cost & Efficiency Metrics

If you’re not measuring cost at the conversation level, it’s easy to burn budget without realizing it.

5.1 Cost per Conversation / Cost per Task

What it is:
Total variable cost to serve a conversation or complete a task, including:

• Model tokens (prompt + completion)
• Embeddings & retrieval calls
• Tool / API costs (e.g., external services)
• Safety / guardrail model calls

Formula:

cost_per_conversation = (total variable cost for conversation)

Aggregate as:

• avg_cost_per_conversation
• median_cost_per_conversation
• Cost distribution for outliers (e.g., long troubleshooting chats)

The key is to join token usage and tool calls with your traces so you can tie cost directly to workflows, intents, and user segments.

---

5.2 Cost per Successful Outcome

Raw cost per conversation is incomplete. You care about cost per success.

cost_per_success = (total_cost_for_successful_sessions) / (# successful_sessions)

This lets you compare:

• Different models (e.g., GPT-4 vs smaller models or finetunes)
• Different RAG strategies (aggressive retrieval vs minimal context)
• Different prompts/workflows (short vs verbose, multi-step vs single-shot)

In Future AGI’s Experiment stage, this is exactly how you pick a “winner” configuration—for quality, cost, latency, and safety.

---

How to Instrument These Metrics in Practice

Metrics only matter if they’re wired into a loop: Datasets → Experiment → Evaluate → Improve → Monitor & Protect.

Step 1: Structuring Traces

Instrument your app so each request generates a trace with:

• User input & metadata (channel, user id, intent)
• All LLM calls (prompt, completion, tokens, latency)
• Tool calls & responses
• RAG context (documents & scores)
• Safety decisions (flags, categories, blocks)
• Final output
• Outcome labels (success/fail, escalation, feedback)

If you’re using OpenAI, Anthropic, Bedrock, Gemini, etc., you can plug in SDK-style instrumentation (e.g., pip install traceAI-openai style) to capture this automatically.

---

Step 2: Building Evaluation Datasets

Use synthetic datasets plus real production traces to evaluate key metrics:

• Generate edge-case conversations for each intent
• Label ground truth or expected behavior (e.g., “must not answer here; escalate”)
• Attach metrics like hallucination rate, task success, and format compliance

These power your pre-production Experiments.

---

Step 3: Running Experiments

For every new agent configuration (model, prompt, tools, retrieval settings):

• Run controlled Experiments on the same dataset
• Compare:
  • Hallucination rate
  • Task success rate
  • Safety violation rate
  • Latency (p50/p95)
  • Cost per conversation / per success

Choose the configuration that meets your guardrails (e.g., hallucination rate < 1%, safety violations ~0, p95 latency under target) and has optimal cost.

---

Step 4: Closing the Loop in Production

Once in production, use Monitor & Protect-style workflows to:

• Continuously track:
  • Hallucinations and safety violations
  • Escalation rate and deflection
  • Latency and error rates
  • Cost per conversation / success
• Set alerts on anomalies (e.g., sudden spike in hallucinations or latency)
• Sample problematic traces into new datasets for the next Evaluate → Improve cycle

This is how you move from “we ship and hope” to “we ship, measure, and iterate.”

---

Example Metric Set by Use Case

To make this concrete, here’s a minimal metric set for a few common LLM app types.

Customer Support Chatbot

• Quality: task_success_rate, hallucination_rate
• Safety: safety_violation_rate (toxicity, privacy, policy)
• Escalation: escalation_rate, deflection_rate
• Latency: p90_total_latency, TTFT
• Cost: avg_cost_per_conversation, cost_per_resolved_ticket

RAG Knowledge Assistant

• Quality: grounding_score, hallucination_rate, relevance_score
• Safety: injection_attempt_rate, safety_violation_rate
• User: thumbs_up_rate / CSAT
• Latency: p90_latency (RAG + model)
• Cost: cost_per_successful_answer

Voice Agent (Phone IVR / Voice Concierge)

• Quality: task_success_rate, step_accuracy (per slot/intent)
• Safety: safety_violation_rate (including self-harm, harassment)
• User: call_completion_rate, escalation_rate to human agent
• Latency: TTFT and p95_latency per turn (critical for UX)
• Cost: cost_per_call, cost_per_success

---

Putting It All Together

If you remember one thing: LLMs are probabilistic, so your metrics must be deterministic.

For a production LLM app, the critical metrics to track include:

• Hallucination rate and grounding score for factual correctness
• Task success rate and deflection/escalation rate for business impact
• Safety violation rate and attack attempt rate for risk
• Latency (TTFT, p95) and error rates for UX and reliability
• Cost per conversation and cost per successful outcome for viability

Instrument these via traces, evaluate them on synthetic + real datasets, and feed them into a continuous Datasets → Experiment → Evaluate → Improve → Monitor & Protect loop.

If you want to see how to wire this up end-to-end—with real traces, deterministic evals, and production guardrails—
Get Started