LLM Observability & Evaluation

Platforms that provide observability, monitoring, and evaluation for machine learning and LLM applications in production, including tracing, quality/safety evals, drift detection, and performance analytics to improve reliability of AI systems and agents.

Lead Story

How do I create an evaluation dataset in Langtrace from production traces and then manually score outputs?

Read full story

LLM Observability & Evaluation

How do I create an evaluation dataset in Langtrace from production traces and then manually score outputs?

More Stories

Latest

How do I contact Langtrace for an Enterprise plan (SOC 2 Type II, custom retention, SLA) and what info should I bring to the call?

Langtrace Enterprise: what’s the self-hosting architecture and what data is stored (prompts, outputs, metadata) for a security review?

Langtrace self-hosting: what are the deployment options and what infra do I need to run it privately?

How do I export Langtrace OpenTelemetry traces to Datadog (or Grafana/Honeycomb) instead of only using the Langtrace UI?

How do I add Langtrace to a TypeScript/Node app (Express or Next.js) to trace LLM calls?

Langtrace pricing: what’s included in Free Forever vs Growth, and what are the span limits?

How do I install and initialize the Langtrace Python SDK in a FastAPI app?

Langtrace Growth plan: will 500k spans/year cover my production traffic, and what happens if we exceed it?

Langtrace vs Arize Phoenix: which is more enterprise-ready for security reviews (SOC 2, self-hosting, data retention)?

How do I sign up for Langtrace and generate an API key for my project?

Langtrace vs Helicone: which has a smoother onboarding for a Next.js/TypeScript app using Vercel AI SDK?

Langtrace vs Langfuse: how do the dashboards compare for debugging multi-step agent traces (tools + vector DB + LLM)?

Langtrace vs Humanloop: which is better for prompt versioning, rollback, and evaluation workflows?

Langtrace vs WhyLabs: which is better for monitoring LLM quality regressions in production?

Langtrace vs Langfuse vs Traceloop: which is best if I need OTEL portability + self-hosting?

Langtrace vs Traceloop: how do their OpenTelemetry implementations differ, and can I export to my existing observability stack?

Langtrace vs Arize Phoenix: which is stronger for RAG evaluations and dataset curation from production traffic?

Langtrace vs Langfuse: which is better for self-hosting and avoiding vendor lock-in?

Langtrace vs Helicone: which one is better for token/cost attribution per user and per endpoint?