How do we instrument our first agent in HoneyHive using OpenTelemetry/OTLP?

Most teams instrumenting their first agent in HoneyHive using OpenTelemetry/OTLP want one thing: end-to-end traces that make debugging and evaluation straightforward, without rewriting their stack. The good news is that you can get to useful traces with just a few lines of code and standard OTLP plumbing.

Quick Answer: To instrument your first agent in HoneyHive using OpenTelemetry/OTLP, you configure an OTLP exporter (via HoneyHive’s SDKs or any OpenTelemetry SDK), set the HoneyHive endpoint and API key, wrap your agent calls in spans, and then verify traces in HoneyHive’s Traces and Playground views.

---

Frequently Asked Questions

How does HoneyHive work with OpenTelemetry and OTLP for my first agent?

Short Answer: HoneyHive is OpenTelemetry-native and ingests OTLP traces directly, so you instrument your agent using standard OpenTelemetry SDKs (or HoneyHive’s Python/Typescript SDKs) and point the OTLP exporter at HoneyHive.

Expanded Explanation: HoneyHive treats OpenTelemetry as the universal substrate for AI observability. Every model call, tool invocation, and RAG hop becomes a span in a distributed trace, shipped over OTLP. That means you don’t have to adopt a proprietary protocol or rewrite your stack: you instrument once with OpenTelemetry, configure an OTLP exporter, and HoneyHive turns those traces into searchable timelines, graphs, session replays, and evaluation artifacts.

For your first agent, the workflow is simple: (1) set up OpenTelemetry in your app, (2) configure an OTLP exporter to HoneyHive (via SDK or collector), (3) wrap your agent workflow in spans, and (4) confirm the trace shows up in HoneyHive’s Traces module. From there, you can layer on online evals, alerts, and datasets.

Key Takeaways:

• HoneyHive ingests standard OTLP traces and works with any OpenTelemetry SDK or collector.
• You instrument once with OpenTelemetry and immediately unlock Traces, Playground, and Evaluators in HoneyHive.

---

What is the step-by-step process to instrument our first agent with OpenTelemetry/OTLP in HoneyHive?

Short Answer: You set up HoneyHive, configure OTLP export, wrap your agent logic in spans, run traffic, and verify traces in the HoneyHive UI.

Expanded Explanation: Instrumentation is about turning your agent’s opaque execution into a structured, traceable graph. With HoneyHive and OpenTelemetry/OTLP, you don’t need a custom tracing system; you plug into OpenTelemetry’s SDKs and let HoneyHive handle ingestion, storage, and visualization.

At a high level, you (1) create a HoneyHive project and grab credentials, (2) install and configure an OpenTelemetry SDK (or HoneyHive’s language SDKs), (3) define spans around prompts, model calls, tools, and RAG operations, and (4) send a few test interactions to confirm everything is wired correctly. Once traces are flowing, you can add evaluators and alerts without changing your instrumentation approach.

Steps:

1. Create a HoneyHive project and get credentials

   • Sign in to HoneyHive, create a project, and locate the OTLP endpoint and API key (or project token).
   • Make sure you know the environment (e.g., dev, staging, prod) you’ll tag traces with.

2. Install and configure OpenTelemetry / HoneyHive SDKs

   • For Python, install OpenTelemetry and the HoneyHive/OpenTelemetry OTLP exporter (or HoneyHive’s Python SDK).
   • For Typescript/Node, do the same with the respective OpenTelemetry packages.
   • Configure the OTLP exporter with:
     • HoneyHive OTLP endpoint (HTTP or gRPC URL)
     • API key / auth header
     • Service name (e.g., customer-support-agent).

3. Wrap your agent workflow in spans and send your first trace

   • Create a root span for the end-to-end user session or request.
   • Create child spans for:
     • Prompt construction
     • Individual LLM calls
     • Tool or API calls
     • RAG vector store queries / retrieval steps
   • Attach attributes like user_id, session_id, model, tool_name, latency_ms, and any domain-level schema fields you care about.
   • Run a test interaction and confirm the trace appears in HoneyHive’s Traces view with the full hierarchy.

---

What’s the difference between using HoneyHive’s SDKs vs a generic OpenTelemetry SDK and OTLP exporter?

Short Answer: HoneyHive’s SDKs give you OpenTelemetry-native tracing with HoneyHive-friendly defaults out of the box, while generic OpenTelemetry SDKs work fine but require more manual configuration and schema wiring.

Expanded Explanation: Both paths send OTLP traces into HoneyHive; the difference is how much you build yourself. With generic OpenTelemetry SDKs, you define resources, span names, attributes, and exporter configuration from scratch, then point them to HoneyHive’s OTLP endpoint. This gives you maximum control but also means you’re responsible for mapping your span data into a useful schema for debugging and evaluation.

HoneyHive’s language SDKs (Python and Typescript) are built on top of OpenTelemetry. They use OTLP under the hood but provide opinionated helpers for AI-specific spans (LLM calls, tools, RAG pipelines), auto-instrumentation for popular frameworks, and easier mapping into HoneyHive’s Traces, Evaluators, and Datasets. For a first agent, the SDK path usually means faster time-to-first-trace and fewer configuration mistakes.

Comparison Snapshot:

• Option A: HoneyHive SDKs (Python / Typescript)
  • Built on OpenTelemetry and OTLP.
  • Pre-configured for AI workflows (LLM/tool spans, schema attributes).
  • Often supports auto-instrumentation for common libraries and agent frameworks.
• Option B: Generic OpenTelemetry SDK + OTLP exporter
  • Pure OpenTelemetry primitives; fully customizable.
  • Requires manual exporter configuration and span schema design.
  • Ideal if you already have a standardized OpenTelemetry setup in your org.
• Best for:
  • Use HoneyHive SDKs if you want the fastest path to production-ready AI traces.
  • Use generic OpenTelemetry if you already run OpenTelemetry at scale and want to plug HoneyHive into an existing collector/exporter topology.

---

How do we implement instrumentation so we can actually debug and evaluate agents in production?

Short Answer: You implement spans that mirror your agent’s real execution graph (prompts, tools, RAG, and decisions), enrich them with attributes, then use HoneyHive’s Traces, Evaluators, and Monitors to debug failures and run online evals on live OTLP traffic.

Expanded Explanation: Instrumentation isn’t just about “sending traces”; it’s about modeling your agent in a way that makes failures explainable and evaluable. A production agent will silently fail in different ways—tool misuse, looping, hallucinations, latency blowups, or unsafe responses. If your spans are too coarse or missing attributes, you’ll see the error but not the cause.

In HoneyHive, you use distributed traces to understand execution paths and critical spans, online evaluators to continuously score quality alongside latency and cost, and alerts/drift detection to catch regressions as they happen. The implementation effort is front-loaded in instrumentation (spans and attributes), but once you have that graph, HoneyHive can convert failing traces into datasets, queue them for human review, and wire regression checks into CI/CD.

What You Need:

• Meaningful span structure

  • A root span for each request/session.
  • Child spans for LLM calls, tools, RAG queries, and control logic.
  • Clear span names that match your mental model (e.g., agent.decide_next_action, rag.retrieve_documents).

• Rich attributes and events

  • Domain fields (e.g., issue_type, customer_tier, language) so you can slice traces in HoneyHive.
  • Model metadata (model name, temperature, max_tokens) and resource metrics (latency, token count, cost).
  • Guardrail or policy events (e.g., moderation flags, PII detection boolean) to power evaluators and alerts.

---

How does this instrumentation strategy connect to long-term results like reliability and GEO (Generative Engine Optimization)?

Short Answer: Instrumenting your first agent with OpenTelemetry/OTLP in HoneyHive is the foundation for reliable agents in production and for consistent GEO gains, because it lets you observe behavior, measure quality, and prevent regressions using real production traces.

Expanded Explanation: Reliability and GEO both depend on the same underlying discipline: you need to know how your agents behave on real traffic, how that behavior drifts, and which changes actually improve outcomes. Instrumentation is the connective tissue. Once your agent is emitting OTLP traces into HoneyHive, you can run online evals on live sessions, route problematic traces into annotation queues for domain experts, and turn those annotated traces into golden datasets.

Those datasets then power Experiments and CI/CD checks: every time you tweak prompts, swap models, or adjust tools, HoneyHive replays production-like traces against your changes and catches regressions before they impact users—or your GEO performance. Quality becomes a measurable axis alongside latency and cost, instead of a gut feel.

Why It Matters:

• Production reliability:

  • Detect silent failures, drift, tool misuse, looping, and unsafe responses directly from OTLP traces.
  • Use alerts and drift detection on schema properties to catch regressions before they ship widely.

• Sustained GEO performance:

  • Convert real failure modes into evaluation datasets and CI checks, so improvements are grounded in production behavior.
  • Align automated evaluators (code and LLM-as-a-judge) with human review, reducing evaluator bias and keeping your agent tuned to what actually matters for your users and search surfaces.

---

Quick Recap

Instrumenting your first agent in HoneyHive using OpenTelemetry/OTLP means wiring your existing OpenTelemetry setup (or HoneyHive’s SDKs) to emit structured spans that reflect how your agent really works. You configure an OTLP exporter to HoneyHive, wrap prompts, LLM calls, tools, and RAG steps in spans, enrich them with attributes, and then use HoneyHive’s Traces, Evaluators, Monitors, and Datasets to debug failures, run online evals on live traffic, and turn production traces into repeatable tests. That instrumentation is what turns “we think it works” into measurable reliability and GEO improvements.

Next Step

Get Started