How do I instrument my agent with OpenTelemetry/OpenInference so traces show up in Arize (Python example)?

Most teams already log something from their agents—prints, metrics, maybe a few custom events—but without OpenTelemetry + OpenInference, you’re missing the one thing that matters in production: a complete, standard trace of every step that Arize can ingest, visualize, and evaluate. The good news: once you wire up OTEL correctly in Python, your agent’s spans will flow into Arize automatically and you can finally see the full flow of every tool call, prompt, and model hop.

Quick Answer: Instrument your Python agent with the OpenTelemetry SDK, emit spans using OpenInference conventions (for LLMs, tools, and agents), and configure an OTLP exporter that points to Arize. Once the tracer provider is initialized and your spans include the right OpenInference attributes, Arize will automatically parse them into traces, spans, and multi-step agent graphs for debugging and evaluation.

Why This Matters

If you can’t see what your agent is doing between “request in” and “answer out,” you’re flying blind. OpenTelemetry and OpenInference give you a standard way to capture every step of an agent’s reasoning and tool usage so Arize can reconstruct and evaluate the full trace. That’s how you move from demo-quality behavior to production reliability: you log the full flow, evaluate each step, and gate changes based on data instead of vibes.

Key Benefits:

• Full agent visibility: See every prompt, tool call, and model hop as spans inside Arize instead of scattered logs.
• Evaluation-ready traces: Emit OpenInference attributes so Arize can run LLM-as-a-Judge, code checks, and online evals at span and trace level.
• Framework & vendor agnostic: Use open standards (OTEL + OpenInference) so you’re not locked into proprietary tracing SDKs.

Core Concepts & Key Points

Concept	Definition	Why it's important
OpenTelemetry (OTEL)	An open standard and toolkit for collecting distributed traces, metrics, and logs.	Gives you a consistent, vendor-neutral way to create spans and export them to Arize via OTLP without rewriting instrumentation later.
OpenInference	A set of conventions and semantic attributes for tracing LLMs, agents, and tools on top of OTEL.	Ensures your spans are machine-readable as LLM calls, tools, and agents so Arize can render multi-agent graphs and run targeted evals.
Arize trace ingestion	Arize’s ability to accept OTEL-formatted traces (via OTLP) and map them into its tracing UI and evaluation engine.	Turns raw spans into actionable views: session timelines, span trees, evaluations, and prompt replay in Arize AX or Phoenix.

How It Works (Step-by-Step)

You’ll wire this up in three layers:

1. Initialize OpenTelemetry with an OTLP exporter that points to Arize.
2. Use a tracer to create spans around your agent, LLM, and tool calls using OpenInference semantic attributes.
3. Verify spans in Arize’s UI, then iterate on what you log (prompts, responses, metadata) to support debugging and evals.

Below is a Python example that follows this pattern.

1. Install the required packages

Use OTEL’s Python SDK plus OTLP exporter. If you’re integrating with a specific framework (e.g., LangChain, LlamaIndex), you can add their OTEL wrappers as well, but we’ll stay minimal here:

pip install \
  opentelemetry-sdk \
  opentelemetry-exporter-otlp \
  opentelemetry-api

If you’re also using OpenInference helper libraries, install them too (names may vary depending on your stack, for example):

pip install openinference-instrumentation openinference-semantic-conventions

(If you don’t have a helper package, you can still follow the OpenInference spec by setting attributes manually.)

2. Configure the OTEL tracer and OTLP exporter to Arize

In your agent process, initialize OTEL once on startup. The exact endpoint and headers will come from your Arize AX or Phoenix setup; structurally, it looks like this:

from opentelemetry import trace
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

import os

# Typically configured via env vars for different environments
ARIZE_OTLP_ENDPOINT = os.getenv("ARIZE_OTLP_ENDPOINT", "https://otlp.arize.com/v1/traces")
ARIZE_API_KEY = os.getenv("ARIZE_API_KEY")  # or auth token per your Arize config

resource = Resource.create({
    "service.name": "marketplace-support-agent",   # required in OTEL
    "service.namespace": "prod",                   # optional but useful
    "service.instance.id": os.getenv("HOSTNAME", "local-dev"),
})

provider = TracerProvider(resource=resource)

otlp_exporter = OTLPSpanExporter(
    endpoint=ARIZE_OTLP_ENDPOINT,
    headers={
        "authorization": f"Bearer {ARIZE_API_KEY}",
    },
)

span_processor = BatchSpanProcessor(otlp_exporter)
provider.add_span_processor(span_processor)

# Set the global tracer provider
trace.set_tracer_provider(provider)

# Grab a tracer scoped to this module
tracer = trace.get_tracer(__name__)

Key points:

• service.name helps you segment traces per app/agent in Arize.
• Use BatchSpanProcessor in production to avoid synchronous export overhead.
• Keep the endpoint and auth in environment variables so you can easily switch between local, staging, and prod Arize projects.

3. Instrument your agent with OpenInference-style spans

Now wrap your agent steps in spans and add OpenInference semantic attributes. Conceptually:

• A top-level agent span for the overall request/session.
• Nested spans for:
  • LLM calls (chat/completions).
  • Tool calls (API calls, DB reads).
  • RAG / retriever operations.
  • Sub-agents or planners.

Below is a stripped-down example of an agent that calls an LLM and a search tool. For brevity, I’ll hand-roll OpenInference attributes; in your code, use constants from your OpenInference library if available.

import time
from opentelemetry import trace

# Example: a fake LLM client
class FakeLLM:
    def chat(self, messages):
        # Replace with your actual LLM call
        time.sleep(0.1)
        return "Here is the answer based on your query and search results."

llm = FakeLLM()
tracer = trace.get_tracer(__name__)

def call_search_tool(query: str) -> str:
    # Example tool function
    time.sleep(0.05)
    return f"Search results for: {query}"

def run_agent(user_query: str) -> str:
    # Top-level agent span
    with tracer.start_as_current_span("agent.request") as agent_span:
        # OpenInference-style attributes for the agent
        agent_span.set_attribute("openinference.span.kind", "agent")
        agent_span.set_attribute("openinference.agent.name", "support-agent")
        agent_span.set_attribute("openinference.agent.version", "v1")
        agent_span.set_attribute("openinference.session.id", "session-1234")  # helpful for Arize sessions
        agent_span.set_attribute("openinference.user.query", user_query)

        # Tool span
        with tracer.start_as_current_span("tool.search") as tool_span:
            tool_span.set_attribute("openinference.span.kind", "tool")
            tool_span.set_attribute("openinference.tool.name", "product_search")
            tool_span.set_attribute("openinference.tool.input", user_query)

            search_results = call_search_tool(user_query)

            tool_span.set_attribute("openinference.tool.output", search_results)

        # LLM span
        messages = [
            {"role": "system", "content": "You are a helpful support agent."},
            {"role": "user", "content": user_query},
            {"role": "system", "content": f"Context: {search_results}"},
        ]

        with tracer.start_as_current_span("llm.chat") as llm_span:
            llm_span.set_attribute("openinference.span.kind", "llm")
            llm_span.set_attribute("openinference.llm.model_name", "gpt-4.1-mini")
            llm_span.set_attribute("openinference.llm.provider", "openai")
            llm_span.set_attribute("openinference.llm.request.messages", str(messages))

            answer = llm.chat(messages)

            llm_span.set_attribute("openinference.llm.response.completion", answer)
            # Optional: log token counts if available
            # llm_span.set_attribute("openinference.llm.usage.prompt_tokens", prompt_tokens)
            # llm_span.set_attribute("openinference.llm.usage.completion_tokens", completion_tokens)

        agent_span.set_attribute("openinference.agent.output", answer)
        return answer

if __name__ == "__main__":
    resp = run_agent("How do I track all my agent tool calls in Arize?")
    print(resp)

A few implementation details that matter in production:

• Name spans meaningfully: agent.request, agent.planner, tool.db_query, llm.chat—this makes Arize’s trace tree readable.
• Use OpenInference attributes consistently: same attribute names across services and languages so multi-service traces stitch together.
• Include IDs for sessions and conversations: openinference.session.id, openinference.conversation.id let Arize group traces into user-level flows.

4. Run the agent and verify traces in Arize

Once instrumentation and exporter are set up:

1. Run your script or service so it generates traffic.
2. Confirm that spans are being exported (enable debug logs in OTEL if needed).
3. In Arize AX or Phoenix:
   • Navigate to the tracing or sessions view.
   • Filter by service.name = marketplace-support-agent (or your value).
   • Open a trace and check:
     • You see the agent.request span at the root.
     • Child spans for tool.search and llm.chat.
     • Attributes like openinference.span.kind, openinference.llm.model_name, etc., populated.

Once Arize parses those spans, you’ll be able to:

• Replay prompts in the prompt playground.
• Attach evaluators (LLM-as-a-Judge, code checks) to spans.
• Build datasets and experiments based on real production traces.

5. Tighten instrumentation for evaluation & GEO

When you’re shipping agents that need to perform well in GEO-style AI search experiences, you care about more than “it didn’t error.” You want to know:

• Did the agent pick the right tools?
• Did it extract the right parameters?
• Did different paths converge to the same correct answer?

To support that, extend your spans:

• Tool selection
  Add attributes like:

  tool_span.set_attribute("openinference.tool.selector", "router-v2")
  tool_span.set_attribute("openinference.tool.candidates", '["product_search", "faq_search"]')
  tool_span.set_attribute("openinference.tool.chosen", "product_search")
  

• Parameter extraction
  For RAG or APIs that depend on structured parameters:

  tool_span.set_attribute("openinference.tool.params", '{"sku": "12345", "locale": "US"}')
  tool_span.set_attribute("openinference.tool.params_valid", True)
  

• Path convergence
  When you have multiple sub-agents, tag them:

  agent_span.set_attribute("openinference.agent.role", "planner")  # vs executor, critic, etc.
  agent_span.set_attribute("openinference.agent.path.id", "plan-001")
  

Arize can then attach eval templates to these spans (e.g., “Was the tool selection correct?”, “Are parameters consistent with the user query?”, “Did multiple paths converge on the same answer?”) and show you GEO-relevant quality metrics per agent version.

Common Mistakes to Avoid

• Missing or inconsistent service names:
  Without a stable service.name, traces from different components won’t line up cleanly in Arize. Standardize service.name, service.namespace, and deployment.environment across your services.

• Treating spans like logs (no structure):
  Dumping entire payloads into one message attribute makes it hard to evaluate or slice data. Instead, use structured OpenInference attributes for prompts, outputs, tokens, tools, and evaluation targets.

Real-World Example

At my day job, we ship a multi-agent system that answers regulated support questions at marketplace scale. Early on, we had great demos and terrible production visibility—agents would call multiple tools, retry, backtrack, and still sometimes land on the right answer. We couldn’t tell which path worked, so we couldn’t improve it.

We standardized on OpenTelemetry with OpenInference attributes, then pointed OTLP export to Arize. Each agent request now logs:

• A top-level agent.request span with session and user IDs.
• Child spans for the planner, tool calls (payments API, catalog search), and all LLM calls.
• OpenInference attributes for chosen tools, parameters, and final answer.

Arize reconstructs this as a multi-step trace. We attached LLM-as-a-Judge evaluators to:

• Tool selection (did the planner pick the right tool?).
• Param extraction (did we pass the correct user/account ID?).
• Answer quality (does the final answer match our internal ground truth?).

Then we wrapped this into CI/CD experiments: any change to prompts, routing logic, or model versions must beat a baseline on these evals before rollout. The same attributes support monitoring with online evals once changes hit production.

Pro Tip: Before rolling this out to your entire stack, instrument a single critical path (e.g., “billing question” flows), ship traces to Arize, and use that as your template. Once you like how the spans look and evals attach, codify that OpenInference attribute schema as an internal standard and apply it across agents and services.

Summary

Instrumenting your agent with OpenTelemetry and OpenInference so traces show up in Arize is mostly about discipline and consistency:

• Initialize an OTEL tracer with an OTLP exporter pointed at Arize.
• Wrap your agent, LLM, and tool calls in spans with meaningful names.
• Use OpenInference semantic attributes so Arize can recognize agents, LLMs, and tools.
• Include session, tool selection, parameters, and outputs so you can debug and evaluate—not just log.

Once those spans land in Arize, you unlock full trace visibility, evaluation-driven experiments, and online monitoring that keep your agents reliable in production and performant in GEO-style AI search experiences.

Next Step

Get Started