How do I get started with Arize Phoenix for LLM tracing on my local machine?

Most teams hit the same wall with local LLM development: the prototype “works” in a notebook, then quietly breaks once you add tools, RAG, and multi-step chains. Arize Phoenix exists for exactly this gap—open-source, self-hosted LLM tracing and evaluation that lets you see every span, prompt, tool call, and response on your own machine before you ever think about production.

Quick Answer: To get started with Arize Phoenix for LLM tracing on your local machine, you install the Phoenix server (via Docker or Python), instrument your LLM app with the Phoenix SDK or OpenTelemetry, and then send traces to your local Phoenix instance while you develop. Within minutes, you can inspect spans, debug prompts, and run evaluations in a browser UI—all without sending data off your box.

Why This Matters

Local LLM tracing is the fastest way to catch hallucinations, broken tool calls, and brittle prompts before they reach users. When you can see the full trace—input, retrieved context, intermediate tool steps, model choices, and final answer—you stop guessing and start engineering agents that are debuggable and repeatable.

With Arize Phoenix running locally, you:

• Keep sensitive data on your machine
• Use open standards (OpenTelemetry / OpenInference) instead of proprietary formats
• Turn messy experiments into structured traces and datasets you can later plug into Arize AX or your own CI pipeline

Key Benefits:

• Full visibility for local LLM apps: Trace every step of your chain or agent, including prompts, tools, and RAG calls, in a local browser UI.
• Production-style debugging on your laptop: Reproduce real issues, inspect spans, and compare runs without standing up a full observability stack.
• Open, portable data: Use standard tracing formats so you’re not locked into a single vendor; traces can later flow into Arize AX or any OTEL-compatible backend.

Core Concepts & Key Points

Concept	Definition	Why it's important
Phoenix server (local UI)	A self-hosted web app (usually on http://localhost:6006) that receives, stores, and visualizes LLM traces and evaluations.	This is where you explore spans, sessions, and multi-step flows—your main window into how your agent behaves.
Spans & traces	Spans are individual units of work (prompt, tool call, retrieval); a trace is the end-to-end tree/graph of spans for a single request or session.	Traces let you see how a user query flows through prompts, tools, and models so you can pinpoint where things go wrong.
Instrumentation	The code or middleware that captures events (prompts, tool calls, model outputs) and sends them to Phoenix using OpenTelemetry/OpenInference conventions.	Without instrumentation, Phoenix has nothing to show; with good instrumentation, you get precise, debuggable traces that mirror production behavior.

How It Works (Step-by-Step)

At a high level, getting started with Arize Phoenix for LLM tracing on your local machine looks like this:

1. Install and run Phoenix locally (Docker or Python).
2. Instrument your LLM application to emit spans/traces.
3. Send test traffic and analyze the traces in the Phoenix UI.

Below is a practical, “from zero to first trace” flow.

1. Install Arize Phoenix Locally

You have two main options on your laptop: Docker (clean and isolated) or Python (handy if you’re already in a virtual environment).

Option A: Run Phoenix with Docker

1. Ensure Docker is installed and running.

2. Pull and run the Phoenix image (example pattern):

   docker run -p 6006:6006 arizephoenix/phoenix:latest
   

3. Once the container starts, open:

   http://localhost:6006
   

You should see the Phoenix UI waiting for traces.

Option B: Install via Python

1. Create and activate a virtual environment (recommended):

   python -m venv .venv
   source .venv/bin/activate  # Linux / macOS
   # or
   .venv\Scripts\activate     # Windows
   

2. Install Phoenix:

   pip install arize-phoenix
   

3. Start the server:

   phoenix serve
   

4. Navigate to:

   http://localhost:6006
   

Either approach gives you a local Phoenix instance ready to receive LLM traces and evaluations.

2. Instrument Your LLM Application

Phoenix is designed to work with open standards—OpenTelemetry and OpenInference—so you can wire it into almost any stack (LangChain, LlamaIndex, custom agents, etc.) without proprietary glue code.

There are two main patterns:

• SDK / native integration for quick starts
• OpenTelemetry-based tracing for more complex or multi-service apps

Below is a simple Python-style walkthrough to match the “how do I get started” intent.

A. Add Phoenix to a Simple LLM Script

Assume you have a basic LLM call:

from my_llm_lib import llm

def answer_question(question: str) -> str:
    return llm(question)

To trace this with Phoenix:

1. Install the tracing dependencies (example, adjust to your stack):

   pip install opentelemetry-sdk opentelemetry-exporter-otlp
   

2. Configure an OTLP exporter pointed at Phoenix
   (Phoenix typically exposes an OTLP endpoint; assume localhost:4317 or as configured):

   from opentelemetry import trace
   from opentelemetry.sdk.trace import TracerProvider
   from opentelemetry.sdk.trace.export import BatchSpanProcessor
   from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
   
   provider = TracerProvider()
   exporter = OTLPSpanExporter(endpoint="http://localhost:4317", insecure=True)
   provider.add_span_processor(BatchSpanProcessor(exporter))
   trace.set_tracer_provider(provider)
   
   tracer = trace.get_tracer(__name__)
   

3. Wrap your LLM call in spans:

   from opentelemetry import trace
   
   tracer = trace.get_tracer(__name__)
   
   def answer_question(question: str) -> str:
       with tracer.start_as_current_span("llm.request") as span:
           span.set_attribute("llm.input", question)
           result = llm(question)
           span.set_attribute("llm.output", result)
           return result
   

4. Run your script, then open Phoenix at http://localhost:6006.
   You should now see traces for each call to answer_question, with the input and output attached as span attributes.

B. Instrument Multi-Step Agents and Tool Calls

For a multi-step agent with RAG and tools, you want more granular spans:

• session / user_request
• retrieval
• tool_call
• llm.response

Example sketch:

def handle_user_query(query: str) -> str:
    with tracer.start_as_current_span("session") as session_span:
        session_span.set_attribute("user.query", query)

        with tracer.start_as_current_span("retrieval") as retrieval_span:
            docs = retriever.search(query)
            retrieval_span.set_attribute("retrieved.count", len(docs))

        with tracer.start_as_current_span("llm.response") as llm_span:
            answer = llm(query=query, context=docs)
            llm_span.set_attribute("llm.answer", answer)

        return answer

Each start_as_current_span call becomes a node in the trace you’ll see in Phoenix, making it obvious where latency or quality issues live.

3. Send Traffic and Explore Traces in Phoenix

With Phoenix running locally and your app instrumented:

1. Generate traffic
   Run a few queries through your app (manual calls, a notebook, or a quick test script).

2. Open the Phoenix UI (http://localhost:6006) and:

   • View traces: end-to-end flows per request
   • Inspect spans: prompts, tool calls, retrieval events
   • Look at attributes: user input, model, temperature, token counts, tool names

3. Iterate on your instrumentation
   If you don’t see the details you want:

   • Add more span attributes (e.g., tool.name, retrieval.latency_ms)
   • Break long chains into sub-spans so the graph is readable
   • Ensure your OTEL exporter endpoint and ports match the Phoenix configuration

As you refine, Phoenix becomes your local control center for agent behavior.

Common Mistakes to Avoid

• Running Phoenix but not pointing your exporter to it:
  If your OTLP exporter is still pointed at some default or a different endpoint, Phoenix will stay empty. Double-check the endpoint and port (4317 for gRPC, 4318 for HTTP by default).

• Instrumenting only the outermost call:
  A single span around the whole agent call hides the real failure modes. Add spans for retrieval, each tool call, and intermediate LLM steps so you can see where hallucinations or timeouts originate.

• Logging sensitive data without redaction:
  Even on your local machine, treat PII carefully. Use attributes selectively and add redaction at the span level (e.g., hashing IDs, truncating inputs) so that later, if you forward traces beyond your laptop, you’re already safe.

Real-World Example

When we first rolled out a marketplace support agent, we prototyped locally with Phoenix before ever wiring it to production data. The agent used RAG over policy docs plus several tools (ticket creation, refund calculator, fraud signal fetcher). We:

1. Ran Phoenix locally on our laptops.
2. Instrumented each tool call and retrieval step with OpenTelemetry spans.
3. Sent synthetic queries that mirrored real support cases (refund edge cases, multi-language issues).
4. Used Phoenix traces to see:
   • Where the agent picked the wrong tool
   • When retrieval fetched outdated policy documents
   • How often the agent “recovered” from a bad intermediate step

We caught a nasty bug where the agent would sometimes call the refund tool before validating eligibility. The trace made it obvious: the tool span fired before the eligibility-check span. Fixing this locally saved us from a very expensive production incident.

Pro Tip: Treat your local Phoenix setup like a “pre-production scope.” Once you have useful spans and attributes locally, you can reuse the same instrumentation patterns in your staging and production environments—whether you stick with Phoenix or plug into Arize AX as the full AI & agent engineering platform.

Summary

Getting started with Arize Phoenix for LLM tracing on your local machine comes down to three moves: run the Phoenix server, instrument your app using open standards, and start sending real traffic through your agent while you watch traces in the UI. Once you see the full flow—prompts, tools, retrieval, and responses—you’ll stop guessing about why an answer went wrong and start systematically removing failure modes.

From there, you can take the same OpenTelemetry/OpenInference traces you’ve honed locally and plug them into a broader loop—experiments, online evals, annotation queues—in Arize AX or your own CI/CD, so your agents stay reliable as they move from laptop to production.

Next Step

Get Started