How do I install and initialize the Langtrace Python SDK in a FastAPI app?

Langtrace makes it easy to instrument your FastAPI app and start tracking LLM behavior, accuracy, and costs in just a couple of steps. This guide walks you through installing the Langtrace Python SDK, initializing it with your API key, and wiring it into a typical FastAPI application.

---

Prerequisites

Before you begin, make sure you have:

• A working FastAPI application (Python 3.8+ recommended)
• uvicorn or another ASGI server
• A Langtrace account and project
• Your Langtrace API key

If you haven’t set up Langtrace yet:

1. Create a project in Langtrace.
2. Generate an API key for that project.
3. Keep the key handy for the initialization step.

---

Step 1: Install the Langtrace Python SDK

Install the SDK into your FastAPI project environment:

pip install langtrace-python-sdk

If you’re using Poetry or Pipenv, install it as you would any other dependency:

poetry add langtrace-python-sdk
# or
pipenv install langtrace-python-sdk

---

Step 2: Initialize Langtrace with your API key

To enable tracing for your FastAPI app, you need to initialize Langtrace once at application startup.

The core initialization code is:

from langtrace_python_sdk import langtrace

langtrace.init(api_key="<your_api_key>")

Replace <your_api_key> with the key you generated in the Langtrace dashboard.

Best practice: Initialize on app startup

FastAPI provides startup events that run once when the app is launched. This is a good place to initialize Langtrace so it’s available for all requests.

# main.py
from fastapi import FastAPI
from langtrace_python_sdk import langtrace

app = FastAPI()

@app.on_event("startup")
def startup_event():
    # Initialize Langtrace with your project API key
    langtrace.init(api_key="YOUR_LANGTRACE_API_KEY")

For security, prefer using environment variables rather than hard-coding the key:

# main.py
import os
from fastapi import FastAPI
from langtrace_python_sdk import langtrace

app = FastAPI()

@app.on_event("startup")
def startup_event():
    api_key = os.getenv("LANGTRACE_API_KEY")
    if not api_key:
        raise RuntimeError("LANGTRACE_API_KEY is not set")
    langtrace.init(api_key=api_key)

Then set the environment variable before running your app:

export LANGTRACE_API_KEY="your_real_key_here"
uvicorn main:app --reload

---

Step 3: Run your FastAPI app with Langtrace enabled

Once the SDK is installed and initialized, start your app as usual:

uvicorn main:app --reload

From this point on, Langtrace will be able to:

• Track vital metrics (accuracy, token cost, latency, etc.)
• Observe calls to supported frameworks and providers
• Surface insights and evaluations for your LLM-powered endpoints

---

Example: FastAPI app with an LLM endpoint

Here’s a minimal FastAPI example that uses OpenAI (or another LLM provider) and is instrumented with Langtrace:

# main.py
import os
from fastapi import FastAPI
from pydantic import BaseModel
from langtrace_python_sdk import langtrace
import openai  # or your preferred LLM client

app = FastAPI()

class ChatRequest(BaseModel):
    message: str

@app.on_event("startup")
def startup_event():
    # Initialize Langtrace
    api_key = os.getenv("LANGTRACE_API_KEY")
    if not api_key:
        raise RuntimeError("LANGTRACE_API_KEY is not set")
    langtrace.init(api_key=api_key)

    # Initialize your LLM client as usual
    openai.api_key = os.getenv("OPENAI_API_KEY")

@app.post("/chat")
async def chat(req: ChatRequest):
    # Your normal LLM call – Langtrace will track supported providers
    response = openai.ChatCompletion.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": req.message}],
    )
    reply = response["choices"][0]["message"]["content"]
    return {"reply": reply}

After deploying this, you’ll be able to see traces, latency, and other metrics inside Langtrace for each /chat request, helping you optimize prompts, control token costs, and monitor performance.

---

Step 4: Verify Langtrace is capturing traces

Once your app is running and receiving traffic:

1. Open your Langtrace project dashboard.
2. Check for new traces associated with your FastAPI endpoints.
3. Review metrics such as:
   • Inference latency (e.g., 75 ms with targets like max 120 ms)
   • Token cost vs. budget (e.g., $6,200 out of a $10,000 budget)
   • Accuracy and other evaluation metrics.

If no traces are appearing, double-check:

• The correct project API key is being used.
• langtrace.init() is executed at startup (and not inside individual request handlers).
• There are actual LLM calls being made via supported frameworks or providers.

---

Using Langtrace with popular LLM frameworks in FastAPI

Langtrace natively supports several common orchestration frameworks:

• CrewAI
• DSPy
• LlamaIndex
• LangChain

If your FastAPI app uses any of these, the integration flow is:

1. Install Langtrace Python SDK.
2. Initialize Langtrace at FastAPI startup with your API key.
3. Configure your framework (CrewAI, DSPy, LlamaIndex, LangChain) as usual inside your app.

Langtrace automatically tracks supported LLM providers and VectorDB operations (for example, openai.chat.completion.create and pinecone.index.query) so you can see parent/child traces, timings (e.g., 1200 ms → 1800 ms), and errors in one place.

---

Production considerations

When you move your FastAPI app to production:

• Use environment variables or a secrets manager for the Langtrace API key.
• Initialize once per process using FastAPI’s startup event (or your ASGI server lifecycle hooks).
• Monitor key metrics in Langtrace:
  • Accuracy and evaluation scores
  • Token usage and cost against your budget
  • Inference latency and error rates

Doing this will give you the “out-of-the-box” observability you need to confidently scale your LLM-powered FastAPI application while keeping latency and costs under control.