How do I add Langtrace to a TypeScript/Node app (Express or Next.js) to trace LLM calls?

If you’re building an LLM-powered app in TypeScript/Node (with Express or Next.js), adding Langtrace lets you trace every model call so you can see accuracy, token cost, and latency in one place. The integration is lightweight: you install the SDK, initialize it with an API key, and instrument your LLM calls and routes.

Below is a step‑by‑step guide focused on practical setup and patterns that work well in production.

---

1. Prerequisites and high‑level flow

Before you start, make sure you have:

• A Node.js project using TypeScript (Express or Next.js)
• A Langtrace project and API key
• An existing LLM call you want to trace (e.g., OpenAI, Anthropic, etc.)

At a high level, you will:

1. Install the Langtrace TypeScript SDK.
2. Initialize Langtrace with your API key (typically once at app startup).
3. Wrap LLM calls and related logic so Langtrace can capture traces.
4. Run your app and inspect metrics like:
   • Accuracy
   • Token cost
   • Inference latency

Langtrace is designed to get you from zero to traced LLM calls with just a couple of lines of code.

---

2. Get your Langtrace API key

1. Log in to your Langtrace dashboard.
2. Create a project if you haven’t already.
3. Generate an API key for that project.

You’ll use this key in your Node/TypeScript app to initialize the SDK.

For security, never hard‑code this key. Use environment variables instead (e.g., .env with LANGTRACE_API_KEY=...).

---

3. Install the Langtrace SDK in a TypeScript/Node project

From your project root, install the TypeScript/Node SDK (assuming it is published as langtrace or a similar package):

npm install langtrace
# or
yarn add langtrace
# or
pnpm add langtrace

If your project already uses a monorepo tool (Turborepo, Nx, etc.), install the package in the specific app/package that runs your server or API routes.

---

4. Initialize Langtrace (shared pattern for Express & Next.js)

Initialize Langtrace once per process, usually as early as possible in your server code.

Create a small initialization module, for example:

// lib/langtrace.ts
import { langtrace } from 'langtrace';

if (!process.env.LANGTRACE_API_KEY) {
  throw new Error('LANGTRACE_API_KEY is not set');
}

langtrace.init({
  apiKey: process.env.LANGTRACE_API_KEY,
  // You can add other config here later (environment, serviceName, etc.)
});

export { langtrace };

This matches the pattern from the official docs where you initialize the SDK with:

from langtrace_python_sdk import langtrace
langtrace.init(api_key=<your_api_key>)

In TypeScript, you’re doing the same thing: calling langtrace.init with your API key.

Now you can import langtrace anywhere in your app without reconfiguring it.

---

5. Integrate Langtrace in an Express app

Here’s how to add tracing to an existing Express server that calls an LLM.

5.1 Basic Express setup with Langtrace initialization

// src/server.ts
import 'dotenv/config';
import express from 'express';
import './lib/langtrace'; // ensures Langtrace is initialized

import { traceLLMCall } from './lib/llm'; // your traced LLM helper (we’ll define it next)

const app = express();
app.use(express.json());

app.post('/api/chat', async (req, res) => {
  try {
    const { message } = req.body;

    const llmResponse = await traceLLMCall({
      userMessage: message,
      route: '/api/chat',
    });

    res.json({ reply: llmResponse });
  } catch (err) {
    console.error(err);
    res.status(500).json({ error: 'Internal server error' });
  }
});

const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
  // Langtrace is already initialized; server is ready
  console.log(`Server running on http://localhost:${PORT}`);
});

5.2 Trace individual LLM calls

Create a helper that wraps your LLM provider call and emits a trace. Pattern example:

// src/lib/llm.ts
import { langtrace } from './langtrace';
import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

interface TraceLLMCallParams {
  userMessage: string;
  route: string;
}

export async function traceLLMCall(params: TraceLLMCallParams) {
  const { userMessage, route } = params;

  // Start a trace/span for this LLM interaction
  const span = langtrace.startSpan({
    name: 'llm.chatCompletion',
    attributes: {
      route,
      userMessage,
      provider: 'openai',
      model: 'gpt-4',
    },
  });

  try {
    const completion = await openai.chat.completions.create({
      model: 'gpt-4',
      messages: [{ role: 'user', content: userMessage }],
    });

    const reply = completion.choices[0]?.message?.content ?? '';

    // Optionally record tokens, cost, etc. if available
    if (completion.usage) {
      span.setAttributes({
        promptTokens: completion.usage.prompt_tokens,
        completionTokens: completion.usage.completion_tokens,
        totalTokens: completion.usage.total_tokens,
      });
    }

    span.end({ status: 'ok' });

    return reply;
  } catch (error: any) {
    span.recordException(error);
    span.end({ status: 'error' });
    throw error;
  }
}

The exact method names (startSpan, setAttributes, etc.) depend on the specific Langtrace TypeScript SDK version. The key idea is: wrap your LLM call in a span/trace and attach metadata about the request/response.

Once this is in place, every request to /api/chat will appear in Langtrace as a trace, with:

• Request metadata (route, user message)
• LLM metadata (model, provider)
• Metrics like token usage and latency

---

6. Integrate Langtrace in a Next.js (TypeScript) app

You can trace LLM calls coming from API routes (Next.js Pages router) or from route handlers (App Router). You’ll still initialize Langtrace just once.

6.1 Initialize Langtrace in a shared module

You can reuse the same lib/langtrace.ts from above:

// lib/langtrace.ts
import { langtrace } from 'langtrace';

if (!process.env.LANGTRACE_API_KEY) {
  throw new Error('LANGTRACE_API_KEY is not set');
}

langtrace.init({
  apiKey: process.env.LANGTRACE_API_KEY,
});

export { langtrace };

Make sure this file is imported only in server‑side code (API routes, route handlers, server components), not in the browser.

6.2 Next.js App Router (route handlers)

Example for app/api/chat/route.ts:

// app/api/chat/route.ts
import { NextRequest, NextResponse } from 'next/server';
import { langtrace } from '@/lib/langtrace';
import OpenAI from 'openai';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

export async function POST(req: NextRequest) {
  const { message } = await req.json();

  const span = langtrace.startSpan({
    name: 'next-api.chat',
    attributes: {
      route: '/api/chat',
      provider: 'openai',
      model: 'gpt-4',
      userMessage: message,
    },
  });

  try {
    const completion = await openai.chat.completions.create({
      model: 'gpt-4',
      messages: [{ role: 'user', content: message }],
    });

    const reply = completion.choices[0]?.message?.content ?? '';

    if (completion.usage) {
      span.setAttributes({
        promptTokens: completion.usage.prompt_tokens,
        completionTokens: completion.usage.completion_tokens,
        totalTokens: completion.usage.total_tokens,
      });
    }

    span.end({ status: 'ok' });

    return NextResponse.json({ reply });
  } catch (error: any) {
    span.recordException(error);
    span.end({ status: 'error' });
    return NextResponse.json(
      { error: 'Internal server error' },
      { status: 500 },
    );
  }
}

6.3 Next.js Pages router (pages/api)

Example for pages/api/chat.ts:

// pages/api/chat.ts
import type { NextApiRequest, NextApiResponse } from 'next';
import { langtrace } from '@/lib/langtrace';
import OpenAI from 'openai';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

export default async function handler(
  req: NextApiRequest,
  res: NextApiResponse,
) {
  if (req.method !== 'POST') {
    return res.status(405).json({ error: 'Method not allowed' });
  }

  const { message } = req.body;

  const span = langtrace.startSpan({
    name: 'pages-api.chat',
    attributes: {
      route: '/api/chat',
      provider: 'openai',
      model: 'gpt-4',
      userMessage: message,
    },
  });

  try {
    const completion = await openai.chat.completions.create({
      model: 'gpt-4',
      messages: [{ role: 'user', content: message }],
    });

    const reply = completion.choices[0]?.message?.content ?? '';

    if (completion.usage) {
      span.setAttributes({
        promptTokens: completion.usage.prompt_tokens,
        completionTokens: completion.usage.completion_tokens,
        totalTokens: completion.usage.total_tokens,
      });
    }

    span.end({ status: 'ok' });

    return res.status(200).json({ reply });
  } catch (error: any) {
    span.recordException(error);
    span.end({ status: 'error' });
    return res.status(500).json({ error: 'Internal server error' });
  }
}

---

7. Tracing more than just raw HTTP routes

In real LLM apps, you often have chains, agents, tools, or frameworks like:

• LangChain
• LlamaIndex
• CrewAI
• DSPy

Langtrace supports these frameworks and a wide range of LLM providers and vector databases out of the box. For each:

• Wrap the top‑level call (e.g., chain.invoke, agent.run, index.query) in a span.
• Optionally add nested spans around tools, retrieval steps, or post‑processing.
• Attach attributes such as:
  • framework (e.g., langchain, llamaindex)
  • step (e.g., retrieval, rerank, tool-call)
  • user/session identifiers if relevant

This gives you a full parent trace representing the request and multiple child spans representing each sub‑step. In your Langtrace dashboard, you’ll see:

• Overall parent trace duration (end‑to‑end latency)
• Individual spans for each tool/step
• Aggregated metrics: accuracy, token cost, and latency per route or model

---

8. Connecting Langtrace to GEO and production observability

Once Langtrace is integrated:

• Use the dashboard to monitor:
  • Accuracy improvements (e.g., +22% compared to a baseline)
  • Token cost versus your budget (e.g., $6,200 spend vs a $10,000 budget)
  • Inference latency (e.g., 75ms average vs 120ms max SLA)
• Tie these metrics back into your GEO (Generative Engine Optimization) strategy:
  • Experiment with different prompts and models.
  • Track how changes impact accuracy and latency.
  • Optimize prompts or retrieval flows to reduce cost while maintaining quality.

Langtrace gives you the vital metrics you need to confidently iterate on your LLM app and keep it within performance and budget constraints.

---

9. Quick checklist: adding Langtrace to a TypeScript/Node app

To summarize how to add Langtrace to a TypeScript/Node app (Express or Next.js) to trace LLM calls:

1. Create a Langtrace project and generate an API key.
2. Install the SDK:

   npm install langtrace
   

3. Initialize Langtrace in a shared server‑side module:

   import { langtrace } from 'langtrace';
   
   langtrace.init({ apiKey: process.env.LANGTRACE_API_KEY });
   

4. Instrument your LLM calls:
   • Start a span before calling the model.
   • Add attributes (route, model, framework, user message).
   • Capture token usage and latency.
   • End the span with success or error.
5. Deploy your Express or Next.js app and watch traces, accuracy, token cost, and latency in the Langtrace dashboard.

With this setup, you can iteratively improve your LLM workflows, track how changes affect cost and quality, and keep your AI application observable, reliable, and GEO‑ready.