How do I add OpenTelemetry tracing in Mastra and export to Datadog/New Relic/SigNoz?

OpenTelemetry tracing is the bridge between Mastra’s AI-aware observability and your existing monitoring stack in Datadog, New Relic, or SigNoz. In Mastra, you configure tracing once via Observability and Exporter classes, then forward those spans to any OpenTelemetry-compatible backend.

Quick Answer: Mastra provides OpenTelemetry-style traces out of the box via Observability and DefaultExporter. To export to Datadog, New Relic, or SigNoz, you add an OpenTelemetry exporter (OTLP HTTP/GRPC) in your Node process and point it at your provider’s OTLP endpoint while keeping Mastra’s own exporters (Default/Cloud) enabled.

---

Quick Answer: Mastra’s observability is OpenTelemetry-aligned and can export traces to any OpenTelemetry-compatible provider (Datadog, New Relic, SigNoz, etc.) by configuring OTLP exporters alongside DefaultExporter/CloudExporter in your Mastra instance.

Frequently Asked Questions

How does OpenTelemetry tracing work in Mastra for Datadog/New Relic/SigNoz?

Short Answer: Mastra captures AI-specific traces via Observability and exporters, and you relay those traces to Datadog/New Relic/SigNoz using standard OpenTelemetry OTLP exporters from your Node.js app.

Expanded Explanation:
Mastra ships with specialized tracing for AI workloads—agents, workflows, RAG, tools, and memory. When you configure Observability with DefaultExporter, Mastra persists traces to your storage backend so you can inspect them in Mastra Studio; CloudExporter optionally sends them to Mastra Cloud.

To integrate with Datadog, New Relic, or SigNoz, you don’t re-implement tracing inside Mastra. Instead, you run an OpenTelemetry SDK/exporter in your Node process that forwards the same spans Mastra generates to your external provider via OTLP (HTTP or gRPC). Those providers already understand OpenTelemetry, so once the exporter is configured, your Mastra traces show up alongside the rest of your service telemetry.

Key Takeaways:

• Mastra owns AI-specific tracing; Datadog/New Relic/SigNoz are just additional OpenTelemetry consumers.
• You keep DefaultExporter/CloudExporter for Mastra observability and add OTLP exporters to stream spans to your observability provider.

---

How do I set up OpenTelemetry in Mastra and export traces step-by-step?

Short Answer: Configure Observability in your Mastra instance with DefaultExporter, then add an OpenTelemetry SDK/exporter in your Node app that sends traces to Datadog/New Relic/SigNoz via OTLP.

Expanded Explanation:
The process breaks cleanly into two parts:

1. Enable Mastra’s observability so your agents, workflows, and tools emit traces.
2. Add an OpenTelemetry pipeline (TracerProvider + OTLP exporter) in your runtime that forwards those spans to your observability vendor.

Mastra doesn’t lock you into a proprietary trace format; its spans can be bridged into any OpenTelemetry-compatible backend. Below is a practical flow you can adapt to your stack (Next.js, Express, Hono, etc.).

Steps:

1. Configure Observability in your Mastra instance

   // src/mastra/index.ts
   import { Mastra } from '@mastra/core';
   import { PinoLogger } from '@mastra/loggers';
   import { LibSQLStore } from '@mastra/libsql';
   import {
     Observability,
     DefaultExporter,
     CloudExporter,
     SensitiveDataFilter,
   } from '@mastra/observability';
   
   export const mastra = new Mastra({
     logger: new PinoLogger(),
     storage: new LibSQLStore({
       id: 'mastra-storage',
       // Important: for serverless, don't use file:./mastra.db
       url: 'file:./mastra.db', // good for local/dev only
     }),
     observability: new Observability({
       configs: {
         default: {
           serviceName: 'mastra',
           exporters: [
             new DefaultExporter(), // persists traces for Mastra Studio
             new CloudExporter(),   // sends traces to Mastra Cloud if MASTRA_CLOUD_ACCESS_TOKEN is set
           ],
           spanOutputProcessors: [
             new SensitiveDataFilter(), // optional: scrub PII/secrets
           ],
         },
       },
     }),
   });
   

   • This gives you Mastra-native traces in Studio and (optionally) Mastra Cloud.
   • For production/high traffic, prefer a ClickHouse-backed storage setup via composite storage, not file:./mastra.db.

2. Install OpenTelemetry SDK + OTLP exporter

   For a generic OTLP HTTP pipeline (works with many providers, including SigNoz and some Datadog/New Relic configs):

   pnpm add \
     @opentelemetry/api \
     @opentelemetry/sdk-trace-node \
     @opentelemetry/exporter-trace-otlp-http
   

3. Initialize OpenTelemetry before your app/agents run

   // tracing.ts
   import { NodeTracerProvider } from '@opentelemetry/sdk-trace-node';
   import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
   import { SimpleSpanProcessor } from '@opentelemetry/sdk-trace-base';
   import { trace } from '@opentelemetry/api';
   
   const provider = new NodeTracerProvider({
     resource: /* optional: set service.name, env, etc. */,
   });
   
   const otlpExporter = new OTLPTraceExporter({
     // For SigNoz or OTLP-compatible backends
     url: process.env.OTEL_EXPORTER_OTLP_ENDPOINT, // e.g. https://otel-collector.mycompany.com/v1/traces
     headers: {
       // If your provider requires auth
       Authorization: `Bearer ${process.env.OTEL_EXPORTER_TOKEN ?? ''}`,
     },
   });
   
   provider.addSpanProcessor(new SimpleSpanProcessor(otlpExporter));
   provider.register();
   
   // Optional: export a tracer if you want to create custom spans around Mastra calls
   export const tracer = trace.getTracer('mastra-app');
   

   Then import this file once at the top-level of your app:

   // server.ts or next.config.mjs or entrypoint
   import './tracing';
   import { mastra } from './src/mastra';
   // ...start HTTP server or framework boot...
   

   At this point:

   • Mastra generates AI-specific traces.
   • Mastra exporters (DefaultExporter, CloudExporter) handle Studio/Cloud.
   • OpenTelemetry’s OTLP exporter streams those traces to Datadog/New Relic/SigNoz via your collector endpoint.

---

What’s the difference between Mastra’s observability and Datadog/New Relic/SigNoz?

Short Answer: Mastra observability is AI-aware and dev-focused (agents, tools, token usage, decisions), while Datadog/New Relic/SigNoz are general-purpose observability platforms; you typically use them together, not instead of one another.

Expanded Explanation:
Mastra’s observability answers questions like:

• What did this agent decide at each step?
• Which tools did it call and with what input/output?
• How many tokens and how much latency did each model call incur?

Datadog, New Relic, and SigNoz provide cross-service observability, correlating your Mastra traces with HTTP requests, databases, queues, and infrastructure metrics. You use Mastra’s Studio/Cloud when you’re debugging agent behavior and workflows; you use Datadog/New Relic/SigNoz when you need a full-system view, SLOs, alerts, and dashboards.

Comparison Snapshot:

• Mastra Observability: AI-first tracing (agents, workflows, RAG, memory) with Studio/Cloud views and processors like SensitiveDataFilter.
• Datadog/New Relic/SigNoz: Platform-wide tracing, metrics, and logs with alerts and dashboards across all services.
• Best for: Use Mastra for deep AI/agent debugging and Datadog/New Relic/SigNoz to correlate AI behavior with the rest of your production stack.

---

How do I implement exports specifically for Datadog, New Relic, or SigNoz?

Short Answer: Use the provider’s OpenTelemetry SDK or OTLP configuration (Datadog: @datadog/datadog-api-client/OTLP; New Relic: OTLP or their Node agent; SigNoz: OTLP to their collector) alongside Mastra’s existing exporters.

Expanded Explanation:
Each provider has its own recommended way to ingest OpenTelemetry traces, but they all accept OTLP. The pattern is:

• Keep Observability with DefaultExporter (and optionally CloudExporter) in Mastra.
• Configure a provider-specific or generic OTLP exporter pointed at that provider’s ingestion endpoint.
• Ensure the exporter initializes before you create and run Mastra agents/workflows.

Below are typical setups (pseudo-config; always check the provider’s latest docs).

What You Need:

• Datadog:

  • Enable OpenTelemetry ingestion or run the Datadog Agent with OTLP.

  • Configure an OTLP exporter to point at the Datadog agent or endpoint:

    const otlpExporter = new OTLPTraceExporter({
      url: process.env.DD_OTLP_HTTP_ENDPOINT, // e.g. http://datadog-agent:4318/v1/traces
      headers: {
        'DD-API-KEY': process.env.DATADOG_API_KEY ?? '',
      },
    });
    

• New Relic:

  • Use their OTLP endpoint or OpenTelemetry integration:

    const otlpExporter = new OTLPTraceExporter({
      url: process.env.NEW_RELIC_OTLP_ENDPOINT, // e.g. https://otlp.nr-data.net/v1/traces
      headers: {
        'api-key': process.env.NEW_RELIC_LICENSE_KEY ?? '',
      },
    });
    

• SigNoz:

  • Point OTLP directly at your SigNoz or OpenTelemetry Collector:

    const otlpExporter = new OTLPTraceExporter({
      url: process.env.SIGNOZ_OTEL_COLLECTOR_ENDPOINT, // e.g. http://otel-collector:4318/v1/traces
    });
    

In all cases, Mastra’s internal observability configuration stays the same; you’re just adding another exporter pipeline that forwards spans to your chosen backend.

---

How should I design an observability strategy for Mastra with OpenTelemetry?

Short Answer: Treat Mastra observability as your AI control surface and OpenTelemetry as your transport layer; use Mastra Studio/Cloud for deep agent debugging and Datadog/New Relic/SigNoz for cross-service monitoring, alerts, and SLOs.

Expanded Explanation:
Production agents are infrastructure. That means you want:

• AI-aware traces and tools for debugging (Mastra).
• Centralized telemetry across services (Datadog/New Relic/SigNoz).
• Storage and exporters that can handle high traffic cost-effectively.

A robust setup looks like:

• Build and iterate:
  • Use DefaultExporter + local LibSQLStore during development.
  • Inspect traces in Mastra Studio to refine agents, workflows, and tools.
• Productionize and test:
  • Migrate observability storage to a production-grade backend (ClickHouse via composite storage recommended for high-volume traces).
  • Add processors like SensitiveDataFilter to guard against prompt injection side effects and PII leaks.
  • Wire OpenTelemetry OTLP exporters to your observability provider.
• Deploy and scale:
  • Run Mastra inside your existing HTTP stack (Next.js, Express, Hono) with tracing initialized on boot.
  • Export traces to Mastra Cloud and OpenTelemetry platforms for dashboards, alerts, and incident response.
  • Continuously define and run evals to track agent quality over time.

Why It Matters:

• You get end-to-end visibility into agent decisions, token usage, and tool calls, not just 500 errors and latency charts.
• You reduce time-to-debug and operational risk when agents move from a demo notebook to production traffic.

---

Quick Recap

Mastra gives you AI-specific observability via Observability, DefaultExporter, and CloudExporter. By layering OpenTelemetry on top, you can export those traces to Datadog, New Relic, SigNoz, or any OpenTelemetry-compatible backend. The pattern is simple: enable Mastra observability, initialize a Node OpenTelemetry pipeline with an OTLP exporter pointing at your provider, and keep using Mastra Studio/Cloud as your AI debugging surface while your observability platform handles cross-service monitoring and alerting.

Next Step

Get Started