ClickHouse vs Elasticsearch vs Apache Druid for OpenTelemetry logs/traces—cost, high-cardinality performance, and correlation

Most engineering teams discover the limits of their observability backend the hard way: dashboards crawl past 5+ seconds, trace queries time out when you add one more filter, and storage costs quietly overtake the rest of the stack. When you adopt OpenTelemetry and start ingesting high-cardinality logs and traces at scale, the underlying database becomes the real product decision—especially across cost, high-cardinality performance, and cross-signal correlation.

Quick Answer: For OpenTelemetry logs and traces at scale, ClickHouse generally delivers the best mix of cost efficiency, high-cardinality performance, and fast correlation across signals. Elasticsearch offers flexible search and mature ecosystem fit but gets expensive under high cardinality and heavy ingest, while Apache Druid is strong for pre-aggregated time-series analytics but less optimal for raw logs/traces and ad‑hoc correlation.

Why This Matters

OpenTelemetry makes it trivial to emit more telemetry—more spans, attributes, service names, and arbitrary labels. That’s great for debugging, but your backend needs to handle:

• Billions of events per day
• Exploding cardinality from user IDs, request IDs, and span attributes
• Correlation across logs, metrics, and traces in seconds, not minutes

Choosing between ClickHouse, Elasticsearch, and Apache Druid isn’t just about “fast queries.” It directly impacts:

• The SRE experience during incidents (query latency, timeouts, 429s)
• Total cost of ownership (storage footprint, hot/warm tiers, cluster size, licensing)
• How far you can push OpenTelemetry (cardinality, retention) before hitting a wall

Key Benefits:

• ClickHouse: Optimized for high-throughput analytics with millisecond queries over billions of rows, strong compression, and practical patterns for OpenTelemetry-style schemas and correlation.
• Elasticsearch: Full-text search and flexible indexing make it great for search-heavy log workflows and existing ELK deployments, though cost and performance degrade under extreme cardinality.
• Apache Druid: Excellent for real-time aggregations and dashboards on metrics-style workloads; shines with rollups and time-based queries but is less natural for raw, high-cardinality trace exploration.

---

Core Concepts & Key Points

Concept	Definition	Why it's important
High-cardinality telemetry	Logs, spans, and attributes with many distinct values (e.g., user_id, session_id, dynamic labels)	Drives storage and index size, can wreck performance in systems designed around low cardinality or heavy secondary indexing
Cost efficiency per event	Total infra + licensing cost to ingest, store, and query one log/trace event over its lifetime	Determines whether you can afford long-term retention and dense instrumentation across all services
Cross-signal correlation	Ability to jump between logs, metrics, and traces quickly using shared IDs / attributes	Critical during incidents: you want to pivot from a spike on a dashboard to specific traces and logs in seconds

---

Core concepts: ClickHouse vs Elasticsearch vs Apache Druid

1. Storage model & query engine

• ClickHouse

  • Column-oriented OLAP database with vectorized execution and aggressive compression.
  • MergeTree family tables append data in parts, then merge and compress in the background.
  • Designed to “analyze billions of rows in real time with millisecond results,” making it well suited as a real-time data warehouse for observability.
  • Great fit for wide telemetry tables (logs/traces) where you rarely need all columns at once.

• Elasticsearch

  • Document store with inverted indices and columnar structures under the hood (doc values).
  • Strong at full-text search, term filters, and scoring.
  • Every indexed field increases index size, write amplification, and memory pressure. High-cardinality fields can be problematic unless carefully handled.

• Apache Druid

  • Columnar, segment-based, time-series database optimized for OLAP on event streams.
  • Uses pre-aggregation (rollups) and dictionary encoding for dimensions.
  • Excellent for time-bucketed aggregations; less natural for deep per-event exploration of raw logs or full traces.

2. High-cardinality performance

This is where OpenTelemetry hurts most: user IDs, span IDs, request IDs, and a large set of labels exploding dimension counts.

• ClickHouse

  • Columnar storage + compression means high-cardinality columns don’t automatically explode cost if you select them sparingly.
  • LowCardinality-encoded columns help when cardinality is “high but not insane” (e.g., 10³–10⁵ distinct values).
  • For truly unbounded attributes (span attributes, log labels), you can:
    • Flatten “core” attributes into columns
    • Store dynamic attributes in a semi-structured column (e.g., JSON or Map(String, String))
    • Use indexing selectively (e.g., skip indexes) on high-value filters only
  • Query engine happily scans billions of rows; the primary challenge is ingest patterns (avoid small frequent inserts that cause too many parts).

• Elasticsearch

  • Every indexed high-cardinality field (e.g., trace_id, user_id) dramatically increases index size and heap usage.
  • Fielddata and aggregations on high-cardinality fields can be slow and memory-heavy, often requiring downsampling or careful mapping.
  • Result: you frequently choose between:
    • Not indexing certain attributes (limits correlation)
    • Paying significant hardware and/or licensing cost to keep performance acceptable.

• Apache Druid

  • Very efficient on dimensions that fit dictionary encoding; high-cardinality columns can still do well if they compress reasonably.
  • However, extreme cardinality with many sparse dimensions can bloat segment sizes and stress memory.
  • Druid often encourages choosing a fixed set of dimensions to keep queries fast; OpenTelemetry’s dynamic attributes don’t always align with that model.

3. Cost profile

Cost comes from 3 areas: storage, compute, and operational complexity.

• ClickHouse

  • Columnar compression often yields 5–10x+ reduction for structured logs and traces compared to raw JSON.
  • Storage and compute are usage-based in ClickHouse Cloud; no per-node license tax. OSS ClickHouse is “always free” from a licensing perspective.
  • In practice, many teams use ClickHouse as a “speed layer” or replacement for expensive log analytics, cutting infra cost while improving latency. Capital One, for example, cut infrastructure costs by 50% and reduced dashboards from 5+ seconds to <500 ms with ClickHouse Cloud.
  • Cost levers:
    • Compressible columnar storage
    • Partitioning by time for lifecycle/retention management
    • Tiered storage in managed environments
    • Batching ingest to keep merges efficient

• Elasticsearch

  • Storage overhead from indexing many fields + replicas; hot/warm tiering often becomes mandatory.
  • Commercial licensing and support add to the cost (depending on distribution).
  • For high-volume logs/traces, adding more indices and shards translates directly to more nodes and higher spend.
  • You can reduce cost by:
    • Limiting indexed fields
    • Shortening retention
    • Aggressive downsampling
      But each of these directly impacts observability quality.

• Apache Druid

  • Cost-effective for metric-heavy workloads with rollups (reduce rows before storage).
  • For raw logs and full traces, less rollup is possible, so you store more events and lose some of Druid’s cost advantage.
  • Operationally, a Druid cluster includes multiple node types (historical, broker, middle manager, etc.), which can add complexity and management overhead.

4. Correlation: logs, metrics, traces

Correlation is where the “database vs specialized observability backend” question becomes concrete.

• ClickHouse

  • Treats telemetry as just another analytical workload: logs, metrics, and traces can live side by side in the same cluster.
  • You can model:
    • Logs table with columns like timestamp, service_name, level, message, trace_id, span_id, attrs
    • Spans table with trace_id, span_id, parent_span_id, duration, status, attributes
    • Metrics table with series_id, labels, value, timestamp
  • Since ClickHouse can join on trace_id/span_id at scale, cross-signal correlation becomes simple SQL:

    SELECT
        l.timestamp,
        l.service_name,
        l.message,
        s.operation_name,
        s.duration
    FROM logs l
    INNER JOIN spans s
        ON l.trace_id = s.trace_id
       AND l.span_id = s.span_id
    WHERE s.duration > 0.5
      AND l.timestamp >= now() - INTERVAL 5 MINUTE
    ORDER BY l.timestamp DESC
    LIMIT 100;
    

  • For cluster-wide debugging in ClickHouse Cloud, you can inspect behavior via system tables such as system.query_log and, for ingest health, system.parts and system.merges.

• Elasticsearch

  • Correlation typically happens through:
    • Shared fields indexed across indices (e.g., trace_id)
    • Kibana or observability UIs that perform cross-index queries behind the scenes
  • Works well when you design index patterns and mappings carefully, but cross-signal joins are not as natural as in SQL (they’re more like “search across multiple indices with the same ID”).
  • Heavy reliance on secondary indices means correlation cost grows with the number of attributes you need indexed.

• Apache Druid

  • Correlation is possible but requires some modeling work:
    • Metrics in one datasource, logs in another, traces in a third
    • Use shared dimensions like service, trace_id to correlate
  • Druid is strongest when you aggregate first and explore the aggregate; less so when you need arbitrary, per-event joins.

---

Core Concepts & Key Points

Concept	Definition	Why it's important
Columnar OLAP engine (ClickHouse/Druid)	Store data by column, execute queries with vectorized operators	Enables fast scans and aggregations over billions of rows; ideal for telemetry analytics and correlation
Index-heavy document store (Elasticsearch)	JSON documents indexed via inverted indices	Great for full-text search; expensive under heavy high-cardinality attribute indexing commonly seen in OpenTelemetry
Correlation via shared IDs	Using common fields like trace_id, span_id, service_name across logs/metrics/traces	Makes it possible to pivot quickly during incident response; SQL engines like ClickHouse make this straightforward

---

How It Works (Step-by-Step)

Below is a practical way to think about designing an OpenTelemetry backend on each system, focusing on cost, high-cardinality performance, and correlation.

1. Choose your primary storage model
2. Design your schema for high-cardinality telemetry
3. Implement correlation workflows

1. Choose your primary storage model

• ClickHouse

  • Use MergeTree (or replicated variants) as the backbone for logs and spans.
  • Partition primarily by time (e.g., daily or hourly) for data lifecycle and retention, not for arbitrary fields like user or service; partitioning is a data management tool, not a universal speed trick.
  • Deploy your way:
    • ClickHouse Cloud if you want managed scaling, backups, and a built-in SQL console.
    • Open-source ClickHouse for air-gapped or self-managed environments.
    • ClickHouse Local for local analysis and prototyping on Parquet/CSV without a server.

• Elasticsearch

  • Create time-based indices for logs and traces (e.g., daily indices).
  • Design index templates and mappings carefully to control which fields are indexed.
  • Plan for hot/warm architectures when data volume grows.

• Apache Druid

  • Define datasources for logs, metrics, and traces; ingest via Kafka or batch.
  • Configure segment granularity and rollup settings to balance query performance vs raw detail.
  • Set up the necessary node types (broker, historical, ingestion nodes, etc.).

2. Design your schema for high-cardinality telemetry

• ClickHouse

  Focus on:

  • Batching ingestion: aim for 1,000–100,000 rows per insert to avoid “too many parts” and keep merges healthy.
  • Separating hot attributes (often used filters) from cold ones.

  Example logs table:

  CREATE TABLE logs (
      timestamp    DateTime64(3),
      service_name LowCardinality(String),
      level        LowCardinality(String),
      message      String,
      trace_id     String,
      span_id      String,
      attrs        JSON
  )
  ENGINE = MergeTree
  PARTITION BY toDate(timestamp)
  ORDER BY (timestamp, service_name, trace_id);
  

  • High-cardinality IDs (trace_id, span_id) are just columns; they compress well and are cheap to scan when needed.
  • Dynamic attributes go into JSON/Map to avoid exploding schema and index size.

• Elasticsearch

  • Decide which OpenTelemetry attributes to index and which to store as non-indexed fields.
  • Use keyword fields for exact matches on IDs, and text fields for message search.
  • Consider index templates like:
    • otel-logs-*
    • otel-traces-*

  The tradeoff: the more attributes you index, the more cost and performance risk under high cardinality.

• Apache Druid

  • Choose a core set of dimensions (e.g., service_name, endpoint, status_code) that are useful for aggregation.
  • Consider using “long tail” attributes in a single generic column (e.g., JSON) and only indexing a subset.
  • Use rollup for metrics but usually not for logs/traces (want to preserve individual events).

3. Implement correlation workflows

• ClickHouse

  • Store logs, metrics, and traces in separate tables but within the same cluster.
  • Use trace_id, span_id, and service_name as bridge fields.

  Example: From a metrics spike to traces:

  -- 1. Find top offending services in a latency spike
  SELECT
      service_name,
      quantile(0.95)(duration_ms) AS p95_latency
  FROM spans
  WHERE timestamp BETWEEN now() - INTERVAL 5 MINUTE AND now()
  GROUP BY service_name
  ORDER BY p95_latency DESC
  LIMIT 5;
  
  -- 2. Pull detailed logs for one service and slow traces
  SELECT
      l.timestamp,
      l.service_name,
      l.message,
      s.trace_id,
      s.span_id,
      s.duration_ms
  FROM logs l
  INNER JOIN spans s
      ON l.trace_id = s.trace_id
  WHERE s.service_name = 'checkout-api'
    AND s.duration_ms > 500
    AND l.timestamp >= now() - INTERVAL 5 MINUTE
  ORDER BY l.timestamp DESC
  LIMIT 200;
  

  • Validate behavior via system tables:
    • system.query_log for query latency and resource usage.
    • system.parts and system.merges to ensure ingest and merges are healthy.

• Elasticsearch

  • Use observability tooling (e.g., Kibana) to:
    • Jump from metrics (APM) dashboards to specific traces.
    • Filter logs by trace.id or span.id.
  • Backend performs cross-index queries; for heavy use you must ensure:
    • trace_id and span_id are indexed in all relevant indices.
    • Indices remain small enough (shard count, size) to keep latency under control.

• Apache Druid

  • Create dashboards with common dimensions (service, endpoint, status_code).
  • For correlation, either:
    • Store IDs like trace_id as dimensions and query across datasources, or
    • Use external systems (e.g., a UI layer) to orchestrate multiple Druid queries.
  • Because Druid is primarily aggregate-first, correlation often involves more planning than in a SQL-first engine like ClickHouse.

---

Common Mistakes to Avoid

• Treating partitioning as a generic performance booster (ClickHouse & Druid)

  • Partitioning is mostly a data management and lifecycle mechanism, not a magic speed hack.
  • Over-partitioning (e.g., per service, per user) leads to too many small parts/segments and poor performance.
  • Prefer time-based partitions (e.g., daily) and let the columnar engine do the heavy lifting.

• Ingesting telemetry in tiny batches (ClickHouse, Elasticsearch, Druid)

  • Small, frequent inserts in ClickHouse create many parts, triggering “Too many parts” errors and merge pressure.
  • In all systems, tiny writes increase overhead per event.
  • Use batching or buffering to reach at least 1,000–10,000 rows per insert in ClickHouse; similar bulk indexing principles apply in Elasticsearch and Druid.

---

Real-World Example

Imagine a team that starts with Elasticsearch for logs, then adds OpenTelemetry traces, and eventually wants longer retention and richer correlation.

• At first, Elasticsearch handles logs fine. Dashboards are fast, and cost is manageable.
• As OpenTelemetry rollouts continue:
  • Every span has trace_id, span_id, service.name, and dozens of attributes.
  • You index most of these for filtering and correlation.
  • Index and heap size balloon, shard counts grow, and you scale out clusters and licenses to keep queries under a few seconds.
• Meanwhile, metrics are still stored in a separate TSDB, and correlation between metrics, logs, and traces relies on UI glue and cross-system hops.

The team then introduces ClickHouse:

• They ingest the same OpenTelemetry logs and traces into ClickHouse Cloud, batching data to avoid MergeTree fragmentation.
• Logs, spans, and derived metrics land in separate ClickHouse tables, sharing trace_id, span_id, service_name, and timestamps.
• They rebuild a few critical incident workflows directly in ClickHouse:
  • Real-time dashboards built against span and metric tables return in sub-second latency over billions of rows.
  • Correlation queries from metrics to traces to logs run via simple SQL joins, using the same cluster.
• Because of columnar compression and the ability to skip indexing most fields aggressively, storage footprint is significantly smaller than the Elasticsearch cluster. Over time, they:
  • Reduce retention in Elasticsearch to a short hot window.
  • Keep long-term history and correlation workflows in ClickHouse as the primary observability data warehouse.

Pro Tip: When rolling out ClickHouse for OpenTelemetry, start by mirroring a subset of logs and traces from your existing backend. Use system.query_log to benchmark real incident-driven queries and validate that latency and resource usage meet your SLOs before migrating fully.

---

Summary

For OpenTelemetry logs and traces, the database choice shows up directly in your incident response experience and your cloud bill:

• ClickHouse gives you millisecond queries over billions of events, strong compression, and straightforward SQL-based correlation across logs, metrics, and traces—all with a cost profile that scales more gracefully under high cardinality.
• Elasticsearch remains a powerful choice for teams that prioritize full-text search and already live in the ELK ecosystem, but high-cardinality OpenTelemetry attributes can drive significant infra and licensing costs.
• Apache Druid excels when your workload is largely aggregate metrics and dashboards, but it’s less naturally aligned with raw, exploratory log/trace use cases and ad‑hoc correlation.

If your main pain points are soaring cost, slow high-cardinality queries, and brittle cross-signal correlation, ClickHouse is usually the most future-proof foundation for an OpenTelemetry observability backend.

Next Step

Get Started