How do I configure tiered storage (SSD + S3) in TigerData Tiger Cloud for long retention?

For long-term telemetry retention, tiered storage in Tiger Cloud lets you keep hot data on high-performance SSD while automatically offloading older chunks to low-cost S3—without changing your SQL or app code. You size for performance, not for “all data forever,” and let Tiger Cloud handle the lifecycle.

Quick Answer: In Tiger Cloud, tiered storage is configured at the service level and driven by data age. You choose a high-performance storage size for compressed SSD, enable tiering, and set retention/tiering policies so older chunks move to “bottomless” S3 while remaining queryable via the same Postgres interface.

The Quick Overview

What It Is: A Postgres-native storage layout where recent data lives on SSD and historical data is automatically tiered to S3 object storage, managed by Tiger Cloud.
Who It Is For: Teams ingesting large time-series/event/tick workloads (IoT metrics, logs, financial ticks) that must keep months or years of history without blowing up storage cost or operational complexity.
Core Problem Solved: Plain Postgres struggles when you try to keep “all telemetry forever” on SSD—storage fills up, queries slow down, and you end up stitching Kafka + lake + warehouse just to keep costs under control. Tiger Cloud’s tiered storage gives you long retention, fast queries, and one Postgres system.

How It Works

Tiger Cloud combines three primitives:

Hypertables (TimescaleDB) for time- and key-based partitioning.
Hypercore row-columnar storage for row-optimized ingest and columnar-optimized analytics.
Tiered storage that automatically moves cold chunks from SSD to S3 while keeping them queryable.

Your workflow looks like this:

Provision a Tiger Cloud service with tiered storage enabled
You pick a service plan (Performance/Scale/Enterprise), size the high-performance storage (SSD), and ensure tiering is turned on so data beyond a given age can live in S3.
Model data with hypertables and policies
You create hypertables for time-series/event/tick data, then configure compression and retention policies. Compression reduces SSD footprint by up to 98%; retention/tiering determines when chunks can be moved to S3 or dropped.
Let Tiger Cloud manage data movement transparently
New data lands on SSD. As it ages, it’s compressed and, once past your “hot” window, automatically tiered to S3 object storage. Queries still run via standard SQL; Tiger Cloud fetches needed chunks from SSD and S3 under the hood.

1. Provisioning a Tiger Cloud service with tiered storage

In Tiger Cloud, each service gives you:

High-performance storage (SSD)
- Up to ~64 TB compressed (typically 80–100 TB uncompressed).
- Metered on your average GB consumption per hour.
Low-cost bottomless storage (S3)
- Used for older, less frequently accessed chunks.
- Charged based on original uncompressed size of the data that’s been tiered.
- No extra data transfer or compute fees for tiered data.

When you create or upgrade a service:

Choose a plan (e.g., Performance, Scale, Enterprise) based on ingest rate, query SLA, and compliance needs (SOC 2, HIPAA on Enterprise).
Set high-performance storage size to cover:
- Your hot/data “working set” (recent N days/weeks you query constantly).
- A buffer for short bursts and backfills.
Ensure tiered storage is enabled in the service configuration so Tiger Cloud can move cold chunks to S3 automatically.

Note: The fastest way to size SSD is to run a realistic workload in the free trial. Because compression often reaches up to 98%, your “effective uncompressed” data volume is much larger than the raw SSD size.

2. Designing hypertables and policies for long retention

Tiered storage works best when your time-series schema follows TimescaleDB best practices.

Create a hypertable

You model your telemetry as a hypertable, partitioned by time and optionally by a dimension like device_id, service, or tenant_id:

CREATE TABLE metrics_raw (
    time        TIMESTAMPTZ       NOT NULL,
    device_id   UUID              NOT NULL,
    metric      TEXT              NOT NULL,
    value       DOUBLE PRECISION  NOT NULL,
    tags        JSONB,
    PRIMARY KEY (time, device_id, metric)
);

SELECT create_hypertable(
    'metrics_raw',
    'time',
    chunk_time_interval => INTERVAL '1 day'
);

Important: Always partition on your primary time column. Tiering and compression operate at the chunk level; predictable time-based chunks make lifecycle behavior reliable and cost-efficient.

Enable compression to shrink SSD footprint

Compression is the first lever for long retention on SSD:

ALTER TABLE metrics_raw
SET (
    timescaledb.compress,
    timescaledb.compress_segmentby = 'device_id',
    timescaledb.compress_orderby   = 'time DESC'
);

SELECT add_compression_policy(
    'metrics_raw',
    INTERVAL '7 days'  -- compress chunks older than 7 days
);

What happens:
- Chunks older than 7 days are compressed in-place.
- Compression can reduce storage usage by up to 98% depending on cardinality and schema.
Why it matters: You might be able to keep weeks or months of compressed data on SSD before you need to rely heavily on S3, which keeps latency lower for most queries.

Configure retention for long-term history

Retention policy governs how long data stays in the system at all:

SELECT add_retention_policy(
    'metrics_raw',
    INTERVAL '365 days'  -- keep 1 year of data
);

When combined with tiering:

Recent data (e.g., 0–7 days): uncompressed on SSD.
Warm data (e.g., 7–90 days): compressed on SSD.
Cold data (e.g., 90–365 days): compressed and tiered to S3.
Older than 365 days: dropped per retention policy.

Note: The exact “SSD vs S3” boundary is controlled by your Tiger Cloud service’s tiering settings (described below), not by Timescale functions themselves. Compression and retention define what to keep and in what format; tiered storage defines where it lives.

3. Configuring tiered storage behavior in Tiger Cloud

Once your schema is in place, you control tiering at the service level in Tiger Console (UI) or via API/CLI.

At a high level, you’ll:

Enable tiered storage (if not already on for your plan).
Set the age threshold for tiering.
Monitor SSD usage vs “effective” dataset size.

While exact UI labels may evolve, the core concepts remain:

Tiering threshold
For example: “Tier chunks older than 90 days to low-cost storage.”
- Chunks older than 90 days are marked for object storage; their SSD footprint is minimized.
- Queries spanning data older than 90 days may fetch chunks from S3, with slightly higher latency than purely-SSD reads, but still via the same SQL.
Independent storage scaling
If your SSD usage grows (higher ingest or longer hot window), you can increase the high-performance storage allocation without touching compute. Cold data in S3 remains unaffected.

Important: Tiger Cloud bills tiered storage based on the original uncompressed size of the data, not on compressed or physical S3 size. This gives you predictable, transparent costs across tables and schemas.

Operationally, here’s the lifecycle:

Ingest hits hypertables → row-store on SSD.
Policy compresses older chunks → columnar compressed on SSD.
Tiering moves very old chunks → compressed representation in S3 (“bottomless”).
Queries run across all of the above → Tiger Cloud handles fetching from SSD + S3; you keep writing normal SQL.

Features & Benefits Breakdown

Core Feature	What It Does	Primary Benefit
High-performance storage (SSD)	Stores hot and warm data with row-columnar layout and compression	Sub-second queries on recent data and high ingest rates for live telemetry
Bottomless S3 tier	Holds compressed, historical chunks based on age/tiering rules	Long-term retention (months/years) without scaling SSD to petabyte levels
Independent storage scaling	Lets you adjust SSD size without redeploying compute or ETL-ing to another DB	Aligns storage spend with actual hot data needs; avoids “over-provision everything” planning

Ideal Use Cases

Best for IoT / metrics platforms needing 1–3 years of history:
Because you can keep the last N days/weeks on SSD for dashboards and alerting, while older metrics live in S3 but remain queryable for audits, capacity planning, and anomaly investigations.
Best for event/log analytics with unpredictable spikes:
Because Tiger Cloud scales SSD independently and tiers older chunks, so you can absorb peaks now and rely on compression + S3 to keep long-term costs predictable.

Limitations & Considerations

Cold-query latency:
Queries that scan mostly S3-tiered data will be slower than SSD-only queries.
- Workaround: Use continuous aggregates for common historical rollups, and keep their materialized views within the SSD window for fast reads.
Policy correctness and backfills:
Aggressive compression/tiering windows can surprise you when backfilling old data or doing late-arriving updates.
- Workaround:
  - Set realistic compression and tiering thresholds for your data arrival patterns.
  - Use separate hypertables or staging tables for large backfills, then move data into main tables.

Pricing & Plans

Tiger Cloud’s storage pricing aligns directly with how tiered storage works:

High-performance SSD storage
- Metered on average GB consumption per hour.
- Designed for up to ~64 TB compressed (80–100 TB uncompressed) per service.
- Compression (up to 98%) lets you keep much more logical data than the raw SSD size suggests.
Low-cost S3 storage
- Charged based on the original uncompressed size of data that has been tiered.
- No additional charges for data transfer or compute related to tiered storage.
- Effectively “bottomless” for long-term retention.

There are no per-query fees, no separate charges for automated backups, and no surprise data egress fees for querying tiered data.

Example positioning:

Performance Plan: Best for teams needing high ingest + fast queries on weeks of data, with tiering used primarily for compliance and rare historical lookbacks.
Scale/Enterprise Plans: Best for organizations needing multi-year telemetry, strict SLAs, SOC 2/HIPAA, and the ability to separate workloads (production vs heavy analytics) while relying heavily on tiering to keep costs down.

Note: The easiest way to understand your cost profile is to sign up for a free trial, run a realistic workload, and inspect storage usage and compression ratios in Tiger Console.

Frequently Asked Questions

How do I decide the right SSD vs S3 split for my workload?

Short Answer: Size SSD for your hot and warm working set (recent data you query daily), then rely on compression and tiering to keep older data in S3.

Details:
Start from query behavior, not from total data size:

Identify the “interactive” window (e.g., last 7, 30, or 90 days) for dashboards, alerting, and debugging. That window should fit comfortably in compressed SSD.
Estimate your daily ingest volume and typical compression ratio (run a trial and check pg_total_relation_size before/after compression).
Add a buffer (e.g., 30–50%) for bursts, schema changes, and backfills.
Set compression policies to fire soon after data leaves the hard real-time window; set tiering age to when queries are rare but still needed.
Use retention policies to drop data that has no business value past a certain age to avoid paying for storage (SSD or S3) you don’t use.

You can adjust SSD size over time without re-architecting; Tiger Cloud lets storage grow with your workload.

Do I need to change my queries or apps to read from S3-tiered data?

Short Answer: No. You keep using standard Postgres/Timescale SQL; Tiger Cloud handles SSD vs S3 under the hood.

Details:
Tiering is a physical storage concern, not a logical one:

Your schema, indexes, hypertables, and continuous aggregates look the same.
Clients connect via psql, PgJDBC, pgx, etc., and run the same SELECT, INSERT, and UPDATE statements.
When a query touches older chunks that are tiered:
- Tiger Cloud fetches and scans them as needed.
- You may see higher latency compared to SSD-only queries, especially for large historical ranges.
To keep UX snappy:
- Use continuous aggregates for pre-computed rollups (daily/weekly metrics).
- Limit interactive dashboards to the hot SSD window.
- Reserve S3-heavy scans for offline jobs and deeper analyses.

No app-side sharding, no dual-writes to a warehouse, and no extra ETL jobs are required just to get long retention.

Summary

Configuring tiered storage in Tiger Cloud is about combining three levers:

Hypertables + compression to keep hot and warm data efficient on SSD (often up to 98% smaller).
Service-level tiering to push older chunks to low-cost “bottomless” S3 while keeping them queryable through Postgres.
Retention policies and independent storage scaling so you store the right data for the right duration, and pay only for the storage you actually use.

You get live telemetry performance on SSD, long-term retention on S3, and one Postgres-native system instead of fragile, multi-database pipelines.

Next Step

Get Started