together.ai vs DeepInfra: SOC 2 Type II, data retention, and enterprise security review—what’s different?

For most AI teams, the real “platform decision” isn’t just about model quality or price—it’s about whether the provider’s security, compliance, and data-retention posture can survive a legal review and a production security audit. That’s where the differences between together.ai and DeepInfra show up most clearly.

Quick Answer: together.ai is an AI Native Cloud with AICPA SOC 2 Type II, tenant-level isolation, and explicit “your data and models remain fully under your ownership” guarantees, designed for high‑traffic, regulated enterprise workloads. DeepInfra is a leaner inference provider; powerful for individual developers and smaller teams, but with a less mature story around audited controls, data governance, and enterprise‑grade security posture.

---

The Quick Overview

• What It Is: A comparison of together.ai and DeepInfra focused on SOC 2 Type II, data retention, and security controls—specifically for teams running production AI workloads.
• Who It Is For: Engineering, security, and data leaders evaluating AI inference platforms for regulated or risk‑sensitive use cases (finance, healthcare, enterprise SaaS, media, etc.).
• Core Problem Solved: How to choose an AI inference platform that hits latency and cost SLOs and passes security, compliance, and data-governance scrutiny.

---

How It Works: Security & Compliance Posture at a Glance

When you evaluate AI infrastructure for production, three questions dominate the security review:

1. Are controls independently audited?
   This is where SOC 2 Type II matters: it’s a third‑party audit of security, availability, and confidentiality controls over time.

2. What happens to my data?
   You need clarity on data retention, training usage, and environment isolation (multi‑tenant vs dedicated).

3. What’s the operational risk at scale?
   You’re evaluating uptime, burst handling, and whether the provider can maintain performance without leaking or co‑mixing data.

Here’s how together.ai and DeepInfra generally align:

1. together.ai: Research‑grade infrastructure with enterprise‑grade security.

   • AICPA SOC 2 Type II attestation.
   • Clear guarantees: Your data and models remain fully under your ownership.
   • Encryption in transit and at rest; tenant-level isolation; region‑aware storage and compute.
   • Designed for production: 99.9% uptime, NVIDIA preferred partner, used by customers like Salesforce AI Research and Cursor for large‑scale, low‑latency inference.

2. DeepInfra: Focused inference platform with lighter enterprise story.

   • Emphasis on speed of access to open models and developer‑friendly pricing.
   • Documentation and marketing tend to focus more on model catalog and price than on audited controls, data residency, and formal governance.
   • Suitable for smaller teams / prototypes where formal compliance is not a blocker; may need additional due diligence for regulated workloads.

3. Enterprise takeaway:

   • If you’re in a compliance‑sensitive environment (SOC 2 expectations, DPA reviews, security questionnaires), together.ai is built to clear that bar.
   • If you’re mostly optimizing for fast access to open‑source models for non‑sensitive workloads, DeepInfra can be a practical option, but you’ll likely layer your own controls on top.

---

Feature-by-Feature: Security, Isolation, and Data Governance

Core Feature	together.ai	DeepInfra (typical posture)	Why It Matters
SOC 2 Type II	Yes – AICPA SOC 2 Type II	Not publicly emphasized as of latest public info	Enterprises often require SOC 2 for vendor approval.
Data Ownership Language	Explicit: “Your data and models remain fully under your ownership.”	Less explicit; must be verified in ToS/DPA	Legal clarity on IP and data rights.
Data Usage for Training	Strong privacy stance; no reuse by default for model training without explicit agreement	Varies by provider configuration; must be read in terms	Critical for customer IP protection and regulatory alignment.
Encryption In Transit/At Rest	Yes – stated explicitly	Likely TLS in transit; at‑rest practices vary, not front‑and‑center in messaging	Baseline security expectation, especially for PII/PHI.
Tenant-Level Isolation	Yes – tenant-level isolation for production inference	Multi‑tenant by default; dedicated options may be limited or less documented	Reduces cross‑tenant risk; important for regulated workloads.
Regional Deployment / Residency	Storage and compute in North America, Europe, or Asia/Middle East to align with data residency needs	Region options may exist but are less tightly framed around compliance	Helps satisfy GDPR and data‑locality requirements.
Dedicated Inference Modes	Dedicated Model Inference, Dedicated Container Inference, GPU Clusters with hard isolation	Primarily shared infrastructure; dedicated capacity options depend on offering evolution	Isolation, performance SLOs, and blast radius control.
Audit & Compliance Story	SOC 2 Type II, HIPAA‑aligned options, NVIDIA preferred partner	Primarily performance / pricing narrative	Security and compliance teams look for third‑party validation, not just vendor claims.
Production Proof Points	Salesforce AI Research: ~33% cost savings, 2x latency reduction; <400ms p95 latency, 6x cost reductions	Case studies less focused on audited security posture	Shows that security measures coexist with high performance at scale.

---

Deep Dive: SOC 2 Type II, Data Retention, and Enterprise Controls

SOC 2 Type II: Why It Changes the Conversation

together.ai

• Holds AICPA SOC 2 Type II attestation.
• Demonstrates that security, availability, and confidentiality controls are not just designed, but operating effectively over a measured period.
• Security teams can map their internal control frameworks directly onto the SOC 2 report (e.g., access control, change management, incident response, logging).

What this means in practice:

• Vendor onboarding is faster: many enterprises require SOC 2 before legal will sign off.
• You can attach together.ai’s SOC report to your internal risk analysis, rather than relying on vendor‑authored security PDFs.
• It signals operational maturity: change control, key management, and incident response are auditable, not ad hoc.

DeepInfra

• As of public documentation, SOC 2 Type II is not a central part of the positioning.
• For non‑regulated workloads, this might be tolerable; for regulated industries, security teams will ask:
  • Is there an independent audit of controls?
  • How are keys, logs, and access handled?
  • How is multi‑tenant risk mitigated?

If SOC 2 is a hard requirement, together.ai aligns with that expectation today.

---

Data Retention, Training Usage, and Governance

together.ai

• Explicit stance: “Your data and models remain fully under your ownership.”
• Focus on strict data privacy controls, with encryption in transit and at rest.
• Clear separation between:
  • Inference data (your prompts/outputs at runtime),
  • Model artifacts (base models, fine‑tuned models, LoRAs),
  • Storage (logs, datasets, intermediate outputs).
• Options for:
  • Region‑aligned storage (NA/EU/Asia/Middle East) to satisfy data residency.
  • Dedicated environments where logs and telemetry are scoped per tenant and per deployment.

For security reviews, this lets you answer:

• “Is customer data used to train shared models?” → Only with explicit agreements.
• “Can we control where logs are stored?” → Yes, via region‑aware deployments and retention controls.
• “Can we remove data if required by contract or regulation?” → Yes; data and models are under your ownership.

DeepInfra

• Typically emphasizes rapid access to open‑source models and performance.
• Data retention and training usage policies must be inferred from ToS and privacy policy; these may not be the first‑class marketing message.
• Security teams will likely request:
  • Explicit statements on training reuse.
  • Log retention periods.
  • Deletion SLAs and any backup policies.

If you need contractual clarity around not using your data to train a shared model and fine‑grained retention controls, together.ai is structured to provide that in enterprise contracts and DPAs.

---

Enterprise Security Controls: Isolation, Encryption, and Uptime

together.ai

• Tenant-level isolation:
  • Serverless Inference for bursty traffic with strong isolation controls.
  • Dedicated Model Inference and Dedicated Container Inference for hard isolation and predictable SLOs.
  • GPU Clusters for teams that want VPC‑like control over training/fine‑tuning plus inference.
• Encryption:
  • Encryption in transit (e.g., TLS) and at rest is explicit in documentation.
• Operational reliability:
  • 99.9% uptime for production inference.
  • Customers report:
    • 2x latency reduction and ~33% cost savings (Salesforce AI Research).
    • 6× cost reduction and <400ms p95 model latency for real‑time applications.
• Compliance extras:
  • SOC 2 Type II.
  • HIPAA‑aligned deployment options (important if you’re handling PHI).
  • NVIDIA preferred partner, which matters if you care about long‑term hardware roadmap and performance stability.

DeepInfra

• Emphasizes performance and model breadth; may rely primarily on shared, multi‑tenant environments.
• Isolation often means logical isolation within shared clusters; “bare metal” isolation or per‑customer clusters may require custom arrangements.
• Encryption in transit is standard; at‑rest controls and key management need to be validated in security due diligence.

If your architecture review requires tenant separation, DDoS resilience, and clearly documented incident response, together.ai is designed with those enterprise checklists in mind.

---

Where together.ai’s Security Model Best Fits

From the perspective of someone who’s migrated a high‑traffic AI product off a patchwork of providers and through multiple security reviews, this is how I’d map the choice.

Best for regulated or risk-sensitive workloads

together.ai is a better fit when:

• You need SOC 2 Type II and potentially HIPAA‑aligned deployments to clear governance and compliance.
• You handle PII/PHI or sensitive enterprise data (finance, healthcare, legal, high‑stakes customer content).
• Your security team insists on:
  • Tenant-level isolation.
  • Explicit “no training on my data” guarantees without a separate agreement.
  • Regional data residency and controlled retention.
• You’re planning:
  • Dedicated Model Inference for stable, high‑throughput services.
  • Dedicated Container Inference for custom runtimes with strict isolation.
  • GPU Clusters for fine‑tuning and large‑scale Batch Inference without moving data between vendors.

Best for early-stage or non-critical experimentation

DeepInfra can be sufficient when:

• Your workloads are non‑sensitive, and you’re in early experimentation or low‑risk internal tools.
• You value fast access to many open models and simple pricing over formal audited security posture.
• You’re willing to build your own:
  • Encryption/key management layers around the service.
  • Logging, monitoring, and network isolation in your own infrastructure.
• You don’t yet have hard requirements for SOC 2, HIPAA alignment, or data residency.

---

Practical Evaluation Checklist

If you’re comparing together.ai and DeepInfra for an enterprise deployment, this is the minimal checklist I’d run:

1. Compliance & Certification

   • SOC 2 Type II report available and under NDA?
   • Any HIPAA‑aligned or sector‑specific deployment options?

2. Data Governance

   • Is there explicit language that your data and models remain under your ownership?
   • Is your data used to train shared models by default?
   • Can you configure data retention and deletion SLAs?

3. Isolation & Deployment Modes

   • Are there dedicated modes (Dedicated Model Inference, Dedicated Container Inference, GPU Clusters) with clear isolation boundaries?
   • Can you choose regions to meet data residency requirements?

4. Security Controls

   • Encryption in transit and at rest?
   • Tenant-level isolation, access control, and logging documented?
   • Incident response, breach notification, and change management policies?

5. Operational Reliability

   • Documented uptime commitments (e.g., 99.9%)?
   • Evidence that security controls don’t compromise latency (e.g., sub‑400ms p95, 2x latency reduction case studies)?

On this checklist, together.ai hits all the enterprise‑critical items out of the box. DeepInfra will require more bespoke verification, especially around SOC 2, data governance, and isolation.

---

Summary

For teams that need a platform that can withstand legal, compliance, and security scrutiny, together.ai’s AI Native Cloud offers:

• SOC 2 Type II attestation.
• Clear data ownership and privacy guarantees: your data and models remain fully under your ownership.
• Encryption in transit/at rest, tenant-level isolation, and region‑aware deployments.
• Multiple deployment modes—Serverless Inference, Batch Inference, Dedicated Model Inference, Dedicated Container Inference, GPU Clusters—that align with both performance and security requirements.

DeepInfra remains attractive for rapid prototyping and non‑sensitive workloads where formal certifications are not yet mandatory. But if your roadmap includes handling regulated data, passing rigorous vendor assessments, and scaling from PoC to always‑on production, together.ai’s security and compliance posture is materially different—and built to clear those enterprise bars.

---

Next Step

Get Started