Embeddings & Reranking Models
ZeroEntropy vs BGE-M3 / Qwen embeddings: should we pay for an API or self-host open models for our workload?
7 min read
Most teams evaluating embeddings for RAG, agents, or internal search end up in the same dilemma: should you keep costs low and sovereignty high by self-hosting open models like BGE-M3 or Qwen, or pay for a managed API like ZeroEntropy’s zembed-1 and outsource the operational pain? The answer isn’t “API vs open-source ideology”—it’s about latency, recall, cost per million tokens, and how much infra you actually want to run.
Quick Answer: If you’re optimizing for retrieval quality, predictable latency, and minimal ops, paying for an embedding API like ZeroEntropy’s zembed-1 usually wins. If your primary constraint is hard data sovereignty and you already have strong MLOps in place, self-hosting open-weight models (including zembed-1) can be the right call—especially for high-volume batch workloads.
Frequently Asked Questions
When does it make sense to pay for ZeroEntropy instead of just using BGE-M3 or Qwen embeddings for free?
Short Answer: Pay for ZeroEntropy when you care about higher retrieval quality, stable latency, and not running a GPU fleet; self-host BGE-M3/Qwen only if you have strict sovereignty needs and a team ready to own infra and evaluation.
Expanded Explanation:
Open models like BGE-M3 and Qwen are excellent baselines, but they ship as raw weights. You’re signing up for benchmarking, scaling, monitoring, and cost-optimization yourself. ZeroEntropy’s zembed-1 is purpose-built for retrieval with state-of-the-art accuracy at $0.05 per million tokens, sub-200ms API latency, and open weights if/when you want to bring it in-house.
In practice, most teams underestimate the hidden cost of “free” embeddings: GPU scheduling, autoscaling, observability, and constant re-benchmarking as workloads shift. If your goal is to ship reliable RAG or search in weeks—not quarters—an API that’s already tuned, benchmarked, and price-efficient is almost always cheaper at the system level.
Key Takeaways:
- Use ZeroEntropy’s API when speed-to-production, retrieval quality, and predictable latency matter more than running your own infra.
- Use self-hosted BGE-M3/Qwen only if you must fully control the environment and already have strong MLOps and GPU capacity.
How do I systematically decide between a managed embedding API and self-hosting open weights?
Short Answer: Evaluate four dimensions—retrieval quality, latency SLOs, data governance, and total cost of ownership (TCO)—then map them to your workload (online vs offline, volume, and sensitivity).
Expanded Explanation:
The API vs self-host decision is a systems design problem, not a feature checklist. Start from your workload: online RAG with p99 latency constraints looks very different from overnight document indexing. Then layer in regulatory and contractual requirements (GDPR, HIPAA, client contracts), plus your team’s operational maturity.
ZeroEntropy is designed for both modes: you can use zembed-1 via ZeroEntropy’s managed endpoints (including EU-region) for low-latency online workloads, and download the open weights to run inside your VPC or on-prem when sovereignty or air-gapped environments are non-negotiable. BGE-M3 and Qwen give you the self-hosting option, but you’ll need to own the rest of the stack.
Steps:
- Classify workloads: Separate online workloads (RAG, search APIs, agents) from offline/batch (indexing archives, log corpora).
- Define constraints: For each workload, specify p50/p99 latency targets, data residency/sovereignty constraints, and uptime expectations.
- Model TCO: Compare API cost per million tokens (e.g., zembed-1 at $0.05/M) against GPU/infra + engineering time required to self-host and scale open models.
How do ZeroEntropy’s zembed-1 embeddings compare to BGE-M3 and Qwen for retrieval workloads?
Short Answer: zembed-1 is optimized for fast, high-precision retrieval with state-of-the-art quality and very low cost; BGE-M3 and Qwen are strong open baselines but require more tuning and infra to hit comparable system-level performance.
Expanded Explanation:
BGE-M3 and Qwen-family embedding models have become popular because they’re open and perform well on general benchmarks. But retrieval systems live and die on top-k precision and latency, not just leaderboard scores. zembed-1 was trained explicitly for text retrieval: higher NDCG@10, calibrated similarity scores, and stable sub-200ms latency on the managed API.
At $0.05 per million tokens, zembed-1 undercuts many high-quality closed APIs (e.g., OpenAI text-embedding-3-large at ~$0.13/M, Cohere embed-v4.0 at ~$0.12/M) while still giving you an open-weight path. You can start with the hosted API and later move to self-hosted zembed-1 if you need strict sovereignty—without swapping models and risking quality regressions.
Comparison Snapshot:
- Option A: ZeroEntropy zembed-1 (API)
- Retrieval-tuned, state-of-the-art accuracy, calibrated scores, sub-200ms latency, $0.05/M tokens, with optional self-hosting.
- Option B: BGE-M3 / Qwen (self-host)
- Strong open models, flexible licensing, but you own infra, scaling, monitoring, and benchmarking.
- Best for:
- zembed-1 API: production RAG/search where quality, latency, and ops simplicity matter.
- BGE-M3/Qwen self-host: teams with strict sovereignty and mature infra willing to absorb engineering + GPU overhead.
How would we actually implement ZeroEntropy vs self-hosted BGE-M3/Qwen in our stack?
Short Answer: With ZeroEntropy, you grab an API key and call the embeddings endpoint or Search API in a few lines of code; with BGE-M3/Qwen, you deploy the model to your own GPUs, wrap it in a service, and manage scaling, observability, and updates.
Expanded Explanation:
ZeroEntropy is built to minimize integration time: you can drop zembed-1 into your stack via a simple SDK or HTTP call and get back embeddings optimized for retrieval. For document-heavy corpora, you can skip building your own pipeline altogether and ingest content directly into the ZeroEntropy Search API, which unifies dense, sparse, and reranking.
Self-hosting BGE-M3 or Qwen means you’ll need to choose a runtime (vLLM, Triton, custom GPU service), manage batching, implement health checks, and wire up autoscaling based on QPS and sequence lengths. That’s absolutely viable if you have the team and the budget—but it’s not “free.”
What You Need:
- For ZeroEntropy (API-first):
- ZeroEntropy API key, SDK or HTTP client, and a simple integration into your indexer/RAG pipeline (plus optional Search API ingestion).
- For self-hosted BGE-M3/Qwen:
- GPU infrastructure (cloud or on-prem), MLOps stack (orchestration, monitoring, logging), and engineers to maintain deployments and evaluate retrieval quality over time.
Strategically, when should we commit to an API, and when should we insist on self-hosting embeddings?
Short Answer: Use APIs like ZeroEntropy for your online, latency-sensitive retrieval and reserve self-hosting (including zembed-1 weights) for highly regulated, batch-heavy, or air-gapped environments where sovereignty beats convenience.
Expanded Explanation:
From a strategy perspective, the goal is not to “pick a side” but to align retrieval infrastructure with business risk and performance requirements. For most products—customer support search, AI assistants for internal docs, compliance tooling—an API with strong SLAs, SOC 2 Type II, HIPAA readiness, and EU-region endpoints is the fastest way to ship and iterate.
At the same time, some deployments (regulated legal, healthcare, government, on-prem enterprise stacks) require that embeddings never leave the organization’s controlled environment. Here, self-hosted open-weight models are mandatory. Because zembed-1 is open-weight, you don’t have to trade off performance to get sovereignty—you can run the same model in your own VPC/on-prem that you prototyped with via API.
Why It Matters:
- Impact 1: System reliability and trust. Better retrieval (higher NDCG@10, calibrated scores, stable p99 latency) means your LLM-based workflows are more accurate and cheaper—fewer tokens wasted on irrelevant chunks.
- Impact 2: Long-term flexibility. A model like zembed-1 that offers both a high-performance API and open weights lets you evolve from hosted to on-prem/VPC without rebuilding your retrieval stack or revalidating quality from scratch.
Quick Recap
The “ZeroEntropy vs BGE-M3/Qwen” decision is really about how much retrieval performance, latency predictability, and operational simplicity you need—and what your data governance constraints look like. For most online RAG and search workloads, using ZeroEntropy’s zembed-1 via API gives you state-of-the-art retrieval quality, sub-200ms latency, and the lowest price point in its class at $0.05 per million tokens, with no GPU ops to own. If you operate in highly regulated, air-gapped, or sovereignty-critical environments and have the infra muscle, self-hosting open-weight models (including zembed-1, BGE-M3, or Qwen) can be the right move—especially for high-volume batch indexing.