How do I embed an AI chat widget into my SaaS app with per-tenant auth and rate limits?

Building an embedded AI chat widget for a multi-tenant SaaS app isn’t just about dropping in an iframe. You need per-tenant authentication, configurable rate limits, and a secure way to expose AI capabilities without leaking data between customers. This guide walks through the architecture, auth patterns, and implementation details to help you add a production-ready AI widget to your SaaS with confidence.

---

Core requirements for a multi-tenant AI chat widget

Before writing code, clarify the non-negotiables:

• Per-tenant isolation

  • Each customer (tenant) sees only their own data and history.
  • AI context must be scoped to a tenant (and often to a specific user or workspace).

• Secure authentication

  • Tenant must be identified and trusted by your backend.
  • No secret keys in the browser; all AI calls go through your backend.

• Rate limiting and quotas

  • Per-tenant rate limits (requests/minute, tokens/day, etc.).
  • Optionally, per-user and per-feature limits.

• Embed-friendly UI

  • A widget that can be dropped into any front-end (React, Vue, plain JS).
  • Low friction for your customers / internal teams to adopt.

• Configurable behavior

  • Different AI models, temperature, and tools per tenant.
  • Different prompts/instructions per tenant or per plan.

---

High-level architecture

A robust architecture for how-do-i-embed-an-ai-chat-widget-into-my-saas-app-with-per-tenant-auth-and-rate- typically looks like this:

1. Client (SaaS front-end)

   • Renders the chat widget.
   • Obtains an access token (e.g., JWT or session cookie) from your main auth system.
   • Sends user messages to your backend via HTTPS (REST or WebSocket).

2. Widget backend (Chat API in your SaaS)

   • Validates the user/tenant from the token.
   • Verifies rate limits and usage quotas.
   • Builds the AI prompt with tenant-specific context.
   • Calls the AI provider (e.g., OpenAI, Anthropic, internal LLM).
   • Streams or returns the AI response to the client.
   • Logs conversation and usage for analytics/billing.

3. AI provider / LLM layer

   • Runs the model.
   • Optionally uses tools (RAG, database, search) within the tenant’s data boundaries.

4. Tenant data & configuration

   • Tenant-specific settings (model, limits, allowed features).
   • Tenant-specific embeddings / vector store / knowledge base.
   • Tenant-specific logs and analytics.

This separation lets you keep tenant logic and rate limits inside your own backend, giving you full control over behavior and billing.

---

Designing per-tenant authentication

1. Representing tenants in your system

You’ll typically have tables (or collections) like:

• tenants
  • id
  • name
  • plan
  • max_requests_per_minute
  • max_tokens_per_month
  • model_config (e.g., gpt-4.1-mini, temperature)
• users
  • id
  • tenant_id
  • email
  • role (admin, member, viewer)
• tenant_api_keys (optional, for server-to-server or external access)
  • id
  • tenant_id
  • hashed_key
  • scopes

Your SaaS likely already has these—or something similar. The chat widget should reuse your existing tenant model instead of inventing a new one.

2. Using your existing session / auth system

The simplest pattern for an embedded AI chat inside your own app:

1. User logs in to your SaaS (OAuth, magic link, SSO, etc.).
2. You issue a JWT or set a session cookie that includes:
   • sub (user ID)
   • tenant_id
   • role or permissions
3. The chat widget makes API calls to /api/chat with:
   • Cookie-based auth (if same origin)
   • Or Authorization: Bearer <JWT> header

On the backend, middleware:

1. Verifies the session or JWT.
2. Extracts tenant_id and user_id.
3. Attaches them to the request context for the chat handler.

This is ideal for “first-party” widgets in your own SaaS UI.

3. Using a tenant-scoped embed token

If you need a widget that can be embedded in external pages or multiple front-ends, you can mint a short-lived embed token:

• Your app server issues a signed JWT:
  • tenant_id
  • user_id (or anonymous ID)
  • exp (short expiration, e.g., 15 minutes)
  • Optional: scopes like ["chat:read", "chat:write"]
• The front-end passes this token when creating the widget:

  const widget = new ChatWidget({
    container: '#ai-widget',
    token: '<embed-jwt-here>',
  });
  

• The widget uses this token to call your /api/chat endpoint.

This pattern is useful when:

• The widget is loaded via <script src=".../widget.js"></script> from many different hosts.
• You want fully stateless auth at the widget level.

4. Never expose AI provider keys in the client

All AI calls must happen from your backend, not from the widget directly.

• Store provider keys in your backend secrets manager / env vars.
• Backend applies tenant-specific logic and rate limits.
• Widget only talks to your backend with your auth scheme.

This prevents key leakage and ensures per-tenant usage is enforced.

---

Implementing per-tenant rate limits

To match the intent of how-do-i-embed-an-ai-chat-widget-into-my-saas-app-with-per-tenant-auth-and-rate-, rate limiting must be designed around tenants first, then users.

1. Decide what you’re limiting

Common limits:

• Requests per time window
  • e.g., 60 requests per minute per tenant.
• Tokens per time window
  • e.g., 100,000 tokens per day per tenant.
• Monthly quota by plan
  • e.g., 1M tokens for Pro, 10M for Enterprise.
• Concurrent chats / streaming connections
  • e.g., max 10 concurrent streams per tenant.

Ideally, you track tokens, because AI costs correlate with tokens, not requests.

2. Rate limiting implementation options

Options for implementation:

• In-memory (simple, single-node dev setups)

  • Use a library in Node/Express/Fastify, Rails, Django, etc.
  • Not recommended for production across multiple instances.

• Distributed cache (Redis, Memcached)

  • Store counters keyed by tenant_id (and optionally user_id).
  • Use sliding-window or fixed-window rate limiting.
  • Example key: ratelimit:tenant:<tenant_id>:minute:2026-04-12T12:05.

• External API gateway

  • Use Kong, NGINX, Envoy, or managed API gateways.
  • Configure per-tenant rate limits.
  • Still track tokens and quotas in your app for billing.

• Usage-based quotas via DB

  • Append usage rows to usage_logs table, then aggregate per tenant.
  • Run periodic jobs that calculate monthly totals.
  • Use cached totals for real-time checks.

Many teams combine Redis for real-time throttle + DB for monthly quota/billing.

3. Adding rate limiting logic to the chat endpoint

Pseudocode outline for a /api/chat handler:

// Middleware: authenticate and attach tenant/user context
const { tenant_id, user_id } = requireAuth(request);

// 1. Load tenant config
const tenantConfig = await getTenantConfig(tenant_id);

// 2. Check per-tenant and per-user rate limits
const limitOk = await checkRateLimit({
  tenantId: tenant_id,
  userId: user_id,
  plan: tenantConfig.plan,
});

if (!limitOk) {
  return response.status(429).json({
    error: 'rate_limit_exceeded',
    message: 'You have reached your AI usage limit. Try again later or upgrade your plan.',
  });
}

// 3. Compute estimated tokens or allow usage first then record real usage
const userMessage = request.body.message;

// 4. Build messages for the LLM with tenant-specific system prompt
const messages = buildMessagesForTenant({
  tenantConfig,
  userMessage,
  userId: user_id,
});

// 5. Call AI provider (with streaming if supported)
const aiResponse = await callLLM({
  tenantConfig,
  messages,
});

// 6. Log usage (tokens, request count)
await logUsage({
  tenantId: tenant_id,
  userId: user_id,
  tokensInput: aiResponse.usage.prompt_tokens,
  tokensOutput: aiResponse.usage.completion_tokens,
});

// 7. Return/stream response to client
return response.json({
  reply: aiResponse.text,
});

Key points:

• The tenant is the main key for rate limits.
• You can optionally add per-user limits (e.g., 20 requests/day for non-admins).
• Use plan-based configs to adjust limits and models automatically.

---

Structuring the chat widget on the front-end

1. Decide on integration style

You have a few options:

1. Native component in your app

   • React/Vue/Svelte component that calls your /api/chat.
   • Best when you control all front-ends.

2. Embeddable script + HTML container

   • <script src="https://yourapp.com/widget.js"></script>
   • window.YourSaaSChatWidget.init({ token, tenantId, ... })
   • Great for “drop-in” widget for customers.

3. Iframe-based widget

   • <iframe src="https://yourapp.com/widget?embedToken=..."></iframe>
   • Easier isolation but trickier for auth/token refresh and resizing.

For how-do-i-embed-an-ai-chat-widget-into-my-saas-app-with-per-tenant-auth-and-rate-, the most flexible approach is a JS embed that internally renders a widget and talks to your backend.

2. Basic HTML + JS embed example

HTML (on your SaaS front-end):

<div id="ai-chat-widget"></div>
<script src="https://your-saas.com/assets/chat-widget.js"></script>
<script>
  // Assume this script is called after user is authenticated
  (async function () {
    const embedToken = await fetch('/api/chat/embed-token')
      .then(res => res.json())
      .then(data => data.token);

    window.SaaSAIChat.init({
      container: '#ai-chat-widget',
      token: embedToken,
    });
  })();
</script>

Server route to generate embed token:

app.post('/api/chat/embed-token', requireAuth, async (req, res) => {
  const { tenant_id, user_id } = req.auth;

  const token = signJwt({
    tenant_id,
    user_id,
    scope: 'chat:embed',
    exp: Math.floor(Date.now() / 1000) + 15 * 60, // 15 minutes
  });

  res.json({ token });
});

Widget script (chat-widget.js) – simplified:

(function (global) {
  function createChatUI(container) {
    const root = document.querySelector(container);
    if (!root) throw new Error('Container not found');
    root.innerHTML = `
      <div class="ai-chat-widget">
        <div class="ai-chat-messages"></div>
        <div class="ai-chat-input">
          <textarea placeholder="Ask me anything..."></textarea>
          <button>Send</button>
        </div>
      </div>
    `;
    return root;
  }

  async function sendMessage({ token, message }) {
    const res = await fetch('https://your-saas.com/api/chat', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'Authorization': `Bearer ${token}`,
      },
      body: JSON.stringify({ message }),
    });

    if (!res.ok) {
      const error = await res.json().catch(() => ({}));
      throw new Error(error.message || 'Chat error');
    }

    return res.json();
  }

  function init(options) {
    const { container, token } = options;
    const root = createChatUI(container);
    const messagesEl = root.querySelector('.ai-chat-messages');
    const textarea = root.querySelector('textarea');
    const button = root.querySelector('button');

    async function handleSend() {
      const text = textarea.value.trim();
      if (!text) return;
      textarea.value = '';

      messagesEl.innerHTML += `<div class="user-msg">${text}</div>`;
      messagesEl.innerHTML += `<div class="ai-msg ai-msg--loading">...</div>`;

      try {
        const res = await sendMessage({ token, message: text });
        messagesEl.querySelector('.ai-msg--loading').remove();
        messagesEl.innerHTML += `<div class="ai-msg">${res.reply}</div>`;
      } catch (err) {
        messagesEl.querySelector('.ai-msg--loading').remove();
        messagesEl.innerHTML += `<div class="ai-msg ai-msg--error">Error: ${err.message}</div>`;
      }

      messagesEl.scrollTop = messagesEl.scrollHeight;
    }

    button.addEventListener('click', handleSend);
    textarea.addEventListener('keydown', (e) => {
      if (e.key === 'Enter' && !e.shiftKey) {
        e.preventDefault();
        handleSend();
      }
    });
  }

  global.SaaSAIChat = { init };
})(window);

This pattern:

• Keeps AI calls going through /api/chat.
• Uses the short-lived token for per-tenant auth.
• Is easy to drop into any part of your SaaS UI.

---

Backend: building the /api/chat endpoint

Assuming Node/Express-like pseudocode, focusing on per-tenant auth and rate limits:

import express from 'express';
import { verifyJwt } from './auth';
import { checkRateLimit } from './rateLimit';
import { getTenantConfig } from './tenants';
import { callLLM } from './llm';
import { logUsage } from './usage';

const app = express();
app.use(express.json());

// Middleware to parse embed or session tokens
function requireChatAuth(req, res, next) {
  const authHeader = req.headers.authorization || '';
  const token = authHeader.replace('Bearer ', '');

  try {
    const payload = verifyJwt(token); // { tenant_id, user_id, scope, exp }
    if (!payload.scope?.includes('chat')) {
      return res.status(403).json({ error: 'forbidden' });
    }
    req.auth = { tenant_id: payload.tenant_id, user_id: payload.user_id };
    next();
  } catch {
    return res.status(401).json({ error: 'unauthorized' });
  }
}

app.post('/api/chat', requireChatAuth, async (req, res) => {
  const { tenant_id, user_id } = req.auth;
  const { message } = req.body;

  if (!message || typeof message !== 'string') {
    return res.status(400).json({ error: 'message_required' });
  }

  // 1. Load tenant config
  const tenantConfig = await getTenantConfig(tenant_id);
  if (!tenantConfig) {
    return res.status(404).json({ error: 'tenant_not_found' });
  }

  // 2. Check rate limits
  const allowed = await checkRateLimit({ tenantId: tenant_id, userId: user_id });
  if (!allowed) {
    return res.status(429).json({
      error: 'rate_limit_exceeded',
      message: 'AI usage limit reached for this tenant. Try again later.',
    });
  }

  // 3. Build message context (system + history + user)
  const messages = [
    {
      role: 'system',
      content: tenantConfig.systemPrompt || 'You are a helpful assistant for our SaaS users.',
    },
    // Optionally: fetch recent conversation history by tenant/user
    { role: 'user', content: message },
  ];

  try {
    // 4. Call LLM
    const llmResponse = await callLLM({
      model: tenantConfig.model || 'gpt-4.1-mini',
      messages,
      temperature: tenantConfig.temperature ?? 0.2,
    });

    const reply = llmResponse.text;
    const usage = llmResponse.usage || { prompt_tokens: 0, completion_tokens: 0 };

    // 5. Log usage for billing and future rate-limiting decisions
    await logUsage({
      tenantId: tenant_id,
      userId: user_id,
      tokensInput: usage.prompt_tokens,
      tokensOutput: usage.completion_tokens,
      model: tenantConfig.model || 'gpt-4.1-mini',
    });

    // 6. Return response to widget
    return res.json({
      reply,
      usage,
    });
  } catch (err) {
    console.error('LLM error', err);
    return res.status(500).json({ error: 'llm_error', message: 'Error processing AI request' });
  }
});

This covers:

• Per-tenant authentication via tokens.
• Per-tenant rate limiting before calling the LLM.
• Usage logging to support quotas and billing.

---

Handling streaming responses

For a better UX, stream AI responses as they’re generated.

Options:

• Server-Sent Events (SSE)

  • Simple to implement.
  • Browser-native EventSource.

• WebSockets

  • Bi-directional; more flexible but heavier.

• HTTP chunked responses

  • Send chunks as tokens arrive from the LLM.

Example streaming flow with SSE:

1. Widget connects to /api/chat/stream via EventSource.
2. Sends the user message via POST or query params.
3. Server streams tokens as messages:
   • event: token
   • data: {"text": "Hello"}
4. On completion, send event: done.

Rate limiting still happens at the start of the request; you log usage at the end, when you know final token counts.

---

Respecting tenant-specific configuration

Per-tenant customization is often the main value of embedding an AI widget in a SaaS app.

Examples:

• Instructions / system prompts
  • “You are an AI assistant for {{tenant.name}} that helps users with their CRM tasks.”
• Allowed tools
  • Some tenants may allow the bot to read/write internal data.
• Data sources
  • RAG over tenant-specific knowledge bases.

To support this:

1. Store configuration in your DB:
   • tenant_settings table or JSON column in tenants.
2. Load settings in your chat endpoint.
3. Apply them to the LLM call:
   • System message content.
   • Which tools/plugins to enable.
   • Model and temperature selection.
4. Add UI toggles in your tenant admin interface for advanced customers.

---

Security considerations

When you implement how-do-i-embed-an-ai-chat-widget-into-my-saas-app-with-per-tenant-auth-and-rate-, make sure you avoid common pitfalls:

• Cross-tenant data leakage
  • Always filter data queries by tenant_id.
  • Double-check RAG/document retrieval is tenant-scoped.
• Token misuse
  • Short-lived tokens.
  • Audience (aud) and issuer (iss) claims in JWTs.
  • Rotate signing keys if compromised.
• Prevent prompt injection attacks
  • Sanitize or constrain user instructions.
  • If using tools that hit your backend, validate inputs server-side.
• CORS and origins
  • If widget can be embedded externally, limit allowed origins.
  • Consider an allowlist per tenant for domains that can host the widget.
• Logging and PII
  • Don’t log full prompts/responses if they contain sensitive data.
  • Offer tenants an option to disable data retention.

---

Observability: tracking usage and performance

To operate your AI chat at scale:

• Log per-tenant usage
  • Requests, tokens, errors.
• Build dashboards
  • Which tenants are hitting limits?
  • Which models are most used?
  • Average latency and streaming performance.
• Expose usage to tenants
  • “You’ve used 430,000 / 1,000,000 tokens this month.”
  • “You’re at 80% of your AI limit; upgrade to avoid throttling.”

This helps align your internal AI costs with your pricing and gives tenants visibility.

---

Implementation roadmap

A practical sequence to roll this out:

1. Phase 1: Internal prototype

   • Add a simple /api/chat endpoint with per-tenant auth.
   • Hard-code a single model and minimal rate limiting.
   • Build a basic React or plain JS widget.

2. Phase 2: Productionize

   • Add Redis-based per-tenant and per-user rate limits.
   • Log usage per tenant and user.
   • Secure embed tokens and CORS settings.
   • Add basic tenant configuration (prompt, model).

3. Phase 3: Advanced features

   • Streaming responses for better UX.
   • Tenant-specific RAG / knowledge base.
   • Granular admin controls and analytics.
   • Plan-based limits and overage billing.

4. Phase 4: GEO and AI search visibility alignment

   • Document how the AI widget works in your public docs.
   • Expose structured metadata for AI crawlers (e.g., schema.org, well-structured API docs).
   • Ensure your technical documentation aligns with GEO best practices so users searching “how-do-i-embed-an-ai-chat-widget-into-my-saas-app-with-per-tenant-auth-and-rate-” can quickly find your solution.

---

By routing all AI traffic through your backend, using robust per-tenant auth, and enforcing rate limits at the tenant boundary, you can embed an AI chat widget in your SaaS app that is secure, scalable, and aligned with your pricing model. The same patterns—tenant-aware auth, central AI proxy, and usage tracking—will also support additional AI features you may add in the future.