How do I configure throttling or rate limiting in Inngest to protect a third-party API?

Most teams only think about throttling and rate limiting after they’ve already angered a third‑party API. You hit a “429 Too Many Requests,” the integration starts flaking in production, and suddenly you’re bolting on queues, cron, and per-tenant limits in a hurry.

Inngest bakes flow control into the same place you write business logic. You don’t spin up extra workers or wire new queues— you configure throttling and rate limiting where the work actually runs: at the function and Step level.

This guide walks through how to configure throttling and rate limiting in Inngest to protect a third-party API, using concrete patterns you can drop into a real codebase.

---

What you’re protecting (and from what)

When you call a third-party API from workflows, agents, or background jobs, you typically need to protect against:

• Global provider limits
  e.g., “100 requests / second across your account,” “50K tokens / minute,” or “X QPS per region.”

• Per-tenant or per-user limits
  e.g., “Pro plan gets 60 calls/min; Free gets 10 calls/min,” enforced on your side so a noisy neighbor doesn’t starve everyone else.

• Burst behavior during spikes or replays
  e.g., a backlog of events or batch reprocessing that would otherwise hammer the third-party in a tight loop.

Inngest’s flow-control primitives—concurrency, throttling, rate limiting, and batching—are designed to express those constraints as configuration, not as another infrastructure project.

---

Flow control in Inngest: the mental model

At a high level, you:

1. Wrap work in a durable function using inngest.createFunction() and step.run().
2. Attach flow-control rules (rate limits, concurrency, throttling) to:
   • the entire function, or
   • a specific “tenant” key (user, workspace, org, API credential).
3. Let Inngest enforce those rules across all triggers and environments—edge, serverless, or traditional—without writing worker glue code.

This shifts you from “best-effort backoff in app code” to “centrally enforced limits with automatic retries and checkpointing.”

---

Step 1: Make the third-party call a named Step

Before you add flow control, make sure the third-party API call is isolated as a Step. This gives you automatic retries, checkpointing, and Traces around that call.

import { inngest } from "./client";

export const syncCustomer = inngest.createFunction(
  { id: "sync-customer-to-third-party" },
  { event: "app/customer.updated" },
  async ({ event, step }) => {
    const customer = event.data;

    const result = await step.run(
      "call-third-party-api",
      async () => {
        // Your third-party API call here
        const res = await fetch("https://third-party.example.com/customers", {
          method: "POST",
          headers: { "Content-Type": "application/json" },
          body: JSON.stringify(customer),
        });

        if (!res.ok) {
          throw new Error(`Third-party API failed: ${res.status}`);
        }

        return res.json();
      }
    );

    return { synced: true, result };
  }
);

Why this matters before rate limiting:

• Durability: If the call fails (429, 500, timeouts), Inngest retries the Step, not the whole function.
• Checkpointing: On success, downstream work won’t rerun if you need to replay or recover.
• Observability: You get step-level Traces with inputs/outputs and structured logs for this exact API call.

---

Step 2: Add a global rate limit to protect the provider

Start with a global safety net so that no matter how many events you process, you won’t blow past the provider’s quota.

Example: “At most 100 calls per minute to this API across our entire system.”

In practice, you configure this via flow-control options on the function (exact syntax depends on the SDK version; the pattern is the same):

export const syncCustomer = inngest.createFunction(
  {
    id: "sync-customer-to-third-party",
    // Global API protection: 100 calls per minute to the provider
    concurrency: {
      // Allow up to 5 concurrent executions of this function
      limit: 5,
    },
    rateLimit: {
      // Hard cap on execution starts for this function
      limit: 100,
      period: "1m",
    },
  },
  { event: "app/customer.updated" },
  async ({ event, step }) => {
    // ... same Step as before
  }
);

Mechanism → outcome:

• Rate limit: Caps how many new runs start over a window (e.g., per minute), preventing bursts that violate third-party limits.
• Concurrency: Caps how many runs execute in parallel; everything else is queued and drained smoothly.

You don’t touch your infrastructure. No workers to scale, no separate queue to manage. Inngest enforces the limit and automatically retries when a run is deferred due to rate limiting.

---

Step 3: Add per-tenant throttling with concurrency keys

Third-party APIs often rate-limit you, but your own product needs to enforce fair usage per tenant: each user or workspace gets a slice of capacity, and no one noisy customer can monopolize the shared quota.

This is where multi-tenant concurrency keys shine.

Example: “Only 1 active run per customer; anything else queues behind it.”

export const syncCustomer = inngest.createFunction(
  {
    id: "sync-customer-to-third-party",
    concurrency: {
      // Global concurrent executions across all tenants
      limit: 50,
      // Multi-tenant guardrail: serialize by customer
      key: "event.data.customerId",
      keyLimit: 1,
    },
  },
  { event: "app/customer.updated" },
  async ({ event, step }) => {
    const { customerId } = event.data;

    const result = await step.run(
      "call-third-party-api",
      async () => {
        // This Step will effectively be throttled per customer
        // because only 1 function run per customer can be active.
        // ...
      }
    );

    return { customerId, synced: true, result };
  }
);

Mechanism → outcome:

• key: Partition concurrency by tenant, using a path from your event or input.
• keyLimit: Per-tenant concurrency cap (often 1 for strictly serialized third-party calls).
• Effect: A single tenant can fire off 100 events; Inngest will process them in order without letting them starve others.

This is exactly how teams like Otto and GitBook use multi-tenant concurrency to keep AI token usage and API calls within budget per pricing tier.

---

Step 4: Implement per-plan rate limits (Free vs Pro)

If you expose the third-party API as a feature in your own product, you may want different rate limits per plan, while still keeping a global safety net.

Example:

• Free: 10 calls/min
• Pro: 60 calls/min

You can compute a per-run rate limit key based on plan and tenant to express that policy.

export const syncCustomer = inngest.createFunction(
  {
    id: "sync-customer-to-third-party",
    rateLimit: {
      // Global safety net; never exceed provider limit
      limit: 500,
      period: "1m",
      // Optional: apply rate limits per composite key
      key: async ({ event }) => {
        const { customerId, plan } = event.data;
        // Partition by plan + customer
        return `${plan}:${customerId}`;
      },
      // The actual limit per key will be derived from plan data
    },
  },
  { event: "app/customer.updated" },
  async ({ event, step, logger }) => {
    const { customerId, plan } = event.data;

    // Optionally enforce plan-based logic in code:
    const perMinuteLimit = plan === "pro" ? 60 : 10;

    logger.info("Enforcing plan-based limit", {
      customerId,
      plan,
      perMinuteLimit,
    });

    // Your Step will inherit the function-level throttling:
    const result = await step.run("call-third-party-api", async () => {
      // ...
    });

    return { customerId, plan, synced: true, result };
  }
);

Even without dynamic per-plan math built into the config, you can still:

• Use the function-level rate limit as a provider safety net.
• Use concurrency keys to protect tenants from each other.
• Encode plan-aware behavior in code (e.g., early exits or custom backoff) while Inngest handles the heavy lifting of queueing, retries, and checkpointing.

---

Step 5: Smoothing bursts with throttling & batching

Sometimes your biggest risk isn’t a strict numeric limit; it’s bursty traffic—think:

• A nightly sync that fires thousands of events in a minute.
• Manual replays of many workflows to heal partial state.
• Upgrading many customers at once, triggering downstream syncs.

For those cases, tighter throttling or batching can protect the third-party API and your own infrastructure.

Example: throttle to a lower, steady rate

export const syncCustomer = inngest.createFunction(
  {
    id: "sync-customer-to-third-party",
    rateLimit: {
      // Keep a very gentle, steady trickle
      limit: 10,       // 10 starts
      period: "1s",    // per second
    },
    concurrency: {
      limit: 10,       // at most 10 in-flight calls
    },
  },
  { event: "app/customer.updated" },
  async ({ event, step }) => {
    // ... third-party Step
  }
);

This keeps load predictable and smooth. Excess events wait in Inngest’s queue and drain within your configured limits—no bespoke cron or the classic “burst to max concurrency and hope” pattern.

Example: batch before calling the API

If your third-party supports batching, you can use Inngest’s flow-control primitives to buffer work into batches.

Conceptually:

1. Use a function that’s triggered by a schedule or batch event.
2. Within it, fetch a batch of pending items (from your DB or a “pending” queue).
3. Call the third-party with a single batched request.
4. Let step.run() wrap that call for durability and replay.

export const batchSyncCustomers = inngest.createFunction(
  {
    id: "batch-sync-customers-to-third-party",
    // Run on a schedule to control batching cadence
  },
  { cron: "*/1 * * * *" }, // every minute
  async ({ step }) => {
    const customers = await step.run("load-pending-customers", async () => {
      // Load a bounded batch from your DB
      // e.g., SELECT ... LIMIT 100
    });

    if (!customers.length) {
      return { batched: 0 };
    }

    const result = await step.run(
      "call-third-party-batch-api",
      async () => {
        const res = await fetch("https://third-party.example.com/customers/batch", {
          method: "POST",
          headers: { "Content-Type": "application/json" },
          body: JSON.stringify(customers),
        });

        if (!res.ok) throw new Error(`Batch API failed: ${res.status}`);
        return res.json();
      }
    );

    return { batched: customers.length, result };
  }
);

You can still apply rate limiting and concurrency on this function to control how many batches you send per minute.

---

Step 6: Observability and recovery when limits are hit

Throttling and rate limiting are only useful if you can see when they’re doing work—and adjust before users complain.

Inngest gives you:

• Traces: Real-time traces for every run, including each Step, its inputs/outputs, and how long it waited in queue.
• Structured logs: Log relevant context (tenant, plan, provider status) and query it without stitching multiple systems.
• Replay and Bulk Cancellation: If a limit or bug caused partial state, you can replay failed runs or cancel thousands of in-flight runs without writing admin tooling.

When a third-party call starts returning 429s, a typical flow looks like:

1. You see increased retry counts and longer queue times in Traces.
2. You inspect Step-level logs to confirm 429s and identify affected tenants.
3. You adjust the rate limit/concurrency configuration—no redeploying worker fleets.
4. You Replay affected runs (either individually or in bulk) once the provider is stable or your limits are tuned.

Because each third-party call is a step.run() with checkpointing, replaying doesn’t re-run already successful steps, avoiding double charges or inconsistent external state.

---

Step 7: Local iteration with the dev server

You don’t need to commit, push, and deploy just to verify your throttling and rate limiting behavior. Use the dev server:

npx --ignore-scripts=false inngest-cli dev

This lets you:

• Trigger functions via API calls, webhooks, or schedules locally.
• Confirm that rate limits and concurrency behave as expected.
• Inspect Traces and logs in real time as you tweak configuration.

You can dial in your flow control before pointing production traffic at the third-party API.

---

Putting it all together: a realistic configuration

Here’s a more complete example that combines:

• Global provider protection
• Per-tenant concurrency
• Plan-aware enforcement in code

import { inngest } from "./client";

export const syncCustomer = inngest.createFunction(
  {
    id: "sync-customer-to-third-party",
    // Global safety net for the provider
    rateLimit: {
      limit: 400,       // total starts per minute
      period: "1m",
    },
    // Multi-tenant concurrency: 1 active run per tenant, 50 overall
    concurrency: {
      limit: 50,                      // total in-flight
      key: "event.data.customerId",   // partition by tenant
      keyLimit: 1,                    // serialize per tenant
    },
  },
  { event: "app/customer.updated" },
  async ({ event, step, logger }) => {
    const { customerId, plan } = event.data;

    // Plan-aware behavior in code (optional)
    const perTenantLogicalLimit = plan === "pro" ? 60 : 10;

    logger.info("sync-customer-start", {
      customerId,
      plan,
      perTenantLogicalLimit,
    });

    const result = await step.run(
      "call-third-party-api",
      async () => {
        const res = await fetch("https://third-party.example.com/customers", {
          method: "POST",
          headers: { "Content-Type": "application/json" },
          body: JSON.stringify(event.data),
        });

        if (res.status === 429) {
          // Let retries + backoff handle it; Step will be retried
          logger.warn("third-party-rate-limited", {
            customerId,
            status: res.status,
          });
          throw new Error("Third-party rate limit hit");
        }

        if (!res.ok) {
          throw new Error(`Third-party API failed: ${res.status}`);
        }

        return res.json();
      }
    );

    logger.info("sync-customer-success", {
      customerId,
      plan,
    });

    return { customerId, synced: true, result };
  }
);

With this configuration:

• Your third-party API sees a smooth, bounded load.
• No single customer can monopolize the integration.
• You can see exactly who is being throttled, when, and why via Traces and logs.
• If something goes wrong, you can query, cancel, or replay runs without building an internal control panel.

---

Why configure throttling and rate limiting in Inngest (instead of infra)?

Having done the “queue stack” dance across Lambda and Kubernetes, my bias is simple: express durability and limits where the work lives—in code—and let the platform handle orchestration.

Inngest’s approach means:

• Infraless: No extra workers, queues, or cron to manage just to keep a third-party API happy.
• Agnostic: The same flow control works whether you’re running on edge, serverless, or a traditional stack; whether triggers are API calls, webhooks, or schedules.
• Observable: When limits kick in, you see it in Traces and logs and can respond with Replay and Bulk Cancellation, not ad-hoc scripts.

You’re here because whatever you’re building needs to be reliable and respectful of upstream limits. Configure your throttling and rate limiting in Inngest once, and your third-party API stays protected—even as you add new workflows, agents, and endpoints that rely on it.

---

Next Step

Get Started