What’s a good pattern for rate limiting background jobs so I don’t get banned by third-party APIs?

Most teams only start thinking about rate limiting after their first 429 storm or “temporarily banned” email from a third-party API. By then, you’re already in incident mode: background jobs are failing, retries are hammering the provider, and you’re trying to translate rate-limit headers into hotfixes at 3 a.m.

You don’t want a one-off fix. You want a repeatable pattern for rate limiting background jobs so you can safely scale and never get banned by third-party APIs again.

Below is the pattern I recommend after years of running multi-tenant workloads on Lambda and k8s, and now shipping everything on Inngest: centralized, code-level rate limiting with durable execution and flow control.

---

The real problem: rate limits + background jobs + retries

Background work is exactly where you’re most likely to violate rate limits:

• It’s batchy: sync jobs, report generation, AI agent runs, ingestion pipelines.
• It’s invisible: no user-facing latency, so you “just queue it and forget it.”
• It retries: failed jobs retry automatically and often double down on a hot API.

Common failure modes:

• Local rate limiting per worker
  Each worker tracks its own counters. With 20 workers, you accidentally do 20× the allowed RPS.

• Naive backoff
  You get 429s, add “wait 1 second and retry” everywhere, and still spike the provider whenever a cron or batch job runs.

• Multi-tenant noisy neighbors
  One large tenant’s job saturates the limit, starving everyone else and triggering bans.

What you actually need is:

1. A single source of truth for rate limits (per API and per tenant).
2. Durable throttling so work waits instead of failing or hammering endpoints.
3. Flow control so one workload or tenant can’t starve the rest.

That’s where Inngest’s pattern comes in.

---

The pattern: central rate limiting + durable steps

The core idea:

Put all calls to a given third-party API behind a single, shared “gate” that enforces rate limits and concurrency — and make the gate durable, so waiting is safe and transparent.

In Inngest, you implement that gate as part of your function’s Steps and Flow Control, not as a sidecar queue + Redis script + custom DLQ.

1. Model work as durable Steps, not raw jobs

Instead of hand-rolled workers, you write functions:

import { inngest } from "@/inngest/client";

export const syncContacts = inngest.createFunction(
  { id: "sync-contacts" },
  { event: "contacts/sync.requested" },
  async ({ step, event }) => {
    const contacts = await step.run("fetch-contacts", async () => {
      // your DB call or internal API
      return getContactsToSync(event.data.accountId);
    });

    await step.run("push-to-crm", async () => {
      // calls to third-party CRM API live here
      await pushContactsToCRM(contacts);
    });
  }
);

Each step.run() is:

• Durable – state is checkpointed; on failure it resumes from the last successful step, not from the beginning.
• Retriable – automatic retries with backoff on configured errors.
• Inspectable – you see step-level inputs/outputs in Traces.

This matters for rate limiting because you can safely pause, queue, and resume work without inventing your own checkpointing.

2. Centralize rate limiting for the third-party API

Now you introduce your rate-limit gate in one place — either via Flow Control or a dedicated “API caller” step.

A. Using concurrency keys (per-API or per-tenant)

For many APIs, the most robust pattern is global or per-tenant concurrency, e.g. “only N in-flight calls to this API at once”:

export const pushToCRM = inngest.createFunction(
  {
    id: "push-to-crm",
    concurrency: {
      // One concurrency “lane” per tenant+API
      key: ({ event }) => `crm:${event.data.accountId}`,
      limit: 2, // at most 2 in-flight CRM calls per tenant
    },
  },
  { event: "crm/push.requested" },
  async ({ step, event }) => {
    return step.run("call-crm-api", async () => {
      return callCRMApi(event.data.payload);
    });
  }
);

Mechanism → outcome:

• Mechanism: concurrency key crm:${accountId}, hard limit of 2.
• Outcome: you never have more than 2 parallel requests to the CRM per tenant, even across many workers/runtimes. No noisy neighbors inside that tenant, and no accidental burst across distributed workers.

Your “higher-level” sync functions just emit events to crm/push.requested instead of calling the API directly. Inngest handles flow control centrally.

B. Using rate-limit windows (tokens per minute)

If the provider uses a window-based limit (e.g. “600 requests per minute”), you can model a token bucket around that step. With Inngest’s Flow Control (multi-tenant concurrency and throttling), you dial in limit + burst without per-worker custom logic.

Conceptually:

• Give the CRM step a max RPS or “max executions per time window.”
• Let Inngest queue additional executions instead of failing them.
• Use automatic retries for transient 429s.

This is precisely the kind of centralized rate limiting teams use Inngest for when dealing with strict LLM token quotas and API RPM limits.

---

Avoiding bans in multi-tenant systems

Most rate-limit incidents I’ve seen come from multi-tenant SaaS:

• One customer kicks off a large import or AI workflow.
• Their workload saturates the API limit.
• Everyone else starts failing, you hit DLQs, and the provider sees sustained abuse.

To avoid this, your pattern needs two layers:

1. Per-tenant concurrency and/or rate limits
   Ensure one tenant can’t monopolize the third-party API.

2. Global guardrails
   Ensure the entire system stays within the provider’s documented limits.

In Inngest, that means:

• Per-tenant keys: crm:${accountId}
• Global keys (if needed): crm:global

Example:

export const pushToCRM = inngest.createFunction(
  {
    id: "push-to-crm",
    concurrency: [
      {
        key: ({ event }) => `crm:${event.data.accountId}`,
        limit: 2, // per tenant
      },
      {
        key: () => "crm:global",
        limit: 20, // across all tenants
      },
    ],
  },
  { event: "crm/push.requested" },
  async ({ step, event }) => {
    return step.run("call-crm", async () => callCRMApi(event.data.payload));
  }
);

Result:

• No tenant can create a “thundering herd” against the API.
• The platform as a whole respects the provider’s cap.
• Excess work is held by Inngest, not dropped or dead-lettered.

---

Handling unpredictable spikes and batch jobs

You also need to consider when background work is triggered:

• Cron jobs (e.g., “sync all accounts every 10 minutes”).
• Bulk operations (e.g., “rebuild all embeddings”).
• Event spikes (e.g., a marketing campaign drives a surge of webhooks).

Patterns that help:

1. Debounce noisy triggers

If a region of your app can emit multiple “do the same thing” events in a short window, debounce them before they become API calls.

Example:

• DB updates emit user/updated events.
• Instead of syncing on every single event, you use an Inngest function that:
  • Collects updates over a short window (e.g. 30–60 seconds).
  • Batches them into a single third-party API call.

This is how teams use Inngest to avoid “needlessly triggered” background processes and pointless duplicate work.

2. Batch where the API allows it

If the provider offers bulk endpoints (e.g., upsert 100 records per request), your function can group units of work before hitting the rate-limited step:

await step.run("batch-push-to-crm", async () => {
  const batches = chunk(contacts, 100);

  for (const batch of batches) {
    await callCRMApi(batch); // still behind concurrency limits
  }
});

Outcome:

• Fewer requests per time window.
• Better headroom under the same rate limit.

3. Make waiting safe via durability

In traditional queue setups, “wait” often means “block a worker” or “hold a job in Redis and hope it doesn’t get lost.” That’s why teams default to retry-on-failure instead of proper throttling.

In Inngest, each step.run() can be safely delayed:

• If the CRM is at capacity, you can:
  • Backoff via retry, or
  • Intentionally “sleep” until the next allowed slot.
• The function’s state is persisted, so you’re not holding open resources.
• If the process dies mid-wait, Inngest resumes from the checkpoint, not from the beginning.

That’s code-level durability instead of infrastructure gymnastics.

---

What to do when you still hit 429s

Even with good flow control, you’ll occasionally bump into a provider’s limits (especially when they change limits or add new tiers).

The pattern:

1. Treat 429s as transient errors
   Configure the relevant step or function so 429s trigger automatic retries with exponential backoff.

2. Inspect in Traces
   Use Inngest Traces to see:

   • Which steps are failing with 429s.
   • Which tenant, event, or batch triggered the hotspot.
   • The exact payload that was sent.

3. Adjust flow control, not application logic
   Once you know the hotspot, you tweak:

   • Concurrency limits.
   • Batching behavior.
   • Scheduling cadence (e.g., spreading heavy crons across time).

4. Replay safely
   For failed runs, use Replay to re-run them once you’ve adjusted the controls, rather than writing bespoke recovery scripts.

Outcome: You fix the incident at the rate-limit layer, not by patching every code path that calls the API.

---

How this pattern prevents bans in practice

Putting it all together:

• Central gate for each third-party API
  All background jobs call the API through a single Inngest function that enforces concurrency and rate limits.

• Durable steps, not fragile jobs
  Each API call is a named Step. If it’s delayed or retried, the rest of the workflow doesn’t restart or duplicate side effects.

• Multi-tenant fairness baked in
  Concurrency keys guarantee that one noisy tenant or workload doesn’t starve everyone else or exhaust global quotas.

• No custom “queue stack”
  You’re not maintaining:

  • Worker fleets
  • Homegrown rate-limit scripts
  • Dead-letter queues
  • Ad-hoc “replay this batch” admin tools

• Operational clarity
  When something goes wrong, you:

  • Open Traces.
  • See every step input/output.
  • Query, cancel, or replay runs in bulk.

This is how teams stay within strict token and request quotas — including for AI/LLM providers where violations can get you rate-limited platform-wide.

---

A simple checklist for safe rate-limited background jobs

If you want a minimum viable pattern to implement today:

1. Identify all third-party APIs you call from background jobs.
2. Create one Inngest function per API that owns the actual HTTP calls.
3. Add concurrency keys:
   • Global key per API.
   • Optional per-tenant key.
4. Batch where possible to reduce total request count.
5. Treat 429 as retriable, with exponential backoff.
6. Use Traces to review failures and adjust flow control instead of patching business logic.
7. Use Replay to safely re-run failed runs after you’ve tuned limits.

With that in place, you stop worrying about “Will this new feature or batch job accidentally get us banned?” and start treating rate limits as just another part of your execution plan — expressed in code, visible in Traces, and enforced by the platform.

---

If you want help designing rate-limited workflows for your specific APIs and quotas, or want to see how multi-tenant flow control looks in practice, you can Get Started with Inngest here:
Get Started