API Rate Limiting: Protect Reliable Customer Access

TL;DR — API rate limiting sets a fair, explicit ceiling on work a caller can ask your service to do in a period. It protects everyone from accidental loops and abusive traffic. A useful limit identifies the affected caller, returns a clear 429 response, and tells a well-behaved client when it can retry.

API rate limiting is capacity management expressed as a product rule. Without it, the busiest integration, a misconfigured script, or a compromised key can consume the database and queue capacity needed by every other customer. With a poorly designed limit, you create a different outage: legitimate customers cannot understand or recover from a rejection.

The customer-facing goal is predictable behavior. The internal goal is to keep one source of demand from exhausting a shared dependency. Those goals meet at the response contract.

Start with the unit of fairness

Limit the thing that maps to responsibility. For a public API that is often an API key, organization, or workspace. An IP address alone is rarely enough: a company can have many users behind one address, and an attacker can have many addresses. For an unauthenticated endpoint, combine IP-based protection with other controls, then tighten after a user identifies themselves.

Do not use one global number because it is easy to configure. Reads and writes have different costs. A search endpoint may need a lower budget than a simple status read; an export may need a separate queued workflow. This is why API pagination and rate limits belong together: bounded pages make work measurable.

The HTTP standard reserves status code 429 Too Many Requests for rate limiting in RFC 6585. Return it only when the caller can act on the response. Include Retry-After when you know the wait, and document any remaining-budget or reset headers you expose. A vague 500 makes a customer retry harder and creates more load.

Pick an algorithm that matches the promise

A fixed window is simple: 100 requests during a clock minute. Its boundary permits a burst at the end of one minute and the start of the next. A sliding window smooths that edge but costs more bookkeeping. A token bucket is often practical: tokens refill steadily, and a short burst is allowed up to the bucket size. The correct choice depends on whether a burst is safe for the downstream service, not on the popularity of the algorithm.

Suppose, hypothetically, that a workspace has a budget of 60 tokenized requests per minute and a bucket of 20. A new batch job may use 20 requests immediately, then must wait for refills. That is a capacity policy, not a universal benchmark. The number should come from observed dependency capacity and a planned reserve for other tenants, then be revised by load testing.

Make rejection recoverable

A customer’s retry loop must be part of your design. Return a machine-readable error code, the affected limit name, and a wait time when available. State whether repeated requests count before or after authentication errors. If an endpoint accepts a write, pair retries with an idempotency key so the customer does not create duplicate orders after a timeout.

Use different limits for different risk classes, but do not make tiers mysterious. A self-serve customer should be able to see their current policy and request an adjustment with evidence. Internally, alert on sustained rejections by tenant and by endpoint. A spike can reveal a customer bug; it can also expose a capacity regression hidden behind the limiter.

Failure modes founders should catch

• A shared “anonymous” bucket lets one noisy visitor throttle everyone.
• A limiter lives only in one application process, so scaling adds an accidental multiplier.
• A response says retry immediately, causing synchronized retry storms.
• Limits protect the API gateway but not expensive fan-out work behind it.
• An exception list becomes permanent and quietly defeats the policy.

The last one is a business failure. An enterprise exception may be justified, but it should have an owner, expiry, observed usage, and an explicit downstream capacity decision.

Acceptance checklist

• Each limit names its subject, endpoint group, period, and owner.
• The server returns 429 and useful retry information for an exhausted budget.
• Limits are coordinated across instances and cover expensive downstream work.
• Retry tests confirm that a client backs off and a repeated write stays safe.
• Dashboards distinguish blocked, successful, and dependency-failed requests.

Rate limiting is not a sign that your product is unfriendly. It is how you keep a customer’s automation from becoming every customer’s incident.