Rate Limiter — lesson

Distributed throttle at ~1M checks/s, <2ms p99. Core trick — token bucket evaluated atomically in Redis via one Lua script; rules versioned and pushed to gateways.

Enforcement path

One Lua eval per check

The gateway checks inline before proxying. Flip the Algorithm toggle to sliding-window counter and the Global exactness toggle to exact — then watch the metrics explain why production picks approx.

1 · Check inline at the edgeGateway runs the check before proxying. Rules come from a local cache (30s TTL + version), so ru…

1/5

React Flow

1 · Check inline at the edge

Gateway runs the check before proxying. Rules come from a local cache (30s TTL + version), so rule reads add no hop.

Read the theoryAPI Design and Gateways →Rate Limiting and Throttling →

Algorithm

Burst = capacity, sustained = refill. One Lua eval.

Global exactness

Divide quota per DC. Bounded over-admission, 1 RTT.

Edge gateways 100Req/s per gateway 10000Active limit-keys 10M

Check QPS avg / peak

1.0M / 3.0M

Redis shards needed

≥ 15

~100k Lua ops/s per shard + replicas

Bucket memory

~3.0 GB

token bucket O(1); sliding-log would be 5×+

Edge→Redis bandwidth

~500 MB/s

must be same-AZ

Multi-DC policy

Divided + headroom

Each DC enforces limit/3 with 20% borrow headroom

Test yourself

1. Why Lua?

Without atomicity, concurrent checks interleave read-refill-write and over-admit several-fold.

2. Fixed-window flaw?

Counters reset at :00 — a spike straddling the boundary passes 2× quota. Sliding counter smooths it.

3. Redis down on a read path?

Reads fail open (serve, alert); money-mutations fail closed. Either way, page on it.

4. Why divide quota per DC?

Linearizability per check at 1M/s is death — accept bounded over-admission and document the bound.

Drag nodes · scroll page normally · add ?elk to URL for auto-layout

Fixed vs log vs counter

Sliding windows without the O(n) bill

Approach	Pros	Cons	Use when
Fixed window	1 key, O(1)	2× burst at :00 boundaries	Coarse quotas
Sliding log	Exact	O(n) memory (16GB+ here)	OTPs, password resets
Sliding counter	O(1), smooth	Tiny edge error	Standard throttles

Counter formula: estimate = cur + prev × overlap. Two integer keys with EXPIRE — no Lua needed. Lua stays for the token bucket, using Redis TIME inside the script so client clocks can't over-admit.

Races, clocks, multi-DC

Fail open or closed? Per endpoint.

Read path fails open (serve, alert); money-mutations fail closed (503 + metric) — decided per rule, not globally. Multi-DC divides quota (each DC limit/3 + 20% headroom); exact global would cost a coordination RTT per request. First bottleneck at 10×: hot-slot on viral keys → gateway pre-aggregation + slot-splitting top-K keys.

Say it in the room

Interview one-liners

Q: Why Lua? / A: Atomic refill-and-debit; without it concurrent checks over-admit.
Q: Exact global limits? / A: No — divide per DC with headroom; exactness costs a RTT per request.
Q: Limiter down? / A: Reads open, money-mutations closed; page either way.

Full write-up: Rate Limiter case note · Theory: Rate Limiting and Throttling, Linearizability · Next: Twitter Timeline playground →