shreyansh

API Design and Gateways

30-Second Glance

  • REST for CRUD over HTTP with universal tooling; gRPC for internal service-to-service (protobuf 3–10× smaller, HTTP/2 multiplexed, ~1–5 ms serialization wins); GraphQL when clients need flexible aggregation and you can afford the gateway complexity.
  • Cursor pagination for anything large or realtime (offset breaks under inserts and costs O(n) scans); idempotency keys make retries safe — the difference between "at-least-once delivery" and "at-least-once charging."
  • Version by additive evolution (never break fields); URI versions (/v2) only at genuine breaking boundaries.
  • The gateway is edge policy in one box: auth, routing, rate limiting, timeouts/retries, and composition (BFF) — so N services look like one product.

REST vs gRPC vs GraphQL

REST gRPC GraphQL
Contract HTTP verbs + JSON; OpenAPI Protobuf IDL + codegen; HTTP/2 Typed schema + query language; single endpoint
Payload/latency JSON verbose; 1 conn per request (HTTP/1.1) unless H2 Binary protobuf (3–10× smaller than JSON); multiplexed streams; browser-hostile Exact-fields fetching saves bytes; resolver fan-out can cost server-side
Streaming SSE/chunked hacks First-class: unary, server/client/bidi streaming (ideal for Stream Processing ingestion) Subscriptions (usually WebSocket transport)
Caching Free via HTTP semantics + CDN (GET, ETags) Hard (POST-like, binary) Hard (single endpoint; needs persisted queries + field-aware caching)
Failure modes Over/under-fetching; chatty clients Tight coupling to schemas; debugging binary needs tooling (grpcurl) N+1 resolvers; expensive arbitrary queries (depth/cost limits required)
Use when Public APIs, CRUD, cacheable reads Internal microservices, polyglot fleets, streaming RPC Aggregating frontends (mobile/web BFF), fast-moving client needs

The FAANG-default topology: gRPC inside, REST (or GraphQL) outside, translated at the gateway/edge. Quote the numbers: protobuf + HTTP/2 typically cuts p99 inter-service latency by single-digit ms per hop and bandwidth by 3–10× vs JSON — which compounds across 5–10-hop fan-outs.

Pagination

  • Offset (?page=3&limit=50): simple, jumpable — and wrong past small scale. Deep offsets are O(n) scans (OFFSET 500000 reads and discards 500k rows), and inserts shift pages (duplicates/skips). Fine for admin UIs; never for user-facing feeds.
  • Cursor/keyset (?after=<opaque_cursor>&limit=50, WHERE (created_at, id) < (?, ?)): stable under writes, index-friendly O(log n + limit), but no page jumps and cursors must be opaque (never expose raw offsets/IDs as API). The default for feeds, timelines, and anything realtime.
  • Numbers: at 1 M-row tables, page 10,000 via offset can take 100s of ms vs single-digit ms for keyset. Add total_count only when cheap (separate cached counter), and cap limit (e.g. 100) to bound fan-out cost.

Idempotency Keys

Retries are mandatory (timeouts, 5xx, LB failover), so non-idempotent endpoints (POST /payments) need keys: client generates Idempotency-Key: <uuid> per intent; server stores (key → status + response) with a TTL (e.g. 24 h) and replays the stored response on duplicates instead of re-executing. State machine per key: new → in-progress → done; concurrent duplicates either block on in-progress or get 409 Conflict — both beat double-charging. Scope keys per endpoint + account, fingerprint the payload (same key + different body = 422), and never reuse keys across distinct user intents. Stripe's payments API is the canonical reference. Note the honest framing: this gives effectively-once execution, not true exactly-once — the underlying transport stays at-least-once.

Versioning

Options in increasing order of pain: additive evolution (only add optional fields; never rename/retype/remove — protobuf field numbers are forever; JSON consumers must ignore unknown fields) → header/content negotiation (Accept: application/vnd.shop.v2+json) → URI versioning (/v2/orders). URI versions are the most visible and most expensive (parallel codepaths, doubled test surface) — use only at genuine breaking boundaries (auth model change, semantic rewrite), and sunset with 6–12 month notice plus usage dashboards per version. For gRPC: additive field rules + new service/method for breaks; never reuse a field number.

The Gateway

One ingress tier (Kong, AWS API Gateway, Apigee, Envoy-based ingress) owning cross-cutting edge policy so services don't reimplement it N times:

  1. Auth: validate JWT/OAuth at the edge with cached JWKS (~ms locally vs a round-trip to the auth service per request); pass down verified identity (headers/claims), never raw tokens.
  2. Routing: host/path → service + version; canary/shadow by header or percentage (ties to the L7 tier in Scaling Out and Load Balancing).
  3. Resilience budgets: per-route timeouts (edge timeout < client timeout, with headroom per hop: e.g. 2 s client → 1.5 s gateway → 1 s service), bounded retries with backoff + jitter (retry only idempotent operations), circuit breaking and bulkheads per backend.
  4. Composition / BFF: one /feed call fans out to user/post/graph services and stitches — saves N mobile round-trips of 50–200 ms each at the cost of gateway-side fan-out latency (bounded with per-call deadlines) and thicker gateway code. GraphQL servers often live exactly here.
  5. Policy + money: rate limiting/quotas per key/plan, request validation, API metering for billing, audit logging.

Quotas worth quoting: API Gateway-style managed tiers default ~10k rps per account/region (raisable) with ~29 s integration timeouts — size your timeouts and load tests against those ceilings, and prefer a self-managed Kong/Envoy fleet past sustained high-throughput.

loading diagram…

Trade-offs

Approach Pros Cons Use when
REST + OpenAPI Universal clients, CDN-cacheable, debuggable Over/under-fetch; JSON cost at scale Public CRUD APIs, webhooks
gRPC internal Compact, fast, typed, streaming Browser/tooling friction; hard to cache Service-to-service, streaming ingestion
GraphQL at edge Clients fetch exactly what renders; fewer round-trips Resolver N+1; query-cost governance needed Aggregating mobile/web frontends
Cursor pagination Stable, index-cheap at depth No jumps; opaque-cursor plumbing Feeds, timelines, large listings
Idempotency keys Safe retries; no double-execution Key store + state machine to build Any mutating endpoint clients retry
Gateway/BFF One policy point; fewer client round-trips Extra hop (~1–5 ms); gateway becomes critical path More than a handful of public-facing services

Interview One-Liners

  • Q: Mobile feed needs user + posts + likes. One endpoint or three?
    A: One composed endpoint (BFF/GraphQL at the gateway): three client round-trips at 100+ ms cellular RTT each vs one gateway fan-out with parallel bounded calls.
  • Q: POST /payments timed out — retry?
    A: Only with the same idempotency key; server replays the stored result. Retrying keyless risks double-charge — this is the effectively-once pattern.
  • Q: Offset or cursor for an infinite-scroll feed?
    A: Cursor: offset degrades O(n) at depth and duplicates/skips rows under concurrent inserts.
  • Q: REST or gRPC between 50 internal services?
    A: gRPC (typed contracts, 3–10× smaller payloads, streaming) with REST/GraphQL translated at the edge gateway for external clients.
  • Q: How do you ship a breaking API change?
    A: Prefer additive evolution; if truly breaking, new /v2 route with dual-serve, per-version usage metrics, and a dated sunset — never mutate v1 semantics in place.
  • Q: Gateway adds latency — why not direct-to-service?
    A: ~1–5 ms per hop buys centralized auth, rate limiting, and routing; without it every service reimplements policy inconsistently — the wrong place to save milliseconds.