Microservices Patterns

Gateway and Service Discovery: The Plumbing Layer

API gateway

Single edge entrypoint (Kong, NGINX/Envoy, AWS API Gateway, Spring Cloud Gateway) owning all cross-cutting concerns: TLS termination, authentication (validate JWT/OAuth2 tokens — see Security and Auth for System Design), rate limiting per key/tenant, request routing and versioning, response aggregation (backend-for-frontend variants per client), and WAF hooks. Numbers: Envoy sidecar adds ~1 ms p99; a centralized gateway adds a network hop (~1–5 ms intra-AZ) but removes auth/TLS code from every service.

Say the trade-off out loud: the gateway is a potential bottleneck and single point of failure — run ≥2 AZs, keep it stateless, push heavy logic (aggregation) to BFF services so the gateway stays a thin policy enforcement point. WebSocket/SSE fanout and large uploads often bypass it via signed URLs straight to object storage.

Service discovery

Container IPs churn on every deploy/scale event, so hardcoded hosts die fast. Two flavors:

• Server-side discovery: client calls a load balancer (ALB, kube-proxy/ClusterIP Service); the LB tracks healthy endpoints via heartbeats. Simple client, extra hop, LB must scale.
• Client-side discovery: client queries a registry (Consul, Eureka, etcd-backed, Kubernetes Endpoints API) and load-balances itself with health-aware pickers (Envoy/xDS, gRPC resolvers). Lower latency, smarter balancing (least-loaded, zone-aware), but every client embeds discovery logic.

In practice Kubernetes DNS + ClusterIP/Headless services plus an ingress controller covers most interviews; name Consul/etcd when multi-cluster. Health checking (liveness vs readiness probes) is what makes discovery correct: readiness gates traffic during deploys, liveness restarts wedged pods. Discovery state itself needs consensus — registries typically run Raft underneath (see Raft Leader Election and Coordination Services).

Distributed Transactions: Saga vs 2PC vs Outbox

The core interview topic. Frame it as: "ACID doesn't cross service boundaries — each service owns its database, so I choose how much consistency to buy."

2PC (and why you almost never use it)

Two-phase commit (deep dive in Two-Phase Commit) gets atomicity via prepare + commit phases under a coordinator. Costs that kill it in microservices: blocking (participants hold locks between phases — a slow coordinator stalls everyone), coordinator as SPOF (needs its own recovery log), and latency (2 round trips minimum, more across regions). Interview line: "2PC gives atomicity but sacrifices availability — during a partition, participants block; that's why it's absent from cross-service designs and survives only inside single databases (Spanner uses it with Paxos-replicated coordinator state and TrueTime to bound the pain)."

Saga

A saga is a sequence of local transactions, each publishing an event that triggers the next step; a failure runs compensating transactions backwards. Two orchestrations:

• Choreography (event-driven): services react to events, no central controller (order service emits OrderCreated → payment service charges → emits PaymentDone → inventory reserves…). Pros: no coupling to a controller, natural on Kafka (see Messaging and Event Queues). Cons: the workflow is implicit — debugging "why did order 42 stall?" means tracing events across services; cyclic confusion and no obvious place for timeouts.
• Orchestration: a saga orchestrator issues commands and tracks state machine explicitly (Temporal/Cadence, Camunda, or a bespoke orchestrator service). Pros: workflow is visible code with timeouts/retries in one place. Cons: orchestrator is coupling + load concentration.

Name the hard truths: sagas give no isolation (another transaction can see the half-completed saga — the hotel booked but flight not yet), compensations must be idempotent and retryable (they can fail too — who compensates the compensation? answer: retry-until-success with DLQ + human ops dashboard), and semantic locks (PENDING states, holds with TTL like airline seat holds) plug the isolation gap. Concrete: e-commerce checkout = saga with orchestrator, compensations = refund payment, release inventory hold, cancel shipment; each step idempotent on order-id.

Transactional outbox (+ inbox)

The outbox solves the dual-write problem: "I updated my DB and published an event — crash between them and they're inconsistent." Instead of publishing inline, the service writes the business row and an outbox row in one local DB transaction; a relay (CDC via Debezium reading the WAL, or a polling publisher) publishes outbox rows to the broker and marks them sent. Atomicity of state + intent-to-publish comes free from the local transaction — no distributed commit. Pair with the inbox pattern on the consumer (store processed message IDs, skip duplicates) for effectively-once processing over at-least-once delivery.

Interview default: outbox for reliable publishing, orchestrated saga for multi-service workflows, 2PC named only to reject it (except single-DB internals).

CQRS: Split Reads from Writes

Command Query Responsibility Segregation: the write model (normalized, constraint-enforcing OLTP) and read model (denormalized, query-shaped projections) are different stores fed by events. Writes go to the command side; events project into read replicas, search indexes, materialized views, or per-query caches.

Why: reads and writes scale differently (100:1 read-heavy social feed — see cache-aside for the caching half of this) and want different schemas (3NF for integrity vs wide denormalized rows for p99). Cost: the read side lags (eventual consistency — "your post appears in search in ~seconds"), projections need versioning/replay (keep the event log — Kafka retention — so a new projection rebuilds from history), and simple CRUD apps don't earn back the complexity. Draw the line: CQRS for hot read paths and divergent models; plain request/response + read replicas for everything else. Event sourcing (store the event log as the source of truth, rebuild state by replay) is CQRS's frequent companion and inherits the same replay/ordering discipline from the Stream Processing world.

Resilience: Circuit Breaker, Bulkhead, Retries

One slow dependency must never take down its callers. The trio, with numbers:

• Circuit breaker (Resilience4j, Polly, Envoy outlier detection): closed (normal, counting failures) → open after threshold (e.g. 50% failures over 20 requests in 10s → fail fast for 30s, returning fallback/cached response) → half-open (probe with limited traffic, close on success). Prevents retry storms and gives the dependency room to recover. Every breaker needs a fallback: stale cache, default value, degraded response — "fail fast" without a fallback is just failing.
• Bulkhead: isolate resources per dependency — separate thread pools / connection pools / concurrency limits (e.g. 20 connections for payments, 50 for search) so the slow payments pool exhausting doesn't starve search. Same logic at deploy level: separate node pools/quotas per critical service. The name is literal: one flooded compartment shouldn't sink the ship.
• Bounded retries with backoff + jitter + budgets: retry only idempotent operations, cap attempts (2–3), exponential backoff with jitter, per-request deadlines (never let 3 retries × 5s timeout = 15s block a 2s SLA), and a global retry budget (e.g. ≤10% of traffic) so retries can't amplify an outage — the classic cascade is retries multiplying load on a recovering service 3x and re-killing it.

Add timeouts everywhere (p99-based, with hedging only for strictly-read tail latency: send a second request at p95 and take the first response — doubles cheap-read load, never use for writes).

Strangler Fig: Getting There from the Monolith

Big-bang rewrites fail; strangulation works: put a facade (the gateway) in front of the monolith, route endpoints to new services one seam at a time, and let the monolith shrink until nothing is left. Mechanics: carve at bounded-context seams (billing, notifications first — low coupling), run old and new paths in parallel with shadow traffic/dark launches to compare, use anti-corruption layers (adapters translating legacy schemas so the new service stays clean), and migrate data with dual-write or CDC-backfill + verification jobs before flipping reads. The interview line: "I'd strangler-fig the monolith starting with the least-coupled domain, validating each cutover with shadow traffic and instant rollback at the gateway."

Trade-offs

Interview One-Liners

• Q: How do two microservices transact without 2PC? A: Orchestrated saga with idempotent compensations, reliable publishing via transactional outbox, dedup via inbox — atomicity traded for availability, and semantic PENDING states covering the isolation gap.
• Q: Why not 2PC across services? A: It blocks lock-holding participants on coordinator health across the network — a partition stalls writes everywhere; it survives only inside single databases that replicate the coordinator itself.
• Q: Choreography or orchestration for checkout? A: Orchestration — checkout has timeouts, parallel branches, and compensations I want as visible code; choreography for simple linear notifications.
• Q: A downstream goes slow — what stops the cascade? A: Timeouts + bounded retries with jitter and a retry budget, circuit breaker with a real fallback, and bulkheaded pools so one dependency can't starve the rest.
• Q: How do you split reads and writes at 100:1? A: CQRS — normalized write store projecting events into denormalized read models, accepting seconds of read lag and funding projection replay from the retained event log.
• Q: How do you migrate the monolith without a rewrite? A: Strangler fig behind the gateway — carve the least-coupled seam first, shadow-traffic validation, CDC data migration, instant rollback by re-routing.
• Q: Where does service discovery state live? A: A strongly-consistent registry (Raft/etcd-style) with liveness vs readiness health checks gating traffic — discovery is only as correct as its health signals.

Related

• Two-Phase Commit — the protocol sagas replace
• Messaging and Event Queues — the transport sagas and outbox publish over
• Stream Processing — event backbone for choreography and CQRS projections
• Coordination Services — discovery/registry foundations
• Raft Leader Election — what keeps the registry consistent
• Security and Auth for System Design — what the gateway enforces
• CAP, PACELC and Consistency Models — why distributed transactions trade what they trade
• cache-aside — the caching half of read scaling