Instagram

1. Requirements

Functional:

1. Upload photo (≤ 20 MB) / video (≤ 4 GB, ≤ 60 s reels longer async) with caption, filters client-side.
2. Home feed (ranked, cursor pages) + profile grid + stories tray + reels tab.
3. Stories: 24 h TTL, viewers list, replies → DMs.
4. Like/comment/follow; counts approximate.
5. Basic safety: NSFW/CSAM hashing + takedown within minutes.

Non-functional:

Out of scope:

• Client-side filter rendering engine.
• Full messaging stack (stories replies hand off to DMs).
• Live streaming (separate infra).

2. Capacity Estimation

Assumptions: 2B users, 500M DAU; 100M photos/day (2 MB avg) + 10M videos/day (30 MB avg); 3 stored variants per photo (~2.6 MB total); 10B media views/day (avg served 500 KB after variant selection); peak 3×.

3. API Design

POST /api/v1/media/upload-slot
{ "bytes": 2100000, "mime": "image/jpeg", "checksum_sha256": "…" }
201 { "media_id": "m_9f2Kd",
      "upload_url": "https://uploads.ig.example/m_9f2Kd?sig=…",  // presigned, 15-min
      "upload_id": "u_77" }                                      // multipart session for video

PUT {upload_url}              // bytes go direct to object storage, resumable chunks
POST /api/v1/media/m_9f2Kd/commit
{ "caption": "…", "filter": "clarendon", "checksum_sha256": "…" }
202 { "media_id": "m_9f2Kd", "status": "PROCESSING" }

GET /api/v1/feed?limit=20&cursor=...        // same contract as News Feed
GET /api/v1/stories/tray                      // active (unexpired) stories of followees
GET /api/v1/media/{id} → 302 to CDN-signed variant URL (size negotiated by client hints)

• Client picks variant by Save-Data / viewport hints (?w=1080); response Vary + separate edge keys per width bucket (3 buckets, not infinite).
• Webhooks/polling for PROCESSING → READY (video): GET /media/{id}/status with 60 s client backoff.

4. Data Model & Storage

-- media_metadata (media_id PK, owner_id, type, status, caption,
--   variants JSON {thumb,feed,full,hls...}, bytes, checksum UNIQUE,
--   created_at, expires_at NULL, INDEX (owner_id, created_at DESC))
-- stories ((owner_id), expires_at, story_id) WITH TTL semantics + viewer counts

• checksum UNIQUE gives free dedup: re-upload of the same bytes returns the existing media_id (quota + storage win).
• Versioned variant URLs (…/v3_…) ⇒ CDN invalidation is a non-problem for edits; takedowns use purge-by-prefix (rare, fine).

5. High-Level Design

Upload: app → upload-slot (id + presigned URL) → client PUTs bytes direct to object storage → commit (checksum verify, metadata INSERT, safety-hash check) → workers (variants/transcode/ML/embeddings) → READY → fan-out id into feeds (News Feed hybrid). Read: feed → hydrate media metadata → 302/client-direct to CDN variant URLs; stories: tray → hot TTL store → CDN.

• Upload service: slot/commit/status only — never handles bytes (stays stateless + tiny).
• Object storage + CDN: bytes plane; origin shield in front of buckets; 3 width buckets cached at edge.
• Worker fleet (async): thumbnail/transcode, NSFW/safety, embeddings for ranking/ads, stories-expiry sweeper.
• Metadata DB (hash-sharded): the source of truth clients hydrate from.
• Feed + stories read path: News Feed hybrid for posts; TTL hot store for stories; viewers-list writes buffered (Kafka → rollup, cap displayed list).

6. Deep Dives

6.1 Upload pipeline: resume, dedup, exactly-once metadata

Flow with failure handling at each step: (1) upload-slot is idempotent on (user, client_mutation_id) — retries return the same URL. (2) Bytes transfer is multipart with per-part ETags; client resumes from the server-reported received-parts list (no restart on flaky mobile). (3) commit verifies SHA-256, then INSERT ... ON CONFLICT(checksum) DO NOTHING — double-commit returns the existing row (exactly-once effect without distributed transactions; note the Atomic Commit and Two-Phase Commit (2PC) alternative and why it's overkill here). (4) Safety-hash check runs before fan-out but after accept — offending media is quarantined (status BLOCKED), never fanned out, reported within minutes. (5) Orphan sweeper: slots with bytes but no commit after 24 h are GC'd (multipart abort + delete). Poison media (worker crash-loop) → dead-letter + FAILED status surfaced to the client with retry.

6.2 CDN strategy: variants, versioning, cost control

• Variant ladder: photo → thumb (150px, ~30 KB), feed (1080px, ~400 KB), full (original); video → HLS 3–5 renditions. Edge caches only the 3 photo buckets + top 2 video renditions; tail renditions collapse to origin shield (hit ratio still >90% by bytes because the head dominates).
• Versioned URLs kill invalidation: edit/re-encode ⇒ new version path; old edge entries expire naturally. Purge reserved for takedowns (purge-by-prefix, audited, rate-limited — a purge storm is a self-DDoS).
• Cost levers (name 2–3 in the interview): AVIF/WebP negotiation (~30% bytes saved), Save-Data downgrade ladder, prefetch only first-viewport media, P2P/off-peak prefetch for drafts — none. Egress dominates the bill, so every 10% byte saving beats any origin optimisation.
• Signed URLs for private accounts/stories (exp=15 min, minted per viewer) vs public long-TTL edge caching — two policies, enforced at the 302 step, not in CDN config.

6.3 Stories TTL at scale + feed reuse statement

• Stories rows carry expires_at = created_at + 24 h; hot store (Redis/Cassandra) applies native TTL so expiry is deletion by clock, not by job. A daily sweeper only reconciles (object-storage delete for variants + viewers-list GC + safety retention holds) — the read path never sees expired rows because every read filters expires_at > now (defence in depth against clock/GC lag).
• Viewers-list: SADD story:{id}:viewers capped — past 10k, store count only (HyperLogLog + exact head). Writes buffered via Kafka; story owner sees ~second-fresh counts, fine.
• Feed reuse (say explicitly): home feed, fan-out hybrid, celebrity pull, cache tiers, cursor pagination are inherited unchanged from the News Feed case — Instagram's novelty is the media plane + stories lifecycle, not a new feed algorithm. Reels = same feed with a video-ranked scorer + prefetch of first video bytes.

7. Bottlenecks, Trade-offs & Monitoring

First bottleneck at 10x: video transcode fleet (300 TB/day ingress ⇒ GPU/CPU burn) → admission control (cap concurrent transcodes per user), prioritise by predicted reach (celebrity videos first), downgrade ladder under backlog (ship 720p now, 4K later). Next: CDN egress bill → format + prefetch discipline (§6.2). Next: metadata DB write heat on viral posts' comment threads → comment pagination + counter batching.

Interview one-liners

• Q: Where do upload bytes flow? / A: Never through origin — presigned direct-to-object-store, resumable; origin only handles slot/commit metadata.
• Q: How do stories disappear? / A: Native TTL + read-time expiry filter; a sweeper only reconciles bytes and viewer lists.
• Q: New feed algorithm for Instagram? / A: No — reuse the News Feed hybrid; Instagram's hard parts are the media plane and stories lifecycle.

Related

• News Feed case — prerequisite: feed, fan-out, ranking, cache tiers
• Twitter Timeline case — sibling: media/graph side-paths, home vs user timelines
• Cache Patterns and Eviction — CDN + edge variant caching
• Partitioning — metadata sharding by media_id
• Atomic Commit and Two-Phase Commit (2PC) — why commit uses idempotent insert instead
• Stream Processing — transcode farm + viewers-list rollups
• Relational Vs Non-Relational DBs — metadata store reasoning