shreyansh

00b Prerequisites Product and Model Selection

Part of Index of Inference Engineering. Book: Ch 0 (Inference) + Ch 1 (Prerequisites).

1. The three layers (memorize)

Every inference platform is three layers that must work together:

  • Runtime — model performance techniques: batching, caching, quantization, speculation, parallelism, disaggregation.
  • Infrastructure — autoscaling at small scale; at scale (few hundred GPUs+) it becomes capacity, multi-cloud, reliability, geo.
  • Tooling — level of abstraction. Black box (give weights, get an API) vs full control. The right DX is in the middle: enough control for mission-critical inference without rebuilding everything.

Interview framing: "Inference engineering spans three layers — runtime, infrastructure, and tooling, from CUDA to Kubernetes."

2. App shape drives the optimization target

Shape Optimize for Notes
Online / interactive TTFT + ITL latency-sensitive, streaming
Offline / batch $/MTok + throughput latency-tolerant, big batches
Consumer bursty, scale-to-zero, cost variable traffic
B2B steady, SLA, regional, security compliance, geo residency

Fill this in for your real app: latency budget → expected concurrency → cost ceiling.

3. Model selection

  • Open vs closed: open buys latency control, cost control, and customization; closed buys raw capability. Match to product, not leaderboard.
  • Evaluate on your task. A 20-100 example eval set beats public benchmarks. Keep it for parity checks (05-Quantization-AWQ-GPTQ-FP8).
  • Fine-tuning for domain quality: LoRA/QLoRA adapters — but prompt/RAG first. No training in this plan, only serving/tuning.
  • Distillation: small model trained on a big model's outputs. Cheaper inference near the teacher's quality — a strong inference-cost lever.

4. Measuring latency and throughput

  • TTFT = queue + prefill. ITL/TPOT = decode. Throughput = aggregate tok/s.
  • Percentiles, never the mean: report p50/p95/p99. The tail defines UX and the SLO.
  • End-to-end includes network + client rendering, not just server time.
  • Perceived TPS is inversely related to ITL (2 ms ITL ≈ 500 TPS).

5. SLOs (define before building)

p95 TTFT < X ms, p99 ITL < Y ms, quality ≥ floor, $/MTok ≤ Z. Every optimization must move one metric without breaking the others — that tension is the whole job.

6. Do (no GPU)

  • Write your app's SLOs: p50/p95/p99 TTFT + ITL + $/MTok.
  • Pick one open model; justify vs one closed model on latency, cost, and control.
  • Build a reusable 20-example eval set for every later parity check.

7. Interview one-liners

  • "Online vs offline changes the objective: TTFT for interactive, $/MTok for batch."
  • "Open models buy latency, cost, and control; closed models buy capability. The choice is product-specific."