Backend → Inference-Platform track for Java + K8s engineers targeting $75-95k+ offshore-remote (5L/mo net).
Rule: If you can't say why you need Triton, use vLLM.
Theory spine: Inference Engineering by Philip Kiely (Baseten Books, 2026) — 259 pp. Read the chapter per week (see the roadmap); use Appendix A as glossary and Appendix B as the reading list. Framing: every inference platform is runtime + infrastructure + tooling, across the stack from CUDA to Kubernetes.
Related: Index of Pattern Vault · Index of High Level Design · Introduction to Spring Boot
- 00 Roadmap and Study Plan
12 weeks, 45 min/day, 30-50 GPU rent. Start here.
- 00b Prerequisites Product and Model Selection
Ch 0-1: runtime/infra/tooling, online vs offline, model selection, SLOs.
- 01 Fundamentals Prefill Decode KV Cache
Ch 2: roofline, ops:byte, TTFT/TPOT, PagedAttention, continuous batching.
- 02 GPU CUDA Basics for Backend Devs
Ch 3-4.1: VRAM math, bandwidth, arch generations, MIG, NCCL, GPU selection.
- 03 vLLM Deep Dive
Ch 4.2-4.3: serve, EngineArgs, prefix-cache, chunked prefill, /metrics.
- 04 SGLang TGI Triton When Why
Ch 4.3-4.4: decision matrix, when to switch off vLLM (Triton = optional Hopper track).
- 05 Quantization AWQ GPTQ FP8
Ch 5.1: FP16 → FP8 → INT4, parity checks.
- 06 Batching Caching Spec Decoding
Ch 5.2-5.4: batching, RadixAttention, MoE/EP, speculative decoding crossover.
- 06b Modalities
Ch 6: VLM, embeddings, ASR, TTS, image/video generation.
- 07 Kubernetes GPU Production
Ch 7.1-7.3: Deployment+KEDA, prod-stack Helm, probes, PVC cache, multi-cloud/procurement.
- 08 Routing Router LiteLLM Dynamo
Ch 5.5+4.4: round-robin → prefix-aware → disaggregated P/D.
- 09 Observability Benchmarking Cost
Ch 7.4+4.5: Grafana, guidellm/genai-perf, /MTok, publish checklist.
- 10 Java Bridge FastAPI Gateway
Ch 7.5: Spring gateway → vLLM core, minimal Python glue.
- 11 Projects P1 P2 P3
portfolio builds that get interviews.
- 12 Interviews Resume Positioning
positioning as Backend/Platform + Inference.
- 13 Resources Glossary
every link, paper, repo, live benchmark (App A/B mapped).