What to read today
Updated daily. Last update: Mon, 26 Oct 2026 · Day 1 · Week 1 · Phase 1 of 4: Resume + proof (Nov–Dec) · Next checkpoint: AI-infra resume done by 30 Nov
Today: Day 1, Mon 26 Oct
Theme: why decode is memory-bound and prefill is compute-bound.
🏢 Office
Already over today. From tomorrow, this slot is for reading and watching only.
🌙 Evening (~1 h, it’s a late start)
| Time | Task |
|---|---|
| 0:00–0:40 | Read Inference Engineering Ch 2 (models and bottlenecks), up to and including prefill vs decode. Stop there. |
| 0:40–1:00 | Code. A small Python script for Llama-3-8B: FLOPs and bytes moved per token for prefill and for decode, arithmetic intensity (FLOPs ÷ bytes) for each, compared with your machine’s compute-to-bandwidth ratio. |
| 1:00–1:05 | Notes. 2–3 lines: the key idea, one question, how it maps to DB / distributed systems ideas. |
Done when: the script prints both intensities and shows decode is memory-bound.
This week (Week 1: 26 Oct – 1 Nov)
| Day | 🏢 Office (read / watch) | 🌙 Evening (build) |
|---|---|---|
| Mon 26 | — | Book Ch 2 (first half) + arithmetic-intensity script |
| Tue 27 | kipply, Transformer Inference Arithmetic | 1 LeetCode · 6.5840: check where MapReduce / Raft 3A stand, continue |
| Wed 28 | How to Scale Your Model: the rooflines chapter | 1 LeetCode · Book Ch 2 (rest) · start GPT-2 inference from scratch: load the real weights into your own PyTorch model, check outputs match Hugging Face |
| Thu 29 | Horace He, Making Deep Learning Go Brrrr | 1 LeetCode · 6.5840 lab |
| Fri 30 | CMU 15-445 Lecture 1 at 1.5x | GPT-2: add a KV cache (15-445 labs wait until Raft is done) |
| Sat 31 | — | 1 LeetCode · 5–6 h: GPT-2 benchmark: tokens/sec and memory with vs without KV cache, across batch sizes · or 6.5840 catch-up |
| Sun 1 | Light | Weekly contest · Resume pass 1: list every bullet missing a number (p99, throughput, users, hours saved) and collect the numbers from work · first weekly notes post |
Roadmap (Oct 2026 – Sep 2027)
| Phase | When | Focus |
|---|---|---|
| 1. Resume + proof | Nov – Dec 2026 | AI-infra resume with numbers · GPT-2 + KV cache benchmark · Narad benchmarks · Raft 3A–3B · LeetCode 5–6/week · DDIA Ch 5, 8, 9 |
| 2. Interview prep | Jan – Mar 2027 | LeetCode daily · system design (classic + ML system design) · project deep-dive stories · behavioral · mock interviews · mini inference engine |
| 3. Applications | Apr – Jun 2027 | Referrals, applications in waves, interviews |
| 4. Offer + move | Jul – Sep 2027 | Negotiate, paperwork, move |
Checkpoints
| By | Checkpoint | Status |
|---|---|---|
| 30 Nov 2026 | AI-infra resume variant done, every bullet has a number | ⬜ |
| 31 Dec 2026 | GPT-2 + KV cache benchmark published · Raft 3B passes | ⬜ |
| 31 Jan 2027 | 30 LeetCode in Jan · ML system design notes for 3 designs | ⬜ |
| 28 Feb 2027 | 6 mock interviews · mini inference engine runs | ⬜ |
| 31 Mar 2027 | Deep-dive stories ready · contest rating 2000+ | ⬜ |
| 30 Jun 2027 | Offer signed | ⬜ |
Reading queue (Phase 1)
- kipply, Transformer Inference Arithmetic
- Horace He, Making Deep Learning Go Brrrr From First Principles
- Google, How to Scale Your Model (rooflines, parallelism)
- Llama 2 / Llama 3 papers: GQA and what it does to KV cache size
- Lilian Weng, Large Transformer Model Inference Optimization
- vLLM / PagedAttention blog post + Anyscale, continuous batching
- Martin Kleppmann, distributed systems lectures (pairs with 6.5840)
- Raft paper (before Raft 3A)
- GPU MODE lectures on YouTube (closer to Phase 2)
- Inference Engineering: Ch 2 ⬜ · Ch 3 ⬜ · Ch 5 ⬜
- Kleppmann, Designing Data-Intensive Applications (1st ed.), one chapter a week from Week 2, office reading: Ch 5 Replication ⬜ · Ch 8 Trouble with distributed systems ⬜ · Ch 9 Consistency and consensus ⬜ (these three go with Raft) · then Ch 3 Storage ⬜ · Ch 7 Transactions ⬜ · Ch 6 Partitioning ⬜ (these go with 15-445)
DDIA: learn by doing
Each chapter read at the office gets one build that week, in the normal evening slots.
| Ch | Read | Build (prove it) | Size |
|---|---|---|---|
| 5 | Replication | Postgres primary + async replica in Docker Compose: measure replication lag under load, kill the primary mid-write and count lost writes, show a read-your-writes violation | 1 evening |
| 8 | Trouble with distributed systems | Last-write-wins with skewed clocks: two Go nodes, one clock +200 ms, show silently lost writes. Then add network partitions with toxiproxy | 1 evening |
| 9 | Consistency and consensus | Raft 3B/3C is the build. Extra: run 6.5840’s porcupine linearizability checker against a deliberately broken KV store and read the failing history | 1 evening |
| 3 | Storage and retrieval | Mini LSM-tree KV store in Go: WAL → memtable → SSTables → compaction → bloom filter. Benchmark against bbolt (B+tree): write amp, read latency | 2–3 Saturdays, after Raft 3B (portfolio piece) |
| 7 | Transactions | Two psql sessions: reproduce lost update, write skew and phantoms under READ COMMITTED / REPEATABLE READ / SERIALIZABLE, and record which level stops which anomaly | 1 evening |
| 6 | Partitioning | Consistent-hashing router in Go with virtual nodes: measure keys moved when a node joins/leaves vs mod-N hashing. Connects to llm-d cache-aware routing | 1 evening |
Reading expert code
After each build, read how experts wrote the same thing.
| After you build… | Read the expert version | Start with |
|---|---|---|
| Arithmetic-intensity script, GPT-2 + KV cache | gpt-fast (PyTorch team, ~1k lines: KV cache, int8/int4, speculative decoding) | model.py, generate.py |
| Mini inference engine (Phase 2) | nano-vllm (~1.2k lines of vLLM ideas), then vLLM itself | nano-vllm engine/; vLLM v1/core/sched/scheduler.py, v1/core/kv_cache_manager.py |
| Raft 3A–3C | etcd-io/raft (the Raft used by etcd, K8s, CockroachDB) | raft.go, log.go, node.go |
| Mini LSM KV store | LevelDB (Jeff Dean & Sanjay Ghemawat, small and clean C++), then Pebble (CockroachDB’s LSM in Go) | LevelDB db/db_impl.cc, table/; Pebble compaction.go |
| 15-445 buffer pool | Postgres buffer manager (clock-sweep eviction, maps to PagedAttention) | src/backend/storage/buffer/bufmgr.c, freelist.c |
| 15-445 B+Tree | bbolt (small Go B+tree), SQLite (heavily commented) | bbolt node.go, cursor.go; SQLite btree.c |
| Go services at work | Go standard library | net/http/server.go, sync/mutex.go, context/ |
How to read it: build your own version first, then read theirs and list 3 things they did differently and why. Trace one request or operation end to end instead of reading files top to bottom. Read the tests, which show what the authors were worried about. Use git blame and the linked PRs to find out why the code is shaped that way. Code reading counts as office reading time.
Rules
- Office is for reading and watching. Evenings are for building.
- After every read: 2–3 lines of notes, rolled up into the Sunday post.
- LeetCode: 5–6 a week in Phase 1, daily from January. Timed, weak topics, explained out loud.
- Falling behind moves the calendar. It never means doing two days at once.
- Eight months of showing up beats two weeks of 14-hour days. Sunday stays light.
Log
| Day | Date | Planned | Done |
|---|---|---|---|
| 1 | Mon 26 Oct | Book Ch 2 (first half) · intensity script | (pending) |