NotesDraft/Book Notes
Designing Data-Intensive Applications·Chapter 3 of 3
Storage and Retrieval
The chapter that made write amplification feel like a physical quantity rather than a piece of vocabulary.
The two designs
| LSM tree | B tree | |
|---|---|---|
| Writes | Sequential, batched | In place, random |
| Reads | Check several levels | One path to a leaf |
| Write amplification | Compaction | Page rewrites plus WAL |
| Space | Better, compacts | Fragmentation |
| Latency | Less predictable | More predictable |
The part worth keeping
Both designs write your data more than once. The question is not whether there is amplification but where you would rather it happen: in a background process you can schedule and throttle (compaction), or inline on the write path (page splits and the write ahead log).
That framing generalises. Checkpointing during training is the same trade. You either pay steadily or you pay in a spike, and which one you want depends entirely on whether anything is waiting on you.
The compaction footgun
Compaction competes with foreground traffic for disk bandwidth. Under sustained heavy writes it can fall behind, and once it does, read performance degrades because there are more levels to check, which slows everything further. It is a feedback loop, and the chapter is refreshingly direct that it is a real operational hazard rather than a footnote.
Left wanting
Very little on how any of this interacts with modern SSD internals, which do their own remapping and have their own amplification. The mental model here is still a spinning disk in places.