MVCC internals — version chains, visibility & the bloat trap
Published
Your bar: on a whiteboard, draw how a row version is stored, how a snapshot decides which version it sees, and why one forgotten transaction bloats a whole database — then name the 3 a.m. page each engine gives you. Grounded in DDIA Ch. 7, the PostgreSQL “Concurrency Control” & “Routine Vacuuming” docs, and InnoDB “Multi-Versioning”. 1
Lesson 4 told you the behaviour; this unpacks the physics. One sentence holds it:
A write makes a new version, not an overwrite
Postgres keeps it in the heap, InnoDB in undo
a snapshot reads via xmin / xmax
dead versions need cleanup — or bloat & wraparound
xmin/xmax + "did that xid commit before me?"
Two ways to store versions — heap vs undo
Same isolation guarantee, opposite on-disk layout. Everything below follows from this fork.

Two MVCC storage models. PostgreSQL keeps every row version in the heap (old versions stay until VACUUM removes them). InnoDB updates the row in place and keeps the old version in an undo log, reconstructing past versions on demand.
Postgres — append to heap An UPDATE does not overwrite: it
inserts a brand-new tuple (xmin=current xid) and stamps the old tuple’s
xmax. Both versions sit in the heap. DELETE just sets
xmax.
InnoDB — update in place + undo The clustered row is modified in place; the prior
version is written as an undo record reached via DB_ROLL_PTR. The
table stays the latest; history lives in the undo log.
Postgres tuple header xmin = xid that created it; xmax =
xid that deleted/superseded it (0 = live). That’s it — two ids per version.
InnoDB hidden fields DB_TRX_ID = last txn to touch the row;
DB_ROLL_PTR = pointer into the undo log. Old reads walk the undo chain
backward.
🗄️ Memory rule: Postgres keeps history in the table (fast latest-row reads, but it bloats and needs VACUUM); InnoDB keeps history in undo (compact table, but old reads pay to walk undo and long readers grow the log).
✓ Postgres inserts a new tuple; InnoDB writes undo first — the old version survives either way
Memory check
- Where do old versions live in each engine? → Postgres heap · InnoDB undo log
- What two ids are in a Postgres tuple header? → xmin, xmax
- Which engine's table grows on every write? → Postgres (append); InnoDB stays compact
Visibility rules — how a snapshot decides
A snapshot isn’t a copy of data — it’s a tiny descriptor that judges each version it meets.

The visibility test. A version is visible only if its creator
(xmin) committed before the snapshot, and its remover (xmax) did
not. Two snapshots see different versions of the same row.
A Postgres snapshot is three things captured at snapshot time, checked against the commit log
(pg_xact/CLOG):
Read Committed A fresh snapshot per statement → can see others’ commits between statements.
Repeatable Read / SI One snapshot taken at first read, reused all txn → one consistent view.
InnoDB read view Same logic: low-water id, high-water id, active set — applied
while walking undo until DB_TRX_ID is visible.
Cost of the check Per tuple, a commit-log (CLOG) lookup to learn “did that xid commit or abort?”
👁️ Memory rule: Visible ⇔ xmin committed in the past AND xmax not committed in the past — two header fields plus one commit-log lookup.
Memory check
- Two conditions for a tuple to be visible? → xmin past+committed; xmax not past
- Why does Read Committed see new commits mid-txn? → fresh snapshot per statement
- What does the engine consult to test commit/abort? → the commit log (CLOG / pg_xact)
VACUUM, purge & the long-transaction bloat trap
Dead versions don’t vanish on their own. One forgotten transaction can bloat a whole database — silently.

The long-transaction bloat trap. VACUUM can only remove versions older
than the oldest running transaction’s snapshot — the xmin horizon. One
idle-in-transaction connection pins that horizon, so dead tuples pile up unbounded, with
no error.
When is a version removable? Once invisible to every snapshot that could
ever run again — its xmax committed before the oldest still-running txn.
The xmin horizon The xmin of the oldest live txn. VACUUM cannot touch
anything that died after it — across the whole DB.
VACUUM vs VACUUM FULL Plain VACUUM (usually autovacuum) frees space for reuse
in the table; the file doesn’t shrink. Only VACUUM FULL rewrites &
shrinks, under an exclusive lock.
InnoDB equivalent The purge thread reclaims undo + delete-marked rows; a long read view shows up as a growing history list length.
🧹 Memory rule: Your cleanup is only ever as current as your
oldest open transaction — one idle in transaction connection bloats
the entire database, with no error.
Defenses (operational, not magical): cap age with
idle_in_transaction_session_timeout; watch pg_stat_activity for
ancient xact_start / backend_xmin; alert on n_dead_tup.
InnoDB: alert on history list length.
Memory check
- What pins the xmin horizon? → the oldest still-running transaction
- Does plain VACUUM shrink the file? → no; only VACUUM FULL does
- InnoDB symptom of the same bug? → growing history list length
The transaction-ID wraparound time bomb
PostgreSQL-specific, famous, and has taken down real companies. The fix is freezing.
The bomb 32-bit xids (~4.2B), compared circularly. After ~2B newer xids, an old
xmin looks future → the row vanishes. Silent, catastrophic data loss.
The defense Freezing: VACUUM marks old tuples “committed &
visible to everyone — ignore xmin.” A frozen tuple is wrap-immune.
Escalation As the oldest unfrozen xid ages (autovacuum_freeze_max_age,
default 200M), Postgres runs aggressive anti-wraparound autovacuums.
The hard stop If freezing falls too far behind, Postgres refuses new xids and goes read-only rather than risk corruption — “DB stopped accepting writes.”
⏰ Memory rule: Wraparound is prevented by freezing — marking old tuples visible-to-all so their soon-to-wrap xmin is ignored. Fall behind and the database goes read-only.
The same long transaction that bloats (§3) also holds back freezing — nudging you toward wraparound. The postmortems (Sentry, Mailchimp, others) all read the same: autovacuum couldn’t keep up. 5
Memory check
- How wide are xids, and how compared? → 32-bit, circularly (~2B past / 2B future)
- What does freezing do? → marks tuples visible-to-all; ignore xmin
- What happens if freezing falls too far behind? → Postgres goes read-only
Loose ends — indexes, SSI & which model wins
Three things to know before you close the book.
Indexes carry no version A PG index points at a heap tuple; visibility lives in the heap, so index-only scans consult the visibility map. InnoDB secondary entries are delete-marked on update and often force a drop to the clustered index + undo.
Indexed-column updates are pricey Under MVCC, updating an indexed column writes new index entries in both engines — extra cost, extra bloat.
SSI is built on MVCC PostgreSQL SERIALIZABLE (SSI) runs on ordinary
snapshots and additionally tracks read/write dependencies, aborting one txn on the
dangerous-structure pattern that causes write skew.
4
Pick your discipline Knowing your engine tells you what you owe it: VACUUM/freeze monitoring for Postgres, undo/history-list + purge monitoring for InnoDB.
🧩 Memory rule: MVCC is the substrate; SSI is the safety layer over it. Neither storage model wins — they trade, and a long transaction starves cleanup in both.
✓ Postgres SSI runs on MVCC snapshots + dependency tracking — same substrate, extra checks
Memory check
- Why is an index-only scan not enough alone? → indexes carry no visibility; consult the visibility map
- What does SSI add on top of MVCC? → read/write dependency tracking + aborts
- Which storage model is "best"? → neither; they trade failure modes
Where each piece bites — production map
The concept, the symptom you actually see, and the lever you pull. This is the difference between reading MVCC and operating it.
| Internal | Production symptom | Lever |
|---|---|---|
| Heap append (PG) | Table + index files grow on every UPDATE | autovacuum tuning; HOT updates |
| Undo log (InnoDB) | Old-version reads slow; undo tablespace grows | shorten long readers; watch purge lag |
| Visibility / snapshot | RR txn doesn’t see a concurrent commit | choose RC vs RR per workload |
| xmin horizon | ”DB got slow and big” — bloat, slow seq scans |
|
| History list length | InnoDB reads degrade, purge can’t advance | kill long-open read views |
| Wraparound / freeze | Warnings → database read-only | anti-wraparound autovacuum; never disable it |
| Indexed-column UPDATE | Write amplification + index bloat | avoid updating indexed cols hot-path |
Pattern to notice: nearly every MVCC incident is one long / idle-in-transaction
connection starving cleanup. Find it in pg_stat_activity first.
Retrieval practice — answer from memory
Reading builds fluency (feels familiar). Recall builds storage (lasts). Pick before you peek — instant feedback.
Q1. On an UPDATE, PostgreSQL and InnoDB differ in that…
Q2. A PostgreSQL tuple is visible to your snapshot only when its…
Q3. A table bloats badly while one connection sits idle-in-transaction because…
Q4. Transaction-ID wraparound is prevented by VACUUM…
Q5. The one-line trade-off between the two storage models is…
Q6. PostgreSQL SERIALIZABLE (SSI) relates to MVCC how?
Reconstruct the deep dive from a blank page
Write the five levels in order, one line each, with nothing on screen. Then reveal and compare — gaps are your next drill.
1 Storage: PG appends to heap (xmin/xmax), InnoDB updates in place + undo (DB_TRX_ID/DB_ROLL_PTR) · 2 Visibility: visible ⇔ xmin past+committed AND xmax not-past; RC = fresh snapshot/stmt, RR = one snapshot · 3 Cleanup: VACUUM/purge can’t pass the xmin horizon = oldest open txn → one idle txn bloats the whole DB · 4 Wraparound: 32-bit circular xids; freezing marks tuples visible-to-all or DB goes read-only · 5 Loose ends: indexes carry no visibility; SSI sits on MVCC; neither model wins — they trade.
I’m your teacher — ask me anything. Say “grill me on MVCC internals” for mixed no-clue questions, or hand me a real incident (“the DB went read-only”, “a table is 4× its row count”) and I’ll walk you from symptom to root cause to the lever you pull. Want the next deep dive — SSI & write skew? Just ask.
Interview — pick your bar
Answer out loud in ~60s, then reveal. Core = recall · Senior = trade-offs & failure modes · Staff = synthesis under ambiguity · System Design = open design round (a different axis, not a harder level).
What does multi-version storage mean — what happens to old data when a row is updated?
xmin = the current xid and the old tuple's xmax is stamped; the old version lives until VACUUM removes it → in InnoDB the clustered row is changed in place but the prior image is written to the undo log, reachable via DB_ROLL_PTR → so a logical row is really a chain of physical versions → "no overwrite" is the core idea — both engines retain the prior version so concurrent readers can still see it.What exactly is a snapshot, and what is in it?
xmin (oldest still-running xid), xmax (next xid not yet assigned), and xip[] (the list of in-progress xids) → together they define "committed as of this instant" → Read Committed takes a fresh snapshot per statement; Repeatable Read / SI takes one snapshot at the first read and reuses it for the whole txn → the snapshot is the txn's frozen view of the commit timeline — it never sees writes that commit after it.State the visibility rule: how does a reader decide a given version is visible?
xmin) must be committed and in the past (not in xip[], below xmax) → the deleting xid (xmax) must be absent, aborted, or still in the future relative to the snapshot → if both hold, that version is the one this reader sees; otherwise it walks to the next version in the chain → commit status comes from the commit log (clog) → "visible = created-by-a-committed-past-txn AND not-yet-deleted-as-of-my-snapshot."What is version GC — VACUUM in Postgres, purge in InnoDB — and why is it needed?
VACUUM FULL is what actually shrinks the file → InnoDB's purge thread reclaims undo records and delete-marked rows once no read view needs them → the cleanup ceiling is the oldest live transaction's horizon: nothing that died after it can be removed → without GC the table bloats and reads slow down.Walk me through exactly what happens on disk when you UPDATE one row in Postgres vs InnoDB.
xmin = current xid), stamps the old tuple's xmax, and writes new index entries unless HOT applies; the old version stays in the heap for VACUUM → InnoDB modifies the clustered-index row in place but first writes an undo record reachable via DB_ROLL_PTR, and old readers walk the undo chain backward → key contrast: Postgres keeps history in the heap (cheap update, expensive cleanup, index churn), InnoDB keeps history in undo (in-place update, cheaper indexes, purge pressure) → in neither case is the prior version destroyed at write time.A table is 5× the size of its live row count and seq scans are slow. Diagnose it.
idle in transaction connection, or a long analytics query, pinning the xmin horizon so nothing that died after it can be removed — and the effect is DB-wide, not just that table → check pg_stat_activity (xact_start, backend_xmin) and n_dead_tup → fix the offending txn, don't just run VACUUM → remember plain VACUUM frees space for reuse but won't shrink the file — only VACUUM FULL does, and it takes an exclusive lock.Why is updating an indexed column more expensive under MVCC — in both engines?
Connect a long-running analytics query to table bloat, and give the operational fix.
xmin horizon, so VACUUM can't reclaim anything that died after the query started — bloat grows for the whole duration of that one read → it also holds back freezing, so it's a wraparound risk too → the read path is fine (snapshots are cheap); the damage is to cleanup → levers: idle_in_transaction_session_timeout and statement timeouts to bound txn age, route heavy analytics to a read replica, and alert on the oldest xact_start / backend_xmin (and InnoDB history list length) → the rule: cleanup is only as current as your oldest open transaction."The database suddenly stopped accepting writes." What's your first hypothesis and how does it tie to MVCC?
Postgres heap-history vs InnoDB undo-history — how does that one design choice ripple through operations?
n_dead_tup, autovacuum lag, backend_xmin, and freeze age, and you never disable autovacuum → InnoDB keeps history in undo: cheaper indexes and in-place clustered updates, but a long read view grows the undo/history list and purge falls behind → you monitor history list length and purge lag, and keep undo bounded → synthesis: different janitor, same root cause — a long transaction starving cleanup → the staff move is operating each engine to its specific failure mode rather than treating MVCC as one thing.How is PostgreSQL SERIALIZABLE built on top of MVCC, and what does it add — and cost?
- Workload shape — read/write ratio, txn duration, hot rows, how much history readers need.
- Versioning model — append-in-heap (Postgres-style) vs in-place + undo (InnoDB-style) and what each costs.
- Visibility & snapshots — per-statement vs per-txn snapshot; what determines a version is visible.
- Garbage collection — who reclaims dead versions, and what pins the cleanup horizon.
- Long-transaction defense — timeouts, replica routing, bounding undo / dead-tuple growth.
- Failure modes — bloat, wraparound/freeze lag, purge lag, index write amplification.
- Observability — oldest open txn, dead-tuple / history-list growth, autovacuum or purge lag.
Design the version-storage and GC strategy for an append-mostly audit-log table that is rarely updated but read by long analytics scans.
READ ONLY DEFERRABLE for a safe snapshot with no SSI cost, and cap them with statement timeouts → aggressively freeze old, immutable rows (they never change) so wraparound is a non-issue and they're skipped by VACUUM → partition by time so old partitions become read-only/freeze-once and cleanup stays local → observability: alert on oldest xact_start / backend_xmin and max xid age → trade-off: replica isolation (fresh-enough, isolates bloat) vs same-DB simplicity (one long scan can bloat the primary).You're choosing between a Postgres-style (heap history) and an InnoDB-style (undo history) engine for a write-heavy, high-contention OLTP service. Walk the decision.
★ Cheat sheet — internal → what it means
Mental models: Row = chain of versions · UPDATE = new version, never overwrite · Postgres = history in the heap · InnoDB = history in undo · Snapshot = xmin/xmax/xip[] judging each version · Visible = xmin past+committed AND xmax not-past · xmin horizon = oldest open txn = cleanup limit · VACUUM/purge = the janitor · Freezing = wraparound vaccine · SSI = safety layer over MVCC.
Translation table
| Term | What it actually is |
|---|---|
| Postgres tuple header: creator xid / deleter xid (0 = live) |
| InnoDB hidden fields: last writer / pointer into undo |
| Snapshot | xmin (oldest running), xmax (next unassigned), xip[] (in progress) |
| xmin horizon | xmin of the oldest live txn — VACUUM’s cleanup ceiling |
| Bloat | Dead tuples a pinned horizon won’t let VACUUM reclaim |
| Freezing | Flag “visible to all; ignore xmin” → immune to wraparound |
| History list length | InnoDB’s bloat signal: undo a long read view can’t release |
| Purge thread | InnoDB’s VACUUM: reclaims undo + delete-marked rows |
| SSI | Serializable on MVCC + read/write dependency tracking |
Three rules for the on-call chair
① Cleanup is only as current as your oldest open transaction — hunt it in
pg_stat_activity first. ② Plain VACUUM frees-for-reuse; only
VACUUM FULL shrinks the file (exclusive lock). ③ Never disable
autovacuum — “DB went read-only” is wraparound, and it’s always “autovacuum
couldn’t keep up.”
★ Review guide & what to read next
5-min (weekly) Don’t re-read — recall. Redraw the version chain with xmin/xmax; state the two visibility gates; say why one idle txn bloats the whole DB; name the wraparound read-only stop.
30-min (when stalled) Blank-page reconstruct all five levels; re-derive how heap-vs-undo forces VACUUM-vs-purge; then trace one real incident from symptom → xmin horizon → lever.
Read next, in order: ① PostgreSQL — Concurrency Control (xmin/xmax, snapshots) · ②
PostgreSQL — Routine Vacuuming & Preventing Wraparound
· ③
MySQL — InnoDB Multi-Versioning
· ④ SSI deep dive (the safety layer over MVCC).
These notes reflect my current understanding and are updated as I learn, build, and discover better explanations.