Caching and consistency system design questions

12 system design interview questions on consistency, contention and caching, each with the correct answer and a full explanation.

Consistency and contention

A Dynamo style store has N = 3 replicas with strict quorums. Which setting guarantees a read sees the latest acknowledged write?
Show answer

Answer

W = 2, R = 2
Reads overlap writes when R + W is greater than N. 2 + 2 = 4 is greater than 3. 1 + 1, 2 + 1 and 1 + 2 all equal at most 3, so a read can hit only replicas that missed the write. Sloppy quorums and hinted handoff weaken even R + W greater than N.

Source: Dynamo: Amazon's Highly Available Key-value Store

The link between two datacenters is cut. Both sides must keep accepting writes. What do you give up?
Show answer

Answer

Linearizability: both sides accept writes and reconcile later
During a partition you choose consistency or availability; staying writable on both sides means the sides diverge until reconciliation. CA is not a real choice when partitions happen. 2PC blocks during a partition, so it cannot keep both sides writable. Raft only lets the majority side write.

Source: Brewer's Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web Services, Jepsen: Consistency Models

An account balance used to approve withdrawals, seat inventory during checkout, the like count shown on a post, claims on unique usernames: name the one that can safely be eventually consistent.
Show answer

Answer

Like count shown on a post
A like count that lags by a few seconds harms nobody. A balance that lags allows overdrafts. Seat inventory that lags oversells. Usernames need a single authority or two people get the same name.

Source: The System Design Primer

Two users booked the same seat. What fixes this code?
seat = db.get(seat_id)
if seat.status == "free":
    db.update(seat_id, status="held", user=u)
Show answer

Answer

A conditional update that also requires status = 'free' and checks rows affected
Both requests read free before either writes. UPDATE seats SET status='held' WHERE id=? AND status='free' makes check and write one atomic step; zero rows affected means someone else won. READ COMMITTED does not stop two transactions reading the same free row. A cache adds another stale copy. Neither update errors, so retry never triggers.

Source: Hello Interview: Design Ticketmaster, PostgreSQL: Transaction Isolation

A wiki page editor: users spend minutes editing, conflicts are rare, and you must not lose an edit silently. What concurrency control fits?
Show answer

Answer

Optimistic concurrency: save with WHERE version = the version read, and reject on mismatch
With rare conflicts, a version check at save time costs nothing and turns a conflict into a visible error. Holding a row lock or a distributed lock across minutes of user think time blocks others and leaks when browsers close. Last write wins silently drops the earlier edit.

Source: Designing Data-Intensive Applications

A worker holds a lock with a 30 second TTL, pauses for 40 seconds in GC, wakes up and writes to storage, overwriting the new lock holder's work. What prevents this?
Show answer

Answer

Fencing tokens: the lock hands out an increasing number and storage rejects older ones
Any pause can outlast a TTL, so the storage must reject the stale writer; a monotonically increasing token checked on each write does that. A longer TTL only needs a longer pause. Redlock still has no fencing, which is Kleppmann's critique. No TTL means a crashed holder blocks forever.

Source: How to do distributed locking

Two requests run this at the same time on a balance of 100, withdrawing 30 and 50. The final balance is 70. What fixes it?
SELECT balance FROM acct WHERE id = 1;
-- app: b = balance
UPDATE acct SET balance = :b - :amt WHERE id = 1;
Show answer

Answer

Do the math in the database: SET balance = balance - :amt
Both read 100 and the second write overwrites the first, a lost update. An atomic UPDATE (or SELECT FOR UPDATE) serializes the change. READ COMMITTED, the PostgreSQL default, does not prevent this pattern. An index speeds lookups but does not lock. READ UNCOMMITTED is weaker, not stronger.

Source: PostgreSQL: Transaction Isolation

Caching

A read heavy profile service uses cache aside with Redis. On a cache miss, what does the service do?
Show answer

Answer

Read the database, store the value in Redis with a TTL, return it
In cache aside the application owns the cache: on miss it reads the source of truth, populates the cache with a TTL and returns. Redis does not load from your database; that would be read through, which needs a cache layer that knows the database. Returning a miss to the client pushes the problem to callers. Rewriting the row changes nothing in the cache.

Source: Cache-Aside pattern

Cache aside, many concurrent writers, staleness must be short. What does the write path do?
Show answer

Answer

Update the database, then delete the cache key
Deleting after the database commit means the next read reloads fresh data, and deletes are idempotent so racing writers cannot leave an older value behind (a slow reader refilling an old value can still race a delete, which memcache leases close). Writing the cache first leaves a value that was never committed if the database write fails. Setting the new value races: two writers can commit A then B but set the cache B then A, leaving A forever. Waiting for TTL means stale reads for the whole TTL. Facebook's memcache uses delete on write for this reason.

Source: Cache-Aside pattern, Scaling Memcache at Facebook

A key read 10,000 times per second expires, and all those requests miss at once and hit the database. What fixes it?
Show answer

Answer

Let one request rebuild the key while the others wait or get the stale value
This is a cache stampede. Request coalescing (a short lock or lease on refill, as in Facebook memcache leases) or probabilistic early refresh means only one rebuild hits the database. A lower TTL makes stampedes more frequent. Replicas only raise the ceiling and still take a burst of identical queries. Memory has nothing to do with expiry.

Source: Scaling Memcache at Facebook

One key gets 500,000 reads per second, more than a single Redis shard can serve. The rest of the cluster is idle. What helps?
Show answer

Answer

Replicate that key: a short TTL in process cache, or copies under suffixed keys on several shards
A single key always maps to one shard, so adding shards or changing the hashing scheme still routes all 500k reads to one node. Spreading copies (key#1 to key#N read randomly) or a local cache in the app tier splits the load. Write through changes write behaviour, not read load on the key.

Source: Scaling Memcache at Facebook, System Design Interview: An Insider's Guide, Volume 1

View counters receive 1 million increments per second. The database handles 20,000 writes per second, and losing a few seconds of counts on a crash is acceptable. Which cache write strategy fits?
Show answer

Answer

Write back: increment in the cache and flush aggregated counts periodically
Write back (write behind) absorbs the 1M/s in memory and turns them into a small number of batched writes, at the cost of losing unflushed data on a crash, which the constraints allow. Write through and cache aside both send every increment to the database, 50x its capacity. Read through only describes the read path.

Source: The System Design Primer

These questions come from the System Design Interview Practice pack in Bertopia.

More system design topics

Practise these on your commute