Failure modes and multi tenancy system design questions
11 system design interview questions on failure modes and multi tenancy, each with the correct answer and a full explanation.
A dependency recovers after an outage, and thousands of clients retry and knock it over again. Which retry policy avoids this?
Show answer
Answer
Exponential backoff with random jitter
Jitter spreads retries in time so clients do not arrive in synchronized waves. Immediate retries hit at full force. A fixed interval keeps every client in lockstep. Backoff without jitter still keeps clients that failed together retrying together.
A request passes through 3 service layers, and each layer makes up to 3 attempts on failure. The bottom database is down. How many calls can one user request make to it?
Show answer
Answer
27
Attempts multiply per layer: 3 x 3 x 3 = 27. Retries at every layer turn an outage into a load storm, so retry at one layer and use retry budgets. 9 counts two layers. 3 counts one. 81 adds a fourth layer that does not exist.
A downstream service starts timing out at 10 seconds. Your request threads pile up waiting and your service stops serving even healthy endpoints. What fixes it?
Show answer
Answer
A circuit breaker that fails fast after an error threshold, plus a shorter timeout
A breaker stops calling a failing dependency so threads return immediately and capacity stays available for other work, probing later to close. A longer timeout holds threads longer. Retries add more waiting calls. A bigger pool fills up just the same, only later.
Your service calls a partner API that allows 1,000 requests per second. Your own traffic can spike 100x. What protects the partner and your users?
Show answer
Answer
Client side rate limit to 1,000 per second with a bounded queue, backoff with jitter and a circuit breaker
You must shape your own outbound traffic to the partner's limit and push back when the queue is full. Plain retries multiply load on an already throttled partner. Autoscaling your side sends even more requests. Caching for a day is only valid if responses are cacheable, which the question does not say.
A server is at 150% of capacity. Latency for every request is climbing past client timeouts. What should it do?
Show answer
Answer
Reject excess requests early, before doing work, and keep serving the rest
Shedding early (429 or 503, ideally by priority) keeps the accepted requests fast. Queueing everything makes every request late, so clients time out and the work is wasted. Accepting everything is the same failure. Restarting drops in flight work and comes back cold to the same load.
Four instances each run at 80% at peak. One dies, the other three overload and fail one by one. What would have prevented the cascade?
Show answer
Answer
Headroom so N minus 1 instances carry peak, plus load shedding above capacity
Losing one of four at 80% puts 107% on each survivor. N minus 1 capacity and shedding keep survivors up. Autoscaling takes minutes to add capacity while the cascade takes seconds. Ejecting slow instances concentrates load on fewer nodes, speeding the cascade. Retries add load.
A multi tenant job service has one shared FIFO queue. One customer submits 100,000 jobs and everyone else waits hours. What fixes it?
Show answer
Answer
Per tenant queues or quotas with fair scheduling across tenants
Fair share across tenants bounds one tenant's impact regardless of their volume. More workers help until the next bigger burst, which still lands at the head. Shortest first starves large jobs and ignores tenants. Rejecting everyone punishes the tenants who did nothing.
A client gives up after 1 second. Service A calls B, which calls C. B and C keep working for 5 more seconds after the client is gone. What fixes this?
Show answer
Answer
Propagate the remaining deadline so each hop stops when the budget runs out
Passing the remaining deadline (as gRPC does) lets each service cancel work nobody will read. Separate 1 second timeouts per hop can add up to 3 seconds and do not shrink. TCP has no request deadline. Longer downstream timeouts guarantee wasted work.
A dependency's p99 latency is 200 ms, its median 40 ms, and it occasionally hangs forever. You call it with no timeout. What timeout do you set?
Show answer
Answer
A little above p99, such as 300 to 500 ms
A timeout just above p99 fails only the truly stuck calls while freeing threads quickly. No timeout lets hung calls hold threads forever. 30 seconds lets a hang tie up threads for half a minute. 30 ms fails most healthy calls.
