Timeouts, retries and connection pool interview questions
10 networking interview questions on timeouts, retries, status codes, connection pools and port exhaustion, each with the correct answer and a full explanation.
Timeouts, retries and status codes
A client retries every 100 ms on 503 from a struggling service. Thousands of clients do the same. What is the correct client behavior?
Show answer
Answer
Exponential backoff with jitter, honor Retry-After, and cap retries with a retry budget
503 means the server is overloaded or down; synchronized fast retries multiply load and keep it down. Backoff spreads retries over time, jitter breaks synchronization, a budget caps the total extra load. Idempotency makes a retry safe, not cheap. A fixed interval still synchronizes clients and never stops. Never retrying gives up on transient failures that a retry would fix.
Your API rejects a client because it exceeded its rate limit. Which response is correct?
Show answer
Answer
429 Too Many Requests with a Retry-After header
429 says the client sent too many requests and Retry-After tells it when to come back. 500 claims a server bug, which pages your on call and makes clients retry blindly. 503 signals the whole service is unavailable, not this client's quota. 400 says the request itself is malformed, which it is not.
A payment POST times out on the client. The client does not know if the charge happened. How do you make the retry safe?
Show answer
Answer
Send a client generated idempotency key; the server stores the result per key and returns the stored result on a repeat
A timeout is ambiguous: the request may have succeeded. An idempotency key lets the server detect the duplicate and replay the original outcome instead of charging twice. Switching verbs does not make the server logic idempotent. Waiting does not resolve the ambiguity. The client logs only show that the response never arrived.
A user request goes edge, A, B, C, then the bottom service D. Edge, A, B and C each retry a failing call to the next hop up to 3 times (4 attempts total). D is down. How many attempts can hit D for one user request?
Show answer
Answer
256
Retries multiply across layers: 4 calling layers with 4 attempts each gives 44 = 256 attempts on D. This is how retry storms keep an outage going. The fix is retrying at one layer only (usually the caller nearest the failure) plus retry budgets. 4 counts a single layer. 12 adds instead of multiplying. 16 covers only two layers.
The edge has a 2 s timeout for a user request. It calls service A, which calls B with its own default 30 s timeout. B hangs. What goes wrong and what is the fix?
Show answer
Answer
A keeps waiting on B for 30 s after the user is gone, tying up threads and connections; propagate a deadline so each hop uses the remaining budget
Timeouts must shrink as you go deeper. Without deadline propagation, work continues long after the caller gave up, and a hanging dependency exhausts A's resources, cascading upward. The edge timeout only frees the edge. Longer timeouts make the pile up worse. Uniform 30 s timeouts ignore that the user budget is 2 s.
Your load balancer returns errors using standard HTTP semantics. In case one, no healthy backend exists. In case two, a backend accepted the request but did not respond in time. Which pair of codes is correct?
Show answer
Answer
503 then 504
503 Service Unavailable means the server or gateway cannot handle it now, which is what an LB returns with no healthy targets. 504 Gateway Timeout means the upstream did not answer in time. 502 means the upstream sent an invalid response or reset the connection. 500 is a generic error in the server itself. Swapping 503 and 504 is the common slip.
A service creates a new HTTP client per outbound request to one downstream host. Under load, calls fail with 'cannot assign requested address' while CPU sits at 20 percent. What is the fix?
Show answer
Answer
Reuse a shared client with a pooled keep-alive connection set, so you stop burning ephemeral ports that sit in TIME_WAIT
Each short lived connection to the same destination uses an ephemeral port that stays in TIME_WAIT on the side that closed first (typically around 60 s on Linux), so a high request rate exhausts the port range. Reusing connections removes the churn. More threads create even more connections. Timeouts do not free ports in TIME_WAIT. Memory is not the bottleneck.
A service's p50 to the database is 3 ms but p99 jumps to 400 ms during peaks. Database query time p99 stays at 5 ms. The connection pool max is 10 and 200 request threads share it. What is the most likely cause?
Show answer
Answer
Requests wait in the pool queue for a free connection; the tail is pool wait time, not query time
When concurrency exceeds pool size, requests queue for a connection, and that wait shows up only in the tail. Measure pool acquire time; size the pool from concurrency and query time (Little's law), within what the database can handle. The database is ruled out by its own 5 ms p99. Pooled connections do not do DNS per query. Slow start affects new connections, and a pool reuses them.
Your HTTP server closes idle keep-alive connections itself after 5 s. Under heavy traffic, ss shows tens of thousands of server side sockets in TIME_WAIT. Why are they on the server?
Show answer
Answer
The server sends the first FIN, and the active closer is the side that holds TIME_WAIT
TIME_WAIT belongs to the active closer and lasts 2 x MSL to absorb delayed segments and make sure the final ACK arrives. A server that closes connections first collects them; letting clients close first or reusing connections longer moves or removes them. It is not fixed to server or client, so both 'always' options are wrong. UDP has no connection state and no TIME_WAIT.
A service handles 1,000 requests per second; each request holds a DB connection for 20 ms. Roughly how many connections are busy on average?
Show answer
Answer
20
Little's law: concurrency = arrival rate x time in system = 1,000 x 0.02 s = 20. Size the pool above that for bursts, not to the request rate. 2 slips a decimal. 200 is 10x too high and can overload the database. 1,000 confuses throughput with concurrency.
