Design a job scheduler, file storage or video service
15 system design interview questions on job schedulers, file storage, video, collaborative docs and LLM serving, each with the correct answer and a full explanation.
A job scheduler: a job must not run twice concurrently, even if a worker dies mid run. What do you use?
Show answer
Answer
A lease with heartbeat renewal and a fencing token, plus idempotent job steps
A lease expires if the worker dies so another can take over, the fencing token stops a paused old worker from writing, and idempotent steps make the rare retry safe. A lock with no expiry is stuck forever after a crash. TTL without fencing lets a zombie worker write after its lease ended. Deleting first loses the job if the worker crashes.
Twenty worker processes poll a PostgreSQL jobs table for rows with run_at at or before now. They keep blocking on and double claiming the same rows. What fixes it?
Show answer
Answer
Claim rows with SELECT ... FOR UPDATE SKIP LOCKED inside a transaction that marks them running
SKIP LOCKED lets each worker lock a different set of due rows without waiting, which is the standard queue on Postgres pattern. Everyone running all due jobs is the double run bug. A single process is a single point of failure. Random sleeps reduce collisions but do not prevent them.
A sync client has a 1 GB file synced. The user changes a few bytes. How do you avoid uploading the whole file again?
Show answer
Answer
Split files into hashed chunks and upload only chunks whose hash is new
Content addressed chunks mean only changed chunks travel and identical chunks across users dedupe for free, as Dropbox does. Uploading then diffing still sends 1 GB. Compression shrinks but still resends everything. A byte diff against one previous version gives no dedupe and breaks if versions diverge.
Where do you keep file bytes and file metadata (names, folders, versions, sharing) in a Drive like service?
Show answer
Answer
Bytes in an object store, metadata in a database
Object stores are cheap and durable for large blobs; metadata needs transactions and queries (move folder, list, share). BLOBs in the database bloat it and slow backups. JSON sidecars cannot do atomic renames or queries. Local disks are lost with the server and not shared.
Mobile users upload 2 GB videos over flaky connections. Uploads keep failing at 80%. What fixes it?
Show answer
Answer
Multipart upload with presigned URLs straight to object storage, retrying only failed parts
Multipart splits the file into parts that upload and retry independently, and presigned URLs keep the bytes off your servers. A single PUT restarts from zero on any drop. Asking for Wi-Fi is not a design. Base64 adds a third to the size and is still one request.
Viewers watch on mobile networks whose bandwidth swings every few seconds. How do you deliver smooth playback?
Show answer
Answer
Transcode several bitrates, cut into short segments, and let the player switch per segment (HLS or DASH)
Adaptive bitrate streaming lets the player pick the next segment's quality as bandwidth changes. One high bitrate file stalls on slow networks. A single measurement at start goes stale. Raw uploads are huge and phones cannot transcode them live.
A small share of titles produces most video traffic, and demand is predictable each evening. How do you serve it cheaply?
Show answer
Answer
Pre-position popular titles on CDN or ISP edge caches during off peak hours
Netflix Open Connect fills edge appliances off peak so evening traffic is served near viewers. Origin serving pays full backbone cost and latency. More origin servers in one region do not fix distance. API servers are not built to serve video bytes.
A collaborative editor must let users edit offline for days and merge peer to peer with no central server. Which approach fits?
Show answer
Answer
CRDTs
CRDTs converge without a central ordering, which suits offline and peer to peer. OT (as in Google Docs) relies on a server to order operations, which the constraints exclude. Last write wins discards one user's work. Locks do not work offline.
An inference service wraps one GPU that processes batches of up to 100 prompts. Callers are synchronous. How do you batch?
Show answer
Answer
Queue requests and dispatch when the batch hits 100 or a max wait such as 20 ms passes
Max size plus max wait trades a small bounded delay for throughput, and each response is routed back by request id. Waiting for 100 makes latency unbounded at low traffic. One request per call wastes most of the GPU. Waiting on a caller's next request does not apply to synchronous callers.
LLM requests produce between 10 and 4,000 output tokens. With static batches, short requests wait for the longest one in their batch. What fixes it?
Show answer
Answer
Continuous (iteration level) batching: add and remove requests from the batch at each decoding step
Scheduling per decoding step lets finished sequences leave and new ones join, as in Orca and later vLLM style servers, so short requests are not held hostage. Larger static batches make the waiting worse. Arrival order does not change who waits. A GPU per request wastes most of the hardware.
An LLM API serves many customers whose requests vary 100x in token count. How do you rate limit fairly by cost?
Show answer
Answer
Per tenant limits on tokens per minute, estimated at admission and corrected after completion
Tokens track GPU cost, so a token budget per tenant keeps a heavy tenant from crowding others. Request counts let one 4,000 token prompt cost the same as a 40 token one. A global queue lets one tenant starve others. Hardware does not remove contention.
