System Design · Intermediate interview

Replication keeps copies of the data on multiple nodes. The common pattern is leader-follower: writes go to the leader and are streamed to read-only followers, which scales reads and provides redundancy for failover. The trade-off is replication lag — followers can be slightly behind — so reads from a follower may be stale, which you handle with read-your-writes routing or by reading critical data from the leader.

Sharding (horizontal partitioning) splits a dataset across multiple databases by a shard key, so each holds a subset — necessary when data or write volume exceeds one node. The challenges are choosing a key that spreads load evenly (avoiding hotspots), cross-shard queries and joins becoming hard, transactions across shards, and resharding as you grow. A good shard key and avoiding cross-shard operations are the crux.

Read-heavy systems lean on caching, read replicas, denormalization, and CDNs to serve reads cheaply, accepting some staleness. Write-heavy systems focus on ingest throughput — sharding/partitioning writes, using write-optimized stores (LSM-tree databases), batching, and async processing via queues, often denormalizing later. You identify the dominant pattern early because it drives the whole architecture.

CAP says that when a network partition occurs, a distributed system can guarantee either Consistency (every read sees the latest write) or Availability (every request gets a response), but not both — you must choose. Without partitions you can have both, so it's really a partition-time trade-off. In practice you pick CP (reject some requests to stay consistent, e.g. a bank ledger) or AP (stay available and reconcile later, e.g. a shopping cart) per use case.

Strong consistency means every read reflects the most recent write, which is simpler to reason about but costs latency and availability because of coordination. Eventual consistency means replicas converge over time, so reads may briefly be stale, in exchange for higher availability and performance. You choose strong for things like balances and inventory, and eventual for things like view counts, feeds, or caches where brief staleness is fine.

Cache-aside (lazy): the app checks the cache, and on a miss loads from the DB and populates it — simple and common. Write-through: writes go to cache and DB together, keeping them in sync at write cost. Write-back: write to cache first, flush to DB later, fast but riskier. Invalidation is the hard part — use TTLs, delete/update keys on write, or version keys — because stale cache and the dual-write consistency problem cause real bugs.

Use a queue to decouple producers from consumers and process work asynchronously — for slow or spiky tasks (emails, image processing, notifications), to smooth load spikes by buffering, to retry failed work, and to fan work out to multiple consumers. It improves resilience and responsiveness, at the cost of eventual consistency and operational complexity. The signal is 'this doesn't need to finish before responding to the user'.

A queue delivers each message to exactly one consumer from a group — good for distributing work (a task is done once). Pub/sub broadcasts each message to all subscribers — good for events many parts of the system react to independently. Queues are for work distribution; pub/sub is for event fan-out, and many brokers support both models.

Common algorithms are token bucket (refill tokens at a steady rate, allow bursts up to the bucket size), leaky bucket (smooth constant outflow), and fixed/sliding window counters. You usually keep the counter in a shared store like Redis so it works across instances, key it by user/IP/API key, and return 429 with a Retry-After when exceeded. Token bucket is the popular default because it permits bursts while bounding the average rate.

Eliminate single points of failure with redundancy: multiple stateless instances behind a load balancer, replicated databases with automatic failover, and deployment across multiple availability zones. Add health checks so traffic routes away from unhealthy nodes, graceful degradation so a failed non-critical dependency doesn't take everything down, and tested failover. Availability is about no single thing being able to take the system down.

An idempotent operation produces the same result whether it's applied once or many times. It matters because networks retry — a client may resend a request it isn't sure completed — and without idempotency you get duplicate charges or records. You implement it with idempotency keys the server records, or by designing operations (like 'set status = paid') to be naturally repeatable.

Trace each component on the critical path and ask 'what happens if this one instance dies?' — anything whose failure stops the system is a SPOF: a lone database, a single load balancer, a shared cache, a master node. You remove it by adding redundancy and failover (replicas, multiple LBs, clustering) or by making the dependency non-critical through graceful degradation. The goal is no single box being irreplaceable.