System Building Blocks

Priority: P1 (HIGH)

Add a component only to remove a named constraint. Every component adds failure modes and cost.

Selection Table

Constraint observed	Component	Cost accepted
One instance saturates CPU or connections	Horizontal scaling behind a load balancer	Statelessness required; session moves to a store
Read QPS exceeds store ceiling; data tolerates staleness	Read-through cache	Stale reads up to TTL; invalidation complexity
Write spikes exceed sustainable store throughput	Queue plus async worker	Eventual completion; needs idempotent consumers
Slow non-critical work blocks the request path	Background job or event	Extra delivery guarantees to reason about
Static or media bytes dominate egress	CDN	Cache purge lag; origin still needed for misses
Cross-cutting authn, quota, routing repeated per service	API gateway	Central chokepoint; fails closed and needs its own failure capacity
One tenant or key can exhaust capacity	Rate limiter or admission control	Rejections need clear client contracts
Cache or shard nodes change count during scaling	Consistent hashing	Rebalancing logic; hot-key skew persists

Caching Rules

• Default to cache-aside for read-heavy paths; use write-through only when read-after-write must be exact.
• Set TTL from tolerable staleness, not from convenience. Every cached key needs a stated TTL and an invalidation trigger.
• Guard the miss path: single-flight or request coalescing prevents a stampede when a hot key expires.
• Never cache what you cannot invalidate. Never cache per-user data under a shared key.

Queue Rules