30 Core System Design Concepts & Mathematical Primer

“"For external public mobile clients, we expose standard REST/JSON over HTTP/3. Internally between Kubernetes microservices, we transcode to binary gRPC over HTTP/2 multiplexed streams to save 70% CPU serialization overhead and 40% network transit bandwidth."”

“"We deploy Envoy Gateway across 3 Availability Zones behind an L4 Network Load Balancer. It terminates TLS 1.3, verifies JWT cryptographic signatures in <0.5ms, and enforces Redis sliding-window token bucket rate-limiting before forwarding to internal pods."”

“"To avoid database session lookups on every API call, we use stateless JWT access tokens signed via asymmetric RS256/Ed25519. The API Gateway validates the signature once at the perimeter and injects sanitized user headers into internal mTLS calls."”

“"We decouple webhook delivery using an asynchronous Kafka worker pool. When an order completes, we publish an event to Kafka. Dedicated worker pods sign the payload with HMAC-SHA256 and POST to client URLs with exponential jittered retries to protect against slow endpoints."”

“"We utilize GraphQL for dynamic frontend mobile screens to eliminate over-fetching and combine 6 REST endpoints into 1 round-trip. On the backend, we enforce strict AST complexity limits and use Dataloader batching to prevent N+1 database query degradation."”

“"We use an L4 AWS Network Load Balancer for ultra-high throughput Anycast TCP ingress, forwarding to Envoy Gateway pods that perform Layer 7 routing using the Least Outstanding Requests algorithm to balance traffic across Kubernetes services."”

“"We deploy Nginx/Envoy as a reverse proxy in front of our application tier. It handles SSL/TLS termination, Brotli compression, request buffering, and prevents direct public internet exposure of our internal Kubernetes service IPs."”

“"We design our application microservices to be 100% stateless, externalizing all state to Redis and PostgreSQL. This enables our Kubernetes Horizontal Pod Autoscaler to dynamically scale from 15 to 100+ pods in seconds based on CPU utilization and request throughput."”

“"Our SLA commitment is 99.99% availability (Four Nines), allowing a maximum of 52.6 minutes of downtime per year. We achieve this through Multi-AZ active-active deployments, N+1 redundancy, automated circuit breakers, and zero-downtime rolling canary deployments."”

“"We audit every hop in our architecture to eliminate SPOFs: Anycast DNS has 300+ PoPs, API Gateways run across 3 AZs, PostgreSQL uses synchronous Multi-AZ standby with automated failover, and Kafka uses 3 KRaft controllers with replication factor 3."”

“"Under the CAP/PACELC theorem, our checkout system is CP (Consistency over Availability) for payment and inventory transactions using PostgreSQL ACID. Our product review feed is AP (Availability over Consistency) using DynamoDB with eventual consistency to maintain ultra-fast sub-10ms response times."”

“"We use PostgreSQL for user accounts, checkout orders, and financial ledger data where relational ACID foreign keys are mandatory. For clickstream analytics, user cart sessions, and product reviews, we use DynamoDB/Redis to achieve horizontal scaling and single-digit millisecond latency."”

“"To guarantee data integrity during order creation, we execute within a Read Committed ACID transaction. We use the Transactional Outbox Pattern to insert both the `orders` record and the `outbox_events` record in the same local transaction, completely eliminating dual-write inconsistencies."”

“"We add composite B+ Tree indexes on `(user_id, created_at DESC)` to support pagination queries in $O(\log N)$ time without disk sorting. The top 3 levels of the B+ Tree reside entirely in PostgreSQL `shared_buffers` RAM, ensuring index lookups execute in <1ms."”

“"When write throughput exceeds 15k QPS or database disk exceeds 10 TB, we shard horizontally by `user_id` using consistent hashing. All orders, payments, and profiles for a user reside on the same physical shard, avoiding costly cross-shard joins and distributed transactions."”

“"We use Consistent Hashing with 200 virtual nodes per physical cache server on a $2^{32}-1$ hash ring. When a Redis node crashes or scales up, only $1/N$ of keys are invalidated, completely preventing cache thundering herd storms on our primary PostgreSQL database."”

“"To maintain search indexes in Elasticsearch and invalidate Redis caches without dangerous dual-writes, we use Debezium CDC to tail the PostgreSQL Write-Ahead Log (WAL). Changes are published to Kafka in <50ms with guaranteed ordering by primary key."”

“"We size our Redis Cluster using the 80/20 rule: 20% of our 500k hot SKUs generate 80% of views. Storing those 100k items in Redis with 1.3x memory overhead requires only 13 GB RAM, while absorbing 95% of database read traffic and dropping P50 latency to 1.2ms."”

“"We use the Cache-Aside pattern for our product catalog: application queries Redis first; on a miss, it fetches from PostgreSQL and populates Redis with a 24-hour TTL. For critical user permissions, we use Write-Through to ensure immediate consistency upon revocation."”

“"We configure Redis with `maxmemory-policy allkeys-lru` capped at 75% of container RAM. When flash sales introduce temporary burst SKUs, Redis automatically evicts cold catalog items based on recency, guaranteeing memory stability without Out-Of-Memory container terminations."”

“"We position Cloudflare/CloudFront Edge PoPs across 300+ global locations. The CDN terminates TLS 1.3 close to the user and serves static assets and cached catalog JSON with an 85% hit rate, reducing 6.6k client RPS down to under 1k RPS reaching our origin API Gateway."”

“"We enforce rate limiting at the API Gateway using a Redis-backed Sliding Window Counter algorithm. Unauthenticated requests are limited to 60 RPM per IP, while authenticated users receive 1,000 RPM. When exceeded, the gateway returns HTTP 429 with a `Retry-After` header."”

“"We use Apache Kafka for event-driven pub/sub. When an order is created, we publish to `order.events` with `acks=all` and replication factor 3. Multiple downstream consumers (Inventory, Notifications, Analytics) read independently via their own consumer group offsets at their own pace."”

“"To prevent costly database queries for non-existent usernames, we maintain a Bloom Filter in Redis. If the Bloom filter returns false, we instantly return 404 in <1ms without touching PostgreSQL. Only if it returns true do we query the database to verify."”

“"To prevent double-charging during mobile network timeouts, the client generates a unique UUIDv4 idempotency key. The payment service acquires a 5-second Redis lock on `idempotency:key`. If already processed, it returns the cached receipt immediately without charging the payment gateway again."”

“"Using Little’s Law ($L = \lambda \times W$), at 6,600 Peak RPS with 50ms average backend latency, our application holds 330 concurrent requests inflight at any millisecond. We use non-blocking Go goroutines / Envoy event loops so this consumes under 2 MB of memory across our cluster."”

“"For real-time driver GPS tracking in our delivery app, we use persistent WebSockets over TLS with a 60-second ping/pong heartbeat. For order status updates, we use Server-Sent Events (SSE) since updates are strictly uni-directional from server to client."”

“"We enforce strict statelessness across all backend microservices. User sessions, shopping carts, and rate limit counters are stored in an external Redis Cluster. This allows any Kubernetes pod to serve any request and enables instantaneous horizontal auto-scaling without sticky session routing."”

“"We implement the Kappa Architecture using Apache Flink consuming from Kafka event streams. Flink calculates real-time 5-minute sliding window fraud detection in <100ms. For cold historical reporting, we sink raw Kafka events into S3/Parquet for querying with Athena/Snowflake."”

“"To locate nearby drivers within 2km, we convert driver GPS coordinates into a 6-character Geohash string (e.g. `dr5ru7`). We query our database for the driver’s current geohash plus the 8 adjacent bounding box neighbors using a fast B+ Tree prefix range query."”