System Design Interview 2025: Top 30 FAANG Questions and the New AI/ML Design Round

Example questions:
- "What are the 3 core actions a user can perform?"
- "What is the read-to-write ratio?"
- "How long must we retain data?"

Things to confirm:
- Availability: 99.9% vs 99.99% (annual downtime 8.7 hours vs 52 minutes)
- Latency: specific targets like p99 < 200ms
- Consistency: strong consistency vs eventual consistency
- Scale: DAU, concurrent users, data volume

Example: Estimating scale for a chat system

- DAU: 50 million
- Average messages per day: 40 per user
- Total messages: 50M x 40 = 2 billion/day
- QPS: 2 billion / 86,400 = ~23,000 QPS
- Peak QPS: 3x average = ~70,000 QPS
- Message size: 100 bytes average
- Daily storage: 2B x 100B = 200GB/day
- Annual storage: 200GB x 365 = 73TB/year

Typical high-level architecture:

Client --> Load Balancer --> API Gateway
                               |
                    +----------+----------+
                    |          |          |
                Service A  Service B  Service C
                    |          |          |
                 DB (SQL)  Cache     Message Queue
                              |          |
                           DB (NoSQL)  Worker

Deep dive areas:

1. Database Schema Design
   - Table structure, indexes, partitioning strategy

2. API Design
   - Endpoints, request/response format, error handling

3. Core Algorithms
   - News Feed: Fan-out on write vs Fan-out on read
   - Search: Inverted index structure
   - Recommendations: Collaborative filtering vs content-based filtering

Checklist:
- Identify and eliminate Single Points of Failure (SPOF)
- Data replication strategy (Leader-Follower, Multi-Leader)
- Failure detection and automatic recovery (Health Checks, Circuit Breaker)
- Monitoring metrics (Latency, Error Rate, Throughput)
- Alerting system (PagerDuty, Slack integration)

Vertical scaling:
  1 server (4 CPU, 16GB RAM) --> 1 server (32 CPU, 128GB RAM)

Horizontal scaling:
  1 server --> 10 servers (each 4 CPU, 16GB RAM) + Load Balancer

L7 load balancer routing example:

/api/users/*  --> User Service Cluster
/api/orders/* --> Order Service Cluster
/api/search/* --> Search Service Cluster
/static/*     --> CDN Origin

Read request flow:
1. Look up data in cache
2. Cache hit --> return data
3. Cache miss --> query DB --> store in cache --> return

Pros: Only caches needed data, DB fallback on cache failure
Cons: First request always slow, cache-DB inconsistency possible

Write request flow:
1. Write data to cache
2. Cache synchronously writes to DB
3. Return completion response

Pros: Cache and DB always consistent
Cons: Increased write latency, caches unused data

Write request flow:
1. Write data to cache
2. Return completion immediately
3. Asynchronously batch-write to DB

Pros: Maximum write performance
Cons: Risk of data loss on cache failure

1. Range-based Sharding
   - user_id 1~1M --> Shard 1
   - user_id 1M~2M --> Shard 2
   - Pros: Efficient range queries
   - Cons: Hotspot risk

2. Hash-based Sharding
   - hash(user_id) % N --> Shard number
   - Pros: Even distribution
   - Cons: Inefficient range queries, complex rebalancing

3. Directory-based Sharding
   - Managed via lookup table
   - Pros: Flexible mapping
   - Cons: Lookup table becomes SPOF

Kafka architecture:

Producer --> Topic (Partition 0) --> Consumer Group A
                                 --> Consumer Group B
         --> Topic (Partition 1) --> Consumer Group A
                                 --> Consumer Group B
         --> Topic (Partition 2) --> Consumer Group A
                                 --> Consumer Group B

- Partition: unit of parallelism
- Consumer Group: independent consumption
- Offset: tracks consumption position

CDN operation:

User (Korea) --> Korea edge server (cache hit) --> immediate response
                                    (cache miss) --> origin (US) --> cache --> response

Pull CDN: Fetches from origin on first request (CloudFront, Cloudflare)
Push CDN: Pre-distributes content (suited for large static files)

REST example:
GET /api/v1/users/123
GET /api/v1/users/123/orders?page=1&limit=10

gRPC example:
service UserService {
  rpc GetUser(GetUserRequest) returns (User);
  rpc ListOrders(ListOrdersRequest) returns (stream Order);
}

GraphQL example:
query {
  user(id: "123") {
    name
    orders(first: 10) {
      id
      total
    }
  }
}

The problem with basic hashing:
  hash(key) % N = server number
  When N changes, almost all keys get remapped

Consistent Hashing:
  - Hash space arranged in a ring (0 to 2^32-1)
  - Both servers and keys are placed on the ring
  - Keys are assigned to the nearest server clockwise
  - Adding/removing a server only affects adjacent segments

Virtual Nodes:
  - Each physical server maps to multiple virtual nodes
  - Distributes load more evenly
  - Number of virtual nodes can be adjusted by server capacity

How it works:
1. Tokens are added to a fixed-size bucket at a constant rate
2. Each request consumes 1 token
3. If no tokens remain, request is rejected (HTTP 429)

Parameters:
- Bucket size: 100 (maximum burst)
- Refill rate: 10/second (average throughput)

How it works:
1. Record each request timestamp in a sorted log
2. Remove entries outside the current window (e.g., 1 minute)
3. Reject if request count within window exceeds the limit

Pros: Precise window boundary handling
Cons: High memory usage (stores all timestamps)

Approach 1: Centralized (Redis-based)
  - All servers share a Redis counter
  - Accurate but Redis becomes bottleneck/SPOF

Approach 2: Local + Synchronization
  - Each server counts locally
  - Periodically syncs with central store
  - Fast but allows slight over-limit

Key design points:
1. Short URL generation: Base62 encoding vs hash (MD5/SHA256 truncation)
2. Collision handling: DB unique constraint + retry vs counter-based
3. Redirection: 301 (permanent) vs 302 (temporary) with rationale
4. Caching: Frequently accessed URLs in Redis (80/20 rule)
5. Analytics: Click count, region, device tracking

Fan-out on Write (Push model):
- When a post is created, immediately write to all followers' feeds
- Pros: Fast reads
- Cons: High write cost for users with many followers (celebrities)

Fan-out on Read (Pull model):
- When a feed is viewed, aggregate in real-time from followed users
- Pros: Low write cost
- Cons: Slow reads

Hybrid (Production answer):
- Regular users: Push model
- Celebrities (1M+ followers): Pull model
- Merge both for the final feed

Architecture:
1. Connection management: WebSocket servers + connection state store
2. Message delivery: Direct for 1:1, fan-out for groups
3. Offline handling: Store in message queue, deliver on reconnection
4. Read receipts: Separate service
5. Message storage: DB optimized for chronological ordering
   - HBase/Cassandra (wide-column stores)

Full recommendation pipeline:

Data Collection --> Feature Store --> Model Training --> Model Registry
     |                                    |                    |
User behavior logs               Offline batch            A/B Testing
Clicks, purchases, views        Daily/weekly update    Champion/Challenger
     |                                    |                    |
     v                                    v                    v
Real-time feature   -->  Candidate    -->  Ranking  -->  Re-ranking  -->  Display
computation              Generation       (Scoring)     (Re-ranking)
                        (Retrieval)

Key components:
1. Feature Store: Offline (batch) + Online (real-time) feature management
2. Candidate Generation: ANN (Approximate Nearest Neighbors) to select ~1000 candidates
3. Ranking Model: Score candidates with deep learning model
4. Re-ranking: Apply business rules, diversity, freshness

LLM Serving Architecture:

Request --> API Gateway --> Request Router --> GPU Cluster
                               |                   |
                          Token Limiter     Model Instances (vLLM)
                          Queue Manager           |
                               |            KV Cache Management
                          Response Streaming <--- |
                               |
                          Cost Tracker

Key optimization techniques:
1. KV Cache: Cache previous tokens' Key/Value to avoid redundant computation
2. Continuous Batching: Dynamically batch requests
3. PagedAttention: Manage memory in page units (vLLM)
4. Quantization: FP16 --> INT8/INT4 to reduce model size
5. Speculative Decoding: Draft with small model, verify with large model

GPU auto-scaling considerations:
- GPU instance startup time: 3-10 minutes (much slower than CPU)
- Predictive scaling: Pre-learn hourly traffic patterns
- Queue-depth based: Trigger scaling on pending request count
- Cost optimization: Spot instances + on-demand fallback

RAG Pipeline Architecture:

Document ingestion --> Chunking --> Embedding generation --> Vector DB storage
                                                                |
User query --> Query embedding --> Similarity search --> Top-K document retrieval
                                                              |
                                  Prompt construction <-------+
                                        |
                                  LLM generation --> Response + source citations

Key design decisions:
1. Chunking strategy: Fixed-size vs semantic (paragraph/section)
2. Embedding model: OpenAI Ada-002, Cohere Embed, open-source models
3. Vector DB: Pinecone (managed), Milvus (self-hosted), pgvector
4. Search approach: Pure vector search vs hybrid (vector + keyword)
5. Re-ranking: Cross-encoder for precise reordering of search results

Real-time ML Pipeline:

Event source --> Kafka --> Stream Processor (Flink) --> Feature computation
                                                            |
                                                       Feature Store
                                                            |
API request --> Model Server --> Inference result --> Business logic
                   |
              Model Registry (MLflow)

Use cases:
- Fraud detection: Payment event --> Real-time features --> Fraud score
- Real-time recommendations: User behavior --> Real-time embedding update --> Refreshed recs
- Dynamic pricing: Supply/demand signals --> Pricing model --> Price adjustment

AI Safety Architecture:

User input --> Input filter --> LLM --> Output filter --> User response
                 |                         |
            Content classifier       Hallucination detector
            Prompt injection detection   Fact checker
            PII detection/masking        Toxicity filter
                 |                         |
            Block or warn            Modify or block

Design points:
1. Multi-layer defense: Filters at input/processing/output stages
2. Async audit: Analyze full conversation logs asynchronously
3. Adaptive rules: Fast rule updates for new attack patterns
4. Human-in-the-loop: Human review for uncertain automated cases
5. Latency budget: Minimize added latency from safety layers (target: under 50ms)

Operation                      Time
---------------------------   ----------
L1 cache reference             1 ns
L2 cache reference             4 ns
Main memory (RAM) reference    100 ns
SSD random read                100 us (microseconds)
HDD seek                       10 ms
Same-datacenter network RTT    0.5 ms
Cross-continent network RTT    150 ms

Memory tips:
- L1 is 100x faster than RAM
- SSD is 1000x slower than RAM
- HDD is 100x slower than SSD
- Network is always slower than local I/O

QPS (Queries Per Second):
  QPS = DAU * average requests per user / 86,400
  Peak QPS = average QPS * 2~5 (depends on service)

Storage:
  Daily data = daily new records * average record size
  Annual data = daily data * 365
  Total storage = annual data * retention period (years) * replication factor (usually 3)

Bandwidth:
  Inbound = write QPS * average request size
  Outbound = read QPS * average response size

Approximate scale of major services:

Service       DAU          Avg QPS     Storage
------       --------     ---------   --------
Twitter      500M         300K        Several TB/day
Instagram    2B           1M          Tens of TB/day
YouTube      2B+          500K+       500 hours uploaded/min
WhatsApp     2B+          Millions    10B messages/day
Google Search 8.5B queries/day  100K+  Hundreds of PB indexed

Powers of 2 (frequently used):
- 2^10 = 1,024 (~1 thousand)
- 2^20 = 1,048,576 (~1 million)
- 2^30 = 1,073,741,824 (~1 billion)
- 2^40 = ~1 trillion (1TB)

Weekly study plan:

Weeks 1-2: Foundational Concepts (DDIA key chapters)
  - Scalability, availability, consistency principles
  - Database, cache, message queue basics

Weeks 3-4: Framework Practice (Alex Xu Vol.1)
  - Internalize the 45-minute framework
  - Solve 5 easy problems

Weeks 5-8: Problem-Solving Practice (Alex Xu Vol.2 + Codemia)
  - 10 medium-difficulty problems
  - 3-4 problems per week, using a timer

Weeks 9-10: Advanced Topics + AI/ML Systems
  - Deep dive into distributed systems
  - LLM serving, recommendation systems

Weeks 11-12: Mock Interviews
  - 3-5 mock interviews with peers
  - Focused review of weak areas

Good clarification questions:
- "Should I focus on file upload or download?"
- "What is the user scale? Are we talking 100 million users?"
- "Is real-time collaborative editing in scope?"
- "Do we need to support both mobile and web?"

Trade-off analysis example:

Question: "Would you choose Cassandra or MySQL for message storage?"

Good answer structure:
1. Confirm requirements: "Chat messages are write-heavy,
   mostly queried chronologically, and availability matters
   more than strong consistency."

2. Compare options:
   - MySQL: ACID guarantees, supports joins, but horizontal
     scaling is difficult and sharding is complex
   - Cassandra: Easy horizontal scaling, write-optimized,
     great for time-series data, but no joins

3. Conclusion: "Given our requirements prioritize write
   performance and horizontal scalability, I would choose
   Cassandra. For data with complex relationships like user
   profiles, I would use a separate MySQL instance."

Scaling scenario framework:

Current scale (1M DAU):
- Single DB, 2 read replicas, 1 cache server

10x growth (10M DAU):
- Introduce DB sharding (user ID-based)
- Expand cache cluster (Redis Cluster)
- Add CDN (static assets)
- Read/write separation

100x growth (100M DAU):
- Multi-region deployment
- Microservice decomposition
- Event-driven architecture migration
- Dedicated search engine (Elasticsearch)

Effective communication techniques:

1. Use the whiteboard
   - Always explain with diagrams
   - Show data flow with arrows
   - Label components clearly

2. Set checkpoints
   - "If this looks good so far, I will move on"
   - "Should I go deeper into this component?"

3. Share your thought process
   - Think out loud instead of going silent
   - "I see two options: A does... while B does..."