Recently, I was asked a question that I actually really liked:
“You built a system using a simple MVC architecture, deployed on EC2 with RDS. It worked fine initially. Now it’s getting millions of users how would you scale it? What changes across each layer?”
I like this because it’s not about theory it’s about how systems actually evolve in the real world.
So here’s how I think about it.
Starting Point (What we had)
A typical setup:
Monolithic MVC app Single (or few) EC2 instances RDS as the database Everything tightly coupled
This setup is honestly perfect in the beginning. Fast to build, easy to debug, easy to deploy.
But once traffic starts growing, you start hitting limits.
1. Compute Layer (First thing that breaks)
Initially, you might just vertically scale your EC2 (bigger instance), but that only gets you so far.
What I’d do next:
Add a load balancer Introduce multiple EC2 instances Use auto-scaling based on traffic
At this point, your app becomes horizontally scalable.
Later on, I’d move towards containers (Docker) and something like Kubernetes or ECS for better control over deployments and scaling.
2. Application Layer (Monolith doesn’t scale forever)
I wouldn’t jump to microservices immediately that’s a trap.
First step:
Break the monolith into clean modules (auth, posts, search, etc.) Make boundaries clear
Then over time:
Extract high-load parts (e.g., search, notifications) into separate services
This way:
You scale only what needs scaling Failures don’t take down the entire system
3. Database Layer (Usually the biggest bottleneck)
RDS will start struggling under heavy read/write load.
Steps I’d take:
Add read replicas (scale reads) Optimize queries + proper indexing Introduce caching to reduce DB hits
If scale keeps growing:
Shard the database Or move certain workloads to specialized stores
4. Caching Layer (Massive impact)
This is where you get huge wins.
Adding something like Redis:
Cache frequently accessed data Store sessions Reduce repeated DB queries
In a lot of cases, this alone can drastically improve performance.
5. Async Processing (Don’t block requests)
Not everything needs to happen instantly.
I’d move heavy or non-critical tasks to background jobs:
Emails Notifications Data processing
Using queues (like Sidekiq/SQS/Kafka) keeps your APIs fast and responsive.
6. CDN & Static Assets
Serving everything from your app servers doesn’t make sense at scale.
I’d:
Move static assets to a CDN Cache images, JS, CSS at the edge
This reduces load on your servers and improves global latency.
7. Observability (You can’t scale blind)
At this level, debugging becomes harder.
So I’d add:
Centralized logging Metrics and alerts Request tracing
Because at scale, problems aren’t obvious anymore you need visibility.
Final Thought
The biggest mistake people make is trying to “design for millions” on day one.
In reality:
You start simple You scale based on real bottlenecks You evolve your architecture step by step
That’s how real systems grow.