What is the difference between scaling up and scaling out?
Scaling up means a bigger machine; scaling out means more machines. They fail at different points and impose completely different demands on your software.
Scaling up — vertical. More cores, more memory, faster storage. Its advantages are considerable and frequently dismissed: no application changes required; no distributed systems problems; simpler operations, debugging and monitoring; and strong consistency for free, because there is one copy of everything.
Its limits: a ceiling exists, and cost rises non-linearly as you approach it — the largest machines cost far more per unit of capacity. It also leaves a single point of failure, and resizing usually requires downtime.
Scaling out — horizontal. More instances behind a load balancer. It scales much further, tolerates individual machine failure, and allows incremental capacity changes and gradual rollouts.
Its costs, which are the real story: the application must be stateless, or state must be externalised to a shared store — and that shared store becomes the new bottleneck. You inherit distributed systems problems: partial failure, network latency, consistency, distributed transactions, clock skew and debugging across machines. Operational complexity rises sharply.
The database is where this actually bites. Stateless application servers scale out easily; databases do not. The usual progression is read replicas, then caching, then sharding — which introduces cross-shard queries, rebalancing and a choice of shard key that is difficult to change later.
The advice that holds up. Scale up first. Modern hardware is extremely capable, and a very large proportion of systems that adopted distributed architecture would have run comfortably on one large machine with far less complexity. Scale out when you need availability beyond one machine, when you exceed what one machine can do, or when traffic is variable enough that elasticity saves real money.
In practice most systems do both — a few large instances, scaled out modestly, which is frequently the cheapest and simplest point on the curve.