powershelldba.de

Why 80% Beats 100%: The Queueing Formula Behind SQL Server and Overloaded Teams

Utilization and response time do not have a straight-line relationship. One formula from queueing theory explains why a SQL Server pushed to 95% CPU falls over, and why a team booked at 100% capacity misses every deadline, for exactly the same mathematical reason.

The Formula Nobody Applies Outside Operations Research

Take the simplest possible model of a server handling requests: work arrives at some average rate, gets processed at some average rate, and utilization is just the ratio of the two. Call the arrival rate λ (lambda), the service rate μ (mu), and utilization ρ (rho) = λ / μ. Basic queueing theory (the M/M/1 model, one of the first things taught in operations research) gives the average time a request spends in the system:

W = (1 / μ) / (1 - ρ)

The interesting part isn't the (1 / μ) term, that's just how long the work takes when nothing is waiting. The interesting part is the 1 / (1 - ρ) multiplier. As utilization climbs, that denominator shrinks toward zero, and the multiplier does not grow gently. It goes vertical.

Utilization (ρ) Wait Multiplier
50%2.0×
70%3.3×
80%5.0×
90%10.0×
95%20.0×
99%100.0×
Going from 80% to 90% utilization doesn't add 10% more wait time. It doubles it. Going from 90% to 99% doesn't add 9% more. It multiplies it by ten.

This is not a metaphor and it is not specific to computers. It's a property of any system where work arrives somewhat randomly and gets processed by a resource with finite capacity: a CPU core, a checkout line, a support queue, a single engineer's calendar. The math doesn't care what the server is.

What This Looks Like on a SQL Server

Every SQL Server DBA has seen the shape of this curve without necessarily naming it. A server sitting at 60-70% sustained CPU feels fine. Push the same workload to 90%+ and it doesn't degrade proportionally, it falls off a cliff: SOS_SCHEDULER_YIELD waits climb, the scheduler's runnable queue backs up, worker threads start queuing behind THREADPOOL, and query response times that were previously milliseconds start reading in seconds.

This is exactly the 1 / (1 - ρ) curve, instantiated in SQLOS. It's also why capacity-planning guidance that says "keep sustained CPU under 70-80%" is not conservative hand-waving. It's reading the curve and staying on the flat part of it deliberately, because the steep part offers almost no warning before it arrives. A monitor alerting at 90% is already alerting from inside the expensive half of the graph.

The Same Curve Applies to People

Queueing theory doesn't know or care whether the server processing requests is a CPU core or a person. A team, or an individual, running at 100% utilization, every hour of every day already assigned to something, is mathematically in the same position as a database server pinned at 100% CPU. Not "a bit busier." In the region where the wait-time curve goes vertical.

The practical consequence: a fully booked team's response time to anything new (an urgent bug, an unplanned request, a genuine emergency) isn't "a little slower than usual." In the strict M/M/1 model, a server sitting at literally 100% utilization has an unstable, unboundedly growing queue; nothing new ever gets serviced in finite expected time, because everything already in the queue has to clear first, and more keeps arriving. Real teams aren't a pure math model, but the direction is the same: the closer to fully booked, the longer anything new sits waiting before anyone can even start on it.

Why This Gets Ignored in Both Worlds

In infrastructure, this happens because the useful warning signal (the curve turning steep) shows up disturbingly close to the failure point, and dashboards default to flagging round numbers like 90% rather than the point where the multiplier is actually accelerating.

In management, it happens because "utilization" gets treated as a productivity metric to maximize, not a risk parameter to manage. A team at 100% billable or 100% allocated looks maximally efficient on a spreadsheet. What it actually is, per the formula above, is a team with no slack left to absorb anything unplanned, running exactly the failure mode a capacity planner would flag immediately if the "server" in question were a piece of hardware instead of a person.

What Headroom Actually Buys You

Neither SQL Server capacity planning nor team capacity planning is served by chasing 100%. Deliberately keeping sustained utilization in the 70-80% range, on a server or on a calendar, isn't waste. It's the only region of the curve where response time stays predictable and a genuine emergency doesn't have to wait behind everything that was already scheduled first.

The number to watch isn't "are we busy." It's "how close are we to the part of the curve where busy stops being linear." Those are very different questions, and only one of them actually predicts what happens next.