Appendix E
Formal scalability and reliability models#
- Created
- Updated
Use this appendix when a scaling or workflow decision needs more than the introductory Little's Law or Amdahl's Law treatment in the main chapters. The models below expose two different ways that adding work can reduce success: serial reliability compounds across critical steps, while contention and coordination can flatten or reverse throughput gains.
E.1 Author perspective — Use models to see the limits#
I do not use these laws directly in my everyday development work. I include them because they have educational value: they make the limits behind scalability visible.
Parallelization cannot remove work that must remain sequential. Additional workers still share resources and require coordination. Longer workflows accumulate more opportunities for failure. These constraints exist regardless of how quickly code can be generated or how much infrastructure can be provisioned.
The purpose of the models and labs is therefore not formula memorization or precise prediction. It is to make relationships concrete enough to challenge claims such as “more agents will make the work proportionally faster” or “adding capacity will continue increasing throughput.”
Use the equations to recognize the shape of a limit and identify what should be measured. Real decisions still require evidence from the actual system.
E.2 Choose a model#
Use the question to select a model, then check that its assumptions fit the observation. The table below collects the relationships introduced across the main chapters and this appendix.
| Law | Exact relationship | Main assumptions | Useful when | Cannot establish |
|---|---|---|---|---|
| Little's Law (Little, 1961) | L = λW, where L is average work in progress, λ is average throughput or arrival rate, and W is average time in the system |
Stable long-run averages, one consistent system boundary, and conserved work | Relating queue or review backlog, throughput, and elapsed time | The cause of delay, tail latency, or stability from one snapshot |
| Amdahl's Law (Amdahl, 1967) | S(N) = 1 / ((1 - p) + p/N), where p is the parallelizable fraction and N is the parallel capacity |
Fixed workload and fractions, with no added parallel overhead in the bound | Bounding speedup for request paths, batch work, deployments, or agent workflows | Contention, coordination cost, load growth, or the actual parallel fraction |
| Lusser's Law (Tait, 1995) | R_system = ∏ R_i for independent components that must all succeed in a series system |
Every named step is critical and step failures are independent | Checking whether a request, deployment, or agent workflow can meet an end-to-end reliability target | Correlated failures, recovery behavior, or omitted dependency paths |
| Universal Scalability Law (Gunther, 2008) | C(N) = N / (1 + α(N - 1) + βN(N - 1)), where α models contention and β coordination or coherency cost |
One workload and configuration, with parameters fitted to representative throughput measurements | Describing capacity that rises, flattens, and then falls as concurrency grows | The causal mechanism or safe extrapolation beyond the measured range |
For each use, identify the variables, calculation, observation boundary, and what remains unexplained. The Benchmarking chapter works through Little's Law, and the Agent Fleets chapter introduces Amdahl's bound.
E.3 Lusser's Law and the reliability chain#
A workflow whose accepted result depends on every critical step has the structure of a series system.
If step i has reliability r_i, the workflow reliability is:
R_workflow = ∏ r_i
When n independent critical steps have the same reliability r, this reduces to R_workflow = r^n.
This product rule is commonly called Lusser's Law or the product law for series reliability (Tait, 1995).
It explains why a small per-step failure probability can dominate a long workflow even when no individual step looks especially risky.
Redundancy within one step has a different topology.
Jason Gorman's compact formulation uses C for the probability that the first prediction is correct and P for the conditional probability that an incorrect prediction is caught.
For completion reliability, catching the prediction must also restore the step to a correct state; under that stronger interpretation:
r = C + (1 - C)P = 1 - (1 - C)(1 - P)
The complement expression describes the prediction-or-recovery paths within one step; Lusser's Law describes the product across serial steps.
The distinction matters because detection alone does not complete a workflow.
If D is the probability that an incorrect prediction is detected and K is the probability of successful repair after detection, then the completion reliability of one step is:
r = C + (1 - C)DK
An incorrect result that is detected but not repaired can stop safely, while an undetected result can propagate. Safety, liveness, and correctness should therefore be reported separately rather than collapsed into one success percentage.
E.4 Reliability chain lab#
The first preset below reproduces the compact idealization with 95% initial correctness, 90% catch-and-repair coverage, and 100 independent critical steps. Before changing the controls, predict whether the workflow remains above 50% correct completion. Then select Caught, not repaired and compare correct completion with safe stopping.
Lusser’s Law · Rworkflow = ∏ri
Track the error that survives every safeguard
- Step gate
- 99.500%
- Correct completion
- 60.6%
- Safe stop
- 0.0%
- Wrong result
- 39.4%
At 99.500% per step, 100 independent critical steps complete correctly 60.6% of the time.
The model assumes identical independent steps, treats every step as necessary, and assigns the first terminal outcome to the whole run.
Real agent workflows can share model blind spots, context, tools, and upstream state; steps can have different reliabilities; and later validation can sometimes repair earlier failures.
Use the curve as a design probe, not a reliability forecast.
Estimate C, D, and K from repeated accepted outcomes at a declared boundary, and use ∏ r_i when observed steps differ instead of replacing them with an unsupported average.
E.5 Contention curve lab#
C(N) = N / (1 + α(N - 1) + βN(N - 1))
Here α represents contention or serialization cost and β represents coherency or coordination cost.
The curve can rise, flatten, and eventually fall: adding workers may reduce total capacity when coordination grows faster than useful work.
Universal Scalability Law · C(N)
See when adding concurrency starts taking capacity away
- Selected capacity
- 5.57×
- Peak at
- 22
- Peak capacity
- 7.33×
- Next worker
- helps
At eight workers, added concurrency still increases modeled capacity.
First set both costs to zero and inspect the ideal linear curve. Then add contention, add coherency cost, and find the first concurrency level after the modeled peak. The parameters should ultimately be fitted from measured throughput at several concurrency levels; arbitrary sliders illustrate the model's shape but do not predict a real deployment.
Case study: limiting concurrency to protect throughput
PlanetScale describes a Vitess and MySQL production incident in which a long-running transaction, immediate application retries, and a transaction pool raised to about 10,000 allowed more than 10,000 requests to accumulate inside the storage engine. Snapshot reads grew more expensive as the long transaction preserved an increasingly long version history; more requests exceeded their latency limit and retried. Work in progress rose while completed query throughput fell. Increasing the pool had silenced pool-full errors but weakened the boundary protecting the database.
The team later reduced the pool to roughly 1,000 and changed full-pool behavior so requests could wait for a slot within a bounded timeout. During a later burst on another shard, demand peaked around 25,000 slot requests per second while reported query throughput stayed near 60,000 per second and fewer than 200 statements executed inside MySQL at once. If those figures describe the same stable boundary and observation window, Little's Law gives an upper estimate of
W < 200 / 60,000seconds, or about 3.3 milliseconds. The important separation is between clients seeking work, work admitted to the constrained resource, and useful work completed.A queue does not create capacity; it moves waiting to a boundary where it can be observed and limited. This report changed pool size and waiting behavior together, the later burst did not reproduce the original incident, and no tail-latency distribution was reported. It also uses the Universal Scalability Law as an explanatory lens rather than fitting its parameters from measurements. Treat the numbers as evidence for this workload, not as a universal pool target. The durable lesson is that admission control can raise throughput when in-flight requests slow one another down, provided waiting is bounded, overload remains visible, and the underlying long transaction and retry behavior are still addressed.
E.6 Treat the model as a challenge to the design#
A formula is useful only when its boundary, variables, assumptions, and evidence are explicit. Use measured inputs where the model requires them, keep recovery and correlated failures visible, and do not extrapolate beyond the observed workload merely because the fitted curve is convenient.
E.7 References#
- Little, J. D. C. (1961). A Proof for the Queuing Formula: L = λW. Operations Research, 9(3), 383–387. https://doi.org/10.1287/opre.9.3.383
- Amdahl, G. M. (1967). Validity of the Single Processor Approach to Achieving Large Scale Computing Capabilities. Proceedings of the April 18–20, 1967, Spring Joint Computer Conference, 483–485. https://doi.org/10.1145/1465482.1465560
- Tait, N. R. S. (1995). Robert Lusser and Lusser’s Law. Safety and Reliability, 15(2), 15–18. https://doi.org/10.1080/09617353.1995.11690648
- Gunther, N. J. (2008). A General Theory of Computational Scalability Based on Rational Functions. CoRR, abs/0808.1431. https://arxiv.org/abs/0808.1431