Elements of Scalable Web Development

Chapter 3

How can we measure web applications and development workflows?#

Created
Updated

A page that feels fast on one developer's machine may behave differently under another workload, device, or network. Benchmarking makes that comparison explicit; observability helps investigate the result. Both need checks that the measured property still represents the outcome people want.

3.1 Benchmark to test a claim#

A measurement records a value for a property using a stated procedure. A benchmark uses declared conditions to compare that property across workloads, systems, or revisions. Begin with the decision the comparison should support: for example, whether the event tracker still records events within an agreed latency objective as concurrent demand increases.

Define the protocol before selecting a runner:

  • Decision and claim: What comparison or threshold would change an engineering decision?
  • Workload: Which event-recording, querying, filtering, or browser actions are performed, and in what mix?
  • Load: How many users, requests, events, or concurrent sessions apply the workload, and for how long?
  • Boundary and conditions: Which application revision, environment, data volume, cache state, dependencies, and client conditions are included?
  • Metrics and distribution: Which latency, throughput, error, resource, or queue measurements answer the question, and which percentiles or ranges preserve important variation?
  • Acceptance and limits: What result would support or challenge the claim, and what does the setup leave untested?

Tools such as Artillery and k6 can generate load, schedule scenarios, and collect observations from that protocol. They cannot decide whether the workload represents real use, whether the boundary matches the question, or whether the comparison supports a statistically defensible conclusion. An agent can help draft a script, but the generated workload, metrics, thresholds, and interpretation still have to be reviewed against the intended decision.

3.1.1 Simulated benchmark lab#

The demonstration below models 60 requests to an event-recording endpoint. Change the number of concurrent clients, the share of requests that take a slow storage path, and the p95 latency objective. The values are deterministic and run only in the browser, so this is a teaching model rather than a capacity test.

Pause & think

Calculate the average and the tail

A fixed sample contains 20 requests: 18 take 40 ms each and two take 240 ms each. Use the nearest-rank percentile: sort the values, then take rank ceil(0.95 × 20). The objective requires latency strictly below 120 ms. Calculate the mean and p95, then choose which meet the objective. This sample is separate from the adjustable simulation in Explore.

Show worked answer

The total is 18 × 40 + 2 × 240 = 1200 ms, so the mean is 60 ms. Rank 19 is 240 ms. The average meets the objective while p95 does not; an average can conceal a slow minority.

Tail latency simulator

An average can hide slow requests

Choose a claim before measuring A deterministic teaching model of 60 POST /events requests. It does not contact a server.

01 / Define the run

Workload controls

Predict the result

02 / Read the distribution

Completion time per request

Typical Slow path
Average
—
p50
—
p95
—
Slowest
—

A benchmark should test an explicit claim. Which measurement do you expect to cross the objective?

Predict the default result before running the simulated workload, then compare the average with p95. Treat the first result as a baseline, change one condition, and predict again before recording the comparison. Record the workload, load, metric, and acceptance threshold together.

Synthetic benchmarks deliberately simplify reality. They can omit workload mixes, client behavior, cache histories, data shapes, provider quotas, dependency contention, and failure combinations that appear in production. A result therefore supports a claim only for its declared conditions. Compare synthetic evidence with field telemetry where available, and revise the protocol when real behavior exposes a missing condition.

3.1.2 Benchmark browser journeys with Playwright and Lighthouse#

Server latency does not describe the whole user experience. A browser still has to download resources, parse and execute JavaScript, calculate styles, lay out the page, paint it, and respond to input. Three related tools expose different parts of that work:

  • The browser's Performance APIs can measure marks, durations, resources, and selected rendering events in the page itself.
  • Playwright drives a real browser through a repeatable user journey and checks whether the expected behavior occurred.
  • Lighthouse audits a page in a controlled lab setup and reports performance metrics and diagnostics.

These tools are complementary rather than interchangeable. A Playwright test can prove that filtering the event list produces the expected rows, but its elapsed test time is not a Web Vital. A Lighthouse navigation can diagnose initial page load, but it does not prove that the application's business flow works. Lighthouse user flows can audit interactions, although they require browser automation outside the page; a normal web page cannot grant itself the required browser-control privileges.

Browser performance workbench

The workbench below runs completely in the current page. It builds and filters an event list either as one uninterrupted task or in batches that yield to the browser. It measures from the start of the work until the next painted frame, repeats the same flow five times, and reports both total time and the longest JavaScript slice observed by its own instrumentation.

Browser performance workbench

Repeat one interaction before trusting it

Ready for five local runs Measurements come from this browser and this device. No data leaves the page.

01 / Define the workload

Event-list interaction

Update strategy

Predict first: which strategy has the shorter total time, and which has the shorter uninterrupted slice?

02 / Inspect the runs

Time until the next painted frame

Local evidence
Median total
—
Fastest
—
Slowest
—
Median longest slice
—

One run is an observation. Repeated runs show whether that observation was typical.

Before each measurement, predict which strategy will finish sooner and which will leave the main thread available sooner. Run both strategies with the same number of events. Then increase the workload and check whether the conclusion remains the same.

This is a real measurement, but it is deliberately narrow. The timer includes work performed by this demo on this browser and device; it is not a Lighthouse score, INP measurement, or claim about other users. The chunked strategy may take longer overall because it waits between batches while still improving responsiveness by avoiding one long uninterrupted task. This distinction between throughput and responsiveness is often more useful than asking which version is simply "faster."

For client-side collection in a real application, the small web-vitals library implements LCP, INP, CLS, FCP, and TTFB using browser APIs and follows the same measurement rules used by Google's web-performance tools. Those measurements describe actual page visits. Lighthouse instead provides synthetic lab evidence under declared conditions.

From page metrics to product metrics

A page-level metric may describe a different element from the one the user came to see. For the event tracker, define a milestone such as the event list becoming visible and usable. Element Timing can report rendering of a supported annotated element; application instrumentation and interaction checks can establish the remaining behavior. Choose the signal from the product question, name its collection rule, and record the browsers and users represented. Paint evidence alone cannot establish that a region is interactive or correct.

Audit a representative page

Use PageSpeed Insights for the lowest-setup version of the exercise. It runs Lighthouse remotely, so the target page must be deployed at a publicly reachable URL; a development server at localhost is not reachable by the service.

  1. Choose one representative public route and state why it matters to a user.
  2. Select the mobile profile and run the same URL five times without changing the application.
  3. Record LCP, TBT, CLS, transferred bytes, and the overall performance score for every run.
  4. Report the median and the fastest-to-slowest range for each time-based metric.
  5. Select one Lighthouse diagnostic, make one change, deploy it, and repeat the five runs under the same conditions.
  6. State the narrow conclusion supported by the comparison and at least one factor it does not control.

The PageSpeed Insights result can include Lighthouse lab data and, for sufficiently popular public pages, field data derived from real Chrome visits. Keep those sources separate: lab and field measurements use different populations and conditions.

If the application is not public, open it in Chrome, select Developer Tools → Lighthouse, choose the mobile profile, and generate a report locally. Close unrelated applications where practical, do not run audits concurrently, and keep the browser version, profile, URL, application version, and machine consistent.

Performance values vary even when the application has not changed because the browser, client hardware, resource contention, network, and server introduce noise. Heričko et al. (2021) found that five consecutive Lighthouse runs aggregated with the median greatly reduced variability in their experiment.

Study a repeatable Playwright journey

PageSpeed Insights is convenient for navigation audits, but it cannot exercise a private or multi-step project flow. The standalone Playwright lab separates a small event-filter fixture from the browser journey that exercises it, so you can inspect locators, interaction, assertions, and trace collection without first adapting the example to a framework. Study the source or download the standalone lab.

The example establishes that filtering produces the expected visible row. It does not measure application loading, network behavior, or Web Vitals because it opens a local fixture. After it passes, replace the fixture URL with the project's real route and adapt the accessible names while preserving the same user-visible contract. Inspect the trace to verify what was acted on and rendered, then use Lighthouse or another suitable performance tool separately; the flow can be correct but slow, or appear fast because it failed to perform the intended work.

For a repeatable CI gate, Lighthouse CI can collect runs and assert a budget; measure the runner's normal variation before choosing its threshold.

3.1.3 Check measurements against a model#

Measurements become more useful when they can be checked against a model. Little's Law states that, for a stable system over a suitable observation period, average work in progress (L) equals the average arrival rate (λ) multiplied by average time in the system (W) (Little, 1961):

L = λW

For an event pipeline, L can be the average number of events waiting or being processed, λ the average arrival rate, and W the average time from arrival to completion. If 20 events arrive per second and each spends half a second in the system, the expected average work in progress is 10 events.

Queue balance lab

Use the lab below to check whether three observed averages are mutually consistent. Change one value at a time and explain whether the mismatch suggests overload, incompatible measurement windows, or a missing boundary.

Little’s Law · L = λW

Check whether the queue tells one story

Measurements agree

01 / Define the observation window

10 = 20 × 0.5 expected work in progress = arrival rate × time in system
Expected L
10.0
Observed L
10
Gap
0.0

The three averages are mutually consistent for a stable observation window.

Pause & think

Check whether the averages agree

At one declared boundary you observe L = 25 operations, λ = 40 operations/s, and W = 0.25 s. Calculate λW, then select the conclusion supported by these three values alone.

Show worked answer

λW is 10, not 25. Check compatible boundaries, observation windows, units, omitted waiting, and whether arrivals and completions balance. The discrepancy alone does not locate a bottleneck or show that adding workers will help.

The law does not identify the cause of latency, prove that a queue is healthy, or promise that the system can sustain any chosen arrival rate. Before applying it, state the system boundary and check that arrivals and completions were approximately balanced during the observation window. A large mismatch is a reason to inspect the measurement setup or system stability, not permission to force the numbers to fit.

3.2 Measurements can lead you astray#

Goodhart's law describes how a metric can stop tracking the desired outcome when it becomes a target (Goodhart, 1975). Software dashboards and quality gates make such proxies easy to optimize.

Consider a few examples:

  • Test coverage can reveal unexercised code, but a coverage target can reward assertions that execute lines without checking meaningful behavior.
  • A Lighthouse score can direct attention to performance problems, but optimizing only the score can remove valuable functionality or tune a page for the audit rather than its users.
  • Mean latency can improve while tail latency or error rate becomes worse.
  • Agent task pass rate can rise because tasks, judges, or outputs become tailored to the evaluation set rather than because the workflow handles new work better.
  • Commit count, generated lines, and closed tickets can all increase while review and maintenance costs grow elsewhere.

Goodhart's law does not imply that targets or metrics are useless. It means that the measurement system changes once people or agents can adapt to it. A quality gate should therefore be treated as a partial model of the desired outcome, not as the outcome itself.

For an important target, record four things:

  1. Outcome: What user or operational result is the metric intended to represent?
  2. Proxy: Why should the chosen measure move with that outcome under the current conditions?
  3. Gaming path: How could the number improve without the outcome improving?
  4. Counterevidence: Which second signal, failure review, or qualitative check would expose that divergence?

Keep raw distributions and failed examples available instead of retaining only an aggregate score. Review the proxy when the workload, system, incentives, or development process changes. An effective metric is not permanently valid; it is a hypothesis that must survive contact with the behavior it influences.

3.2.1 Field note — Slideotter#

Slideotter let me test whether a local language model could support presentation authoring. Using Qwen3.5-9B through the project's generation harness, I found that it could usually produce semantically sensible drafts: the slides contained information related to the requested topic and formed a recognizable presentation. That established adequacy, not excellence. Some slides remained weak or misunderstood my intent.

This conclusion came from my editorial review rather than a separate evaluation. The observed result belonged to the complete configuration—model, prompts, schemas, validators, and review workflow—so I could not confidently attribute poor outputs to the model rather than the harness. Structural and rendering checks could establish that a deck was valid; they could not establish that it was a good response to what I meant.

3.2.2 Goodhart pressure test#

The challenge below turns that four-part review into two decisions. First choose a proxy and predict how an optimizer will respond when the number becomes a target. Then inspect the displaced cost and revise the gate so that improving the proxy cannot compensate for losing the intended outcome. The scenarios use deterministic illustrative values to make the feedback loop visible; they are not empirical comparisons between tools or teams.

Goodhart pressure test

Can your metric survive an optimizer?

Choose a proxy to put under pressure A deterministic two-round challenge. The values are illustrative, not observed results.
  1. 01 Target
  2. 02 Pressure
  3. 03 Repair

Write the target contract

Which proxy gets rewarded?

Choose one optimization target
Intended outcome

Choose a proxy to reveal the result it is supposed to represent.

Predict what target pressure will do

Optimizer dashboard

Waiting for a target

UNPRESSURED
Targeted proxy
—

— ?

Intended outcome
—

— ?

Counter-signal
—

— ?

Displaced cost
—

— ?

Optimizer trace

No behavior to inspect yet. Select a proxy, predict the result, and apply pressure.

Try at least two targets. Notice that monitoring the proxy is still useful after the repair, but it no longer decides success alone.

3.2.3 Resource proxies are not the outcome#

Energy use is harder to measure than latency or transferred bytes, and neither of those proxies is an automatic substitute for it (Kalliola & Vepsäläinen, 2025).

EcoAssist illustrates the need to check the product alongside resource use (Barrocas et al., 2026). Its before-and-after benchmark reported reduced local CPU and GPU energy on selected standalone webpages using one M4 MacBook and Chrome testbed. In a separate lab study, some participants observed removed content or changed layouts after optimization. The measurements concern those pages and that device, exclude the optimizing model's full footprint, and are not a whole-system carbon estimate. An optimization must preserve the intended behavior as well as improve the selected resource measure.

Run one bounded optimization experiment:

  1. Choose one representative Playwright journey and preserve its baseline.
  2. Make one targeted AI-assisted frontend optimization.
  3. Run the same journey under the same declared conditions before and after the change. If the chosen signal varies, repeat each condition at least five times.
  4. Record direct energy only if the platform exposes a defensible measure. Otherwise label transferred bytes, JavaScript or asset size, CPU time, or long-task time as resource proxies, not energy or carbon.
  5. For a deterministic measure such as built asset size, report one value per condition. For a run-varying measure, report the median and range.
  6. Re-run the functional checks, inspect for visual or accessibility regressions, note rejected suggestions or regressions, and state what the comparison cannot establish.

3.3 Measurement alone cannot explain behavior#

A benchmark can reveal that the request distribution changed, but it does not necessarily explain why one request was slow. Telemetry provides evidence about a running system through signals such as traces, metrics, and logs. Observability is the broader capability to use that evidence to investigate system behavior, including failures you did not predict in advance.

OpenTelemetry is a vendor-neutral framework for generating, processing, and exporting telemetry. It supplies APIs, SDKs, semantic conventions, and a Collector, but it is not itself a storage or visualization backend. That distinction matters: instrumenting an application creates evidence, while operating the pipeline and asking useful questions turns that evidence into an observability practice.

3.3.1 Simulated OpenTelemetry trace lab#

The trace explorer below reuses the event-recording request from the benchmark demonstration. Switch between a healthy write, a slow write, and a successful request that required a retry. Select individual spans to inspect their duration, status, source, and attributes. A span represents one timed unit of work and carries its own span ID, while every span in the same trace shares a trace ID. The spans are not necessarily sent as separate network messages: an SDK can batch completed span records into one OTLP export request before a Collector processes and routes them.

OpenTelemetry trace explorer

Follow one request, span by span

Slow storage path found A client-only trace simulation. No telemetry is collected, exported, or stored.
Applicationstarts + ends spans OTel SDKbatches span records Collectorreceives OTLP + routes Backendstores + explores

The rows below are spans from one trace. Select a row to inspect the record the SDK could include in a batched OTLP export.

Trace

POST /events · 286 ms

4bf92f3577b34da6a3ce929d0e0e4736

Select any span row to inspect its telemetry record.

The slow trace localizes most of the request time to connection waiting and event storage. That is a stronger lead than the end-to-end latency alone, but it still does not establish root cause. The next investigation might inspect database saturation, pool configuration, query behavior, or competing workloads. Also notice that useful domain spans and attributes require conscious instrumentation, while generic HTTP spans can often be produced automatically.

3.4 Observe the agentic workflow too#

Observe the development workflow as well as the application. In both, outcome checks establish what worked and traces help explain the path to that result.

Table 3.1. Parallel evidence roles in a running web application and its agentic development workflow. #
Evidence role Web application Agentic development workflow
Outcome to protect Useful user and operational behavior Accepted implementation and maintained project goal
Controlled comparison Benchmark run under a declared workload Bounded task or evaluation trial
Diagnostic evidence Logs, metrics, request traces, and profiles Prompts, tool calls, diffs, checks, and retries
Configuration Revision, deployment, dependencies, and resource limits Model, agent harness, tools, starting revision, and limits
Intervention Operator diagnosis, mitigation, and recovery Human review, correction, approval, and escalation

Keep the task, one attempted trial, its actual outcome, and the trace that led there distinct. Grade the outcome; use the trace to diagnose failures, account for cost and intervention, and check requirements such as permission compliance. Bind the result to the model and runtime where exposed, harness, tools, starting revision, resource limits, and routing or fallback.

Use the same reporting categories across A, B, and C: accepted outcome, checks, elapsed time, retries, cost or resource use, human correction, and known limits. Generated lines and transcript length can hide work shifted into review or maintenance; they do not replace these outcome measures. Appendix A develops grader design, repeated trials, uncertainty, and configuration qualification.

From a coding task to software delivery

A faster patch is only one part of getting a useful change into production. DORA's five delivery metrics cover change lead time, deployment frequency, failed deployment recovery time, change fail rate, and deployment rework rate (Harvey, 2026). Interpret them together for one application or service over a declared period. They describe delivery throughput and instability, while the task record above describes one attempt at producing a change.

Consider an invented timeline: work starts at 09, a patch is committed at 09, and it reaches production at 15. The twenty-minute interval ends at a commit; DORA change lead time runs from that commit to deployment, five hours and forty minutes later. If the deployment fails and service recovers at 15, record the thirty-minute recovery separately. Failed and incident-driven deployments also belong in the period's failure and rework counts, with their denominators stated.

A short project may have too few deployments or failures for stable rates. Use this timeline to interpret the measurement boundaries; individual rankings and an uncontrolled before/after chart cannot establish an AI effect.

3.4.1 Field note — Preserve accepted context, not every interaction#

I have not established a practice of preserving complete agent transcripts or detailed workflow telemetry. When returning to a project, I rely mainly on code, documentation, and commit history to recover the implemented state and accepted intent.

Architecture Decision Records are especially useful. Vibe Template's ADR workflow keeps proposed, accepted, implemented, and superseded decisions distinct. I let agents draft records, but negotiate important choices carefully; implementation can then expose what an unresolved decision still needs to settle.

Those records preserve the rationale I accepted, not every prompt, disagreement, retry, or abandoned path. They cannot reconstruct the complete agent workflow. For the course comparison, preserve the selected task's trace and correction record while the work happens, rather than trying to infer them later from the accepted repository.

Pause & think

Choose a valid comparison

You want to test whether batching improves useful write throughput while keeping p95 end-to-end latency below 100 ms and errors at or below 0.1%. Select the comparison that isolates batching. Then evaluate a run with higher throughput, p95 = 90 ms, and errors = 0.8% against those declared criteria.

Show worked answer

Keep the workload, dataset, environment, and acceptance criteria comparable while changing batching. A run with p95 = 90 ms passes the latency bound, but 0.8% errors violates the 0.1% error bound. Higher throughput does not compensate for failing a declared acceptance criterion.

3.5 Project artifact#

Add implementation A's measurement baseline to the Establish evidence bundle:

  • the representative runtime workload, load, conditions, prediction, thresholds, and results, including distributions for varying measurements;
  • one explanatory telemetry signal and inspected failure, with the first warning that behavior is leaving the measured conditions;
  • one representative AI-assisted task and its configuration, trace, checks, retries, correction, cost or resource use, and acceptance decision.

Tie the records to the application revision and retain the reporting frame for B and C. State what each comparison leaves untested.

3.6 Summary#

A benchmark tests a claim under declared conditions; telemetry helps investigate the behavior it reveals. Preserve variation and check that an improved proxy still represents the desired outcome. For agent-assisted work, evaluate the accepted artifact and use configuration, attempts, and interventions to explain how it arose.

3.7 References#

  1. Kalliola, J., & Vepsäläinen, J. (2025). Challenges Related to Approximating the Energy Consumption of a Website. IEEE Access, 13, 139001–139017. https://doi.org/10.1109/ACCESS.2025.3596459
  2. Barrocas, A., Jardim Nunes, N., Nisi, V., & Martelaro, N. (2026). EcoAssist: Embedding Sustainability into AI-Assisted Frontend Development. Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems, 1–16. https://doi.org/10.1145/3772318.3791330
  3. Little, J. D. C. (1961). A Proof for the Queuing Formula: L = λW. Operations Research, 9(3), 383–387. https://doi.org/10.1287/opre.9.3.383
  4. Goodhart, C. A. E. (1975). Problems of Monetary Management: The U.K. Experience. In Papers in Monetary Economics (Vol. 1). Reserve Bank of Australia. https://www.econbiz.de/Record/problems-of-monetary-management-the-u-k-experience-goodhart-charles/10002525062
  5. Heričko, T., Šumak, B., & Brdnik, S. (2021). Towards Representative Web Performance Measurements with Google Lighthouse. Proceedings of the 2021 7th Student Computer Science Research Conference, 39–42. https://doi.org/10.18690/978-961-286-516-0.9
  6. Harvey, N. (2026). DORA’s Software Delivery Performance Metrics. DORA Guides. https://dora.dev/guides/dora-metrics/