Elements of Scalable Web Development

Chapter 3

How can we measure web applications and development workflows?#

Created
Updated

A page that feels fast on one developer's machine may behave differently under another workload, device, or network. Benchmarking makes that comparison explicit; observability helps investigate the result. Both need checks that the measured property still represents the outcome people want.

3.1 Benchmark to test a claim#

A measurement records a value for a property using a stated procedure. A benchmark uses declared conditions to compare that property across workloads, systems, or revisions. Begin with the decision the comparison should support: for example, whether the event tracker still records events within an agreed latency objective as concurrent demand increases.

Define the protocol before selecting a runner:

  • Decision and claim: What comparison or threshold would change an engineering decision?
  • Workload: Which event-recording, querying, filtering, or browser actions are performed, and in what mix?
  • Load: Do arrivals follow previous completions or an independent schedule, at what concurrency or rate, and for how long?
  • Boundary and conditions: Which application revision, environment, data volume, cache state, dependencies, and client conditions are included?
  • Metrics and distribution: Which latency, throughput, error, resource, or queue measurements answer the question, and which percentiles or ranges preserve important variation?
  • Acceptance and limits: What result would support or challenge the claim, and what does the setup leave untested?

Tools such as Artillery and k6 can generate load, schedule scenarios, and collect observations from that protocol. They cannot decide whether the workload represents real use, whether the boundary matches the question, or whether the comparison supports a statistically defensible conclusion. An agent can help draft a script, but the generated workload, metrics, thresholds, and interpretation still have to be reviewed against the intended decision.

3.1.1 Check whether the generator hides overload#

A closed-loop worker waits for a result before issuing its next operation; an open-loop generator schedules arrivals independently of completions. Suppose four closed-loop workers immediately repeat requests with no think time or overhead. At 0.1 seconds per request they offer about 40 requests per second; at one second they offer only four. The same concurrency setting has reduced demand just when the application became slower. By contrast, an independent stream of 20 requests per second facing a service capacity of four creates about 16 additional waiting or rejected requests per second, under this simplified constant-rate model. Neither experiment substitutes for the other: the open-versus-closed workload study by Schroeder, Wierman, and Harchol-Balter demonstrates why the arrival model changes the interpretation.

The shared project baseline uses a small closed-loop workload and fixes the timing boundary, deadlines, outcome counts, and percentile rule. Its higher-concurrency profile is not evidence of sustaining an independent production arrival rate. Use the bounded arrival-controlled pressure run when testing queues, overload, or admission control. Count offered, admitted, and completed work separately: a lower latency among successful requests can coexist with more rejected or unfinished work. Timeouts supply deadline lower bounds, not observed completion times; report them beside the completed-request latency distribution.

3.1.2 Worked case — A larger buffer changes the result#

In an October 11, 2026 investigation of Campfire rewrites, Sarnacki (2026) reports changing a Rust implementation's notification buffer at a fixed arrival rate of 100 POSTs per second. The table below contains his reported observations. The article does not pin the implementation commits; treat this as a case to interpret, with incomplete reproduction information.

Table 3.1. Reported buffer trade-off in Sarnacki's Campfire investigation; these are practitioner observations, not measurements reproduced for this book. #
Buffer capacity Notifications delivered Maximum latency
256 About 14% 11 seconds
16,384 About 90% More than 130 seconds

More delivered notifications can coexist with updates too late to use. A missing notification also does not establish lost durable data: the described client reconnects and fetches current messages. Specify persistence, timely visibility, and reconnect recovery separately before benchmarking a replacement.

For your own comparison, pin both implementations and the load script, hold offered load fixed, and report admissions, errors, update latency, memory, and recovered event IDs. Which buffer meets your declared update deadline, and what further observation would establish recovery?

3.1.3 Simulated benchmark lab#

The demonstration below supplies 60 authored request durations for an event-recording endpoint. Change the share that takes a slow storage path and the p95 latency objective. Inspect the ordered records, the percentile rank and how many requests miss the objective. This is a deterministic percentile investigation; it does not model concurrency, queueing or a system's capacity.

Pause & think

Calculate the average and the tail

A fixed sample contains 20 requests: 18 take 40 ms each and two take 240 ms each. Use the nearest-rank percentile: sort the values, then take rank ceil(0.95 × 20). The objective requires latency strictly below 120 ms. Calculate the mean and p95, then choose which meet the objective. This sample is separate from the adjustable simulation in Explore.

Show worked answer

The total is 18 × 40 + 2 × 240 = 1200 ms, so the mean is 60 ms. Rank 19 is 240 ms. The average meets the objective while p95 does not; an average can conceal a slow minority.

Tail latency simulator

An average can hide slow requests

Choose a claim before measuring A constructed population of 60 requests: 24–34 ms plus 188 ms for the selected slow paths. It does not contact a server.

01 / Define the run

Workload controls

Predict the result

02 / Read the distribution

60 records, ordered fastest to slowest

Typical Slow path

The outlined bar is rank 57, the nearest-rank p95. Three slower records can sit beyond it.

Inspect all ordered records

Make a prediction to reveal the population.

Average
—
p50
—
p95
—
Slowest
—

A benchmark should test an explicit claim. Which measurement do you expect to cross the objective?

Predict the default result before running the simulated workload, then compare the average with p95. Treat the first result as a baseline, change one condition, and predict again before recording the comparison. Record the duration population, percentile convention and acceptance threshold together; a separate executed workload is needed for a capacity claim.

Synthetic benchmarks deliberately simplify reality. They can omit workload mixes, client behavior, cache histories, data shapes, provider quotas, dependency contention, and failure combinations that appear in production. A result therefore supports a claim only for its declared conditions. Compare synthetic evidence with field telemetry where available, and revise the protocol when real behavior exposes a missing condition.

3.1.4 Benchmark browser journeys with Playwright and Lighthouse#

Server latency does not describe the whole user experience. A browser still has to download resources, parse and execute JavaScript, calculate styles, lay out the page, paint it, and respond to input. Three related tools expose different parts of that work:

  • The browser's Performance APIs can measure marks, durations, resources, and selected rendering events in the page itself.
  • Playwright drives a real browser through a repeatable user journey and checks whether the expected behavior occurred.
  • Lighthouse audits a page in a controlled lab setup and reports performance metrics and diagnostics.

These tools are complementary rather than interchangeable. A Playwright test can prove that filtering the event list produces the expected rows, but its elapsed test time is not a Web Vital. A Lighthouse navigation can diagnose initial page load, but it does not prove that the application's business flow works. Lighthouse user flows can audit interactions, although they require browser automation outside the page; a normal web page cannot grant itself the required browser-control privileges.

Browser performance workbench

The workbench below runs completely in the current page. It builds and filters an event list either as one uninterrupted task or in batches that yield to the browser. It measures from the start of the work through two animation-frame callbacks after the DOM work, repeats the same flow five times, and reports total elapsed time and the longest instrumented JavaScript slice. The callbacks provide rendering opportunities, not a timestamp proving that pixels reached the display (HTML rendering algorithm).

Browser performance workbench

Compare two browser update strategies

Ready for five local runs Measurements come from this browser and this device. No data leaves the page.

01 / Define the workload

Event-list interaction

Update strategy

Predict first: which strategy has the shorter total time, and which has the shorter instrumented JavaScript slice?

02 / Inspect the runs

Elapsed time including frame waits

Local evidence
Median total
—
Fastest
—
Slowest
—
Longest JS slice (median)
—

Five runs show variation in this browser. Frame callbacks are a scheduling proxy, not proof of pixel presentation; slices exclude later layout and rendering.

Before each measurement, predict which strategy will finish sooner and which will leave the main thread available sooner. Run both strategies with the same number of events. Then increase the workload and check whether the conclusion remains the same.

This is a real measurement, but it is deliberately narrow. The timer includes work performed by this demo on this browser and device; it is not a Lighthouse score, INP measurement, or claim about other users. The chunked strategy may take longer overall while shortening the measured JavaScript segments and leaving opportunities for other browser work. The longest-slice metric excludes the forced layout after those segments and other rendering work; the demo does not measure input delay, so it cannot by itself establish improved responsiveness. This distinction between throughput and responsiveness is often more useful than asking which version is simply "faster."

For client-side collection in a real application, the small web-vitals library implements LCP, INP, CLS, FCP, and TTFB using browser APIs and follows the same measurement rules used by Google's web-performance tools. Those measurements describe actual page visits. Lighthouse instead provides synthetic lab evidence under declared conditions.

From page metrics to product metrics

A page-level metric may describe a different element from the one the user came to see. For the event tracker, define a milestone such as the event list becoming visible and usable. Element Timing can report rendering of a supported annotated element; application instrumentation and interaction checks can establish the remaining behavior. Choose the signal from the product question, name its collection rule, and record the browsers and users represented. Paint evidence alone cannot establish that a region is interactive or correct.

Use Appendix B's browser measurement procedures for repeatable page audits and a standalone Playwright journey. Choose the procedure that answers the product question and report the conditions and limits of its evidence.

3.1.5 Check measurements against a model#

Measurements become more useful when they can be checked against a model. Little's Law states that, for a stable system over a suitable observation period, average work in progress (L) equals the average arrival rate (λ) multiplied by average time in the system (W) (Little, 1961):

L = λW

For an event pipeline, L can be the average number of events waiting or being processed, λ the average arrival rate, and W the average time from arrival to completion. If 20 events arrive per second and each spends half a second in the system, the expected average work in progress is 10 events.

Derive averages from one observation window

The record below is an authored example with one arrival and one terminal completion per event, no retries and no work present before time zero. Times are seconds on the same clock; the boundary includes both waiting and processing.

Table 3.2. An event cohort with explicit arrival and completion times. #
Event Arrives Completes
A 0 2
B 1 3
C 2 4
D 3 5

For the window from 0 inclusive to 6 exclusive, calculate the arrival rate, each event's time inside the boundary, and the time-weighted number of unfinished events. The successive one-second occupancy intervals contain 1, 2, 2, 2, 1, 0 events: eight event-seconds in total. Thus L = 8/6 events, λ = 4/6 events/s and mean W = 2 seconds agree. This exact accounting for an empty-to-empty cohort does not establish that a running service has reached a stable long-run regime.

Now shorten the window to 0 inclusive through 3 exclusive. Which events have arrived but not completed, and which portions of their residence times lie outside the window? The occupancy area is five event-seconds, while the only completion strictly inside the window is A. Multiplying the arrival rate by A's two-second residence time does not recover 5/3 events. Explain the incomplete population before interpreting that mismatch as overload. Finally, state what would change if the timer measured only processing but occupancy included queued events.

Pause & think

Check whether the averages agree

At one declared boundary you observe L = 25 operations, λ = 40 operations/s, and W = 0.25 s. Calculate λW, then select the conclusion supported by these three values alone.

Show worked answer

λW is 10, not 25. Check compatible boundaries, observation windows, units, omitted waiting, and whether arrivals and completions balance. The discrepancy alone does not locate a bottleneck or show that adding workers will help.

The law does not identify the cause of latency, prove that a queue is healthy, or promise that the system can sustain any chosen arrival rate. Before applying it, state the system boundary and check that arrivals and completions were approximately balanced during the observation window. A large mismatch is a reason to inspect the measurement setup or system stability, not permission to force the numbers to fit.

3.1.6 Follow the queue into development#

A queue can contain events waiting for processing or changes waiting for review. Before calculating, name the work and measurement boundary in each case: generating a change and accepting it are different completion events. Try the question without opening the references, then use their assumptions to explain what carries across the two examples.

Pause & think

Follow work from events to review

A stable event pipeline completes 20 events/s with average end-to-end time 0.5 s. A stable development workflow accepts four changes/hour with average time from submission to acceptance of three hours. Both times include waiting inside their respective boundaries. Calculate the average work in progress implied by Little's Law for each. Then suppose submissions rise to six changes/hour while acceptance stays at four, with no discarded changes. Can the earlier steady-state calculation describe this growing backlog indefinitely?

Show worked answer

The event calculation is 20 × 0.5 = 10 events. The development calculation is 4 × 3 = 12 changes. The same relationship applies to different work when boundaries, windows, and units agree. After the increase, unfinished changes accumulate at two/hour, so the earlier stable calculation cannot describe the backlog indefinitely. Check arrival/completion balance and waiting time again. Neither calculation identifies the cause of delay or proves the waiting time is acceptable.

Explain one difference between the event pipeline and human review that could make a capacity prediction unreliable. The numerical check establishes the stated relationship; your explanation should also identify its limits.

3.2 Measurements can lead you astray#

Goodhart's law describes how a metric can stop tracking the desired outcome when it becomes a target (Goodhart, 1975). Software dashboards and quality gates make such proxies easy to optimize.

Consider a few examples:

  • Test coverage can reveal unexercised code, but a coverage target can reward assertions that execute lines without checking meaningful behavior.
  • A Lighthouse score can direct attention to performance problems, but optimizing only the score can remove valuable functionality or tune a page for the audit rather than its users.
  • Mean latency can improve while tail latency or error rate becomes worse.
  • Agent task pass rate can rise because tasks, judges, or outputs become tailored to the evaluation set rather than because the workflow handles new work better.
  • Commit count, generated lines, and closed tickets can all increase while review and maintenance costs grow elsewhere.

Goodhart's law does not imply that targets or metrics are useless. It means that the measurement system changes once people or agents can adapt to it. A quality gate should therefore be treated as a partial model of the desired outcome, not as the outcome itself.

For an important target, record four things:

  1. Outcome: What user or operational result is the metric intended to represent?
  2. Proxy: Why should the chosen measure move with that outcome under the current conditions?
  3. Gaming path: How could the number improve without the outcome improving?
  4. Counterevidence: Which second signal, failure review, or qualitative check would expose that divergence?

Keep raw distributions and failed examples available instead of retaining only an aggregate score. Review the proxy when the workload, system, incentives, or development process changes. An effective metric is not permanently valid; it is a hypothesis that must survive contact with the behavior it influences.

3.2.1 Field note — Slideotter#

Slideotter let me test whether a local language model could support presentation authoring. Using Qwen3.5-9B through the project's generation harness, I found that it could usually produce semantically sensible drafts: the slides contained information related to the requested topic and formed a recognizable presentation. That established adequacy, not excellence. Some slides remained weak or misunderstood my intent.

This conclusion came from my editorial review rather than a separate evaluation. The observed result belonged to the complete configuration—model, prompts, schemas, validators, and review workflow—so I could not confidently attribute poor outputs to the model rather than the harness. Structural and rendering checks could establish that a deck was valid; they could not establish that it was a good response to what I meant.

3.2.2 Challenge a coverage target#

In this authored case, a team requires at least 95% line coverage. A candidate adds tests that execute the transition handler but never assert the resulting state or event history. Before revealing the explanation, predict whether the candidate could meet the target while missing a regression, and name one observation that would challenge your prediction.

Reveal the authored explanation

Executing a line does not establish that its result is correct. A test can increase coverage while accepting an invalid transition or a duplicate event. Raising the coverage target alone does not close that gap. A separate assertion about the promised behavior can expose it, but the assertion must itself be challenged with a faulty implementation and a correct alternative. This is a worked explanation; no test or model ran in this page.

For optional executed evidence, open concern-directed-mutation/README.md in the course workspace and use your model and harness. Strengthen one black-box assertion so that it passes the baseline, rejects a named faulty variant with an ordinary assertion failure, and still passes the harmless storage refactor. Preserve the first result and explain the behavior it distinguishes in your existing specification and decision notes. Monitoring coverage can remain useful; the independent behavior check tests a different claim. One detected fault and one tolerated refactor do not establish a complete acceptance gate.

3.2.3 Resource proxies are not the outcome#

Energy use is harder to measure than latency or transferred bytes, and neither of those proxies is an automatic substitute for it (Kalliola & Vepsäläinen, 2025).

An optimization must preserve the intended behavior as well as improve the selected resource measure.

Run one bounded optimization experiment:

  1. Choose one representative Playwright journey and preserve its baseline.
  2. Make one targeted AI-assisted frontend optimization.
  3. Run the same journey under the same declared conditions before and after the change. If the chosen signal varies, repeat each condition at least five times.
  4. Record direct energy only if the platform exposes a defensible measure. Otherwise label transferred bytes, JavaScript or asset size, CPU time, or long-task time as resource proxies, not energy or carbon.
  5. For a deterministic measure such as built asset size, report one value per condition. For a run-varying measure, report the median and range.
  6. Re-run the functional checks, inspect for visual or accessibility regressions, note rejected suggestions or regressions, and state what the comparison cannot establish.

3.3 Measurement alone cannot explain behavior#

A benchmark can reveal that the request distribution changed, but it does not necessarily explain why one request was slow. Telemetry provides evidence about a running system through signals such as traces, metrics, and logs. Observability is the broader capability to use that evidence to investigate system behavior, including failures you did not predict in advance.

OpenTelemetry is a vendor-neutral framework for generating, processing, and exporting telemetry. It supplies APIs, SDKs, semantic conventions, and a Collector, but it is not itself a storage or visualization backend. That distinction matters: instrumenting an application creates evidence, while operating the pipeline and asking useful questions turns that evidence into an observability practice.

Before opening a trace, choose the first signal you would inspect for a slow request, stale summary or missing event. State which observation could distinguish competing explanations; the name of a signal alone does not establish a cause.

3.3.1 Simulated OpenTelemetry trace lab#

The trace explorer below reuses the event-recording request from the benchmark demonstration. Compare requests A, B and C without revealing the authored explanation first. Inspect child spans for duration, status, source and attributes, then predict which work accounts for the additional server time. Only after inspecting the record, reveal the case reading and compare it with your explanation. A span represents one timed unit of work and carries its own span ID, while every span in the same trace shares a trace ID. The HTTP server span measures this server's handling of the request; it does not include the complete client-observed request or browser interaction. It is the root in this simulation because no incoming parent context is modeled; in a distributed trace it may be a child of an upstream span. The spans are not necessarily sent as separate network messages: an SDK can batch completed span records into one OTLP export request before a Collector processes and routes them.

OpenTelemetry trace explorer

Follow one request, span by span

Inspect the evidence first A client-only trace simulation. No telemetry is collected, exported, or stored.
Applicationstarts + ends spans OTel SDKbatches span records Collectorreceives OTLP + routes Backendstores + explores

The rows cover one server's handling of an inbound HTTP request. Browser work and time outside this server span are excluded. Select a row to inspect the record the SDK could include in a batched OTLP export.

Server trace

POST /events · 286 ms on server

4bf92f3577b34da6a3ce929d0e0e4736

Select any span row to inspect its telemetry record.

Try this in your project: instrument one operation, record a normal request and a deliberately delayed or failed attempt, then inspect the exported spans. Preserve the request result and explain what a missing span would prevent you from concluding. The records above are authored examples.

A long span can localize elapsed time without establishing root cause. Explain what additional evidence would distinguish a saturated resource, a configuration limit, slow work or a competing workload. A successful HTTP response can also contain an earlier failed attempt; retain both outcomes when interpreting the trace.

For optional local practice, ask your model to instrument one operation in your own project and export spans to a local console or file. Keep the accepted behavior and workload fixed, then add one controlled delay or one fail-once dependency in a disposable setup. Retain the original and changed records, including the failed attempt, and check shared trace IDs, parent-child links and the timer boundary. Use an independent acceptance check to establish the operation's final outcome. A telemetry backend is unnecessary for this bounded check; the local delay does not establish the cause of an unrelated production incident. Also notice that useful domain spans and attributes require conscious instrumentation, while generic HTTP spans can often be produced automatically.

3.3.2 Use runtime evidence during repair#

Telemetry can also become input to an agent repairing the application. The TeleGen preprint links browser actions to runtime observations and compresses them into repair briefs (Luo et al., 2026). It reports gains on two web-generation benchmarks with different success measures; those results do not establish lower production maintenance cost.

The optional repair-evidence investigation compares task feedback alone, raw telemetry, and a brief for the same failure. Appendix A explains the controls and acceptance boundary.

3.4 Observe the agentic workflow too#

Observe the development workflow as well as the application. In both, outcome checks establish what worked and traces help explain the path to that result. Use these evidence roles to evaluate loop engineering: changing a trigger, repair policy, or stop condition needs outcome and effort evidence, including unsuccessful runs.

Table 3.3. Parallel evidence roles in a running web application and its agentic development workflow. #
Evidence role Web application Agentic development workflow
Outcome to protect Useful user and operational behavior Accepted implementation and maintained project goal
Controlled comparison Benchmark run under a declared workload Bounded task or evaluation trial
Diagnostic evidence Logs, metrics, request traces, and profiles Prompts, tool calls, diffs, checks, and retries
Configuration Revision, deployment, dependencies, and resource limits Model, agent harness, tools, starting revision, and limits
Intervention Operator diagnosis, mitigation, and recovery Human review, correction, approval, and escalation

Keep the task, one attempted trial, its actual outcome, and the trace that led there distinct. Grade the outcome; use the trace to diagnose failures, account for cost and intervention, and check requirements such as permission compliance. Bind the result to the model and runtime where exposed, harness, tools, starting revision, resource limits, and routing or fallback.

Use the same reporting categories across A, B, and C: accepted outcome, checks, elapsed time, retries, cost or resource use, human correction, and known limits. Generated lines and transcript length can hide work shifted into review or maintenance; they do not replace these outcome measures. Appendix A develops grader design, repeated trials, uncertainty, and configuration qualification.

From a coding task to software delivery

A faster patch is only one part of getting a useful change into production. DORA's five delivery metrics cover change lead time, deployment frequency, failed deployment recovery time, change fail rate, and deployment rework rate (Harvey, 2026). Interpret them together for one application or service over a declared period. They describe delivery throughput and instability, while the task record above describes one attempt at producing a change.

Consider an invented timeline: work starts at 09, a patch is committed at 09, and it reaches production at 15. The twenty-minute interval ends at a commit; DORA change lead time runs from that commit to deployment, five hours and forty minutes later. If the deployment fails and service recovers at 15, record the thirty-minute recovery separately. Failed and incident-driven deployments also belong in the period's failure and rework counts, with their denominators stated.

A short project may have too few deployments or failures for stable rates. Use this timeline to interpret the measurement boundaries; individual rankings and an uncontrolled before/after chart cannot establish an AI effect.

3.4.1 Field note — Preserve accepted context, not every interaction#

I have not established a practice of preserving complete agent transcripts or detailed workflow telemetry. When returning to a project, I rely mainly on code, documentation, and commit history to recover the implemented state and accepted intent.

Architecture Decision Records are especially useful. Vibe Template's ADR workflow keeps proposed, accepted, implemented, and superseded decisions distinct. I let agents draft records, but negotiate important choices carefully; implementation can then expose what an unresolved decision still needs to settle.

Those records preserve the rationale I accepted, not every prompt, disagreement, retry, or abandoned path. They cannot reconstruct the complete agent workflow. For the course comparison, preserve the selected task's trace and correction record while the work happens, rather than trying to infer them later from the accepted repository.

Pause & think

Choose a valid comparison

You want to test whether batching improves useful write throughput while keeping p95 end-to-end latency below 100 ms and errors at or below 0.1%. Select the comparison that isolates batching. Then evaluate a run with higher throughput, p95 = 90 ms, and errors = 0.8% against those declared criteria.

Show worked answer

Keep the workload, dataset, environment, and acceptance criteria comparable while changing batching. A run with p95 = 90 ms passes the latency bound, but 0.8% errors violates the 0.1% error bound. Higher throughput does not compensate for failing a declared acceptance criterion.

3.5 Project artifact#

Add implementation A's measurement baseline to the Establish evidence bundle:

  • the representative runtime workload, load, conditions, prediction, thresholds, and results, including distributions for varying measurements;
  • one explanatory telemetry signal and inspected failure, with the first warning that behavior is leaving the measured conditions;
  • one representative AI-assisted task and its configuration, trace, checks, retries, correction, cost or resource use, and acceptance decision.

Tie the records to the application revision and retain the reporting frame for B and C. State what each comparison leaves untested.

3.6 Summary#

A benchmark tests a claim under declared conditions; telemetry helps investigate the behavior it reveals. Preserve variation and check that an improved proxy still represents the desired outcome. For agent-assisted work, evaluate the accepted artifact and use configuration, attempts, and interventions to explain how it arose.

3.7 References#

  1. Kalliola, J., & Vepsäläinen, J. (2025). Challenges Related to Approximating the Energy Consumption of a Website. IEEE Access, 13, 139001–139017. https://doi.org/10.1109/ACCESS.2025.3596459
  2. Little, J. D. C. (1961). A Proof for the Queuing Formula: L = λW. Operations Research, 9(3), 383–387. https://doi.org/10.1287/opre.9.3.383
  3. Goodhart, C. A. E. (1975). Problems of Monetary Management: The U.K. Experience. In Papers in Monetary Economics (Vol. 1). Reserve Bank of Australia. https://www.econbiz.de/Record/problems-of-monetary-management-the-u-k-experience-goodhart-charles/10002525062
  4. Sarnacki, P. (2026, October). I’m Sorry, but You Still Have to Think. It’s All About the Bit. https://itsallaboutthebit.com/i-am-sorry-but-you-still-have-to-think/
  5. Luo, Y., Zhang, H., Shen, J., Ding, Z., & Shang, W. (2026, October). TeleGen: Improving LLM-Based Web Application Generation via Runtime Telemetry. arXiv preprint arXiv:2610.04981. https://doi.org/10.48550/arXiv.2610.04981
  6. Harvey, N. (2026). DORA’s Software Delivery Performance Metrics. DORA Guides. https://dora.dev/guides/dora-metrics/