HTTP/2 Stream Resets Are Fetch Evidence, Not Generic Retries

A production crawler can lose data without a target page changing, a selector breaking, or a proxy failing. The request may have reached the right origin over a healthy HTTP/2 connection, but one stream was refused, reset, replayed, deduplicated, or settled differently than the last runtime build. If the fetch layer records only status: failed and retry: true, the incident looks like ordinary flakiness. In reality, the transport contract changed under the extractor.

That distinction matters this week because two fresh Node ecosystem releases put HTTP client behavior back in the operational path. Undici v8.10.0, published August 3, 2026, includes HTTP/2 and retry fixes such as ensuring every request settles, retrying refused HTTP/2 streams, detaching an upgrade close handler after GOAWAY, forwarding informational responses, preserving DNS origin hostnames on sockets, and namespacing HTTP/2 options. Node.js 26.7.0, published August 5, 2026, is not a crawler release, but it is exactly the kind of runtime update that moves production fetch fleets when base images, serverless runtimes, or worker pools roll forward.

The thesis is not "Undici changed, so log more." The useful lesson is narrower: HTTP/2 failures are often connection-wide in appearance but stream-specific in cause, and crawler operators need evidence that separates source volatility from client-runtime semantics before they tune retries, quarantine a domain, or blame an extraction rule.

Why HTTP/2 changes the retry question

With HTTP/1.1, many fetch incidents are easy to describe at the connection level: a socket timed out, a TLS handshake failed, a response body truncated, or a server returned a status code. HTTP/2 multiplexing makes the evidence more layered. Multiple requests can share one connection while retaining separate streams, priorities, flow-control state, reset reasons, and lifecycle timing. A single origin connection may remain usable while one stream is refused or reset.

That is good for throughput. It is awkward for data quality. If a product-price crawl sends 200 detail-page requests over a small pool of HTTP/2 sessions, one refused stream should not automatically mean the whole origin is unhealthy. Conversely, a sequence of stream resets after a browser or runtime upgrade may be a client compatibility issue rather than a content change. Without stream-level fields, both cases collapse into the same retry bucket.

Undici v8.10.0 is a useful signal because its release notes are full of the kinds of edge cases operators usually discover only during incidents: a request that must always settle, a refused HTTP/2 stream that should be retried, an idle socket validation stall, DNS origin hostname preservation on sockets, and retry handling around unusual response shapes. These are not SEO topics. They are the plumbing that decides whether a crawler records a missing product, waits forever, retries the wrong operation, or labels a source as degraded.

The failure modes to separate

For Better Fetch-style browser-grade infrastructure, the operational mistake is treating every lower-level fetch failure as either "network flake" or "bot defense." At least five categories deserve separate labels:

The important trade-off is restraint. Retrying every stream reset can amplify load, duplicate side effects, and hide a real access-control boundary. Not retrying a refused idempotent stream can produce false extraction gaps. The safe middle is to retry only operations that the pipeline has classified as idempotent, within the site's published limits and your customer authorization, while logging enough evidence to prove why the retry happened.

Add stream evidence before changing policy

A useful fetch record should make the transport layer reviewable without exposing secrets or encouraging access-control bypass. At minimum, add these fields to HTTP and browser-network telemetry:

• runtime.name, runtime.version, undici.version or equivalent client build identifier
• fetch_mode: direct HTTP, browser navigation, browser subresource, discovered API call, replayed API call
• protocol: h1, h2, h3, unknown, plus ALPN result when available
• origin_key: scheme, host, port, proxy route class, and region label
• connection_id and connection_generation, not raw socket internals
• stream_id or client-side stream correlation identifier when available
• h2_event: refused stream, reset, GOAWAY, flow-control timeout, headers received, body ended
• retry classifier: none, idempotent stream retry, connection refresh, policy stop, manual review
• request idempotency classification and whether a body was replayed
• elapsed times for queue wait, DNS, connect, TLS, headers, body, render, and extraction
• final extraction outcome, not just final HTTP outcome

These fields let an operator answer the incident question that matters: did the source stop serving the data, did our runtime change how it observes the source, or did one transport stream fail while the broader crawl remained healthy?

A canary plan for Node and HTTP-client rollouts

Before rolling a Node, Undici, Puppeteer, Playwright, browser, or worker image update through the whole crawl fleet, run a small canary that is designed to catch evidence shifts rather than only crashes.

1. Pick a fixed corpus: static HTML, JSON APIs, large bodies, redirects, conditional requests, range requests, authenticated-but-authorized pages, and JavaScript-rendered pages.
2. Run the old and new client versions from the same regions and proxy route classes.
3. Compare protocol negotiation, connection reuse, stream reset counts, request settlement, body byte counts, redirect chains, and extraction hashes.
4. Force safe negative cases in a controlled test service: refused streams, GOAWAY, delayed bodies, informational responses, HEAD responses, 206 responses, and idle connection reuse.
5. Fail the rollout if extraction yield changes without a matching content hash change, or if unsettled requests increase beyond a small threshold.
6. Keep browser automation separate from direct HTTP in the report. A browser page can hide a direct HTTP regression by successfully rendering cached or alternate data.

This test plan is intentionally about evidence, not bypass. It does not try to defeat rate limits, WAFs, login walls, or site policy. It asks whether your own client stack still records the same facts when the network behaves in realistic but imperfect ways.

How to classify incidents after the rollout

When a crawl starts producing empty records after an HTTP-client update, avoid the default escalation path of adding retries and browser renders. Start with three questions:

• Did failures cluster by runtime version, protocol, connection generation, region, or origin key?
• Did failed requests receive any headers, informational responses, stream events, or body bytes before extraction failed?
• Did browser-rendered fetches and direct API fetches diverge, or did both paths show the same source state?

If the failures cluster by runtime and HTTP/2 event, quarantine the new fetch runtime for that source class and replay a small idempotent sample with the previous build. If failures cluster by source response and appear across runtimes, treat it as a source or policy change. If failures appear only in extraction after equivalent bytes arrived, send it to parser review rather than transport retry.

The durable lesson from the fresh Undici and Node releases is that fetch infrastructure has release notes too. Production web-data systems should read those notes as evidence of possible observation changes, then encode the lesson into telemetry and canaries. Stream-level HTTP/2 behavior is not an implementation detail when it decides whether a crawler records a fact, retries a request, or raises a false source incident.