Better Fetch

· Paul Crossland

TLS Groups Belong in Fetch Incident Logs

Node and Undici updates show why crawler incidents need TLS, HTTP/2, proxy, and stream evidence before retry tuning.

A production fetch failure often arrives as a simple symptom: more timeouts, more empty bodies, more 403 responses, or more retries on a set of domains. The tempting response is also simple: change proxy pools, increase timeouts, or retry with a browser. But recent runtime updates are a reminder that many incidents begin below the parser and below the target application's visible HTML.

On July 8, Node.js released v26.5.0. Two details matter for web-data operators: Node now reports negotiated TLS groups, and the release updates its bundled Undici dependency to 8.7.0. Undici's own v8.7.0 release on July 4 includes fixes around HTTP/2 aborts, late response events, requeueing requests after GOAWAY, proxy tunneling detection, open-ended Range values, cookie path validation, unparseable Set-Cookie expiry attributes, EventSource credentials, and late readable chunks.

Those are not scraping features. That is exactly why they matter. Most fetch fleets depend on runtime behavior that was never designed as a data-quality interface: TLS negotiation, HTTP/2 lifecycle, proxy tunnel setup, cookie parsing, abort semantics, and stream teardown. When those layers change, the same URL, region, session, and extractor can produce different evidence.

The thesis for operators is practical: treat transport negotiation and stream lifecycle as incident evidence, not as invisible plumbing. If you cannot answer which TLS group, HTTP version, proxy tunnel mode, abort path, and cookie-parse outcome produced a fetch artifact, you will overfit retry policy to bugs you cannot see.

The repeated mistake: calling every transport difference a site change

Recent Better Fetch posts have covered browser version drift, automation library revisions, runtime-specific fetch behavior, redirects, consent, language, and AI traffic policy. The repeated angle to avoid is "another release happened, therefore pin and log versions." Version logging is necessary, but it is not enough here.

The surplus signal in the latest Node and Undici updates is more specific: fetch correctness depends on evidence that many systems never store. A request can fail because the origin changed policy, but it can also fail because an HTTP/2 session received GOAWAY, a proxy tunnel was classified differently, a stream was aborted in a different phase, a late response event arrived after completion, or a cookie parser accepted or rejected state that later shaped the page.

If your observability flattens all of that into network_error, blocked, or retry_exhausted, your incident response will drift toward superstition. You will compare different client states, tune the wrong layer, and sometimes make the system less respectful of origin capacity by adding retries where classification would have been better.

Why TLS groups are operationally useful

Node v26.5.0's note that TLS can report negotiated groups is easy to read as a low-level security improvement. For fetch infrastructure, it is also a diagnostic field.

TLS negotiation is part of the client profile a server and intermediary observe. It can vary by runtime version, OpenSSL build, operating system, proxy path, and managed platform. Most teams already log TLS protocol and cipher when debugging serious connection issues. Negotiated groups add another clue when a subset of connections behaves differently after a base-image rollout or runtime upgrade.

This does not mean operators should chase a magic TLS fingerprint or try to impersonate another client. The useful question is narrower: did the client profile that produced this artifact change at the same time the data quality changed?

For example, a domain may start timing out only from a new worker image; a proxy provider may route some sessions through a different TLS stack; or a managed Node upgrade may change low-level negotiation while the application code and target site did not deploy. TLS evidence helps separate "same client, different response" from "different client, different response." The operational goal is not to optimize TLS by hand. It is to preserve enough evidence to avoid false attribution.

HTTP/2 lifecycle bugs look like flaky targets

Undici v8.7.0 contains several HTTP/2 and stream-lifecycle fixes that map directly to crawler symptoms. Destroying a stream on abort, guarding against a response event after completion, and requeueing after a GOAWAY session can change whether a fetch ends as a clean retry, a crash, a hung job, a partial body, or a duplicate-looking failure.

At production volume, those distinctions matter. A small percentage of HTTP/2 edge cases can become a large percentage of the incident queue on high-cardinality crawl jobs. Worse, they often cluster by origin, CDN, proxy, or region, which makes them look like target-specific blocking or rate limiting.

Before changing crawler behavior, ask what actually happened at the transport layer:

SymptomPossible transport-level causeEvidence to capture
Empty or truncated bodyAbort, late chunks, stream teardown, content decoding failurebytes read, declared length, abort reason, stream close event, decoder error
Sudden retry spikeHTTP/2 GOAWAY, dropped tunnel, connect timeout, session reuse issueHTTP version, session id, retry reason, socket reused, proxy tunnel state
Parser misses after runtime updateCookie parsing or redirect behavior changed before renderSet-Cookie parse warnings, cookie jar diff, redirect chain, runtime version
Long-tail hangsRequest not failed cleanly after tunnel or stream lifecycle edgedeadline, phase at timeout, active socket count, pending queue depth
Inconsistent EventSource or streaming dataCredentials or stream consumption behavior changedrequest credentials mode, last event id, chunks consumed after close

This table is intentionally about diagnosis, not evasion. If a source is enforcing access control, respect it. Better transport evidence helps you distinguish that policy outcome from your own client regression.

Cookie parse outcomes belong beside cookie values

Undici's v8.7.0 release also includes fixes for cookie path validation and unparseable Set-Cookie Expires attributes. Cookie handling is often treated as a browser-only concern, but many pipelines use Node fetches for discovery, API replay, preflight checks, session warming, or cheap monitoring before launching a browser.

A cookie parser does not merely store text. It decides whether state exists for the next request. If a runtime starts rejecting a path with non-ASCII octets, ignoring an invalid expiry instead of failing differently, or normalizing duplicate cookie behavior in another layer, downstream fetches can land in a different session. That can change consent state, language, prices, logged-in versus logged-out content, rate limits, or which internal API calls appear during render.

For high-value sources, log cookie processing as an event stream rather than only logging the final jar:

set_cookie_header_hash
cookie_name
accepted: true|false
reject_reason
path
domain
same_site
secure
expires_parse_result
source_url
runtime
http_client_version

You do not need to store sensitive cookie values to make incidents debuggable. Hash names or values where necessary, apply retention limits, and protect session material. The key is to know whether state changed because the site sent different headers or because the client interpreted the same headers differently.

A transport evidence checklist

Add a small transport block to every fetch attempt record, especially for browser-discovery, API-replay, and server-side preflight paths:

  1. Runtime and client: Node version, Undici version when available, worker image digest, deployment id, operating system, and architecture.
  2. TLS: protocol, cipher, negotiated group when exposed, SNI, certificate issuer, validation error, and whether a custom CA bundle or proxy MITM path was used.
  3. HTTP: negotiated version, ALPN result, request method, redirect count, response status, response headers hash, content encoding, declared length, bytes consumed, and decompression error.
  4. Proxy path: provider, egress region, tunnel mode, connect time, tunnel failure reason, socket reuse, and remote address family.
  5. HTTP/2 lifecycle: session reuse, stream id when available, GOAWAY seen, abort signal reason, reset code, and whether the request was requeued.
  6. Stream lifecycle: first byte time, last byte time, close event, abort event, late chunks ignored, and whether the consumer stopped early.
  7. Cookie state: accepted cookies, rejected cookies with reasons, jar generation, and whether the next request reused or isolated the jar.
  8. Classification: separate origin_policy, transport_failure, client_runtime_regression, proxy_path_failure, timeout, and extractor_failure instead of one generic failure bucket.

This checklist is not free. It adds storage, privacy review, and cardinality. Start with sampled detailed records for errors and canary runs, then promote the fields that repeatedly shorten incidents.

Test the client boundary before scaling retries

When a fetch incident begins, run a boundary test before changing retry policy:

  1. Replay a small failing URL set with the previous runtime and the current runtime from the same region.
  2. Keep proxy, session, headers, and extractor version fixed where policy allows.
  3. Compare TLS fields, ALPN, HTTP version, redirect chain, cookie acceptance, response bytes, and stream lifecycle events.
  4. Repeat with direct browser navigation if the server-side path is used only as a discovery step.
  5. If the target clearly returns an access-control or policy response, classify it as such and do not attempt to defeat it.
  6. Only tune retries after distinguishing transient transport failure from stable policy, parser drift, and client-runtime regression.

The durable lesson from the Node and Undici releases is not that every team should jump to the newest runtime immediately. It is that runtime upgrades expose how much fetch infrastructure depends on hidden negotiation and lifecycle behavior. Browser-grade fetching is not only about rendering JavaScript. It is about preserving the evidence chain from connection setup through final extracted record.

When that evidence chain includes TLS groups, HTTP/2 lifecycle, proxy tunnel classification, stream events, and cookie parse decisions, incidents become less mysterious. More importantly, the fix becomes safer: fewer blind retries, fewer false accusations against target sites, and better separation between respectful policy handling and client bugs you own.