The Browser Network Archive Is a Derived Artifact

A crawler finishes with a HAR file, a trace, and a passing extraction. It is tempting to treat those artifacts as a transcript of what crossed the network. That assumption can fail in three different places: the browser may send headers that its instrumentation API does not report, a response-body API may reacquire bytes after the original body was evicted, and the archive writer may fail while the browser context is closing.

The resulting file can be useful without being complete or transaction-identical.

Fresh Playwright and WebKit work exposes each boundary. A Playwright change merged August 11 records that WebKit on macOS 26 and later can send Sec-WebSocket-Key, Connection, and Upgrade during a WebSocket handshake while omitting them from the request reported to HAR and protocol consumers. The related WebKit pull request, also opened August 11, proposes delaying the report until transaction metrics are available instead of inventing values that the earlier API cannot observe.

A separate Playwright change merged August 11 adjusts when Chromium response bodies may be fetched again after eviction. On August 13, another merged Playwright fix changed HAR failure handling so a save error is raised after context shutdown rather than leaving the context permanently stuck in closing. These are changes on development branches, not proof that any particular stable package already contains them.

The practical thesis is: browser network evidence is produced by a fallible acquisition pipeline, so fetch systems should distinguish what the engine reported from what was derived, reacquired, unavailable, or never finalized before using an archive to prove a data fact.

This is not guidance for collecting secrets, replaying unsafe requests, or probing past a denial. Capture only public or properly authorized traffic, minimize sensitive headers and bodies, and preserve login, challenge, payment, rate-limit, and publisher-policy outcomes as stop conditions.

The repeated angle to avoid

Recent Better Fetch posts already cover request metadata, runtime-specific HTTP behavior, WebSocket flow observability, browser control-plane telemetry, lifecycle cleanup, frame completeness, and evidence planes. The repeated article would say to retain HAR files and log browser versions.

The new surplus is an acquisition verdict for the artifact itself. A field can be absent because the source did not send it, because an engine API did not expose it, because a safety policy declined to reacquire it, or because archive finalization failed. Those meanings cannot share one empty value.

Source map

Fresh primary evidence from the last seven days:

Older background comes from Playwright's HAR recording documentation, which says the HAR is saved when the browser context closes, and its network documentation, which describes response-body and WebSocket observation surfaces. The operational synthesis is not in either document: consumers need to preserve how each piece of network evidence was acquired.

A capture has at least three failure boundaries

Reporting

The network stack and the instrumentation surface are not identical. In the WebKit case, CFNetwork adds real WebSocket handshake headers after the request representation available to the earlier callback. The server can receive a valid handshake while HAR and protocol clients see an incomplete request.

That makes header absent in HAR a statement about the capture surface, not automatically about the transaction. It also explains why fabricating a plausible Sec-WebSocket-Key would be worse than leaving an explicit gap: a synthetic value would make the archive internally convincing while no longer describing the request that ran.

Platform belongs in this verdict. The report distinguishes macOS 26 and later from macOS 15 and links the difference to the loader and framework used. Two runs with the same Playwright and WebKit versions may therefore have different evidence capability.

Body acquisition

A browser can receive a response and later evict its buffered body. Asking for response.body() after that point is no longer necessarily a passive read. Earlier Playwright work introduced a restricted fallback because silently issuing another network request could repeat a server-side effect. The August 11 follow-up shows the classification problem: a gate based on Sec-Fetch-Dest failed on a plain-HTTP Android test origin where Chromium did not send that header, so static response bodies came back empty. The fix also considers Chromium's resource type.

Reacquisition can be appropriate only for a request proven safe under the collector's policy. Even then, the returned bytes belong to a second observation. Cache state, time, authorization, personalization, and source content may have changed. Store them as reacquired, not as the original response body. If safety cannot be established, body_unavailable_safety_policy is a valid result; a blank body must not become a blank product description.

Finalization

HAR generation continues after the interesting page work appears finished. Entries and attached bodies still have to be serialized, synchronized, and written. The August 13 fix shows why teardown success and artifact success require separate verdicts: a save error previously interrupted context closure and left later teardown waiting on a permanent closing state.

The corrected ordering improves lifecycle behavior, but it does not turn a failed HAR into a complete one. A fetch job should not publish high-confidence records merely because navigation and extraction succeeded if its evidence contract required an archive that did not finalize.

Give every artifact an acquisition manifest

Attach a compact manifest to each HAR, trace, or network-derived extraction batch:

• artifact_id, type, schema version, path or object reference, digest, size, and retention class;
• automation library, browser product and revision, operating system, network loader, and protocol adapter;
• capture start, last network event, context-close start and finish, write start and finish, and outer watchdog result;
• requested capture mode, body policy, attachment policy, redaction policy, and archive-writer version;
• per evidence class counts for engine_reported, collector_derived, reacquired, unavailable, redacted, and unknown;
• reacquisition method, original request identifier, idempotency verdict, cache involvement, and resulting transaction identifier;
• expected and observed WebSocket handshake fields, with capability status rather than secret values in broad logs;
• entry count, body-reference count, missing-body count, serialization errors, write errors, and parse result;
• final verdict: complete_for_contract, complete_with_declared_gaps, unfinalized, corrupt, policy_limited, or unknown.

Do not put cookies, authorization values, WebSocket keys, private payloads, or full authenticated URLs in general telemetry. Keep names, counts, classifications, hashes, and restricted artifact references. Apply redaction before long-term storage, but record that redaction occurred so a deliberate omission is not mistaken for collector loss.

An owned-fixture capture test

Build the canary on infrastructure you control:

1. Serve a normal document, a static subresource, a cacheable API response, and a WebSocket endpoint with a known handshake outcome.
2. Record direct server-side receipt separately from HAR, trace, and automation events. Compare field presence without treating the server log as a substitute for browser timing evidence.
3. Run the matrix across supported operating systems, browser engines, and automation builds. Assert that capability differences become manifest verdicts rather than false source differences.
4. Make one static body unavailable to the later reader. Verify that any reacquisition receives a new transaction identifier and that unsafe or unclassified requests are not repeated.
5. Change a harmless fixture response between original delivery and permitted reacquisition. Confirm that the system preserves both generations and never attributes the later bytes to the first request.
6. Write the HAR to an intentionally failing owned test sink. Require bounded context closure, a surfaced save error, and unfinalized; file existence or test-body success must not pass the evidence check.
7. Terminate one run before context close and corrupt another completed copy. Verify that the parser and publication gate distinguish absent, partial, and invalid artifacts.
8. Repeat before automation, browser, operating-system, capture-mode, or archive-writer promotion.

The operator decision rule

When an archive disagrees with server behavior or extracted data, identify the earliest acquisition boundary that cannot support the claim. If the engine did not report a field, classify it as unobservable on that runtime rather than source-absent. If a body was reacquired, compare it as a separate transaction generation. If safety policy prevented reacquisition, keep the field unknown. If finalization failed, quarantine any result whose contract required that artifact.

A HAR or trace remains valuable because it organizes browser-visible evidence. Reliability comes from refusing to treat organization as omniscience. Record the reporting surface, retrieval path, and finalization result, and the archive can say exactly what it proves without pretending to be the wire.