Better Fetch

· Paul Crossland

The URL Passed. The Request Graph Did Not.

Fresh browser-agent guardrail gaps show why URL policy must cover every producer, scheme, and redirect hop before a request is sent.

A browser agent can begin on an allowed page and still send a later request outside the boundary its operator intended. The initial navigation passes. Ordinary subresources may even be blocked correctly. Then an audit tool asks the browser to load a secondary resource, a redirect selects another origin, or a hostless URL enters through a special-case branch. The job still says allowed, even though that verdict described only one node in a larger request graph.

Two fresh reports expose different versions of this problem. A confirmed Chrome DevTools MCP issue opened August 12 reports that Lighthouse-owned secondary loads can cross --allowed-url-pattern after an allowed origin redirects them. Ordinary excluded subresources were blocked, but the audit path used a DevTools loading command outside the same URL-rule profile. The P3 issue remains open; maintainers also stress that the option is a guardrail, not a complete network sandbox.

Separately, Browser Use 0.13.8 was released August 16. An August 17 Browser Use pull request verified that the release and current main still allowed data: and blob: URLs before domain rules. It added regression tests but was closed, unmerged, in favor of an older open fix.

The practical thesis is: a browser URL policy is enforceable only when every network-capable producer and every transition receives a pre-request decision; admitting the first page cannot authorize the redirect, tool, worker, scheme, or secondary load that follows.

This is defensive guidance for owned or properly authorized automation. It is not a recipe for crossing domain restrictions, smuggling credentials, or probing past a denial. Unexpected destinations, access-control outcomes, challenges, payment boundaries, and source-policy restrictions should stop or narrow a run.

The repeated angle to avoid

Recent Better Fetch posts already cover agent control points, MCP evidence planes, browser route graphs, service-worker state, frame churn, and archive completeness. The repeated article would say to configure an allowlist and log final URLs.

The new surplus is policy closure over the request graph. A final URL log is too late to prevent a request, and a page-level interceptor may not govern traffic initiated by a developer tool, browser subsystem, local tool server, or special URL scheme. The policy contract has to name its producers, transitions, credential rules, and containment layer before allowed is a meaningful verdict.

Source map

Fresh primary evidence from the last seven days:

SourceDateContribution
Chrome DevTools MCP issue 25672026-08-12Shows a confirmed difference between guarded ordinary page traffic and Lighthouse secondary browser loads after redirects; documents the intentionally different DevTools loading path and the issue's limited threat model.
Browser Use 0.13.82026-08-16Identifies the current released baseline against which the hostless-scheme behavior was checked. The release notes do not claim a fix.
Browser Use pull request 54832026-08-17Rechecks the early data: and blob: allowance on 0.13.8/current main and contributes a regression matrix. It was closed as a duplicate and not merged.

Older background comes from the Chrome DevTools MCP security policy, which explicitly limits URL patterns to attached DevTools targets and recommends OS or VM controls for a full network sandbox, and the older Browser Use fix proposal, which remains open. Neither source supplies the production request-graph contract below.

Model the fetch as edges, not one URL

Represent each possible outbound action as an edge:

EdgeProducerPolicy question
Initial navigationagent or deterministic runnerIs this canonical URL, scheme, origin, path, task, and principal in scope?
Redirect hopbrowser, HTTP client, or toolMay the current destination select this next destination, with this credential class?
Page subresourcedocument, frame, worker, or service workerIs this resource type and destination allowed for this document generation?
Tool-owned secondary loadaudit, performance, accessibility, preview, or metadata toolDoes the tool use the same enforcement path, and is the load necessary for the approved task?
Hostless or nested schemebrowser or agent actionIs the scheme explicitly supported, and can its effective origin be validated?
Backend side requestMCP server, Node process, browser service, or extractorIs this egress covered outside the browser's target-level controls?

In both reports, initial_url_allowed: true says nothing about the later edge: one changes the producer, while the other changes the URL representation.

Redirects deserve their own decision. “Origin A may be visited” does not imply “A may choose arbitrary future destinations.” Unless both are explicit, evaluate each hop before sending it. A final-response check cannot unsend a request or retract attached credentials.

Schemes need policy, not exceptions. A hostname matcher cannot classify a representation with no ordinary hostname. Under an active domain policy, use unsupported_or_denied unless the scheme has explicit semantics, a validated effective origin, and a tested enforcement path.

Separate four enforcement layers

A reliable design uses several controls because no one hook sees every producer:

  1. Task admission. Bind the run to approved origins, paths, actions, principals, session class, data class, and tool set. This prevents a prompt from redefining scope.
  2. Pre-request edge policy. Canonicalize the candidate without resolving away evidence, classify its producer and scheme, evaluate every redirect before following it, and decide whether credentials may accompany that exact edge.
  3. Response consumption policy. Before an audit, extractor, model, or report consumes bytes, confirm that the response chain remained in policy. This limits downstream exposure but is not a substitute for pre-request enforcement.
  4. Independent egress containment. Apply network controls at the process, container, VM, proxy, or service boundary. Browser-target interception is useful, but the Chrome project's own security policy says it is not a complete sandbox.

Fail closed when the policy engine cannot observe an edge. An ungoverned secondary loader can be disabled, isolated, or marked policy_incomplete; it should not inherit the page navigation's verdict.

Credentials should narrow with the graph. Record a credential class rather than secret values, and decide independently whether cookies, authorization, client certificates, or account state may cross each redirect or tool-owned load. “The browser would normally attach it” is transport behavior, not authorization.

Log the policy decision before the network outcome

Keep one record per attempted edge:

  • run_id, edge_id, parent edge, task, principal class, and browser-context generation;
  • producer class: page, frame, worker, service worker, DevTools command, audit tool, MCP server, or backend client;
  • original candidate, canonicalization result, scheme, destination class, and matched policy rule ID;
  • transition type: initial, redirect, subresource, popup, secondary tool load, replay, or backend request;
  • credential class requested and credential class permitted, without storing secret values;
  • decision time, allow, deny, review, or unknown, and normalized reason;
  • request-start evidence, redirect status, response-consumption decision, and whether the destination observed a request in owned tests;
  • enforcement layer and version that made the decision;
  • final run verdict: closed_graph, denied_edge, policy_incomplete, containment_only, or unknown.

A final URL is still useful, but only as an outcome. The decision record proves whether the system evaluated the edge before traffic left.

An owned-fixture request-graph test

Build the canary with synthetic credentials and servers you control:

  1. Create allowed and denied fixture origins plus an egress recorder. Start with an allowed page and prove a direct denied subresource produces zero recorder hits.
  2. Test a same-policy redirect, an allowed-to-denied redirect, and a multi-hop chain. Require a decision before every hop and zero denied-origin hits.
  3. Run every enabled secondary feature—performance audit, metadata inspection, screenshot helper, accessibility tool, download handler, and API discovery—and inventory which component actually performs its requests.
  4. Exercise document, frame, dedicated-worker, and service-worker producers. Attribute each request to its producer instead of assuming page listeners cover them all.
  5. Test supported non-HTTP and hostless URL classes as data, not executable examples. Under an active domain policy, unknown or unvalidated classes must fail closed.
  6. Repeat with no credentials and with synthetic cookie-bearing state. Assert credential permission separately from URL permission.
  7. Deliberately disable the browser-level interceptor while retaining external egress containment. Verify the outer layer blocks the denied destination and reports which inner evidence went missing.
  8. Run the matrix after browser, agent framework, MCP server, audit bundle, proxy policy, or URL-matcher changes. Open upstream fixes are not deployment guarantees; test the exact build you operate.

The operator decision rule

When an agent run touches an unexpected destination, find the earliest edge that lacked a pre-request verdict. If the destination came from a redirect, do not treat the initial allow as inherited authority. If a tool-owned loader used another network path, classify the tool as uncovered until its path is governed or independently contained. If a scheme cannot be interpreted by the active policy, deny it rather than routing around the matcher. If only a post-response check caught the problem, treat the request as already sent and review credential exposure and artifact retention accordingly.

An allowlist is useful when it describes the traffic it actually controls. Browser agents turn one navigation into a graph of documents, redirects, tools, workers, protocols, and backend calls. Reliability comes from closing policy over that graph—and from saying policy_incomplete whenever the system cannot.