Better Fetch

· Paul Crossland

The Agent Saw It. The Accessibility Tree Did Not.

A fresh Playwright mismatch shows why agent snapshots need visibility, accessibility, and actionability labels before extraction.

A browser agent can be shown text that its role-based browser tools cannot find. The text may exist in the DOM and even remain visually rendered, while aria-hidden="true" removes it from the accessibility tree. If the agent's diagnostic snapshot presents that text without its hidden state, a reasonable next action can still be wrong: query a heading that is not exposed, extract a stale duplicate, or conclude that the automation library lost an element.

A fresh Playwright report demonstrates the contradiction. In issue 42223, opened August 12, locator.ariaSnapshot() correctly excluded a subtree marked aria-hidden="true", but the error-context.md page snapshot included its heading and paragraph. The report explains that this misled both a human and coding agents because role-based locators could not find content that the failure artifact appeared to expose.

The distinction is intentional, not simply a broken accessibility query. An open Playwright pull request from August 16 says the AI-mode snapshot includes ARIA-hidden elements when they are visually visible. It proposes adding an [aria-hidden] annotation while leaving default snapshots unchanged. The patch is current design evidence, not a released guarantee; operators should test the exact Playwright build they deploy.

The practical thesis is: an AI-oriented page snapshot is a composite debugging representation, not proof of accessibility, visibility, actionability, or extraction validity, so every important claim should retain the semantic plane that made it observable.

This is not a reason to collect intentionally hidden data or work around a site's controls. Operate only on public or properly authorized pages. Login surfaces, challenges, payment boundaries, denied interactions, and publisher restrictions remain stop conditions regardless of what appears in a DOM or diagnostic artifact.

The repeated angle to avoid

Recent Better Fetch posts already cover accessibility sampling, browser fidelity tiers, iframe completeness, control-plane telemetry, session state, and MCP evidence planes. The repeated article would say that accessibility trees are useful or that agents need better logs.

The new surplus is a snapshot contract. A single agent-facing representation can deliberately combine nodes from different semantic planes. Annotation can make that combination legible, but the fetch system still has to decide which plane is acceptable for navigation, interaction, extraction, and publication.

Source map

Fresh primary evidence from the last seven days:

SourceDateContribution
Playwright issue 422232026-08-12Reproduces the mismatch between ariaSnapshot(), role-based queries, and the AI-oriented failure snapshot in Playwright 1.62.1.
Playwright pull request 422682026-08-16Documents that AI mode intentionally retains visually visible ARIA-hidden nodes and proposes an explicit annotation. It remains open.

Older background comes from the WAI-ARIA definition of aria-hidden, which excludes an element and its descendants from the accessibility tree without necessarily hiding them visually, and Playwright's ARIA snapshot documentation. Neither says that a composite AI snapshot is suitable as an extraction contract. That decision belongs to the data system.

One node can have five different states

Do not reduce a browser node to present: true. Record the plane answering each question:

PlaneQuestionA positive result does not prove
DOMIs the node in the current document generation?That it is rendered, exposed, current, or permitted to collect
VisualDoes it produce visible pixels in the intended viewport?That assistive APIs expose it or an action can reach it
AccessibilityDoes the accessibility tree expose its role, name, and state?That it is unobscured, enabled, or the source of a business fact
ActionabilityCan the approved automation path focus, hit-test, and operate it now?That its text is authoritative or publication-ready
Data provenanceWhich response, script state, or application transition supplied the value?That the user-facing representations agree with it

aria-hidden creates a legitimate split between the visual and accessibility planes. Other mechanisms create different splits: hidden, CSS visibility, off-screen responsive copies, inert, disabled controls, overlays, detached frames, stale virtualized rows, and DOM nodes left behind during a transition.

An AI snapshot optimized for diagnosis may include more context than a role-based query. That is useful. It becomes dangerous only when the consumer silently upgrades "included in the snapshot" to "accessible to the user," "safe to click," or "valid to publish."

Where composite snapshots fail in production

A modal is a common example. While the dialog is active, an application may mark the background root aria-hidden="true". A composite snapshot can show both the dialog and background navigation. An agent might repeatedly target the background control, interpret locator failure as flakiness, or extract a background value that belongs to the pre-dialog state.

Responsive and hydrated applications create another failure. Desktop and mobile variants may coexist in the DOM, with only one intended for the current state. A snapshot can contain two prices, labels, or pagination controls. Choosing the first textual match turns representation ambiguity into a false fact.

The incident loop can then amplify the mistake. A coding agent reads error-context.md, sees the text, changes from a role locator to a broad text or CSS locator, and makes the test pass by selecting a node that the original user-facing contract deliberately excluded. The repair removes the symptom while weakening the evidence standard.

Annotations help humans and agents explain this conflict. They do not decide whether hidden content is stale, decorative, temporarily unavailable, incorrectly authored, or necessary to the data product. That requires an explicit acceptance policy.

Put a plane verdict on each important field

For browser-backed extraction, attach a compact observation block to every high-value field or evidence group:

  • document_generation, frame role, origin class, and snapshot timestamp;
  • snapshot producer, mode, schema version, automation version, and browser build;
  • DOM presence, computed visibility, bounding-box and viewport intersection;
  • accessibility inclusion, role, accessible name, and inherited aria-hidden source;
  • hidden, inert, disabled, occluded, detached, and hit-test outcomes;
  • provenance pointer to the network response, DOM region, structured state, or screenshot crop;
  • agreement result across required planes;
  • final verdict: meets_contract, visual_only, accessibility_only, stale_or_duplicate, policy_stop, or unknown.

Do not broadly log hidden text, credentials, private account content, or full snapshots. Keep metadata for all runs and retain bounded artifacts only for authorized canaries and incidents.

The extraction contract should name its required plane. Article text may be acceptable when visibly rendered even if no interaction is required. A button used to load more results needs actionability and a post-action state transition. A claim that a label is available to assistive technology needs accessibility evidence. A price may require agreement between rendered text and its data-bearing response. No universal plane wins every case.

An owned-fixture snapshot test

Build a small fixture rather than learning these semantics on third-party pages:

  1. Render visible accessible text, visible aria-hidden text, CSS-hidden text, an inert control, an occluded button, and desktop/mobile duplicates with distinct markers.
  2. Capture the normal ARIA snapshot, the AI-oriented snapshot, DOM state, a screenshot, hit-test results, and the data-bearing response.
  3. Assert which markers appear in each representation and whether hidden states are explicitly labelled. Never infer semantics from snapshot membership alone.
  4. Open a modal that applies aria-hidden to the background. Verify that background fields become ineligible for action and that pre-modal values are not attributed to the modal generation.
  5. Deliberately fail a role locator. Give the resulting artifact to the same agent workflow used in production and reject any suggested repair that merely broadens selection to an ineligible node.
  6. Repeat after Playwright, browser, snapshot-schema, agent, or prompt changes. The August 16 patch is open, so require the annotation in a canary before depending on it.
  7. Fail closed when a required plane cannot be observed. Store unknown instead of choosing whichever duplicate text is easiest to locate.

The operator decision rule

When an agent says an element exists but an action or extractor cannot find it, compare planes before changing selectors. If it is DOM-present and visually rendered but ARIA-hidden, decide whether the task requires visual evidence, accessibility semantics, or interaction; do not pretend those contracts are equivalent. If duplicate nodes disagree, use document generation and data provenance to identify the intended representation. If a locator repair crosses from an eligible node to hidden, stale, or policy-bound content, reject the repair.

A richer agent snapshot can improve diagnosis because it preserves context that a strict accessibility tree omits. Reliability comes from keeping that context labelled. The browser did not produce one universal truth: it produced DOM, pixels, accessibility semantics, action state, and network provenance. Production fetch systems should preserve those distinctions until the data contract—not the agent's snapshot—decides what the result proves.