Resolve Agent Identity Before Trusting Fetch Evidence

Agentic web retrieval is starting to carry more than a URL and a task prompt. A fetch run may soon include an asserted agent identity, a wallet-mediated credential, a customer-scoped permission, a delegated tool session, or a machine-readable claim about who is allowed to request what. That can be useful for compliant automation. It can also create a new class of quiet data-quality failures: the crawler trusts an identity label without recording how that label was resolved, which document version was used, what verification method was selected, or whether the credential context matched the target operation.

Two fresh W3C publications make the problem timely. On August 6, 2026, W3C announced that it invites implementations of Decentralized Identifier Resolution v1. The accompanying Candidate Recommendation, DID Resolution v1.0, describes the process of obtaining a DID document and accompanying metadata for a specific decentralized identifier. On July 30, 2026, W3C published a Group Note Draft for Verifiable Credentials Overview v1.1, with the draft overview document describing a family of mechanisms for cryptographically secure, privacy-respecting, machine-verifiable credentials on the Web.

Those documents are not crawler manuals. The production implication is narrower and practical: when a fetch run depends on agent identity, delegated authority, or machine-verifiable claims, identity resolution becomes fetch evidence. Operators need to log the resolver path, DID document metadata, verification method, credential context, and policy decision before treating a response as comparable with ordinary anonymous or first-party fetches.

This is not a guide to impersonate users, evade access controls, automate credential abuse, or force access to protected services. The safe frame is the opposite: use identity evidence only for permitted retrieval, contractual data exchange, auditability, and stopping when authorization is missing or ambiguous.

The repeated angle to avoid

Recent Better Fetch posts have already covered agent control points, comment-triggered agents, auth boundaries, session evidence, MCP/browser state, browser drift, resolver view, WAF windows, and runtime HTTP semantics. The repeated angle would be another broad claim that "agents need provenance" or "auth state matters."

The surplus here is a specific missing layer: resolution evidence. A session log can say which cookies were present. An agent log can say which tool was called. An auth classifier can say the page requested sign-in. None of those facts answer whether the identity used by an agent was resolved through the expected method, whether the DID document changed, whether a verification method was dereferenced correctly, whether credential metadata was fresh, or whether the target policy allowed that identity class to collect the requested representation.

Source map

Fresh primary evidence from the last seven days:

Older background context is the existing production pattern: many crawlers already maintain anonymous sessions, consent state, customer credentials, API keys, OAuth tokens, browser profiles, and audit logs. The new synthesis is that agentic systems may mix those with standards-based identity and credential artifacts. Without resolution evidence, two runs that appear to use the same agent can actually have different trust inputs.

Why identity resolution changes the fetch contract

A URL and a browser profile are no longer always enough to describe a fetch. In a credentialed data exchange, the representation returned by a site may depend on a chain of facts:

1. which agent, organization, or delegated service is claiming identity;
2. how that identifier was resolved;
3. which DID document or credential metadata was current at fetch time;
4. which verification method or service endpoint was selected;
5. which credential claims were presented, if any;
6. what the target's policy allowed for that identity class;
7. whether the response was public, personalized, partner-scoped, or denied.

If those facts are not in the fetch record, downstream consumers see only ordinary symptoms. A response body changes. A JSON field appears for one customer but not another. A screenshot contains a partner banner. A 403 looks like a bot block. A 200 contains a limited data view. A retry from a different worker produces different content because it resolved the identity through a different cache, method, or credential set.

The mechanism matters because identity artifacts are not static strings. Resolution can involve metadata, document versions, dereferencing, service endpoints, verification relationships, cache lifetime, network availability, and method-specific behavior. Even when every component behaves correctly, a fetch pipeline can mix incompatible evidence if it compares records from different resolution contexts.

What to log when identity affects a run

Treat identity-dependent fetching as a separate mode with explicit evidence. At minimum, the fetch record should include:

Do not log secrets, private keys, bearer tokens, full credential payloads, or personally sensitive claims in ordinary crawl telemetry. Store sensitive artifacts in the appropriate vault or audit system and put only references, hashes, and policy outcomes in the fetch row.

Failure modes that look like scraping problems

Identity-resolution failures rarely announce themselves as "identity-resolution failures" to the extractor. They usually masquerade as ordinary web-data drift.

A stale resolver cache can make an agent present an old service endpoint or verification method. The fetch still runs, but the target returns a limited representation. A method implementation upgrade can change dereferencing behavior, so one worker accepts a document shape that another rejects. A credential overview or schema change can alter claim interpretation, making a partner-scoped request look unauthorized. A policy engine can correctly deny a run because the presented identity is allowed to test availability but not collect full content. A privacy-preserving credential can intentionally avoid disclosing a claim that an extractor assumed would be visible.

The wrong response is to increase concurrency, rotate regions, modify headers, or escalate to a fuller browser without understanding the authorization result. That turns a governance problem into an access-control incident. The safer response is to classify the run: public fetch, credentialed fetch, limited representation, denied representation, or identity evidence incomplete.

A resolution-aware test plan

Before enabling identity-dependent fetches in production, run a small canary matrix:

1. Anonymous control: fetch the same URL with no identity artifact and record the public representation.
2. Known-good identity: fetch with a test identity whose permission is documented and stable.
3. Limited identity: fetch with an identity that should receive a reduced or partner-scoped representation.
4. Expired or revoked credential: verify the system stops and classifies the denial instead of retrying blindly.
5. Resolver unavailable: simulate resolver timeout and require a clear resolution_status, not a generic network error.
6. Document changed: rotate a test DID document or verification method and confirm the digest, method ID, and policy decision change in logs.
7. Extractor comparison: compare extracted fields only within the same representation_class unless the test explicitly expects a difference.

The goal is not to make identity a magic pass through every site. The goal is to prevent a credentialed or agent-scoped run from being mixed with anonymous public evidence without a label.

Operational decision rule

Use a simple rule in incident review: if a response was produced by an identity-dependent run, do not compare it with anonymous, cookie-only, or different-credential runs until resolution evidence matches at the policy-relevant level.

That means matching at least identity mode, resolver implementation family, resolution status, document metadata digest, verification method relationship, credential schema, authorization decision, and representation class. If any of those differ, the incident is not yet an extraction regression. It is an identity-context difference that may be expected, denied, stale, or unsafe to automate further.

For Better Fetch-style infrastructure, this fits the broader pattern: browser version, session state, DNS view, WAF windows, runtime HTTP semantics, and agent triggers are all part of the evidence chain. Identity resolution is another link. Once agents and credentials can shape what a site is willing to return, a trustworthy fetch system has to preserve how that identity was resolved before it publishes the data that came back.