better fetch

Overview

Better Fetch fetches URLs through a real, stealth Chromium browser. Use it when a target needs JavaScript rendering, regional routing, sticky sessions, screenshots, or Cloudflare clearance. The base URL is https://api.betterfetch.co and endpoints are versioned under /v1.

Every request runs in a real browser profile — not incognito. Requests with a session reuse a warm, pooled context keyed by a stable fingerprint; if a response looks blocked and a residential exit is in play, the service automatically retries on a fresh session and escalates to a headed browser.

Authentication

All fetch requests require a bearer token. Create and revoke keys on the API keys page. Keep keys server-side — never in browser JavaScript, query strings, or shared logs.

Authorization: Bearer <your-api-key>
Content-Type: application/json

Keys require an active subscription and are metered against your plan's monthly quota. A call is counted when it is accepted, regardless of the fetch outcome.

POST /v1/fetch

POST/v1/fetch

Fetch a URL through the browser and return rendered page data: target status, final URL, title, rendered HTML, headers, timing, block classification, and optionally the raw body, parsed JSON, captured network calls, or a screenshot. Unknown request fields are rejected with 400.

curl -sS -X POST "https://api.betterfetch.co/v1/fetch" \
  -H "Authorization: Bearer $BETTER_FETCH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "wait_until": "domcontentloaded",
    "timeout_ms": 60000
  }'

Request fields

Only url is required. Everything else has a sensible default.

urlstringdefault: required

HTTP or HTTPS URL to fetch.

wait_untilstringdefault: "load"

Browser navigation state to wait for: load, domcontentloaded, networkidle, or commit. Prefer domcontentloaded for fast document and API responses.

wait_selectorstring

CSS selector to wait for after navigation. Use this for pages that render content after the initial load.

wait_msnumber

Extra fixed wait after navigation, in milliseconds. Capped by timeout_ms. Use only when there is no reliable selector.

timeout_msnumberdefault: 30000

Navigation and selector timeout in milliseconds. Maximum 120000. Keep it as low as practical.

return_response_textbooleandefault: false

Include the raw navigation response body in body_text. JSON responses include it automatically, but setting this is robust even when the target sends a wrong content type.

return_cf_clearancebooleandefault: false

Return Cloudflare cf_clearance token data when the browser receives that cookie. When false, cookies are not read or returned.

capture_networkbooleandefault: false

Capture matching browser network calls and return them in network. Defaults to XHR/fetch only — useful for API discovery and debugging.

network_resource_typesstring[]default: ["xhr","fetch"]

Playwright resource types to capture, e.g. xhr, fetch, document, script, or websocket. Keep this narrow for most workloads.

network_include_bodiesbooleandefault: true

Include capped response bodies for captured network responses.

network_include_headersbooleandefault: false

Include request and response headers for captured entries. Off by default because headers can contain cookies, bearer tokens, or other secrets.

network_max_entriesnumberdefault: 100

Maximum matching network entries to return. Range 1–500.

network_max_body_bytesnumberdefault: 262144

Maximum bytes kept from each captured response body. Range 0–1048576.

screenshotbooleandefault: false

Include a PNG screenshot encoded as base64. Screenshots can make responses much larger, so request them only when needed.

full_pagebooleandefault: false

Capture the full scrollable page when screenshot is true.

countrystring

Two-letter country code for managed regional routing, e.g. us, gb, de, au, ca. Use when the target response depends on geography.

sessionstring

Sticky routing key. Reuse it to keep the same residential exit IP (and a warm browser context) for roughly 10 minutes; use distinct values to rotate independently.

geoipbooleandefault: true when routed

Align browser timezone, locale, and WebRTC IP with the proxy exit IP when regional routing is used. Send false to skip the exit-IP lookup.

localestringdefault: automatic

Browser locale, e.g. en-GB. Overrides the GeoIP-derived locale. Works with or without regional routing.

timezonestringdefault: automatic

Browser timezone, e.g. Europe/London. Overrides the GeoIP-derived timezone.

user_agentstringdefault: browser default

Custom user agent applied to the browser context. Usually leave unset — it forms part of a session's warm-context identity.

extra_headersobjectdefault: {}

Additional HTTP headers applied inside the browser context.

humanizebooleandefault: auto

Human-like mouse, keyboard, and scroll behavior. Defaults to true for HTML page fetches and false for JSON/API calls. Set explicitly to override.

Response fields

ok: true means Better Fetch completed the browser request — check status and blockedfor the target's verdict.

okboolean

true when Better Fetch completed the browser request — not whether the target accepted it. Check status and blocked for the target's verdict.

statusnumber | null

HTTP status from the target navigation response.

final_urlstring

Final browser URL after redirects.

titlestring

Page title after rendering. Empty string if unavailable mid-navigation.

htmlstring

Rendered DOM HTML from the browser.

body_textstring | null

Raw navigation response body when requested or when the target response is JSON.

json_parse_okboolean

Whether body_text parsed as JSON.

jsonany | null

Parsed JSON payload when json_parse_ok is true; otherwise null.

headersobject

Response headers from the target navigation response (string values).

screenshot_b64string | null

Base64-encoded PNG when screenshot is true; otherwise null.

cf_clearancestring | null

Cloudflare cf_clearance token value when return_cf_clearance is true and the target issued it.

cf_clearance_cookieobject | null

Storage-ready cookie metadata (name, value, domain, path, expires, httpOnly, secure, sameSite) when requested and present.

cf_clearance_sessionstring | null

The session that produced the clearance result. Blocked retries may rotate to a fresh session before the final result.

networkarray

Captured browser network entries when capture_network is true. Otherwise omitted.

blockedboolean

true when the response looks like a bot wall or unsolved challenge — even when the target returns HTTP 200. A solved page that merely carries Turnstile DOM is not blocked.

headedboolean

Whether the attempt that produced this result ran a headed browser (set on escalated retries).

pooledboolean

true when served from a warm pooled context (session requests); false for sessionless ephemeral contexts.

attemptsnumber

Total attempts including the first. Greater than 1 means the service retried after a block on a fresh session.

timing_msnumber

Time spent inside the fetch request (final attempt).

Examples

Fetch rendered HTML

The basic call: navigate with a real browser and return the rendered DOM.

curl -sS -X POST "https://api.betterfetch.co/v1/fetch" \
  -H "Authorization: Bearer $BETTER_FETCH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "wait_until": "domcontentloaded",
    "timeout_ms": 60000
  }' | jq '.status, .title, .final_url'

Fetch from a region

Use country when the target response depends on geography, and session when several requests should look like the same visitor from that region.

curl -sS -X POST "https://api.betterfetch.co/v1/fetch" \
  -H "Authorization: Bearer $BETTER_FETCH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "country": "gb",
    "session": "example-gb",
    "wait_until": "domcontentloaded",
    "timeout_ms": 60000
  }'

Session names may be friendly strings like example-gb; the service normalizes them before passing them to the proxy provider.

Fetch a JSON API

Set return_response_text for JSON endpoints. Better Fetch includes the raw body in body_text, attempts to parse it, and returns the parsed value in json.

curl -sS -X POST "https://api.betterfetch.co/v1/fetch" \
  -H "Authorization: Bearer $BETTER_FETCH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://api.example.com/data",
    "return_response_text": true,
    "extra_headers": { "Accept": "application/json" },
    "timeout_ms": 60000
  }' | jq 'if .json_parse_ok then .json else { status, blocked, body_text } end'

If json_parse_ok is false, inspect status, blocked, headers, body_text, and html — the target may have returned HTML, a challenge page, or non-JSON text.

Wait for rendered content

For client-rendered pages, wait for a selector instead of adding a long fixed delay.

curl -sS -X POST "https://api.betterfetch.co/v1/fetch" \
  -H "Authorization: Bearer $BETTER_FETCH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://app.example.com/items/123",
    "wait_until": "domcontentloaded",
    "wait_selector": "#content",
    "timeout_ms": 90000
  }'

Capture network calls

Capture the XHR/fetch calls a page makes while rendering — the fastest way to discover a site's internal APIs.

curl -sS -X POST "https://api.betterfetch.co/v1/fetch" \
  -H "Authorization: Bearer $BETTER_FETCH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://app.example.com/items/123",
    "wait_until": "networkidle",
    "timeout_ms": 90000,
    "capture_network": true,
    "network_max_entries": 50
  }' | jq '.network[] | { method, url, status, json }'

Enable network_include_headers only when you need it — headers can contain credentials.

Capture a screenshot

Return a base64-encoded PNG of the rendered page.

curl -sS -X POST "https://api.betterfetch.co/v1/fetch" \
  -H "Authorization: Bearer $BETTER_FETCH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "screenshot": true,
    "full_page": true,
    "wait_until": "domcontentloaded"
  }' | jq -r '.screenshot_b64'

Collect a Cloudflare clearance token

Return the cf_clearance cookie after rendering the target page, with storage-ready metadata.

curl -sS -X POST "https://api.betterfetch.co/v1/fetch" \
  -H "Authorization: Bearer $BETTER_FETCH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://www.example.com/",
    "country": "us",
    "session": "clearance-us",
    "wait_until": "domcontentloaded",
    "timeout_ms": 90000,
    "return_cf_clearance": true
  }' | jq '{ status, blocked, cf_clearance, cf_clearance_cookie }'

cf_clearance is null when the target doesn't issue the cookie or the challenge remains unsolved. Store cf_clearance_session too — retries may rotate sessions.

Errors

Every non-2xx response uses a single JSON envelope. Switch on the stable error code, not the message.

{ "ok": false, "error": "unauthorized", "message": "invalid or missing bearer token", "status": 401 }
StatusCodeMeaning
400bad_requestInvalid JSON, unknown request field, missing or non-HTTP url, or an out-of-range parameter.
401unauthorizedMissing, incorrect, or revoked bearer token.
402payment_requiredValid key but no active subscription.
429quota_exceededMonthly call quota exhausted; resets at the next billing cycle. A call is counted when accepted, regardless of fetch outcome.
502fetch_failedBrowser launch, navigation, proxy, or target fetch failed. The message includes the underlying detail.
504timeoutRequest timed out at the API layer.

Target errors are different from API errors: if Better Fetch returns HTTP 200 and the JSON contains "status": 403 or "blocked": true, the API worked and the target denied the browser request.

GET /v1/health

GET/v1/health

Liveness check. No authentication required.

curl -sS "https://api.betterfetch.co/v1/health"

{ "ok": true, "version": "0.3.30", "proxy_configured": true, "pool_size": 2, "pool_max": 8 }

Tips

  • Prefer wait_until: "domcontentloaded" for fast document and API responses, and wait_selector for pages that render after load.
  • Pass a session for hard targets: it enables a warm pooled context, a stable fingerprint, and automatic block-retry with headed escalation.
  • Requests with neither country nor session skip block-retry — there is no residential exit to rotate.
  • Check blocked to detect bot walls even when the target returns HTTP 200; attempts > 1 means the service already retried on fresh exits.
  • Leave user_agent, locale, and timezone unset unless required — they form part of a session's warm-context identity, so changing them splits the warm pool.
  • First requests after a deploy can be slower while Chromium starts; reused session requests are served from a warm pool.
  • Do not send raw proxy credentials. Regional routing is selected with country and session; credentials are managed by the service.

Looking for the raw spec? The OpenAPI 3.1 document and generated reference are served by the API itself.