Stack Overflow Scraper

Search public Stack Overflow questions and optionally fetch accepted or top answers through the Stack Exchange API, returning titles, tags, scores, views, answer counts, owner fields, timestamps, body text, and answer bodies.

Developer Toolsv0.1.1~2 credits/runApify StoreSource on GitHub

Overview

Stack Overflow Scraper gives agents a reliable path into developer Q&A signals without maintaining a custom crawler. Search by keyword, tag, or both, then optionally pull answer rows for the returned questions. The tool normalizes public Stack Exchange API records into compact question and answer payloads for technical research, developer support triage, content planning, and product opportunity discovery.

Last validated: Jul 3, 2026

Playground

Input

mode"search" | "answers"default: "search"

Run a question search or fetch answers for known question IDs.

sitestringdefault: "stackoverflow"

Stack Exchange API site slug. Defaults to stackoverflow.

sort"relevance" | "activity" | "creation" | "votes"default: "relevance"

Question search sort field.

querystring

Free-text Stack Overflow search query. Used in search mode.

taggedstring

Comma, space, or semicolon separated Stack Overflow tags. Used in search mode.

max_resultsintegerdefault: 5

Maximum questions returned in search mode.

question_idsstring

Numeric question IDs separated by commas, spaces, or semicolons. Required in answers mode.

include_answersbooleandefault: false

When true in search mode, fetch a bounded batch of top-voted answers for the returned questions.

max_answers_per_questionintegerdefault: 1

Maximum top-voted answers per question when fetching answers.

Output

modestringrequired

Mode used for the run

sitestringrequired

Stack Exchange site slug

countintegerrequired

Number of returned primary records

answersobject[]

Stack Overflow answer records

questionsobject[]

Stack Overflow question records

source_urlstringrequired

Fetched Stack Exchange API URL

answer_countinteger

Number of answer records returned when answers were requested

quota_remaininginteger

Stack Exchange API quota remaining as reported by the response

Examples

typescript-questions

{
  "mode": "search",
  "query": "typescript error",
  "tagged": "typescript",
  "max_results": 3
}

Use cases

Developer pain-point research

Search for recurring questions around an API, framework, error message, or product category and hand an agent structured titles, tags, scores, views, and body text.

Documentation gap analysis

Fetch public questions and top answers for a technology so docs teams can see where developers struggle and which explanations already work.

Support and DevRel monitoring

Monitor new or relevant Stack Overflow questions by keyword or tag and route high-signal issues into support, community, or DevRel workflows.

FAQ

Does Stack Overflow Scraper need a Stack Apps API key?

No. Version 0.1 uses public Stack Exchange API endpoints through Better Fetch and does not require a Stack Apps key or authenticated Stack Overflow account.

Can it fetch every answer and comment?

Not yet. Version 0.1 searches questions and can fetch a bounded batch of top-voted answers. Comments, all-answer pagination, profile enrichment, and cross-site fan-out should be added as separate validated slices.

Can I search sites other than Stack Overflow?

The input includes a site parameter for Stack Exchange site slugs, but the page and benchmark are optimized for Stack Overflow. Broader Stack Exchange coverage should be verified per site before being promoted.

Use it anywhere

MCP (Claude, Cursor, any client)

# Add the Better Fetch MCP connector (or paste the URL into
# Claude → Settings → Connectors → Add custom connector):
claude mcp add --transport http better-fetch https://betterfetch.co/api/mcp \
  --header "Authorization: Bearer bf_your_key_here"

# Then ask for the tool by name: stack_overflow_scraper

REST

curl -sS -X POST "https://betterfetch.co/api/tools/stack_overflow_scraper/run" \
  -H "Authorization: Bearer bf_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{"input": {"mode":"search","query":"typescript error","tagged":"typescript","max_results":3}}'

Run locally

git clone https://github.com/better-fetch/stack-overflow-scraper && cd stack-overflow-scraper && npm i
BETTER_FETCH_API_KEY=bf_your_key_here npx bf-tool run --input '{"mode":"search","query":"typescript error","tagged":"typescript","max_results":3}'