POST/api/v1/scrape

Scrape

Fetch a URL and extract structured data using AI. The API handles fetching, rendering to markdown, and extraction in a single request.

Try it without a key

A small set of sample URLs run without an API key, in JSON format only, and don't count against any quota:

https://news.ycombinator.com

https://example.com

https://www.producthunt.com

Any other URL requires a key from /dashboard/keys. Try sample URLs live in the playground.

Parameters

Parameter	Type	Required	Description
url	string	Required	A public HTTP/HTTPS URL. Private and internal addresses are rejected, including when reached via a redirect.
extract	string	Optional	Natural language description of what to extract. Max 4,000 characters.
schema	object	Optional	A JSON Schema subset to enforce the output shape: types `object`, `array`, `string`, `number`, `integer`, `boolean`, `null`; keywords `properties`, `required`, `items`, `enum`, `description`. Max 10 KB.
format	string	Optional	`json` (default), `markdown`, or `both`. `markdown` runs no model and returns `data: null`.
options	object	Optional	`timeout` — page fetch timeout in ms, 1000–30000 (default 15000).

Request Example

curl

curl -X POST https://contrie.com/api/v1/scrape \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ck_live_your_key_here" \
  -d '{
    "url": "https://news.ycombinator.com",
    "extract": "Get the top 5 stories with title, URL, and points",
    "options": {
      "timeout": 20000
    }
  }'

Response

The HTTP status is always 200 for a completed extraction — success reflects whether the result cleared the quality bar, not whether the request worked. Branch on metadata.qualityScore rather than the status code.

200 OK

{
  "success": true,
  "data": {
    "stories": [
      {
        "rank": 1,
        "title": "Show HN: I built an open-source AI code editor",
        "url": "https://techcrunch.com/2024/01/ai-code-editor",
        "points": 847
      }
    ]
  },
  "metadata": {
    "url": "https://news.ycombinator.com",
    "extractionMethod": "ai",
    "model": "minimax/minimax-m3:free",
    "qualityScore": 92,
    "grounding": 0.95,
    "escalations": 0,
    "latencyMs": 2340,
    "costUsd": 0,
    "tokensUsed": { "input": 3120, "output": 410 },
    "trace": [
      { "stage": "heuristic", "detail": "no structured data found" },
      { "stage": "markdown", "detail": "page reduced to 4.1KB markdown" },
      { "stage": "classify", "detail": "simple" },
      { "stage": "route", "detail": "hedged", "tier": "free" },
      { "stage": "extract", "model": "minimax/minimax-m3:free", "detail": "3120 in / 410 out", "ms": 1890 },
      { "stage": "validate", "detail": "quality 92, grounding 0.95", "quality": 92 }
    ],
    "cached": false,
    "credits": 1
  }
}

Key Response Fields

Field	Meaning
extractionMethod	`heuristic`, `ai`, `hybrid`, or `markdown`.
qualityScore	0–100. Schema validity (40%) + completeness (40%) + sanity/grounding (20%). Threshold for `success` is 70.
grounding	0–1, the fraction of extracted string/number values found verbatim in the page text. Undefined when nothing was checkable. Below 0.5 checkable, the score is capped at 60 so the pipeline escalates instead of returning invented data.
escalations	How many times the pipeline moved to a stronger model tier. Max 2.
costUsd	Model list-price spend for the call. 0 on free-tier models.
trace	One entry per pipeline stage (`heuristic`, `markdown`, `classify`, `route`, `extract`, `validate`, `escalate`).
heuristicData	JSON-LD, microdata, Open Graph, or Twitter Card data found on the page, if any.
cached	Always `false` for live extractions. Sample URLs return recorded runs instead of live calls.

Streaming

Send Accept: application/x-ndjson to receive each pipeline stage as it completes instead of waiting for the final result. See Streaming for the line format and examples.

Error Responses

Errors return { success: false, error: { code, message } }. Full descriptions are on the Error Codes page.

Status	Code
400	INVALID_REQUEST, INVALID_URL
401	UNAUTHORIZED
422	FETCH_FAILED, RENDER_REQUIRED
429	RATE_LIMITED, PAGE_LIMIT_REACHED
503	SERVICE_UNAVAILABLE, MODELS_UNAVAILABLE
500	EXTRACTION_FAILED