Streaming

Get each pipeline stage as it completes instead of waiting for the whole request to finish. Works on /api/v1/scrape and /api/v1/extract.

How It Works

Send Accept: application/x-ndjson instead of the default application/json. The response is HTTP 200 with content type application/x-ndjson, one JSON object per line:

A { "event": "trace", ... } line as each pipeline stage completes
Exactly one final line: either { "event": "result", "status": ..., ... } or { "event": "error", "status": ..., "success": false, "error": {...} }

The final line carries a status field with the HTTP status a non-streamed call would have returned, plus the full normal response body (or error body) spread into the same line. The HTTP status of the streaming response itself is always 200 — auth and request-parsing errors happen before the stream starts, so those still come back as a normal JSON error response, not a stream line.

Trace Stages

Stage	Meaning
heuristic	Structured data (JSON-LD, microdata, Open Graph) found on the page, if any
markdown	Page reduced to structure-preserving markdown — this is what the model reads
classify	simple / complex / visual / adversarial
route	Model chosen; free-tier first attempts are hedged across two models
extract	Tokens used, or `"failed: ..."`
validate	Quality score and grounding, or why it fell below 70
escalate	The next model, when quality falls short (max 2 escalations)

curl

curl -N -X POST https://contrie.com/api/v1/scrape \
  -H "Content-Type: application/json" \
  -H "Accept: application/x-ndjson" \
  -H "Authorization: Bearer $CONTRIE_API_KEY" \
  -d '{
    "url": "https://news.ycombinator.com",
    "extract": "Get the top 5 stories with title, URL, and points"
  }'

The -N flag disables curl's output buffering so lines print as they arrive.

stream

{"event":"trace","stage":"heuristic","detail":"no structured data found"}
{"event":"trace","stage":"markdown","detail":"page reduced to 4.1KB markdown"}
{"event":"trace","stage":"classify","detail":"simple"}
{"event":"trace","stage":"route","detail":"hedged","tier":"free"}
{"event":"trace","stage":"extract","model":"minimax/minimax-m3:free","detail":"3120 in / 410 out","ms":1890}
{"event":"trace","stage":"validate","detail":"quality 92, grounding 0.95","quality":92}
{"event":"result","status":200,"success":true,"data":{"stories":[...]},"metadata":{...}}

JavaScript

const res = await fetch("https://contrie.com/api/v1/scrape", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    Accept: "application/x-ndjson",
    Authorization: `Bearer ${process.env.CONTRIE_API_KEY}`,
  },
  body: JSON.stringify({
    url: "https://news.ycombinator.com",
    extract: "Get the top 5 stories with title, URL, and points",
  }),
});

const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = "";

while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });

  const lines = buffer.split("\n");
  buffer = lines.pop(); // keep the last, possibly incomplete line

  for (const line of lines) {
    if (!line) continue;
    const event = JSON.parse(line);
    if (event.event === "trace") {
      console.log(`[${event.stage}] ${event.detail}`);
    } else if (event.event === "result") {
      console.log("done:", event.data);
    } else if (event.event === "error") {
      console.error(event.error.code, event.error.message);
    }
  }
}