Streaming

Receive high-level request progress followed by the final result. Works on /api/v1/scrape and /api/v1/extract.

How It Works

Send Accept: application/x-ndjson instead of the default application/json. The response is HTTP 200 with content type application/x-ndjson, one JSON object per line:

A { "event": "trace", ... } line when public request progress changes
Exactly one final line: either { "event": "result", "status": ..., ... } or { "event": "error", "status": ..., "success": false, "error": {...} }

The final line carries a status field with the HTTP status a non-streamed call would have returned, plus the full normal response body (or error body) spread into the same line. The HTTP status of the streaming response itself is always 200 — auth, request-parsing and source-fetch errors happen before the stream starts, so those still come back as a normal JSON error response with their headers (a source's Retry-After included), not a stream line.

Public progress

The compatibility event name remains trace. A trace line's stage is loading or processing, with a fixed display label; there is no complete trace line, because completion is issued only after the result has been accounted for. The terminal result line reports it instead, and its metadata.trace ends with complete for an accepted answer. Progress contains no partial answer. Stages can be absent or coalesced; use the terminal result or error to determine the outcome.

To verify the signed receipt on a result line, remove its event and status members first: the signature covers the normal response body, not the stream framing. See Evidence.

curl

curl -N -X POST https://www.contrie.com/api/v1/scrape \
  -H "Content-Type: application/json" \
  -H "Accept: application/x-ndjson" \
  -H "Authorization: Bearer $CONTRIE_API_KEY" \
  -d '{
    "url": "https://news.ycombinator.com",
    "extract": "Get the top 5 stories with title, URL, and points"
  }'

The -N flag disables curl's output buffering so lines print as they arrive.

stream

{"event":"trace","stage":"loading","detail":"Loading source"}
{"event":"trace","stage":"processing","detail":"Processing request"}
{"event":"result","status":200,"success":true,"data":[{"title":"…","url":"…","points":119}],"metadata":{"qualityScore":80,"credits":1,"evidenceStatus":"signed","trace":[…,{"stage":"complete","detail":"Request complete"}],…}}

A refused answer still ends with a result line, carrying success: false and its status (422 for RESULT_NOT_ACCEPTED). An error line ends the stream when the request fails without a result after the stream has started, for example when usage accounting is unavailable:

stream (error)

{"event":"trace","stage":"loading","detail":"Loading source"}
{"event":"error","status":503,"success":false,"error":{"code":"SERVICE_UNAVAILABLE","message":"Processing is temporarily unavailable. Try again later."}}

JavaScript

const res = await fetch("https://www.contrie.com/api/v1/scrape", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    Accept: "application/x-ndjson",
    Authorization: `Bearer ${process.env.CONTRIE_API_KEY}`,
  },
  body: JSON.stringify({
    url: "https://news.ycombinator.com",
    extract: "Get the top 5 stories with title, URL, and points",
  }),
});

if (!res.headers.get("content-type")?.includes("application/x-ndjson")) {
  throw new Error("Request did not start: " + res.status);
}
const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = "";
let terminal = false;
function handle(line) {
  if (!line.trim() || terminal) return;
  const event = JSON.parse(line);
  if (event.event === "trace") {
    console.log(event.stage); // loading or processing
  } else if (event.event === "result" || event.event === "error") {
    terminal = true;
    if (!event.success) throw new Error(event.error?.message ?? "Not accepted");
    console.log(event.data ?? event.markdown);
  }
}
try {
  while (!terminal) {
    const { done, value } = await reader.read();
    buffer += decoder.decode(value, { stream: !done });
    const lines = buffer.split("\n");
    buffer = lines.pop() ?? "";
    for (const line of lines) handle(line);
    if (done) { handle(buffer); break; }
  }
  if (!terminal) throw new Error("Connection ended before the result");
} finally {
  await reader.cancel();
  reader.releaseLock();
}