Skip to main content
POST
General-purpose URL scrape

When to use

  • Research / planning before an agent exists
  • One-off page extraction without creating a kb_ingest crawler job
For agent KB ingest crawls, keep using POST /workspaces/{workspaceId}/crawler/jobs (type: kb_ingest when toAgentId is set).

Request

Response

A scrape_jobs/{jobId} row is recorded with type: "general" and agentId: null for auditability.

Authorizations

Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json
url
string<uri>
required

Response

Successful response

success
boolean
required
content
string
required
images
string[]
required
jobId
string