Skip to main content
POST
Recrawl Document
For AI agents, use https://docs.tavus.io/openapi.yaml for the full HTTP API contract.

Authorizations

x-api-key
string
header
required

Path Parameters

document_id
string
required

Unique id of the crawl-backed website document to refresh. Use when the source site changed, you want to refresh content on a schedule, or retry after crawl or processing errors.

The document must be in ready or error (otherwise 409). It must have been created with a crawl configuration unless you supply crawl in the request body for this call.

The same document cannot be recrawled more than once within each 1-hour cooldown (429 if invoked too soon).

Body

application/json

Optional body. Omit entirely to reuse the crawl depth / max_pages stored from document creation, or include crawl to override those values for this run only.

After 202, status is typically recrawling until processing finishes. If you set callback_url when creating the document, webhooks report progress until the document returns to ready or error.

Poll Get Document for current status, crawl_count, and last_crawled_at.

Account limits: at most 5 concurrent crawls per user and at most 100 crawl-backed documents per user.

crawl
object

Optional depth and max_pages for this recrawl only; overrides stored crawl settings from document creation when provided. If omitted, the original crawl configuration is used.

What runs: the same starting URL as the original crawl, links followed within these limits, fresh page content processed, existing vectors replaced when processing completes, and crawl_count / last_crawled_at updated (see the 202 payload and Get Document while status is recrawling).

Response

Recrawl initiated successfully

document_id
string

Unique identifier for the document

Example:

"d8-5c71baca86fc"

document_name
string

Name of the document

Example:

"Company Website"

document_url
string

URL of the document

Example:

"https://example.com/"

status
enum<string>

After a successful recrawl request, typically recrawling until processing completes, then ready or error. Other values: started, processing.

Available options:
started,
processing,
ready,
error,
recrawling
Example:

"recrawling"

progress
integer | null

Processing progress as a percentage (0-100). Null when processing has not started or is complete.

Example:

null

error_message
string | null

Error code indicating why processing failed. Only present when status is error. Possible values include: file_download_failed, file_format_unsupported, file_size_too_large, file_empty, invalid_file_url, document_processing_failed, website_processing_failed, chunking_failed, embedding_failed, vector_store_failed, contact_support.

created_at
string

ISO 8601 timestamp of when the document was created

Example:

"2024-01-01T12:00:00Z"

updated_at
string

ISO 8601 timestamp of when the document was last updated

Example:

"2024-01-15T10:30:00Z"

callback_url
string

If set on Create Document, Tavus POSTs status updates here while this recrawl runs through completion.

Example:

"https://your-server.com/webhook"

tags
string[]

Array of document tags

Example:
crawl_config
object

The crawl configuration being used for the recrawl

crawled_urls
string[] | null

List of URLs from the previous crawl (will be updated when recrawl completes)

Example:
last_crawled_at
string | null

ISO 8601 timestamp of the previous crawl

Example:

"2024-01-01T12:05:00Z"

crawl_count
integer

Number of times the document has been crawled (will increment when recrawl completes)

Example:

1