Skip to main content
Ingestion turns a raw file into searchable, grounded knowledge. This page covers the developer flow — requesting an upload URL, uploading bytes, and polling status — plus what happens in the background pipeline and how quotas apply.

​
Upload flow

Uploads use a two-step presigned pattern so file bytes go straight to object storage, and processing begins automatically once the file lands.
1

Request an upload URL

POST /documents/presign returns a document_id, an upload_url, and an expiry.
2

Upload the file

PUT the raw file bytes to upload_url before it expires.
3

Processing starts automatically

Once the bytes land, the background pipeline begins on its own — no extra call needed.
4

Poll status

GET /documents/status?document_id=... until the status is processed.
The upload_url is time-limited. Upload the file promptly after presigning; if the URL expires before you upload, request a new one.

​
The processing pipeline

Once a file lands, it moves through a background pipeline that prepares it for grounded retrieval. Each stage runs on background workers, so large files process without blocking your integration.
1

Parse

Text is extracted from the source. PDFs — including scanned documents via OCR — audio files (transcribed to text), and Office and structured documents are all supported.
2

Summarize

A concise summary is generated so agents can see at a glance what the document covers.
3

Chunk

Content is split into content-aware chunks that respect the document’s structure, which improves retrieval quality.
4

Embed

Each chunk is embedded into a vector representation for semantic search.
5

Index

Chunks are indexed into the knowledge base’s private vector partition, isolated from every other knowledge base.

​
Status values

The status field tracks a document through ingestion:
Wait until a document is processed before querying against it — chunks are searchable only after indexing completes. If a document ends up failed, or you abandon an upload at pending_upload, remove it: incomplete documents still count toward your quota.

​
Other ingestion methods

Beyond file uploads, you can bring in web content and manage existing documents:
POST /documents/upload-url fetches a URL and ingests its readable text as a document.
POST /documents/upload-url-site discovers URLs from a site. It does not ingest those pages; submit each selected URL separately to POST /documents/upload-url.
POST /documents/:id/reprocess runs a document through the pipeline again — useful after a failed status.
GET /documents/:id/download returns a presigned URL to retrieve the original file.
DELETE /documents/:id soft-removes the document from search and cleans up its stored bytes and index entries.

​
Quotas

Uploads are checked against your tenant’s limits before they are accepted:
  • Document count — the number of documents the tenant may hold.
  • Storage — the total bytes the tenant may store.
When a limit is exceeded, the API returns a clear 403 so you can surface the reason and prompt the user to free up space.
failed and abandoned pending_upload documents count toward your document and storage quotas until removed. Clean them up as part of your ingestion flow to avoid hitting limits unexpectedly.

​
Endpoint summary

​
Next steps

Documents in a knowledge base

How documents power the RAG corpus.

Documents API

Full request and response schemas.

Retrieval & grounding

How indexed chunks become cited answers.

Quickstart

Upload a document and run a chat end to end.