Skip to main content
Ingestion turns a raw file into searchable, grounded knowledge. This page covers the developer flow — requesting an upload URL, uploading bytes, and polling status — plus what happens in the background pipeline and how quotas apply.

​
Upload flow

Uploads use a two-step presigned pattern so file bytes go straight to object storage, and processing begins automatically once the file lands.
1

Request an upload URL

POST /documents/presign returns a document_id, an upload_url, and an expiry.
2

Upload the file

PUT the raw file bytes to upload_url before it expires.
3

Processing starts automatically

Once the bytes land, the background pipeline begins on its own — no extra call needed.
4

Poll status

GET /documents/status?document_id=... until the status is processed.
# 1. Request an upload URL
curl -X POST "$AUTOSAGE_URL/documents/presign" \
  -H "Authorization: Bearer $AUTOSAGE_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "kb_id": "KB_ID",
    "tenant_id": "TENANT_ID",
    "filename": "handbook.pdf",
    "mime_type": "application/pdf",
    "size_bytes": 482913
  }'

# 2. Upload the bytes to the returned upload_url
curl -X PUT "UPLOAD_URL" \
  -H "Content-Type: application/pdf" \
  --data-binary @handbook.pdf

# 3. Poll until processed
curl "$AUTOSAGE_URL/documents/status?document_id=DOCUMENT_ID" \
  -H "Authorization: Bearer $AUTOSAGE_KEY"
The upload_url is time-limited. Upload the file promptly after presigning; if the URL expires before you upload, request a new one.

​
The processing pipeline

Once a file lands, it moves through a background pipeline that prepares it for grounded retrieval. Each stage runs on background workers, so large files process without blocking your integration.
1

Parse

Text is extracted from the source. PDFs — including scanned documents via OCR — audio files (transcribed to text), and Office and structured documents are all supported.
2

Summarize

A concise summary is generated so agents can see at a glance what the document covers.
3

Chunk

Content is split into content-aware chunks that respect the document’s structure, which improves retrieval quality.
4

Embed

Each chunk is embedded into a vector representation for semantic search.
5

Index

Chunks are indexed into the knowledge base’s private vector partition, isolated from every other knowledge base.

​
Status values

The status field tracks a document through ingestion:
StatusMeaning
pending_uploadPresigned; waiting for the file bytes to be uploaded.
queuedBytes received; queued for processing.
processingMoving through parse → summarize → chunk → embed → index.
processedFully indexed and ready to query.
failedProcessing did not complete.
Wait until a document is processed before querying against it — chunks are searchable only after indexing completes. If a document ends up failed, or you abandon an upload at pending_upload, remove it: incomplete documents still count toward your quota.

​
Other ingestion methods

Beyond file uploads, you can bring in web content and manage existing documents:
POST /documents/upload-url fetches a URL and ingests its readable text as a document.
POST /documents/upload-url-site discovers URLs from a site. It does not ingest those pages; submit each selected URL separately to POST /documents/upload-url.
POST /documents/:id/reprocess runs a document through the pipeline again — useful after a failed status.
GET /documents/:id/download returns a presigned URL to retrieve the original file.
DELETE /documents/:id soft-removes the document from search and cleans up its stored bytes and index entries.

​
Quotas

Uploads are checked against your tenant’s limits before they are accepted:
  • Document count — the number of documents the tenant may hold.
  • Storage — the total bytes the tenant may store.
When a limit is exceeded, the API returns a clear 403 so you can surface the reason and prompt the user to free up space.
failed and abandoned pending_upload documents count toward your document and storage quotas until removed. Clean them up as part of your ingestion flow to avoid hitting limits unexpectedly.

​
Endpoint summary

ActionMethod & path
Request an upload URLPOST /documents/presign
Upload file bytesPUT <upload_url>
Check processing statusGET /documents/status?document_id=...
Scrape a single URLPOST /documents/upload-url
Discover site URLsPOST /documents/upload-url-site
Reprocess a documentPOST /documents/:id/reprocess
Download original fileGET /documents/:id/download
Delete a documentDELETE /documents/:id

​
Next steps

Documents in a knowledge base

How documents power the RAG corpus.

Documents API

Full request and response schemas.

Retrieval & grounding

How indexed chunks become cited answers.

Quickstart

Upload a document and run a chat end to end.