Embedding Models

DocuStore turns every text chunk, page summary, and document summary into a vector, and every search query into a vector in the same space. The text embedding model decides that space. Chemical structures use a separate model (ChemBERTa) and are not affected by anything on this page.

Two rules follow from "the same space":

• Documents and queries must use the same model. Changing the model means re-embedding every document (see Changing models).
• Every model produces 768-dimensional vectors (EMBEDDING_DIMENSIONS), so one set of Qdrant collections serves any choice. API models are asked for 768 dimensions and every response is checked.

Local models (default)

Local models run inside the DocuStore workers via sentence-transformers. No GPU is required, and no text leaves your infrastructure.

Both models run at the same speed on CPU and are downloaded from Hugging Face on first use. Neither needs trust_remote_code. Nomic's search_query: / search_document: prefixes are applied automatically; granite uses none.

On BEIR, measured through DocuStore's own embedding path (chunks capped at 512 tokens, which covers DocuStore's ~250-token chunks):

EMBEDDING_MODEL_PROVIDER=sentence-transformers
EMBEDDING_MODEL_NAME=ibm-granite/granite-embedding-english-r2
EMBEDDING_DIMENSIONS=768
EMBEDDING_DEVICE=cpu  # or cuda, mps

Any other model works if transformers supports its architecture natively and it can produce 768 dimensions. Set EMBEDDING_QUERY_PREFIX / EMBEDDING_DOCUMENT_PREFIX if it was trained with prefixes.

API models

Set EMBEDDING_MODEL_PROVIDER to call a hosted model instead. This moves embedding work off your CPUs; document text is sent to the provider.

EMBEDDING_MODEL_PROVIDER=openai
EMBEDDING_MODEL_NAME=text-embedding-3-small
EMBEDDING_API_KEY=sk-...

Any OpenAI-compatible endpoint works with provider=openai and EMBEDDING_BASE_URL, for example Azure AI Foundry's v1 API. Use your deployment name as the model:

EMBEDDING_MODEL_PROVIDER=openai
EMBEDDING_BASE_URL=https://<resource>.openai.azure.com/openai/v1/
EMBEDDING_MODEL_NAME=<your text-embedding-3-small deployment>
EMBEDDING_API_KEY=<azure key>

Only API keys are supported for now. Managed-identity (keyless) sign-in is not.

Gemini Embedding 2 takes its task as a text prefix rather than a parameter; DocuStore adds the right prefixes for it automatically. A missing key does not stop the services from starting: embedding calls fail with a clear "not configured" error instead.

Cost

Embedding costs are small next to the chat model. At OpenAI's list price for text-embedding-3-small ($0.02 per million tokens):

• a search embeds one query of about 20 tokens,
• a chat turn embeds a few queries, about 300 tokens,
• a 10-page deck embeds about 21,000 tokens (chunks plus summaries), roughly $0.0004.

Re-embedding a workspace costs the same as uploading it again.

Changing models

Vectors from different models cannot be compared, so after changing EMBEDDING_MODEL_NAME or EMBEDDING_MODEL_PROVIDER:

1. Deploy the new configuration.
2. In each workspace, a workspace admin opens Settings → Status → Re-embed Documents.

Until the re-embed finishes, search results for documents still embedded with the old model are unreliable. The Status page shows how many stored vectors come from a different model, and the API logs embedding_model_mismatch at startup, so a forgotten re-embed is visible.

Per-workspace models (self-serve)

When workspace keys are enabled (USER_LLM_KEYS_ENABLED=true, see Workspace keys), each workspace can choose its own embedding model under Settings → AI Provider → Search embeddings:

• Only workspace admins can change it. Members see which model is in use.
• An admin can only pick a model the workspace holds a key for.
• Changing it for a workspace that already has documents asks for confirmation and re-embeds every document on the workspace's key.
• If the workspace's key for its model is removed, uploads, chat, and search return "not configured" instead of silently falling back to another model.