Chat Pipeline
Agentic RAG chat system with three operating modes, grounding verification, and SSE streaming.
The DocuStore chat system is an agentic RAG (Retrieval-Augmented Generation) pipeline that answers questions grounded in the user's document corpus. It supports three operating modes, inline citation verification, and real-time token streaming via Server-Sent Events.
Operating Modes
| Mode | Pipeline | Best For |
|---|---|---|
| Quick | 5-step single-shot retrieval | Fast factual lookups |
| Thinking | 6-stage agentic retrieval with LLM tool-calling loop | Complex multi-faceted questions |
| Deep Thinking | Thinking + page image analysis | Questions about figures and visual content |
The mode is selected per-message via the mode field. When omitted, the server default (CHAT_DEFAULT_MODE, default thinking) is used.
Thinking Mode Pipeline (6 Stages)
Stage 1: Query Planning
Three analysis tracks run in parallel:
- structflo-ner -- Extracts compound names, targets, and gene names from the query.
- GLiNER2 -- Extracts author names for tag-based filtering.
- LLM Planning -- Classifies query type (factual, comparative, exploratory, compound, follow-up), generates sub-queries, and selects a retrieval strategy.
NER entities are accumulated across conversation turns. Follow-up questions inherit entities from prior grounded turns, enabling multi-turn topic continuity.
Stage 2: Agentic Retrieval
An LLM tool-calling loop iteratively searches the document corpus:
- Seed -- Carry forward citations from the previous turn; detect
[N]references in the question. - Auto-seed -- Vector search with NER-derived filters, plus an unfiltered search for broad recall.
- Bioactivity pre-fetch -- Deterministic MongoDB lookup for compound bioactivity data.
- LLM loop -- Up to 5 iterations where the LLM evaluates accumulated results and decides whether to call additional tools or finish.
Available tools in the retrieval loop:
| Tool | Description |
|---|---|
search_documents | Chunk + summary search via Qdrant |
search_summaries | Summary-only search |
get_page_content | Full page text from MongoDB |
search_structured_bioactivity | Direct MongoDB bioactivity lookup |
finish_retrieval | Signal that retrieval is complete |
The RetrievalAccumulator handles cross-iteration deduplication, budget tracking (max chars), and query deduplication to prevent repeated searches.
Stage 3: Context Assembly
Retrieved results are deduplicated (chunks take priority over summaries for the same page), tiered by relevance, and formatted with [N] citation indices:
| Tier | Criteria | Budget |
|---|---|---|
| HIGH | Rerank > 0.7 or similarity > 0.85 | Full text |
| MEDIUM | Rerank > 0.4 or similarity > 0.6 | 1000 chars |
| LOW | Below thresholds | 200 chars |
Stage 4: Adaptive Synthesis
A system prompt is selected based on query type (factual, comparative, exploratory, compound, follow-up). The LLM generates a draft answer with [N] citations. Tokens are not streamed to the client at this stage.
Stage 5: Inline Verification
An algorithmic citation coverage check runs first. If coverage is low and the query is factual, a selective LLM verification follows. If the answer is not grounded:
- Strategy A (Broaden) -- Re-run retrieval with unfiltered search enabled.
- Strategy B (Augment) -- Add unsupported claims to the query and re-run.
Stage 6: Answer Formatting
The draft is reformatted for coherence. This is where tokens stream to the client as SSE events. Citation indices are re-extracted to produce the final used_citations list.
SSE Event Schema
All modes emit the same event types:
| SSE Event | Content |
|---|---|
agent_step | Step progress with optional thinking_content |
retrieval_results | Citation sources for the frontend sidebar |
query_context | Extracted entities, authors, query type |
token | Streaming answer delta (only during formatting) |
structured_block | Rich content blocks (tables, molecules) |
grounding_result | is_grounded boolean + confidence score |
done | Final metadata: message_id, token counts, duration, sources |
Follow-up Handling
Multi-turn conversations use several continuity mechanisms:
- History window -- Last 10 messages loaded from MongoDB, compressed to a 4000-character budget.
- NER accumulation -- Entities from prior grounded turns are inherited or merged into the current query.
- Citation continuity -- Previous turn's citations are seeded into the retrieval accumulator at score 0.5.
- Explicit references --
[N]references in the question trigger a full page content fetch.
Configuration
| Variable | Default | Description |
|---|---|---|
CHAT_DEFAULT_MODE | thinking | Default pipeline mode |
CHAT_MAX_RETRIES | 1 | Grounding verification retries |
CHAT_AGENT_MAX_ITERATIONS | 5 | Max tool-calling iterations |
CHAT_AGENT_ITERATION_TIMEOUT_S | 30 | Per-iteration timeout (seconds) |
CHAT_AGENT_TOTAL_TIMEOUT_S | 120 | Total retrieval loop timeout |
CHAT_CONTEXT_BUDGET_CHARS | 12000 | Max chars for assembled context |
CHAT_DEEP_THINKING_MAX_IMAGES | 5 | Max page images in deep thinking |
CHAT_FOLLOW_UP_CONTEXT_BUDGET | 4000 | Char budget for conversation history |
CHAT_LLM_PROVIDER | Falls back to LLM_PROVIDER | LLM provider for chat (can differ from batch) |
CHAT_LLM_MODEL_NAME | Falls back to LLM_MODEL_NAME | Model for chat responses |
CHAT_LLM_TEMPERATURE | 0.3 | Higher temperature for conversational style |