Blob Storage

DocuStore stores uploaded documents (PDFs) and rendered page images in blob storage. The BlobStore port supports both local filesystem storage (for development) and S3-compatible object storage (for production).

Storage Layout

When a document is uploaded, the blob store organizes files in a predictable structure:

{blob_root}/
  artifacts/
    {artifact_id}/
      source.pdf            # Original uploaded PDF
      pages/
        0.png               # Full-resolution page image (page 0)
        0_thumb.jpg          # Thumbnail (page 0)
        1.png               # Full-resolution page image (page 1)
        1_thumb.jpg          # Thumbnail (page 1)
        ...

Page images are generated during the PDF processing workflow. Full-resolution PNGs are used for deep thinking mode (page image analysis), while JPEG thumbnails (~200px wide) are served for list views and table previews.

Local Filesystem (Development)

The default configuration stores blobs on the local filesystem. This is the simplest option and requires no additional infrastructure.

# Default: stores in services/blobs/ directory
BLOB_BASE_URL=file:///path/to/docu-store/services/blobs

The file:// URL scheme tells DocuStore to use the local filesystem adapter. The path must be absolute.

S3-Compatible Storage (Production)

For production deployments, configure an S3-compatible object store (AWS S3, MinIO, Cloudflare R2, etc.):

BLOB_BASE_URL=s3://my-docustore-bucket/artifacts

Additional S3 configuration is passed via blob_storage_options or standard AWS environment variables:

AWS_ACCESS_KEY_ID=AKIA...
AWS_SECRET_ACCESS_KEY=...
AWS_DEFAULT_REGION=us-east-1

For S3-compatible services (MinIO, R2), set the endpoint URL:

AWS_ENDPOINT_URL=https://minio.example.com:9000

API Endpoints

The API provides streaming endpoints for serving blob content to the frontend:

Stream PDF

GET /artifacts/{artifact_id}/pdf

Returns the raw PDF binary as application/pdf with an inline content disposition. The browser can render this directly in an iframe or PDF viewer.

Stream Page Image

GET /artifacts/{artifact_id}/pages/{page_index}/image

Returns the full-resolution PNG page image. Add ?size=thumb for the lightweight JPEG thumbnail.

# Full resolution (for detail views, deep thinking)
GET /artifacts/{artifact_id}/pages/0/image

# Thumbnail (for list views, previews)
GET /artifacts/{artifact_id}/pages/0/image?size=thumb

The thumbnail endpoint falls back to the full PNG if a thumbnail has not been generated.

How Blobs Are Created

The blob store is used at two points in the processing pipeline:

1. Document Upload

When a document is uploaded via POST /artifacts/upload, the ArtifactUploadSaga orchestrates:

1. Store the file bytes in blob storage at artifacts/{artifact_id}/source.pdf.
2. Create the Artifact aggregate with the storage_location pointing to the stored file.
3. The Artifact.Created event triggers the processing pipeline.

2. PDF Processing

The ArtifactProcessingWorkflow reads the PDF from blob storage and:

1. Extracts text from each page via PyMuPDF.
2. Renders each page to a PNG image (2x zoom for quality).
3. Generates a JPEG thumbnail for each page.
4. Stores images and thumbnails back to blob storage.
5. Creates Page aggregates with references to the artifact.

Configuration

Considerations

• Blob storage is append-mostly. Documents are uploaded once and read many times. Updates are rare (only when re-processing overwrites page images).
• Page images can be large. A 12-page PDF at 2x zoom generates approximately 30-50 MB of PNG images. Thumbnails add roughly 500 KB per document.
• Local storage is not suitable for multi-instance deployments. If running multiple API server instances, use S3-compatible storage so all instances can access the same blobs.
• Blobs are referenced by storage location, not copied. The artifact aggregate stores the blob path (e.g., artifacts/{id}/source.pdf), and all access goes through the BlobStore port. Migrating storage backends requires updating the blob root URL, not the aggregate data.