Intellidoc Backend

Enterprise Content Management MVP with semantic search capabilities.

Features

• Upload PDFs and images
• OCR text extraction from images using Tesseract
• LLM-assisted metadata suggestion using Google Gemini (enhanced with OCR text)
• Semantic and exact metadata search
• Image similarity search using OpenCLIP
• Vector embeddings with Qdrant
• SQLite database for metadata storage

Requirements

• Python 3.8+
• Qdrant vector database
• Ollama if using in Self Hosted fashion
• Google API key (optional, for LLM suggestions)

Installation

Quick Setup

1. Run the development setup script:

python setup_dev.py

Manual Setup

1. Install dependencies:

pip install -r requirements.txt

2. Set up environment variables (create .env file):

# Environment variables for ECM MVP
GOOGLE_API_KEY=your_google_api_key_here

# Ollama settings (set USE_OLLAMA=true to use Ollama instead of Gemini)
USE_OLLAMA=false
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_MODEL=gemma:2b

QDRANT_HOST=localhost
QDRANT_PORT=6333
DATABASE_URL=sqlite:///./ecm_mvp.db
UPLOADS_DIR=uploads

3. Start Qdrant:

docker run -d --name qdrant -p 6333:6333 qdrant/qdrant

4. Run the application:

python -m uvicorn app.main:app --reload

Development

VS Code Setup

• Install recommended extensions (see .vscode/extensions.json)
• Use F5 to start debugging
• Available debug configurations:
  • Python: FastAPI - Debug the main application
  • Python: Uvicorn Server - Debug with uvicorn
  • Python: Test Setup - Debug the setup script

Tasks (Ctrl+Shift+P > Tasks: Run Task)

• Install Dependencies - Install Python packages
• Start Qdrant - Start Qdrant container
• Start FastAPI Server - Start the development server
• Test Setup - Run setup validation
• Format Code - Format code with Black

API Endpoints

Upload Workflow

• POST /api/v1/upload - Upload file and get metadata suggestions
• POST /api/v1/import/{document_id} - Import document with final metadata

Search

• POST /api/v1/search - Search documents by metadata and/or image

Document Viewing

• GET /api/v1/documents/{document_id} - Get document details
• GET /api/v1/documents/{document_id}/pages/{page_number}/image - Get page image
• GET /api/v1/documents/{document_id}/download - Download original file

Usage Example

1. Upload a file:

curl -X POST "http://localhost:8000/api/v1/upload" \
  -H "Content-Type: multipart/form-data" \
  -F "file=@document.pdf"

2. Import with metadata:

curl -X POST "http://localhost:8000/api/v1/import/1" \
  -H "Content-Type: application/json" \
  -d '{
    "metadata": [
      {"key": "document_type", "value": "invoice"},
      {"key": "company", "value": "Acme Corp"}
    ]
  }'

3. Search documents:

curl -X POST "http://localhost:8000/api/v1/search" \
  -H "Content-Type: application/json" \
  -d '{
    "metadata": [
      {"key": "document_type", "value": "invoice", "value_semantic": false}
    ]
  }'

API Testing

Use the api_tests.http file with VS Code REST Client extension for interactive API testing.

Project Structure

├── app/                    # Main application code
│   ├── routes/            # API route handlers
│   ├── models.py          # SQLAlchemy models
│   ├── schemas.py         # Pydantic schemas
│   ├── database.py        # Database connection
│   ├── config.py          # Configuration settings
│   ├── embedding_service.py  # Text/image embeddings
│   ├── qdrant_client.py   # Vector database client
│   ├── llm_service.py     # Gemini LLM integration
│   ├── file_processor.py  # File handling utilities
│   └── main.py           # FastAPI application
├── .vscode/               # VS Code configuration
├── uploads/               # File storage (auto-created)
├── requirements.txt      # Python dependencies
├── api_tests.http        # API test cases
└── setup_dev.py         # Development setup script