feat: Add document classification use case with Handit self-improving AI

[coderTtxi12]

🎯 Add Document Classification Self-Improving AI Use Case

📋 Overview

This PR adds a comprehensive use case demonstrating how Handit.ai transforms static document processing AI into a self-improving system that learns and optimizes over time.

🚀 What's New

Complete Document Processing Use Case

• 3-Phase Handit Transformation: Observability → Evaluation → Optimization
• Real Performance Metrics: 85% → 94% accuracy (+11%), 40% cost reduction
• Static vs Self-Improving AI: Clear before/after comparison
• Production Code Examples: Actual implementation with startTracing(), trackNode(), fetchOptimizedPrompt()

Single-Purpose Evaluator System

• Coherence Evaluator: Logical structure and flow assessment
• Correctness Evaluator: Factual accuracy and prompt compliance
• Formatting Evaluator: Structure and style adherence
• Hallucination Evaluator: Fabrication detection and grounding verification

Real-World Scenarios

• Invoice image processing with vision AI optimization
• Multi-sheet Excel expense report analysis
• Legal contract analysis with automatic term extraction

🔧 Technical Features

Phase 1: AI Observability

// Complete tracing integration
await startTracing({ agentName: 'document_classification' });
await trackNode({ input, output, nodeName, nodeType, executionId });
await endTracing({ executionId, agentName });

Phase 2: LLM-as-Judge Evaluation

• Automated quality assessment on every document
• Single-purpose evaluators for actionable insights
• Real-time quality scoring (0-10 scale)

Phase 3: Self-Improving AI

• AI-generated prompt optimization based on evaluation data
• Background A/B testing without user impact
• Release Hub for controlled deployment

🎮 Interactive Demo

• Updated demo repository link: https://github.com/Handit-AI/handit-demo-docs-classification.git
• Live demonstration of all 3 phases working together
• Real document processing with multi-format support

📸 Visual Assets

• Processing Dashboard: Real-time metrics and optimization tracking
• Complete Flow Visualization: Step-by-step processing trace
• Quality Analytics: Real-time evaluation and trend analysis

🔗 Files Changed

New Files

• app/use_cases/document_classifier_agent/page.mdx - Complete use case documentation
• public/assets/use_cases/processing_dashboard.png - Dashboard visualization
• public/assets/use_cases/complete_flow.png - Processing flow diagram
• public/assets/use_cases/real_time.png - Real-time metrics screenshot

Modified Files

• app/use_cases/_meta.ts - Added to navigation
• Updated image references and demo links

🎯 Key Value Propositions

1. Self-Improving Intelligence: AI that learns from every document
2. Zero-Effort Optimization: System improves while you focus on building
3. Complete Quality Control: LLM-as-Judge evaluation on every operation
4. Controlled Deployment: You decide when optimizations go live
5. Measurable ROI: Clear performance and cost improvements

🚀 Ready for Review

This use case provides a complete, production-ready example of how Handit transforms static AI into continuously improving intelligence, with real metrics and actionable implementation guidance.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
handit-ai-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 11, 2025 8:35am