Global jump UI

.kiro/specs/artifact-thumbnails-gallery/requirements.md

@@ -0,0 +1,101 @@
+# Requirements Document
+
+## Introduction
+
+This document specifies the requirements for artifact thumbnail generation and an artifact-based gallery search feature. The thumbnail task extracts small WebP images at artifact timestamps for use in search result galleries. The gallery search enables browsing artifacts as a visual grid with thumbnails.
+
+## Glossary
+
+- **Artifact_Thumbnail**: A small WebP image extracted from a video frame at an artifact's timestamp
+- **Artifact_Gallery**: A visual grid display of artifact search results with thumbnails
+- **Timestamp_Deduplication**: The process of generating only one thumbnail per unique timestamp, even when multiple artifacts share that timestamp
+
+## Requirements
+
+### Requirement 1: Thumbnail Extraction Task
+
+**User Story:** As a system operator, I want thumbnails generated for all artifact timestamps, so that search results can display visual previews.
+
+#### Acceptance Criteria
+
+1. THE system SHALL provide a `thumbnail.extraction` task type that generates thumbnails for a video's artifacts
+2. THE task SHALL query all artifacts for the video and collect unique `start_ms` timestamps
+3. THE task SHALL skip thumbnail generation for timestamps that already have a thumbnail file on disk (idempotent)
+4. THE task SHALL generate WebP format thumbnails with maximum width of 320 pixels (height proportional)
+5. THE task SHALL store thumbnails at `/data/thumbnails/{video_id}/{timestamp_ms}.webp`
+6. THE task SHALL target thumbnail file size of approximately 10-20KB each
+7. THE task SHALL use ffmpeg for frame extraction
+
+### Requirement 2: Thumbnail Deduplication
+
+**User Story:** As a system operator, I want to avoid generating duplicate thumbnails, so that storage is used efficiently.
+
+#### Acceptance Criteria
+
+1. WHEN multiple artifacts share the same `start_ms` timestamp, THE task SHALL generate only one thumbnail for that timestamp
+2. WHEN a thumbnail file already exists for a timestamp, THE task SHALL skip extraction for that timestamp
+3. THE task SHALL log the count of skipped (already existing) thumbnails vs newly generated thumbnails
+
+### Requirement 3: Thumbnail Serving Endpoint
+
+**User Story:** As a frontend developer, I want an API endpoint to retrieve thumbnails, so that I can display them in the UI.
+
+#### Acceptance Criteria
+
+1. THE backend SHALL provide a `GET /api/v1/thumbnails/{video_id}/{timestamp_ms}` endpoint
+2. WHEN the thumbnail exists, THE endpoint SHALL return the WebP file with appropriate content type
+3. WHEN the thumbnail does not exist, THE endpoint SHALL return 404
+4. THE endpoint SHALL set appropriate cache headers for browser caching (e.g., 1 week)
+
+### Requirement 4: Artifact Gallery Search API
+
+**User Story:** As a user, I want to search artifacts and see results as a visual gallery, so that I can quickly browse matching content.
+
+#### Acceptance Criteria
+
+1. THE backend SHALL provide a `GET /api/v1/artifacts/search` endpoint for gallery-style artifact search
+2. THE endpoint SHALL accept parameters: `kind`, `label`, `query`, `min_confidence`, `filename`, `limit`, `offset`
+3. THE endpoint SHALL return results ordered by global timeline (file_created_at, video_id, start_ms)
+4. EACH result SHALL include: `video_id`, `artifact_id`, `start_ms`, `thumbnail_url`, `preview`, `video_filename`
+5. THE `thumbnail_url` SHALL point to the thumbnail serving endpoint for that artifact's timestamp
+6. THE endpoint SHALL support pagination via `limit` and `offset` parameters
+7. THE endpoint SHALL return total count of matching artifacts for pagination UI
+8. WHEN `filename` parameter is provided, THE endpoint SHALL filter results to videos whose filename contains the search string (case-insensitive)
+
+### Requirement 5: Artifact Gallery UI Component
+
+**User Story:** As a user, I want a visual gallery interface to browse artifact search results, so that I can find content by looking at thumbnails.
+
+#### Acceptance Criteria
+
+1. THE frontend SHALL provide an `ArtifactGallery` component that displays search results as a thumbnail grid
+2. THE gallery SHALL display thumbnails in a responsive grid layout (adapting to screen width)
+3. EACH thumbnail card SHALL display: the thumbnail image, artifact label/text preview, video filename, timestamp
+4. WHEN the user clicks a thumbnail, THE application SHALL navigate to the player page at that video and timestamp
+5. THE gallery SHALL display a loading state while fetching results
+6. THE gallery SHALL display "No results found" when the search returns empty
+7. THE gallery SHALL support infinite scroll or pagination for browsing large result sets
+
+### Requirement 6: Gallery Search Form
+
+**User Story:** As a user, I want to filter the artifact gallery by type and search criteria, so that I can find specific content.
+
+#### Acceptance Criteria
+
+1. THE gallery page SHALL include a search form with artifact type selector
+2. THE search form SHALL include appropriate input fields based on artifact type (label for objects, query for transcript/ocr)
+3. THE search form SHALL include a confidence threshold slider for applicable artifact types
+4. THE search form SHALL include a filename filter input to search within specific videos
+5. WHEN the user submits the search form, THE gallery SHALL update to show matching results
+6. THE search form state SHALL be preserved in the URL for shareable links
+
+### Requirement 7: Thumbnail Fallback
+
+**User Story:** As a user, I want to see a placeholder when a thumbnail is not available, so that the gallery layout remains consistent.
+
+#### Acceptance Criteria
+
+1. WHEN a thumbnail fails to load (404 or error), THE gallery SHALL display a placeholder image
+2. THE placeholder SHALL indicate the artifact type (e.g., icon for object, face, transcript)
+3. THE gallery layout SHALL not break when thumbnails are missing
+

.kiro/specs/artifact-thumbnails-gallery/tasks.md

@@ -0,0 +1,249 @@
+# Implementation Tasks: Artifact Thumbnails & Gallery
+
+## Overview
+
+Tasks for implementing artifact thumbnail generation and the artifact gallery search feature.
+
+## Tasks
+
+### Backend: Thumbnail Extraction Task
+
+- [ ] 1. Create thumbnail extractor module
+  - Create `ml-service/src/workers/thumbnail_extractor.py`
+  - Define constants: THUMBNAIL_DIR, THUMBNAIL_WIDTH (320), THUMBNAIL_QUALITY (75)
+  - Create output directory structure
+  - _Requirements: 1.1, 1.4, 1.5, 1.6_
+
+- [ ] 2. Implement timestamp collection
+  - Query all artifacts for video
+  - Extract unique start_ms timestamps
+  - Deduplicate timestamps (multiple artifacts at same ms)
+  - _Requirements: 1.2, 2.1_
+
+- [ ] 3. Implement idempotent thumbnail generation
+  - Check if thumbnail file exists before extraction
+  - Skip existing thumbnails
+  - Log skipped vs generated counts
+  - _Requirements: 1.3, 2.2, 2.3_
+
+- [ ] 4. Implement ffmpeg frame extraction
+  - Build ffmpeg command for WebP output
+  - Use scale filter for 320px width
+  - Handle extraction errors gracefully
+  - Add timeout (10 seconds per frame)
+  - _Requirements: 1.7_
+
+- [ ] 5. Register thumbnail task type
+  - Add `thumbnail.extraction` to task registry
+  - Configure as low priority task
+  - _Requirements: 1.1_
+
+- [ ] 6. Write tests for thumbnail extraction
+  - Test timestamp deduplication
+  - Test idempotent behavior (skip existing)
+  - Test ffmpeg command construction
+  - Test error handling
+  - _Requirements: 1, 2_
+
+### Backend: Thumbnail Serving API
+
+- [ ] 7. Create thumbnail controller
+  - Create `backend/src/api/thumbnail_controller.py`
+  - Add router with `/thumbnails` prefix
+  - _Requirements: 3.1_
+
+- [ ] 8. Implement thumbnail serving endpoint
+  - Add `GET /{video_id}/{timestamp_ms}` endpoint
+  - Return FileResponse with WebP content type
+  - Return 404 if thumbnail not found
+  - Set cache headers (1 week)
+  - _Requirements: 3.1, 3.2, 3.3, 3.4_
+
+- [ ] 9. Register thumbnail router
+  - Import router in main_api.py
+  - Include router with `/v1/thumbnails` prefix
+  - _Requirements: 3.1_
+
+- [ ] 10. Write tests for thumbnail serving
+  - Test successful thumbnail retrieval
+  - Test 404 for missing thumbnail
+  - Test cache headers
+  - _Requirements: 3_
+
+### Backend: Artifact Search API
+
+- [ ] 11. Create artifact search controller
+  - Create `backend/src/api/artifact_search_controller.py`
+  - Add router with `/artifacts` prefix
+  - Define response schemas
+  - _Requirements: 4.1_
+
+- [ ] 12. Implement search endpoint
+  - Add `GET /search` endpoint
+  - Accept kind, label, query, filename, min_confidence, limit, offset params
+  - Map kind to artifact_type
+  - _Requirements: 4.1, 4.2_
+
+- [ ] 13. Implement search query building
+  - Build base query joining artifacts and videos
+  - Add label filter for object/place
+  - Add query filter for transcript/ocr
+  - Add filename filter (ILIKE)
+  - Add min_confidence filter
+  - _Requirements: 4.2, 4.8_
+
+- [ ] 14. Implement pagination and ordering
+  - Order by global timeline (file_created_at, video_id, start_ms)
+  - Apply limit and offset
+  - Return total count for pagination UI
+  - _Requirements: 4.3, 4.6, 4.7_
+
+- [ ] 15. Build response with thumbnail URLs
+  - Construct thumbnail_url for each result
+  - Include video_filename, file_created_at
+  - Return ArtifactSearchResponse
+  - _Requirements: 4.4, 4.5_
+
+- [ ] 16. Register artifact search router
+  - Import router in main_api.py
+  - Include router with `/v1/artifacts` prefix
+  - _Requirements: 4.1_
+
+- [ ] 17. Write tests for artifact search
+  - Test search by kind
+  - Test label filter
+  - Test query filter
+  - Test filename filter
+  - Test min_confidence filter
+  - Test pagination
+  - Test ordering
+  - _Requirements: 4_
+
+### Frontend: ArtifactGallery Component
+
+- [ ] 18. Create ArtifactGallery component skeleton
+  - Create `frontend/src/components/ArtifactGallery.tsx`
+  - Define props interface
+  - Set up state for results, loading, pagination
+  - _Requirements: 5.1_
+
+- [ ] 19. Implement search form
+  - Add artifact type selector dropdown
+  - Add label/query input based on type
+  - Add filename filter input
+  - Add confidence slider for applicable types
+  - _Requirements: 6.1, 6.2, 6.3, 6.4_
+
+- [ ] 20. Implement API integration
+  - Build search request with form state
+  - Call `/api/v1/artifacts/search` endpoint
+  - Handle loading state
+  - Handle errors
+  - _Requirements: 6.5_
+
+- [ ] 21. Implement thumbnail grid
+  - Create responsive grid layout
+  - Render ThumbnailCard for each result
+  - Handle thumbnail load errors with placeholder
+  - _Requirements: 5.2, 7.1, 7.2, 7.3_
+
+- [ ] 22. Create ThumbnailCard component
+  - Display thumbnail image
+  - Show label/text preview
+  - Show video filename
+  - Show timestamp (MM:SS format)
+  - Handle click to navigate
+  - _Requirements: 5.3, 5.4_
+
+- [ ] 23. Implement pagination/infinite scroll
+  - Add pagination controls or infinite scroll
+  - Load more results on scroll/click
+  - _Requirements: 5.7_
+
+- [ ] 24. Implement empty and loading states
+  - Show loading spinner during fetch
+  - Show "No results found" for empty results
+  - _Requirements: 5.5, 5.6_
+
+- [ ] 25. Implement URL state preservation
+  - Sync form state to URL query params
+  - Read initial state from URL on mount
+  - _Requirements: 6.6_
+
+- [ ] 26. Apply component styling
+  - Use dark theme colors
+  - Responsive grid (auto-fill, minmax 200px)
+  - Card hover effects
+  - _Requirements: 5.2_
+
+### Frontend: Gallery Page
+
+- [ ] 27. Create Gallery page
+  - Create `frontend/src/pages/GalleryPage.tsx`
+  - Render ArtifactGallery component
+  - Handle artifact click navigation to player
+  - _Requirements: 5.4_
+
+- [ ] 28. Add gallery page route
+  - Add `/gallery` route to app router
+  - Link to gallery from navigation/header
+  - _Requirements: 5.1_
+
+### Testing
+
+- [ ] 29. Write ArtifactGallery unit tests
+  - Test search form rendering
+  - Test grid layout
+  - Test thumbnail card rendering
+  - Test placeholder on image error
+  - Test loading state
+  - Test empty state
+  - _Requirements: 5, 6, 7_
+
+- [ ] 30. Write integration tests
+  - Test search flow end-to-end
+  - Test navigation to player
+  - Test URL state preservation
+  - _Requirements: 5, 6_
+
+### Checkpoints
+
+- [ ] 31. Checkpoint: Thumbnail extraction works
+  - Run thumbnail task on test video
+  - Verify thumbnails created in /data/thumbnails
+  - Verify deduplication
+  - Verify idempotency
+  - _Requirements: 1, 2_
+
+- [ ] 32. Checkpoint: Thumbnail serving works
+  - Test endpoint returns thumbnails
+  - Test 404 for missing
+  - Test cache headers
+  - _Requirements: 3_
+
+- [ ] 33. Checkpoint: Artifact search API works
+  - Test all filter combinations
+  - Test pagination
+  - Test thumbnail URLs in response
+  - _Requirements: 4_
+
+- [ ] 34. Checkpoint: Gallery UI works
+  - Test search form
+  - Test thumbnail grid display
+  - Test navigation to player
+  - _Requirements: 5, 6, 7_
+
+- [ ] 35. Final checkpoint: All tests pass
+  - Run ml-service tests
+  - Run backend tests
+  - Run frontend tests
+  - Run lint checks
+  - _Requirements: All_
+
+## Notes
+
+- Backend tasks (1-17) can be done before frontend tasks (18-28)
+- Thumbnail extraction (1-6) is independent of thumbnail serving (7-10)
+- Artifact search API (11-17) is independent of thumbnail tasks
+- Gallery UI (18-28) depends on both thumbnail serving and artifact search APIs
+- Consider running thumbnail extraction as a batch job for existing videos