diff --git a/.kiro/specs/global-jump-navigation/design.md b/.kiro/specs/global-jump-navigation/design.md new file mode 100644 index 0000000..1cb546a --- /dev/null +++ b/.kiro/specs/global-jump-navigation/design.md @@ -0,0 +1,688 @@ +# Design Document: Global Jump Navigation + +## Overview + +The Global Jump Navigation feature enables cross-video artifact search and navigation using a unified API endpoint. Users can search for objects, faces, text, and scenes across their entire video library in chronological order. The design leverages existing projection tables (object_labels, face_clusters, transcript_fts, ocr_fts, scene_ranges, video_locations) to provide fast queries without requiring new data structures. + +The architecture follows a service-oriented pattern with clear separation between API layer, business logic, and data access. All queries use a deterministic global timeline based on file_created_at (EXIF/filesystem date) as the primary ordering mechanism. + +## Architecture + +### High-Level Flow + +``` +Client Request (GET /jump/global) + ↓ +GlobalJumpController (validates parameters, routes request) + ↓ +GlobalJumpService (orchestrates search logic) + ↓ +Projection Table Queries (object_labels, face_clusters, transcript_fts, ocr_fts, etc.) + ↓ +Video Metadata Join (file_created_at, filename) + ↓ +GlobalJumpResult (formatted response) + ↓ +Client Response (video_id, jump_to, preview) +``` + +### Service Architecture + +``` +GlobalJumpService +├── jump_next() - Navigate forward in global timeline +├── jump_prev() - Navigate backward in global timeline +├── _search_objects_global() - Query object_labels projection +├── _search_faces_global() - Query face_clusters projection +├── _search_transcript_global() - Query transcript_fts projection +├── _search_ocr_global() - Query ocr_fts projection +├── _search_scenes_global() - Query scene_ranges projection +├── _search_locations_global() - Query video_locations projection +└── _to_global_result() - Format database results to response schema +``` + +## Components and Interfaces + +### 1. GlobalJumpController + +**File:** `backend/src/api/global_jump_controller.py` + +**Responsibility:** HTTP request handling, parameter validation, response formatting + +**Key Methods:** +- `global_jump(kind, direction, from_video_id, from_ms, label, query, face_cluster_id, min_confidence, limit)` → GlobalJumpResponseSchema + +**Validation Rules:** +- `kind` must be one of: object, face, transcript, ocr, scene, place, location +- `direction` must be: next or prev +- `from_video_id` must reference an existing video +- `from_ms` must be non-negative integer (optional) +- `label` and `query` are mutually exclusive +- `min_confidence` must be between 0 and 1 (optional) +- `limit` must be between 1 and 50 + +**Error Responses:** +- 400: Invalid parameters +- 404: Video not found +- 500: Database or service error + +### 2. GlobalJumpService + +**File:** `backend/src/services/global_jump_service.py` + +**Responsibility:** Business logic for cross-video navigation, query orchestration + +**Key Methods:** + +```python +async def jump_next( + kind: str, + from_video_id: str, + from_ms: int | None = None, + label: str | None = None, + query: str | None = None, + face_cluster_id: str | None = None, + min_confidence: float | None = None, + limit: int = 1, +) -> list[GlobalJumpResult] +``` + +Returns list of GlobalJumpResult objects ordered by global timeline (ascending for next). + +```python +async def jump_prev( + kind: str, + from_video_id: str, + from_ms: int | None = None, + label: str | None = None, + query: str | None = None, + face_cluster_id: str | None = None, + min_confidence: float | None = None, + limit: int = 1, +) -> list[GlobalJumpResult] +``` + +Returns list of GlobalJumpResult objects ordered by global timeline (descending for prev). + +**Internal Methods:** + +Each search method follows the same pattern: +1. Get current video metadata (file_created_at) +2. Build base query on projection table +3. Apply filters (label, confidence, text query) +4. Apply direction-specific WHERE clause +5. Order by global timeline +6. Limit results +7. Format and return + +### 3. GlobalJumpResult (Data Model) + +**File:** `backend/src/models/global_jump.py` + +```python +@dataclass +class GlobalJumpResult: + video_id: str + video_filename: str + file_created_at: datetime | None + jump_to: JumpTo + artifact_id: str + preview: dict + +@dataclass +class JumpTo: + start_ms: int + end_ms: int +``` + +### 4. Response Schemas (Pydantic) + +**File:** `backend/src/api/schemas/global_jump_schemas.py` + +```python +class JumpToSchema(BaseModel): + start_ms: int + end_ms: int + +class GlobalJumpResultSchema(BaseModel): + video_id: str + video_filename: str + file_created_at: datetime | None = None + jump_to: JumpToSchema + artifact_id: str + preview: dict + +class GlobalJumpResponseSchema(BaseModel): + results: list[GlobalJumpResultSchema] + has_more: bool +``` + +## Data Models + +### Global Timeline Ordering + +All queries use this deterministic ordering: + +**For "next" direction (ascending):** +```sql +ORDER BY + videos.file_created_at ASC, + videos.video_id ASC, + projection_table.start_ms ASC +``` + +**For "prev" direction (descending):** +```sql +ORDER BY + videos.file_created_at DESC, + videos.video_id DESC, + projection_table.start_ms DESC +``` + +### Query Pattern for "Next" Direction + +```sql +SELECT + projection.artifact_id, + projection.asset_id, + projection.start_ms, + projection.end_ms, + projection.[kind_specific_fields], + videos.filename, + videos.file_created_at +FROM projection_table projection +JOIN videos ON videos.video_id = projection.asset_id +WHERE + (videos.file_created_at > :current_file_created_at) + OR ( + videos.file_created_at = :current_file_created_at + AND videos.video_id = :current_video_id + AND projection.start_ms > :from_ms + ) + OR ( + videos.file_created_at = :current_file_created_at + AND videos.video_id > :current_video_id + ) + AND [kind_specific_filters] +ORDER BY + videos.file_created_at ASC, + videos.video_id ASC, + projection.start_ms ASC +LIMIT :limit +``` + +### Query Pattern for "Prev" Direction + +```sql +SELECT + projection.artifact_id, + projection.asset_id, + projection.start_ms, + projection.end_ms, + projection.[kind_specific_fields], + videos.filename, + videos.file_created_at +FROM projection_table projection +JOIN videos ON videos.video_id = projection.asset_id +WHERE + (videos.file_created_at < :current_file_created_at) + OR ( + videos.file_created_at = :current_file_created_at + AND videos.video_id = :current_video_id + AND projection.start_ms < :from_ms + ) + OR ( + videos.file_created_at = :current_file_created_at + AND videos.video_id < :current_video_id + ) + AND [kind_specific_filters] +ORDER BY + videos.file_created_at DESC, + videos.video_id DESC, + projection.start_ms DESC +LIMIT :limit +``` + +### Projection Table Schemas (Existing) + +**object_labels:** +- artifact_id (PK) +- asset_id (FK to videos) +- label (string) +- confidence (float 0-1) +- start_ms (int) +- end_ms (int) + +**face_clusters:** +- artifact_id (PK) +- asset_id (FK to videos) +- cluster_id (string) +- confidence (float 0-1) +- start_ms (int) +- end_ms (int) + +**transcript_fts:** +- artifact_id (PK) +- asset_id (FK to videos) +- text (string) +- text_tsv (tsvector for FTS) +- start_ms (int) +- end_ms (int) + +**ocr_fts:** +- artifact_id (PK) +- asset_id (FK to videos) +- text (string) +- text_tsv (tsvector for FTS) +- start_ms (int) +- end_ms (int) + +**scene_ranges:** +- artifact_id (PK) +- asset_id (FK to videos) +- scene_index (int) +- start_ms (int) +- end_ms (int) + +**video_locations:** +- artifact_id (PK) +- asset_id (FK to videos) +- latitude (float) +- longitude (float) +- country (string) + +## Correctness Properties + +A property is a characteristic or behavior that should hold true across all valid executions of a system—essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees. + +### Property 1: Global Timeline Ordering Consistency + +*For any* two results returned from a global jump query with direction="next", if result A has an earlier file_created_at than result B, then result A should appear before result B in the results list. If they have the same file_created_at, then the one with the earlier video_id should appear first. If they have the same video_id, then the one with the earlier start_ms should appear first. + +**Validates: Requirements 6.1, 6.2, 6.3, 6.4** + +### Property 2: Reverse Direction Ordering + +*For any* two results returned from a global jump query with direction="prev", if result A has a later file_created_at than result B, then result A should appear before result B in the results list. If they have the same file_created_at, then the one with the later video_id should appear first. If they have the same video_id, then the one with the later start_ms should appear first. + +**Validates: Requirements 6.5** + +### Property 3: Filter Consistency for Confidence + +*For any* global jump query with a min_confidence parameter, all returned results should have confidence >= min_confidence. No result should have confidence below the specified threshold. + +**Validates: Requirements 1.3, 2.2** + +### Property 4: Filter Consistency for Labels + +*For any* global jump query with a label parameter, all returned results should have label exactly matching the specified label. No result should have a different label. + +**Validates: Requirements 1.1, 1.2** + +### Property 5: Filter Consistency for Text Search + +*For any* global jump query with a query parameter (for transcript or OCR search), all returned results should contain the search term in their text field. The full-text search should match the query. + +**Validates: Requirements 3.1, 3.2, 4.1, 4.2** + +### Property 6: Empty Result Handling + +*For any* global jump query that finds no matching results, the response should contain an empty results array and has_more=false, without raising an error. + +**Validates: Requirements 8.5, 12.5, 14.4** + +### Property 7: Limit Enforcement + +*For any* global jump query with limit=N, the response should contain at most N results. If exactly N results are returned and more exist, has_more should be true. If fewer than N results exist, all available results should be returned and has_more should be false. + +**Validates: Requirements 5.4, 5.5** + +### Property 8: Video Existence Validation + +*For any* global jump query with a from_video_id that does not exist in the database, the system should return a 404 error with message "Video not found". + +**Validates: Requirements 8.1** + +### Property 9: Parameter Validation - Invalid Kind + +*For any* global jump query with an invalid kind parameter (not one of: object, face, transcript, ocr, scene, place, location), the system should return a 400 error with message "Invalid artifact kind". + +**Validates: Requirements 8.2** + +### Property 10: Parameter Validation - Invalid Direction + +*For any* global jump query with an invalid direction parameter (not "next" or "prev"), the system should return a 400 error with message "Direction must be 'next' or 'prev'". + +**Validates: Requirements 8.3** + +### Property 11: Parameter Validation - Conflicting Filters + +*For any* global jump query where both label and query parameters are specified, the system should return a 400 error with message "Cannot specify both label and query". + +**Validates: Requirements 8.4** + +### Property 12: Response Schema Completeness + +*For any* global jump result returned, the response should include all required fields: video_id, video_filename, file_created_at, jump_to (with start_ms and end_ms), artifact_id, and preview. No required field should be missing or null (except file_created_at which may be null). + +**Validates: Requirements 5.3, 7.1, 7.2, 7.3, 7.4, 7.5, 13.1** + +### Property 13: Arbitrary Position Navigation + +*For any* global jump query with from_video_id and from_ms parameters, the search should start from that position in the global timeline. Results should be chronologically after (for "next") or before (for "prev") that position. + +**Validates: Requirements 11.1, 11.2, 11.3** + +### Property 14: Optional from_ms Parameter + +*For any* global jump query where from_ms is omitted, the search should start from the beginning of the video (for "next") or the end of the video (for "prev"). The query should execute without error. + +**Validates: Requirements 11.5** + +### Property 15: Boundary Condition - from_ms Beyond Duration + +*For any* global jump query where from_ms is beyond the video duration, the system should treat it as the end of that video and search forward (for "next") or backward (for "prev") accordingly without error. + +**Validates: Requirements 11.4** + +### Property 16: Filter Change Independence + +*For any* two consecutive global jump queries with the same from_video_id and from_ms but different filters (e.g., label="dog" vs label="cat"), the results should be independent. The second query should not be affected by the first query's filter. + +**Validates: Requirements 12.1, 12.2, 12.3, 12.4** + +### Property 17: Result Chaining + +*For any* global jump result R, if we make a subsequent query with from_video_id=R.video_id and from_ms=R.end_ms, the new results should be chronologically after R in the global timeline. + +**Validates: Requirements 13.5, 14.2, 14.3** + +### Property 18: Cross-Video Navigation + +*For any* global jump query that returns a result from a different video than from_video_id, that result should be the first matching artifact in the next/previous video in the global timeline (based on direction). + +**Validates: Requirements 14.1, 14.5** + +### Property 19: Backward Compatibility + +*For any* request to the existing GET /videos/{video_id}/jump endpoint, the system should continue to return results scoped to that video only, without being affected by the new global jump feature. + +**Validates: Requirements 10.1, 10.3** + +### Property 20: Global Jump Independence + +*For any* request to the new GET /jump/global endpoint, the system should return results across all videos without affecting the behavior of the existing single-video jump endpoint. + +**Validates: Requirements 10.2, 10.4** + +## Error Handling + +### Error Categories + +**Validation Errors (400):** +- Invalid kind parameter +- Invalid direction parameter +- Conflicting parameters (label and query both specified) +- Invalid min_confidence (not between 0-1) +- Invalid limit (not between 1-50) +- Invalid from_ms (negative) + +**Not Found Errors (404):** +- from_video_id does not exist + +**Server Errors (500):** +- Database connection failure +- Unexpected query execution error +- Service initialization failure + +### Error Response Format + +```python +class ErrorResponse(BaseModel): + detail: str + error_code: str + timestamp: datetime +``` + +### Error Messages + +| Scenario | Message | Code | +|----------|---------|------| +| Invalid kind | "Invalid artifact kind. Must be one of: object, face, transcript, ocr, scene, place, location" | INVALID_KIND | +| Invalid direction | "Direction must be 'next' or 'prev'" | INVALID_DIRECTION | +| Conflicting filters | "Cannot specify both label and query parameters" | CONFLICTING_FILTERS | +| Video not found | "Video not found" | VIDEO_NOT_FOUND | +| Invalid confidence | "min_confidence must be between 0 and 1" | INVALID_CONFIDENCE | +| Invalid limit | "limit must be between 1 and 50" | INVALID_LIMIT | + +## Testing Strategy + +### Unit Testing Approach + +Unit tests verify specific examples, edge cases, and error conditions: + +1. **Parameter Validation Tests** + - Test each invalid parameter combination + - Verify correct error codes and messages + - Test boundary values (limit=1, limit=50, confidence=0, confidence=1) + +2. **Query Logic Tests** + - Test ordering correctness with mock data + - Test filter application (label, confidence, text query) + - Test direction reversal (next vs prev) + - Test boundary conditions (from_ms at video end, etc.) + +3. **Integration Tests** + - Test with real database (using test fixtures) + - Test cross-video navigation + - Test empty result handling + - Test with multiple videos in different chronological orders + +4. **Error Handling Tests** + - Test 404 for non-existent video + - Test 400 for invalid parameters + - Test graceful handling of database errors + +### Property-Based Testing Approach + +Property-based tests verify universal properties across many generated inputs: + +1. **Timeline Ordering Property Test** + - Generate random videos with random file_created_at dates + - Generate random artifacts in each video + - Run global jump queries + - Verify results are ordered by global timeline + - Minimum 100 iterations + +2. **Filter Consistency Property Test** + - Generate random artifacts with various labels and confidence scores + - Run queries with different filters + - Verify all results satisfy the filter criteria + - Minimum 100 iterations + +3. **Direction Symmetry Property Test** + - Generate random video/artifact data + - Run "next" query from position A + - Run "prev" query from result position + - Verify we can navigate back to original position + - Minimum 100 iterations + +4. **Boundary Handling Property Test** + - Generate videos with various durations + - Test from_ms at boundaries (0, duration, beyond duration) + - Verify no errors and correct results + - Minimum 100 iterations + +5. **Empty Result Property Test** + - Generate scenarios with no matching results + - Verify empty results array and has_more=false + - Verify no errors raised + - Minimum 100 iterations + +### Test Configuration + +- **Framework:** pytest with pytest-asyncio for async tests +- **Property Testing:** hypothesis for property-based tests +- **Database:** PostgreSQL test container +- **Fixtures:** Pre-populated test data with known ordering +- **Minimum iterations:** 100 per property test +- **Timeout:** 5 seconds per test + +### Test Tags + +Each property test includes a comment tag: +```python +# Feature: global-jump-navigation, Property N: [Property Title] +``` + +## Performance Considerations + +### Query Optimization + +1. **Composite Indexes** (Phase 3) + - `idx_object_labels_label_global` on (label, asset_id, start_ms) + - `idx_face_clusters_cluster_global` on (cluster_id, asset_id, start_ms) + - `idx_videos_created_at_id` on (file_created_at, video_id) + +2. **Full-Text Search Optimization** + - Existing GIN indexes on transcript_fts.text_tsv and ocr_fts.text_tsv + - PostgreSQL plainto_tsquery for text normalization + +3. **Query Execution** + - Use LIMIT to restrict result set early + - Avoid full table scans through proper indexing + - Target query execution time: < 500ms for typical queries + +### Scalability + +- Design supports millions of artifacts across thousands of videos +- Projection tables already denormalized for cross-video queries +- No new data structures required for MVP +- Future: Materialized views for very large datasets (Phase 3) + +## Future Enhancements (Out of Scope for MVP) + +### Phase 4: Multi-Filter Searches + +- Support AND/OR logic for combining multiple filters +- Example: "next frame with DOG AND CAT" +- Example: "next frame with Kubernetes text AND plant object" +- Requires query builder for complex filter combinations +- May require new database views or query optimization + +### Phase 5: Embedding-Based Search + +- Add pgvector extension for similarity search +- Store face embeddings for fuzzy face matching +- Implement CLIP embeddings for visual similarity +- Add embedding-based global jump queries + +### Phase 6: Advanced Filtering + +- Geo-spatial queries for location-based navigation +- Date range filtering +- Confidence score distribution analysis +- Multi-label filtering (AND/OR logic) + +### Phase 7: Performance Optimization + +- Materialized views for frequently accessed queries +- Query result caching +- Asynchronous index building +- Partitioning large projection tables by date + +## Backward Compatibility + +The new global jump feature is fully additive: +- Existing `/videos/{video_id}/jump` endpoint remains unchanged +- No modifications to existing projection tables +- No breaking changes to existing APIs +- New `/jump/global` endpoint is independent + +## API Endpoint Specification + +### GET /jump/global + +**Query Parameters:** +- `kind` (required): artifact type (object, face, transcript, ocr, scene, place, location) +- `direction` (required): next or prev +- `from_video_id` (required): current video ID +- `from_ms` (optional): current position in milliseconds (default: 0 for next, video end for prev) +- `label` (optional): filter by label (objects, places) +- `query` (optional): text search query (transcript, ocr) +- `face_cluster_id` (optional): filter by face cluster +- `min_confidence` (optional): minimum confidence threshold (0-1) +- `limit` (optional): max results (default: 1, max: 50) + +**Response:** +```json +{ + "results": [ + { + "video_id": "abc-123", + "video_filename": "beach_trip.mp4", + "file_created_at": "2025-05-19T02:22:21Z", + "jump_to": { + "start_ms": 15000, + "end_ms": 15500 + }, + "artifact_id": "artifact_xyz", + "preview": { + "label": "dog", + "confidence": 0.95 + } + } + ], + "has_more": true +} +``` + +**Status Codes:** +- 200: Success +- 400: Invalid parameters +- 404: Video not found +- 500: Server error + +## User Experience Flow + +### Initial Search Scenario + +1. **User initiates search** from the UI (e.g., clicks "Find next DOG" button, or uses a search form/bar) + - *Note: The specific UI implementation (search bar, form, buttons, etc.) is a frontend concern and not specified in this backend design* +2. **Frontend calls** `GET /jump/global?kind=object&label=dog&direction=next&from_video_id={current_video}&from_ms={current_position}` +3. **Backend returns** first matching result with video_id, filename, and jump_to timestamps +4. **Frontend behavior:** + - If result.video_id == current_video: Seek to result.jump_to.start_ms in current video + - If result.video_id != current_video: Navigate to new video and seek to result.jump_to.start_ms +5. **Video player** displays the artifact at the specified timestamp + +### Continuous Navigation Scenario + +1. **User watches** the artifact at jump_to.start_ms to jump_to.end_ms +2. **User clicks** "Find next DOG" again +3. **Frontend calls** `GET /jump/global?kind=object&label=dog&direction=next&from_video_id={result.video_id}&from_ms={result.end_ms}` +4. **Backend returns** next matching result (chronologically after the previous result) +5. **Process repeats** - navigate to new video or seek in current video + +### Filter Change Scenario + +1. **User is watching** a video with DOG search active +2. **User changes filter** to search for CAT instead +3. **Frontend calls** `GET /jump/global?kind=object&label=cat&direction=next&from_video_id={current_video}&from_ms={current_position}` +4. **Backend searches** for CAT starting from current position +5. **Results are independent** of previous DOG search - no state carried over + +### No Results Scenario + +1. **User searches** for a rare object (e.g., "giraffe") +2. **Backend returns** empty results array with has_more=false +3. **Frontend displays** "No results found" message +4. **User can** change filters and try again, or navigate manually + +### Timeline Exploration Scenario + +1. **User wants to explore** the entire video library chronologically +2. **User calls** `GET /jump/global?kind=object&label=dog&direction=next&from_video_id={first_video}&from_ms=0` +3. **Backend returns** first DOG occurrence in the library +4. **User clicks** "Next" repeatedly to browse all DOG occurrences in chronological order +5. **Each click** uses the previous result's position as the starting point for the next search diff --git a/.kiro/specs/global-jump-navigation/requirements.md b/.kiro/specs/global-jump-navigation/requirements.md new file mode 100644 index 0000000..6909f1e --- /dev/null +++ b/.kiro/specs/global-jump-navigation/requirements.md @@ -0,0 +1,186 @@ +# Requirements Document: Global Jump Navigation + +## Introduction + +Global Jump Navigation enables users to search for and navigate to specific artifacts (objects, faces, text, scenes) across their entire video library in chronological order. Instead of being limited to jumping within a single video, users can now query "next video with DOG" or "next occurrence of WORD" and seamlessly navigate across videos. This feature leverages existing projection tables (object_labels, face_clusters, transcript_fts, ocr_fts, scene_ranges, video_locations) to provide fast, cross-video navigation. + +## Glossary + +- **Artifact**: A detected or extracted element within a video (object, face, text, scene, location) +- **Projection Table**: Pre-computed, denormalized table containing artifact data across all videos (e.g., object_labels, transcript_fts) +- **Global Timeline**: Chronological ordering of all videos based on file_created_at (EXIF/filesystem date), then by artifact timestamp within each video +- **Jump**: Navigation action to move to a specific artifact occurrence +- **Kind**: Type of artifact being searched (object, face, transcript, ocr, scene, place, location) +- **Direction**: Navigation direction (next or prev) along the global timeline +- **Confidence**: Probability score (0-1) indicating detection confidence for objects and faces +- **Face Cluster**: Grouping of face detections representing the same person across videos +- **FTS**: Full-Text Search capability for text-based artifacts (transcript, OCR) +- **Asset ID**: Unique identifier for a video (video_id) +- **Artifact ID**: Unique identifier for a specific artifact occurrence + +## Requirements + +### Requirement 1: Cross-Video Navigation by Object Label + +**User Story:** As a video analyst, I want to navigate to the next or previous occurrence of a specific object (e.g., "dog", "car") across all my videos, so that I can quickly find all instances of that object without manually searching each video. + +#### Acceptance Criteria + +1. WHEN a user requests the next occurrence of an object label THEN THE GlobalJumpService SHALL query the object_labels projection table and return the first matching object in chronological order after the current position +2. WHEN a user requests the previous occurrence of an object label THEN THE GlobalJumpService SHALL query the object_labels projection table and return the first matching object in reverse chronological order before the current position +3. WHEN a user specifies a minimum confidence threshold THEN THE GlobalJumpService SHALL filter results to only include objects with confidence >= the specified threshold +4. WHEN multiple objects match the search criteria THEN THE GlobalJumpService SHALL order results by file_created_at (ascending for next, descending for prev), then by video_id, then by start_ms within the video +5. WHEN a user navigates to a result in a different video THEN THE System SHALL return the video_id, video_filename, file_created_at, and jump_to timestamps (start_ms, end_ms) + +### Requirement 2: Cross-Video Navigation by Face Cluster + +**User Story:** As a video editor, I want to find all occurrences of a specific person (face cluster) across my video library, so that I can quickly compile scenes featuring that person. + +#### Acceptance Criteria + +1. WHEN a user requests navigation by face cluster ID THEN THE GlobalJumpService SHALL query the face_clusters projection table and return matching face detections in chronological order +2. WHEN a user specifies a minimum confidence threshold for face detection THEN THE GlobalJumpService SHALL filter results to only include faces with confidence >= the specified threshold +3. WHEN a user navigates to a face result THEN THE System SHALL return the face cluster ID, confidence score, and temporal boundaries (start_ms, end_ms) +4. WHEN multiple face detections match the search criteria THEN THE GlobalJumpService SHALL order results by file_created_at, then video_id, then start_ms + +### Requirement 3: Cross-Video Full-Text Search in Transcripts + +**User Story:** As a researcher, I want to search for specific words or phrases across all video transcripts and navigate between occurrences, so that I can find all mentions of a topic across my video collection. + +#### Acceptance Criteria + +1. WHEN a user provides a text query THEN THE GlobalJumpService SHALL perform full-text search across the transcript_fts projection table using PostgreSQL FTS capabilities +2. WHEN a user requests the next occurrence of a search term THEN THE GlobalJumpService SHALL return the first matching transcript segment in chronological order after the current position +3. WHEN a user requests the previous occurrence of a search term THEN THE GlobalJumpService SHALL return the first matching transcript segment in reverse chronological order before the current position +4. WHEN multiple transcript segments match the search query THEN THE GlobalJumpService SHALL order results by file_created_at, then video_id, then start_ms +5. WHEN a user navigates to a transcript result THEN THE System SHALL return the matched text snippet and temporal boundaries + +### Requirement 4: Cross-Video Full-Text Search in OCR + +**User Story:** As a document analyst, I want to search for text that appears in video frames (OCR) across all videos, so that I can find all instances of specific text or document content. + +#### Acceptance Criteria + +1. WHEN a user provides a text query for OCR search THEN THE GlobalJumpService SHALL perform full-text search across the ocr_fts projection table +2. WHEN a user requests navigation by OCR text THEN THE GlobalJumpService SHALL return matching OCR segments in chronological order following the global timeline +3. WHEN multiple OCR segments match the search query THEN THE GlobalJumpService SHALL order results by file_created_at, then video_id, then start_ms +4. WHEN a user navigates to an OCR result THEN THE System SHALL return the matched text and temporal boundaries + +### Requirement 5: Global Jump API Endpoint + +**User Story:** As a frontend developer, I want a unified API endpoint for cross-video navigation, so that I can implement global jump functionality without managing multiple endpoints. + +#### Acceptance Criteria + +1. THE System SHALL provide a GET /jump/global endpoint that accepts query parameters for kind, direction, from_video_id, from_ms, label, query, face_cluster_id, min_confidence, and limit +2. WHEN a request is made to /jump/global THEN THE System SHALL validate all input parameters and return a 400 error if required parameters are missing or invalid +3. WHEN a valid request is made to /jump/global THEN THE System SHALL return a JSON response containing results array with video_id, video_filename, file_created_at, jump_to (start_ms, end_ms), artifact_id, and preview data +4. WHEN the limit parameter is specified THEN THE System SHALL return at most that many results (default 1, maximum 50) +5. WHEN results are returned THEN THE System SHALL include a has_more boolean indicating whether additional results exist beyond the limit + +### Requirement 6: Global Timeline Ordering + +**User Story:** As a user, I want consistent, predictable ordering when navigating across videos, so that I can understand the sequence of results and navigate intuitively. + +#### Acceptance Criteria + +1. THE GlobalJumpService SHALL order all results using a deterministic global timeline based on file_created_at (EXIF/filesystem date) as the primary sort key +2. WHEN two videos have the same file_created_at THEN THE GlobalJumpService SHALL use video_id as a secondary sort key to ensure deterministic ordering +3. WHEN navigating within the same video THEN THE GlobalJumpService SHALL use start_ms as the tertiary sort key to order artifacts chronologically +4. WHEN a user requests "next" THEN THE GlobalJumpService SHALL return results in ascending order along the global timeline +5. WHEN a user requests "prev" THEN THE GlobalJumpService SHALL return results in descending order along the global timeline + +### Requirement 7: Result Preview Data + +**User Story:** As a user, I want to see relevant preview information about each result before navigating to it, so that I can verify it's the result I'm looking for. + +#### Acceptance Criteria + +1. WHEN returning an object detection result THEN THE System SHALL include preview data containing the label and confidence score +2. WHEN returning a face cluster result THEN THE System SHALL include preview data containing the cluster ID and confidence score +3. WHEN returning a transcript search result THEN THE System SHALL include preview data containing the matched text snippet +4. WHEN returning an OCR search result THEN THE System SHALL include preview data containing the matched text snippet +5. WHEN returning a scene result THEN THE System SHALL include preview data containing the scene index or description + +### Requirement 8: Error Handling for Global Jump + +**User Story:** As a developer, I want clear error messages when global jump queries fail, so that I can debug issues and provide helpful feedback to users. + +#### Acceptance Criteria + +1. IF a user requests navigation from a non-existent video_id THEN THE System SHALL return a 404 error with message "Video not found" +2. IF a user provides an invalid kind parameter THEN THE System SHALL return a 400 error with message "Invalid artifact kind" +3. IF a user provides an invalid direction parameter THEN THE System SHALL return a 400 error with message "Direction must be 'next' or 'prev'" +4. IF a user provides conflicting parameters (e.g., both label and query) THEN THE System SHALL return a 400 error with message "Cannot specify both label and query" +5. IF no results are found matching the criteria THEN THE System SHALL return a 200 response with empty results array and has_more=false + +### Requirement 9: Performance and Scalability + +**User Story:** As a system administrator, I want global jump queries to execute efficiently even with large video libraries, so that users experience responsive navigation. + +#### Acceptance Criteria + +1. WHEN a global jump query is executed THEN THE System SHALL complete within 500ms for typical queries (< 10,000 videos) +2. WHEN querying by object label THEN THE System SHALL use composite indexes on (label, asset_id, start_ms) to optimize performance +3. WHEN querying by face cluster THEN THE System SHALL use composite indexes on (cluster_id, asset_id, start_ms) to optimize performance +4. WHEN querying by text (transcript or OCR) THEN THE System SHALL use existing GIN indexes on full-text search vectors +5. WHEN querying by video ordering THEN THE System SHALL use composite indexes on (file_created_at, video_id) to optimize timeline ordering + +### Requirement 10: Backward Compatibility + +**User Story:** As a developer, I want the new global jump feature to coexist with existing single-video jump functionality, so that I can migrate gradually without breaking existing code. + +#### Acceptance Criteria + +1. THE existing GET /videos/{video_id}/jump endpoint SHALL remain unchanged and functional +2. THE new GET /jump/global endpoint SHALL be additive and not modify existing video jump behavior +3. WHEN a user uses the existing single-video jump endpoint THEN THE System SHALL continue to return results scoped to that video only +4. WHEN a user uses the new global jump endpoint THEN THE System SHALL return results across all videos without affecting single-video jump functionality + +### Requirement 11: Jump from Arbitrary Timeline Position + +**User Story:** As a user, I want to start a global jump search from any point in the timeline (not just the current video), so that I can explore different branches of the timeline without navigating to that video first. + +#### Acceptance Criteria + +1. WHEN a user specifies from_video_id and from_ms parameters THEN THE GlobalJumpService SHALL treat that position as the starting point for the global timeline search +2. WHEN a user requests "next" from an arbitrary position THEN THE GlobalJumpService SHALL return results chronologically after that position +3. WHEN a user requests "prev" from an arbitrary position THEN THE GlobalJumpService SHALL return results chronologically before that position +4. WHEN a user specifies a from_video_id that exists but from_ms is beyond the video duration THEN THE System SHALL treat it as the end of that video and search forward/backward accordingly +5. THE from_ms parameter SHALL be optional; if omitted, the search SHALL start from the beginning (for "next") or end (for "prev") of the specified video + +### Requirement 12: Dynamic Filter Changes + +**User Story:** As a user, I want to change search filters while navigating the timeline, so that I can switch from searching for "dogs" to "cats" without losing my position in the timeline. + +#### Acceptance Criteria + +1. WHEN a user changes the search filter (e.g., from label="dog" to label="cat") THEN THE System SHALL accept the new filter in a subsequent /jump/global request +2. WHEN a user changes filters while maintaining the same from_video_id and from_ms THEN THE GlobalJumpService SHALL search for the new filter starting from that same timeline position +3. WHEN a user changes from one artifact kind to another (e.g., from object to transcript) THEN THE System SHALL route to the appropriate projection table and return results of the new kind +4. WHEN a user changes filters THEN THE System SHALL NOT require re-navigation to a specific video; the timeline position (from_video_id, from_ms) remains the reference point +5. WHEN a user changes filters and no results exist for the new filter THEN THE System SHALL return an empty results array and has_more=false, allowing the user to try different filters + +### Requirement 13: Direct Navigation to Result Artifact + +**User Story:** As a user, I want to navigate directly to the exact timestamp of a search result, so that I can immediately see the artifact I'm looking for without manual seeking. + +#### Acceptance Criteria + +1. WHEN a global jump result is returned THEN THE System SHALL include jump_to object containing start_ms and end_ms for the artifact +2. WHEN a user navigates to a result in a different video THEN THE System SHALL automatically load that video and seek to the start_ms timestamp +3. WHEN a user navigates to a result in the same video THEN THE System SHALL seek to the start_ms timestamp without reloading the video +4. WHEN a user navigates to a result THEN THE System SHALL highlight or focus on the artifact (start_ms to end_ms) to draw attention to the specific occurrence +5. WHEN a user requests the next result after navigating to a result THEN THE GlobalJumpService SHALL use the result's video_id and end_ms as the new from_video_id and from_ms for the next search + +### Requirement 14: Navigate to Next Video in Timeline + +**User Story:** As a user, I want to jump to the next video in the global timeline, so that I can browse through my video library sequentially. + +#### Acceptance Criteria + +1. WHEN a user requests navigation to the next video (without specifying a search filter) THEN THE System SHALL return the first artifact from the next video in chronological order +2. WHEN a user is at the end of a video and requests "next" THEN THE GlobalJumpService SHALL return results from the next video in the global timeline +3. WHEN a user is at the beginning of a video and requests "prev" THEN THE GlobalJumpService SHALL return results from the previous video in the global timeline +4. WHEN a user requests next/prev video navigation and no next/previous video exists THEN THE System SHALL return an empty results array and has_more=false +5. WHEN navigating between videos THEN THE System SHALL maintain the same search filter (kind, label, query) across video boundaries diff --git a/.kiro/specs/global-jump-navigation/tasks.md b/.kiro/specs/global-jump-navigation/tasks.md new file mode 100644 index 0000000..1240c89 --- /dev/null +++ b/.kiro/specs/global-jump-navigation/tasks.md @@ -0,0 +1,346 @@ +# Implementation Plan: Global Jump Navigation + +## Overview + +This implementation plan breaks down the Global Jump Navigation feature into discrete, incremental tasks. The feature enables cross-video artifact search and navigation using a unified API endpoint. Tasks are organized to build functionality piece by piece, with testing integrated at each step. + +The implementation follows a bottom-up approach: start with data models and service logic, then expose through API endpoints, then optimize with database indexes. + +## Tasks + +- [x] 1. Set up project structure and core data models + - Create `backend/src/global_jump/` module directory + - Create `backend/src/global_jump/__init__.py` + - Create `backend/src/global_jump/models.py` with GlobalJumpResult and JumpTo dataclasses + - Create `backend/src/global_jump/exceptions.py` with GlobalJumpException, VideoNotFoundError, InvalidParameterError + - _Requirements: 5.1, 8.1, 8.2, 8.3, 8.4_ + +- [x] 2. Create Pydantic response schemas + - Create `backend/src/global_jump/schemas.py` + - Implement JumpToSchema with start_ms and end_ms fields + - Implement GlobalJumpResultSchema with all required fields (video_id, video_filename, file_created_at, jump_to, artifact_id, preview) + - Implement GlobalJumpResponseSchema with results array and has_more boolean + - Add comprehensive docstrings and field descriptions + - _Requirements: 5.3, 7.1, 7.2, 7.3, 7.4, 7.5, 13.1_ + +- [ ]* 2.1 Write unit tests for response schemas + - **Property 12: Response Schema Completeness** + - **Validates: Requirements 5.3, 7.1, 7.2, 7.3, 7.4, 7.5, 13.1** + +- [x] 3. Implement GlobalJumpService - core infrastructure + - Create `backend/src/global_jump/service.py` + - Implement GlobalJumpService class with __init__ method accepting session and artifact_repo + - Implement _get_video() helper method to fetch video by ID and raise VideoNotFoundError if not found + - Implement _to_global_result() helper method to convert database rows to GlobalJumpResult objects + - Add comprehensive docstrings + - _Requirements: 1.1, 1.2, 2.1, 3.1, 4.1, 6.1_ + +- [ ]* 3.1 Write unit tests for service initialization and helpers + - Test _get_video() with valid and invalid video IDs + - Test _to_global_result() with various artifact types + - _Requirements: 8.1_ + +- [x] 4. Implement object label search in GlobalJumpService + - Implement _search_objects_global() method for "next" direction + - Build query on object_labels projection table joined with videos + - Apply label filter + - Apply min_confidence filter + - Apply direction-specific WHERE clause for "next" (after current position) + - Order by file_created_at ASC, video_id ASC, start_ms ASC + - Limit results + - Return list of GlobalJumpResult objects + - _Requirements: 1.1, 1.3, 1.4, 1.5, 6.1, 6.2, 6.3, 6.4_ + +- [ ]* 4.1 Write property test for object search ordering + - **Property 1: Global Timeline Ordering Consistency** + - **Validates: Requirements 6.1, 6.2, 6.3, 6.4** + +- [ ]* 4.2 Write property test for object confidence filtering + - **Property 3: Filter Consistency for Confidence** + - **Validates: Requirements 1.3, 2.2** + +- [ ]* 4.3 Write property test for object label filtering + - **Property 4: Filter Consistency for Labels** + - **Validates: Requirements 1.1, 1.2** + +- [x] 5. Implement object label search - "prev" direction + - Implement _search_objects_global() method for "prev" direction + - Apply direction-specific WHERE clause for "prev" (before current position) + - Order by file_created_at DESC, video_id DESC, start_ms DESC + - Return list of GlobalJumpResult objects + - _Requirements: 1.2, 6.5_ + +- [ ]* 5.1 Write property test for reverse direction ordering + - **Property 2: Reverse Direction Ordering** + - **Validates: Requirements 6.5** + +- [ ] 6. Implement face cluster search in GlobalJumpService + - Implement _search_faces_global() method for both "next" and "prev" directions + - Build query on face_clusters projection table joined with videos + - Apply face_cluster_id filter + - Apply min_confidence filter + - Apply direction-specific WHERE clause + - Order by global timeline (ascending for next, descending for prev) + - Return list of GlobalJumpResult objects with face-specific preview data + - _Requirements: 2.1, 2.2, 2.3, 2.4_ + +- [ ]* 6.1 Write property test for face cluster search + - Test both "next" and "prev" directions + - Verify confidence filtering + - _Requirements: 2.1, 2.2, 2.4_ + +- [x] 7. Implement transcript full-text search in GlobalJumpService + - Implement _search_transcript_global() method for both "next" and "prev" directions + - Build query on transcript_fts projection table joined with videos + - Use PostgreSQL plainto_tsquery for text normalization + - Apply FTS filter using @@ operator on text_tsv column + - Apply direction-specific WHERE clause + - Order by global timeline + - Return list of GlobalJumpResult objects with text snippet in preview + - _Requirements: 3.1, 3.2, 3.3, 3.4, 3.5_ + +- [ ]* 7.1 Write property test for transcript FTS search + - **Property 5: Filter Consistency for Text Search** + - **Validates: Requirements 3.1, 3.2, 4.1, 4.2** + +- [x] 8. Implement OCR full-text search in GlobalJumpService + - Implement _search_ocr_global() method for both "next" and "prev" directions + - Build query on ocr_fts projection table joined with videos + - Use PostgreSQL plainto_tsquery for text normalization + - Apply FTS filter using @@ operator on text_tsv column + - Apply direction-specific WHERE clause + - Order by global timeline + - Return list of GlobalJumpResult objects with text snippet in preview + - _Requirements: 4.1, 4.2, 4.3, 4.4_ + +- [ ]* 8.1 Write property test for OCR FTS search + - Test both "next" and "prev" directions + - Verify text matching + - _Requirements: 4.1, 4.2, 4.3_ + +- [x] 9. Implement public jump_next() method in GlobalJumpService + - Implement async jump_next() method that routes to appropriate search method based on kind + - Handle kind="object" → _search_objects_global(direction="next") + - Handle kind="face" → _search_faces_global(direction="next") + - Handle kind="transcript" → _search_transcript_global(direction="next") + - Handle kind="ocr" → _search_ocr_global(direction="next") + - Handle kind="scene" → _search_scenes_global(direction="next") + - Handle kind="place" → _search_places_global(direction="next") + - Handle kind="location" → _search_locations_global(direction="next") + - Raise InvalidParameterError for unknown kind + - _Requirements: 1.1, 2.1, 3.1, 4.1, 5.1, 6.1, 12.3_ + +- [ ]* 9.1 Write unit tests for jump_next() routing + - Test each kind parameter routes to correct search method + - Test invalid kind raises InvalidParameterError + - _Requirements: 8.2_ + +- [x] 10. Implement public jump_prev() method in GlobalJumpService + - Implement async jump_prev() method that routes to appropriate search method based on kind + - Handle all kind values with direction="prev" + - Raise InvalidParameterError for unknown kind + - _Requirements: 1.2, 2.1, 3.3, 4.2, 5.1, 6.5, 12.3_ + +- [ ]* 10.1 Write unit tests for jump_prev() routing + - Test each kind parameter routes to correct search method + - Test invalid kind raises InvalidParameterError + - _Requirements: 8.2_ + +- [x] 11. Implement scene and place search methods + - Implement _search_scenes_global() method for both directions + - Implement _search_places_global() method for both directions (using object_labels with place-specific labels) + - Apply appropriate filters and ordering + - Return GlobalJumpResult objects with scene/place-specific preview data + - _Requirements: 5.1, 7.5_ + +- [x] 12. Implement location search method + - Implement _search_locations_global() method for both directions + - Build query on video_locations projection table + - Apply optional geo_bounds filtering if provided + - Order by global timeline + - Return GlobalJumpResult objects with location preview data + - _Requirements: 5.1_ + +- [x] 13. Create GlobalJumpController with parameter validation + - Create `backend/src/global_jump/router.py` + - Implement global_jump() endpoint handler + - Add parameter validation: + - kind must be one of: object, face, transcript, ocr, scene, place, location + - direction must be: next or prev + - from_video_id must be non-empty string + - from_ms must be non-negative integer (optional) + - label and query are mutually exclusive + - min_confidence must be between 0 and 1 (optional) + - limit must be between 1 and 50 + - Raise 400 errors with descriptive messages for validation failures + - _Requirements: 5.1, 5.2, 8.2, 8.3, 8.4_ + +- [ ]* 13.1 Write unit tests for parameter validation + - **Property 9: Parameter Validation - Invalid Kind** + - **Property 10: Parameter Validation - Invalid Direction** + - **Property 11: Parameter Validation - Conflicting Filters** + - **Validates: Requirements 8.2, 8.3, 8.4** + +- [x] 14. Implement GET /jump/global endpoint + - Create APIRouter with prefix="/jump" and tags=["global-navigation"] + - Implement @router.get("/global", response_model=GlobalJumpResponseSchema) + - Call GlobalJumpService.jump_next() or jump_prev() based on direction parameter + - Handle VideoNotFoundError and return 404 response + - Handle InvalidParameterError and return 400 response + - Return GlobalJumpResponseSchema with results and has_more + - Add comprehensive OpenAPI documentation + - _Requirements: 5.1, 5.3, 5.4, 5.5_ + +- [ ]* 14.1 Write integration tests for /jump/global endpoint + - Test successful object search across multiple videos + - Test successful face cluster search + - Test successful transcript search + - Test successful OCR search + - Test 404 for non-existent video + - Test 400 for invalid parameters + - _Requirements: 5.1, 5.2, 5.3, 8.1, 8.2, 8.3, 8.4_ + +- [x] 15. Implement error handling and response formatting + - Ensure all error responses include detail, error_code, and timestamp + - Implement proper HTTP status codes (200, 400, 404, 500) + - Add error logging for debugging + - Test error scenarios + - _Requirements: 8.1, 8.2, 8.3, 8.4, 8.5_ + +- [ ]* 15.1 Write property test for empty result handling + - **Property 6: Empty Result Handling** + - **Validates: Requirements 8.5, 12.5, 14.4** + +- [ ]* 15.2 Write property test for limit enforcement + - **Property 7: Limit Enforcement** + - **Validates: Requirements 5.4, 5.5** + +- [x] 16. Implement optional from_ms parameter handling + - Update GlobalJumpService methods to handle from_ms=None + - For "next" direction: default from_ms to 0 (start of video) + - For "prev" direction: default from_ms to video duration (end of video) + - Update endpoint to accept from_ms as optional query parameter + - _Requirements: 11.5_ + +- [ ]* 16.1 Write property test for optional from_ms parameter + - **Property 14: Optional from_ms Parameter** + - **Validates: Requirements 11.5** + +- [x] 17. Implement boundary condition handling for from_ms + - Update GlobalJumpService to handle from_ms beyond video duration + - Treat from_ms > duration as end of video for "next" direction + - Treat from_ms > duration as end of video for "prev" direction + - No errors should be raised + - _Requirements: 11.4_ + +- [ ]* 17.1 Write property test for boundary conditions + - **Property 15: Boundary Condition - from_ms Beyond Duration** + - **Validates: Requirements 11.4** + +- [x] 18. Implement arbitrary position navigation + - Verify GlobalJumpService correctly handles any from_video_id and from_ms combination + - Results should be chronologically after (for "next") or before (for "prev") that position + - Test with various video orderings and timestamps + - _Requirements: 11.1, 11.2, 11.3_ + +- [ ]* 18.1 Write property test for arbitrary position navigation + - **Property 13: Arbitrary Position Navigation** + - **Validates: Requirements 11.1, 11.2, 11.3** + +- [x] 19. Implement filter change independence + - Verify that changing filters (label, query, kind) doesn't affect timeline position + - Each query is independent and doesn't carry state from previous queries + - Test with multiple consecutive queries with different filters + - _Requirements: 12.1, 12.2, 12.3, 12.4_ + +- [ ]* 19.1 Write property test for filter change independence + - **Property 16: Filter Change Independence** + - **Validates: Requirements 12.1, 12.2, 12.3, 12.4** + +- [x] 20. Implement result chaining capability + - Verify that using a result's video_id and end_ms as the next starting point works correctly + - Results should be chronologically after the previous result + - Test continuous navigation through multiple results + - _Requirements: 13.5, 14.2, 14.3_ + +- [ ]* 20.1 Write property test for result chaining + - **Property 17: Result Chaining** + - **Validates: Requirements 13.5, 14.2, 14.3** + +- [x] 21. Implement cross-video navigation correctness + - Verify that results from different videos are the first matching artifacts in the next/previous video + - Test with multiple videos in different chronological orders + - Verify video_id in results matches expected video + - _Requirements: 14.1, 14.5_ + +- [ ]* 21.1 Write property test for cross-video navigation + - **Property 18: Cross-Video Navigation** + - **Validates: Requirements 14.1, 14.5** + +- [x] 22. Verify backward compatibility with single-video jump + - Ensure existing GET /videos/{video_id}/jump endpoint still works + - Verify single-video jump returns results scoped to that video only + - Verify new global jump doesn't affect single-video jump behavior + - _Requirements: 10.1, 10.2, 10.3, 10.4_ + +- [ ]* 22.1 Write integration tests for backward compatibility + - **Property 19: Backward Compatibility** + - **Property 20: Global Jump Independence** + - **Validates: Requirements 10.1, 10.2, 10.3, 10.4** + +- [x] 23. Checkpoint - Ensure all unit and property tests pass + - Run all unit tests: `cd backend && poetry run pytest tests/global_jump/ -v` + - Run all property tests: `cd backend && poetry run pytest tests/global_jump/ -v -k property` + - Verify 100% test pass rate + - Address any failing tests + - Ensure all tests complete within reasonable time + - _Requirements: All_ + +- [x] 24. Add composite database indexes for optimization + - Create migration file: `backend/alembic/versions/xxx_add_global_jump_indexes.py` + - Add index on object_labels(label, asset_id, start_ms) + - Add index on face_clusters(cluster_id, asset_id, start_ms) + - Add index on videos(file_created_at, video_id) + - Run migration: `cd backend && poetry run alembic upgrade head` + - Verify indexes are created in database + - _Requirements: 9.2, 9.3, 9.5_ + +- [ ]* 24.1 Write performance tests for query execution time + - Test object search completes within 500ms + - Test face search completes within 500ms + - Test transcript search completes within 500ms + - _Requirements: 9.1_ + +- [x] 25. Register GlobalJumpRouter in main application + - Import GlobalJumpRouter in `backend/src/main.py` + - Register router with app: `app.include_router(global_jump_router)` + - Verify endpoint is accessible at GET /jump/global + - Test endpoint with curl or API client + - _Requirements: 5.1_ + +- [x] 26. Add comprehensive API documentation + - Add docstrings to all endpoint handlers + - Add response examples to OpenAPI schema + - Document all query parameters with descriptions + - Document all error responses + - Verify documentation appears in /docs endpoint + - _Requirements: 5.1, 5.3_ + +- [x] 27. Final checkpoint - Run all tests and quality checks + - Run all tests: `cd backend && poetry run pytest tests/ -v` + - Run format check: `cd backend && poetry run ruff format --check src tests` + - Run lint check: `cd backend && poetry run ruff check src tests` + - Fix any formatting or linting issues + - Verify all tests pass + - Ensure code quality gates are met + - _Requirements: All_ + +## Notes + +- Tasks marked with `*` are optional and can be skipped for faster MVP, but comprehensive testing is recommended +- Each task references specific requirements for traceability +- Property tests should run minimum 100 iterations each +- All code must follow FastAPI best practices and PEP 8 style guide +- Use async/await for all I/O operations +- Comprehensive error handling with descriptive messages +- All endpoints must have OpenAPI documentation diff --git a/backend/alembic/versions/g1h2i3j4k5l6_add_global_jump_indexes.py b/backend/alembic/versions/g1h2i3j4k5l6_add_global_jump_indexes.py new file mode 100644 index 0000000..399e255 --- /dev/null +++ b/backend/alembic/versions/g1h2i3j4k5l6_add_global_jump_indexes.py @@ -0,0 +1,66 @@ +"""add_global_jump_indexes + +Revision ID: g1h2i3j4k5l6 +Revises: f9a0b1c2d3e4 +Create Date: 2026-01-30 14:00:00.000000 + +""" +from collections.abc import Sequence + +from alembic import op + +# revision identifiers, used by Alembic. +revision: str = "g1h2i3j4k5l6" +down_revision: str | Sequence[str] | None = "f9a0b1c2d3e4" +branch_labels: str | Sequence[str] | None = None +depends_on: str | Sequence[str] | None = None + + +def upgrade() -> None: + """Upgrade schema. + + Add composite indexes for global jump navigation query optimization. + These indexes support cross-video artifact search with efficient + timeline ordering based on file_created_at. + + Requirements: 9.2, 9.3, 9.5 + """ + # Index for global jump object label queries + # Optimizes: SELECT ... FROM object_labels WHERE label = ? + # ORDER BY asset_id, start_ms + # Supports requirement 9.2: composite index on (label, asset_id, start_ms) + op.create_index( + "idx_object_labels_label_global", + "object_labels", + ["label", "asset_id", "start_ms"], + ) + + # Index for global jump face cluster queries + # Optimizes: SELECT ... FROM face_clusters WHERE cluster_id = ? + # ORDER BY asset_id, start_ms + # Supports requirement 9.3: composite index on (cluster_id, asset_id, start_ms) + op.create_index( + "idx_face_clusters_cluster_global", + "face_clusters", + ["cluster_id", "asset_id", "start_ms"], + ) + + # Index for global timeline ordering on videos table + # Optimizes: ORDER BY file_created_at, video_id for cross-video navigation + # Supports requirement 9.5: composite index on (file_created_at, video_id) + op.create_index( + "idx_videos_created_at_id", + "videos", + ["file_created_at", "video_id"], + ) + + +def downgrade() -> None: + """Downgrade schema. + + Remove composite indexes for global jump navigation. + """ + # Drop indexes in reverse order + op.drop_index("idx_videos_created_at_id", "videos") + op.drop_index("idx_face_clusters_cluster_global", "face_clusters") + op.drop_index("idx_object_labels_label_global", "object_labels") diff --git a/backend/src/api/global_jump_controller.py b/backend/src/api/global_jump_controller.py new file mode 100644 index 0000000..9cfdb8d --- /dev/null +++ b/backend/src/api/global_jump_controller.py @@ -0,0 +1,620 @@ +"""Controller for Global Jump Navigation API.""" + +import logging +from datetime import datetime, timezone + +from fastapi import APIRouter, Depends, Query, status +from fastapi.responses import JSONResponse +from sqlalchemy.orm import Session + +from ..database.connection import get_db +from ..domain.exceptions import InvalidParameterError, VideoNotFoundError +from ..domain.schema_registry import SchemaRegistry +from ..repositories.artifact_repository import SqlArtifactRepository +from ..services.global_jump_service import GlobalJumpService +from ..services.projection_sync_service import ProjectionSyncService +from .schemas import ( + ErrorResponseSchema, + GlobalJumpResponseSchema, + GlobalJumpResultSchema, + JumpToSchema, +) + +logger = logging.getLogger(__name__) + +router = APIRouter(prefix="/jump", tags=["global-navigation"]) + +VALID_KINDS = {"object", "face", "transcript", "ocr", "scene", "place", "location"} +VALID_DIRECTIONS = {"next", "prev"} + +# Error codes for consistent error handling +ERROR_CODES = { + "INVALID_VIDEO_ID": "INVALID_VIDEO_ID", + "INVALID_KIND": "INVALID_KIND", + "INVALID_DIRECTION": "INVALID_DIRECTION", + "CONFLICTING_FILTERS": "CONFLICTING_FILTERS", + "INVALID_FROM_MS": "INVALID_FROM_MS", + "INVALID_CONFIDENCE": "INVALID_CONFIDENCE", + "INVALID_LIMIT": "INVALID_LIMIT", + "VIDEO_NOT_FOUND": "VIDEO_NOT_FOUND", + "INTERNAL_ERROR": "INTERNAL_ERROR", +} + + +def create_error_response( + status_code: int, detail: str, error_code: str +) -> JSONResponse: + """Create a consistent error response with detail, error_code, and timestamp.""" + error_data = ErrorResponseSchema( + detail=detail, + error_code=error_code, + timestamp=datetime.now(timezone.utc), + ) + return JSONResponse( + status_code=status_code, + content=error_data.model_dump(mode="json"), + ) + + +def get_global_jump_service(session: Session = Depends(get_db)) -> GlobalJumpService: + """Dependency injection for GlobalJumpService.""" + schema_registry = SchemaRegistry() + projection_sync = ProjectionSyncService(session) + artifact_repo = SqlArtifactRepository(session, schema_registry, projection_sync) + return GlobalJumpService(session, artifact_repo) + + +@router.get( + "/global", + response_model=GlobalJumpResponseSchema, + summary="Global Jump Navigation", + description="""Navigate across videos to find artifacts in chronological order. + +## Overview + +This endpoint enables **cross-video artifact search and navigation** using a +unified API. Users can search for objects, faces, text (transcript/OCR), +scenes, places, and locations across their entire video library. + +## Global Timeline Concept + +Results are ordered using a **deterministic global timeline** based on: +1. **Primary sort**: `file_created_at` (EXIF/filesystem date of the video) +2. **Secondary sort**: `video_id` (for deterministic ordering when dates match) +3. **Tertiary sort**: `start_ms` (artifact timestamp within the video) + +This ensures consistent, predictable navigation across your video library. + +## Supported Artifact Kinds + +| Kind | Description | Required Parameters | +|------|-------------|---------------------| +| `object` | Detected object labels (e.g., "dog", "car") | `label` | +| `face` | Face cluster detections | `face_cluster_id` | +| `transcript` | Full-text search in video transcripts | `query` | +| `ocr` | Full-text search in on-screen text | `query` | +| `scene` | Scene boundary navigation | None | +| `place` | Detected place labels | `label` | +| `location` | GPS location data | None | + +## Navigation Directions + +- **`next`**: Find artifacts chronologically **after** the current position +- **`prev`**: Find artifacts chronologically **before** the current position + +## Result Chaining (Pagination) + +To navigate through all occurrences sequentially: +1. Make initial request with your starting position +2. Use the returned `video_id` and `jump_to.end_ms` as `from_video_id` + and `from_ms` for the next request +3. Continue until `has_more` is `false` + +## Example Usage + +**Find next dog in video library:** +``` +GET /jump/global?kind=object&label=dog&direction=next&from_video_id=abc-123 +``` + +**Search for word in transcripts:** +``` +GET /jump/global?kind=transcript&query=hello&direction=next&from_video_id=abc +``` + +**Navigate to previous face occurrence:** +``` +GET /jump/global?kind=face&face_cluster_id=face-001&direction=prev&from_video_id=abc +``` +""", + responses={ + 200: { + "description": "Successful response with matching artifacts", + "model": GlobalJumpResponseSchema, + "content": { + "application/json": { + "examples": { + "object_search": { + "summary": "Object search result", + "description": ( + "Example response when searching for objects" + ), + "value": { + "results": [ + { + "video_id": "abc-123", + "video_filename": "beach_trip.mp4", + "file_created_at": "2025-05-19T02:22:21Z", + "jump_to": {"start_ms": 15000, "end_ms": 15500}, + "artifact_id": "obj_xyz_001", + "preview": {"label": "dog", "confidence": 0.95}, + } + ], + "has_more": True, + }, + }, + "transcript_search": { + "summary": "Transcript search result", + "description": "Example response for transcripts", + "value": { + "results": [ + { + "video_id": "def-456", + "video_filename": "meeting_2025.mp4", + "file_created_at": "2025-06-01T10:30:00Z", + "jump_to": {"start_ms": 45000, "end_ms": 48000}, + "artifact_id": "trans_abc_002", + "preview": { + "text": "...discussed the project..." + }, + } + ], + "has_more": False, + }, + }, + "face_search": { + "summary": "Face cluster search result", + "description": ("Example response when searching by face"), + "value": { + "results": [ + { + "video_id": "ghi-789", + "video_filename": "family_reunion.mp4", + "file_created_at": "2025-04-15T14:00:00Z", + "jump_to": { + "start_ms": 120000, + "end_ms": 125000, + }, + "artifact_id": "face_def_003", + "preview": { + "cluster_id": "person_001", + "confidence": 0.89, + }, + } + ], + "has_more": True, + }, + }, + "empty_results": { + "summary": "No results found", + "description": "Response when no matching artifacts exist", + "value": { + "results": [], + "has_more": False, + }, + }, + "multiple_results": { + "summary": "Multiple results with pagination", + "description": ( + "Response with multiple results when limit > 1" + ), + "value": { + "results": [ + { + "video_id": "abc-123", + "video_filename": "video1.mp4", + "file_created_at": "2025-01-01T00:00:00Z", + "jump_to": {"start_ms": 1000, "end_ms": 1500}, + "artifact_id": "art_001", + "preview": {"label": "car", "confidence": 0.92}, + }, + { + "video_id": "abc-123", + "video_filename": "video1.mp4", + "file_created_at": "2025-01-01T00:00:00Z", + "jump_to": {"start_ms": 5000, "end_ms": 5500}, + "artifact_id": "art_002", + "preview": {"label": "car", "confidence": 0.88}, + }, + ], + "has_more": True, + }, + }, + } + } + }, + }, + 400: { + "description": "Invalid request parameters", + "model": ErrorResponseSchema, + "content": { + "application/json": { + "examples": { + "invalid_kind": { + "summary": "Invalid artifact kind", + "description": ( + "Returned when kind is not a valid artifact type" + ), + "value": { + "detail": "Invalid artifact kind. Must be one of: " + "face, location, object, ocr, place, scene, transcript", + "error_code": "INVALID_KIND", + "timestamp": "2025-05-19T02:22:21Z", + }, + }, + "invalid_direction": { + "summary": "Invalid direction", + "description": ( + "Returned when direction is not 'next' or 'prev'" + ), + "value": { + "detail": "Direction must be 'next' or 'prev'", + "error_code": "INVALID_DIRECTION", + "timestamp": "2025-05-19T02:22:21Z", + }, + }, + "conflicting_filters": { + "summary": "Conflicting filters", + "description": ( + "Returned when both label and query are specified" + ), + "value": { + "detail": ( + "Cannot specify both label and query parameters" + ), + "error_code": "CONFLICTING_FILTERS", + "timestamp": "2025-05-19T02:22:21Z", + }, + }, + "invalid_confidence": { + "summary": "Invalid confidence value", + "description": ( + "Returned when min_confidence is outside 0-1 range" + ), + "value": { + "detail": "min_confidence must be between 0 and 1", + "error_code": "INVALID_CONFIDENCE", + "timestamp": "2025-05-19T02:22:21Z", + }, + }, + "invalid_limit": { + "summary": "Invalid limit value", + "description": ( + "Returned when limit is outside 1-50 range" + ), + "value": { + "detail": "limit must be between 1 and 50", + "error_code": "INVALID_LIMIT", + "timestamp": "2025-05-19T02:22:21Z", + }, + }, + "invalid_from_ms": { + "summary": "Invalid from_ms value", + "description": "Returned when from_ms is negative", + "value": { + "detail": "from_ms must be a non-negative integer", + "error_code": "INVALID_FROM_MS", + "timestamp": "2025-05-19T02:22:21Z", + }, + }, + "empty_video_id": { + "summary": "Empty video ID", + "description": ("Returned when from_video_id is empty"), + "value": { + "detail": "from_video_id must be a non-empty string", + "error_code": "INVALID_VIDEO_ID", + "timestamp": "2025-05-19T02:22:21Z", + }, + }, + } + } + }, + }, + 404: { + "description": "Video not found", + "model": ErrorResponseSchema, + "content": { + "application/json": { + "example": { + "detail": "Video not found", + "error_code": "VIDEO_NOT_FOUND", + "timestamp": "2025-05-19T02:22:21Z", + } + } + }, + }, + 500: { + "description": "Internal server error", + "model": ErrorResponseSchema, + "content": { + "application/json": { + "example": { + "detail": "An unexpected error occurred", + "error_code": "INTERNAL_ERROR", + "timestamp": "2025-05-19T02:22:21Z", + } + } + }, + }, + }, +) +async def global_jump( + kind: str = Query( + ..., + description=( + "Type of artifact to search for. Determines which projection table " + "is queried and what filter parameters are applicable." + ), + examples=["object", "face", "transcript", "ocr", "scene", "place", "location"], + ), + direction: str = Query( + ..., + description=( + "Navigation direction along the global timeline. 'next' returns " + "artifacts chronologically after the current position, 'prev' returns " + "artifacts chronologically before." + ), + examples=["next", "prev"], + ), + from_video_id: str = Query( + ..., + description=( + "Starting video ID for the search. The search begins from this video's " + "position in the global timeline. Must be a valid, existing video ID." + ), + examples=["abc-123", "video_001"], + ), + from_ms: int | None = Query( + None, + description=( + "Starting timestamp in milliseconds within the from_video_id. " + "If omitted, defaults to 0 for 'next' direction (start of video) " + "or video duration for 'prev' direction (end of video). " + "Values beyond video duration are treated as end of video." + ), + examples=[0, 15000, 120000], + ge=0, + ), + label: str | None = Query( + None, + description=( + "Filter by artifact label. Used with kind='object' or kind='place'. " + "Case-sensitive exact match. Cannot be combined with 'query'." + ), + examples=["dog", "car", "person", "beach", "office"], + ), + query: str | None = Query( + None, + description=( + "Full-text search query. Used with kind='transcript' or kind='ocr'. " + "Supports PostgreSQL full-text search syntax. Cannot be used together " + "with 'label' parameter." + ), + examples=["hello world", "project meeting", "kubernetes"], + ), + face_cluster_id: str | None = Query( + None, + description=( + "Filter by face cluster ID. Used with kind='face'. " + "Face clusters group face detections representing the same person." + ), + examples=["person_001", "face_cluster_abc"], + ), + min_confidence: float | None = Query( + None, + description=( + "Minimum confidence threshold (0.0 to 1.0). Filters results to only " + "include artifacts with confidence >= this value. Applicable to " + "kind='object' and kind='face'." + ), + examples=[0.5, 0.8, 0.95], + ge=0.0, + le=1.0, + ), + limit: int = Query( + 1, + description=( + "Maximum number of results to return (1-50). Default is 1 for " + "single-step navigation. Use higher values to preview multiple " + "upcoming results. Check 'has_more' in response to determine if " + "additional results exist beyond this limit." + ), + examples=[1, 5, 10, 50], + ge=1, + le=50, + ), + service: GlobalJumpService = Depends(get_global_jump_service), +) -> GlobalJumpResponseSchema | JSONResponse: + """ + Navigate across videos to find artifacts in chronological order. + + This endpoint provides cross-video artifact search and navigation using a + unified API. It enables users to search for objects, faces, text (transcript/OCR), + scenes, places, and locations across their entire video library. + + **Global Timeline Ordering:** + Results are ordered using a deterministic global timeline based on: + 1. `file_created_at` (EXIF/filesystem date) - primary sort + 2. `video_id` - secondary sort for deterministic ordering + 3. `start_ms` - tertiary sort for artifacts within the same video + + **Navigation Flow:** + 1. Specify starting position with `from_video_id` and optionally `from_ms` + 2. Choose `direction` ('next' or 'prev') to search forward or backward + 3. Filter by artifact type using `kind` and appropriate filter parameters + 4. Use returned `video_id` and `jump_to.end_ms` for subsequent navigation + + **Pagination:** + The `has_more` field indicates whether additional results exist beyond the + requested `limit`. To paginate through all results: + - Use the last result's `video_id` as `from_video_id` + - Use the last result's `jump_to.end_ms` as `from_ms` + - Continue until `has_more` is `false` + + Args: + kind: Type of artifact (object, face, transcript, ocr, scene, place, location) + direction: Navigation direction ('next' or 'prev') + from_video_id: Starting video ID for the search + from_ms: Starting timestamp in milliseconds (optional) + label: Filter by artifact label (for object/place kinds) + query: Full-text search query (for transcript/ocr kinds) + face_cluster_id: Filter by face cluster ID (for face kind) + min_confidence: Minimum confidence threshold 0-1 (for object/face kinds) + limit: Maximum number of results to return (1-50, default 1) + service: Injected GlobalJumpService instance + + Returns: + GlobalJumpResponseSchema: Contains results array and has_more pagination flag + + Raises: + 400 Bad Request: Invalid parameters (kind, direction, conflicting filters, etc.) + 404 Not Found: Specified from_video_id does not exist + 500 Internal Server Error: Unexpected server error + """ + # Validate from_video_id is non-empty + if not from_video_id or not from_video_id.strip(): + logger.warning("Validation error: from_video_id is empty") + return create_error_response( + status_code=status.HTTP_400_BAD_REQUEST, + detail="from_video_id must be a non-empty string", + error_code=ERROR_CODES["INVALID_VIDEO_ID"], + ) + + # Validate kind parameter + if kind not in VALID_KINDS: + valid_kinds_str = ", ".join(sorted(VALID_KINDS)) + logger.warning(f"Validation error: invalid kind '{kind}'") + return create_error_response( + status_code=status.HTTP_400_BAD_REQUEST, + detail=f"Invalid artifact kind. Must be one of: {valid_kinds_str}", + error_code=ERROR_CODES["INVALID_KIND"], + ) + + # Validate direction parameter + if direction not in VALID_DIRECTIONS: + logger.warning(f"Validation error: invalid direction '{direction}'") + return create_error_response( + status_code=status.HTTP_400_BAD_REQUEST, + detail="Direction must be 'next' or 'prev'", + error_code=ERROR_CODES["INVALID_DIRECTION"], + ) + + # Validate label and query are mutually exclusive + if label is not None and query is not None: + logger.warning("Validation error: both label and query specified") + return create_error_response( + status_code=status.HTTP_400_BAD_REQUEST, + detail="Cannot specify both label and query parameters", + error_code=ERROR_CODES["CONFLICTING_FILTERS"], + ) + + # Validate from_ms is non-negative + if from_ms is not None and from_ms < 0: + logger.warning(f"Validation error: negative from_ms '{from_ms}'") + return create_error_response( + status_code=status.HTTP_400_BAD_REQUEST, + detail="from_ms must be a non-negative integer", + error_code=ERROR_CODES["INVALID_FROM_MS"], + ) + + # Validate min_confidence is between 0 and 1 + if min_confidence is not None and (min_confidence < 0 or min_confidence > 1): + logger.warning(f"Validation error: invalid min_confidence '{min_confidence}'") + return create_error_response( + status_code=status.HTTP_400_BAD_REQUEST, + detail="min_confidence must be between 0 and 1", + error_code=ERROR_CODES["INVALID_CONFIDENCE"], + ) + + # Validate limit is between 1 and 50 + if limit < 1 or limit > 50: + logger.warning(f"Validation error: invalid limit '{limit}'") + return create_error_response( + status_code=status.HTTP_400_BAD_REQUEST, + detail="limit must be between 1 and 50", + error_code=ERROR_CODES["INVALID_LIMIT"], + ) + + try: + # Route to appropriate service method based on direction + if direction == "next": + results = service.jump_next( + kind=kind, + from_video_id=from_video_id, + from_ms=from_ms, + label=label, + query=query, + face_cluster_id=face_cluster_id, + min_confidence=min_confidence, + limit=limit + 1, + ) + else: + results = service.jump_prev( + kind=kind, + from_video_id=from_video_id, + from_ms=from_ms, + label=label, + query=query, + face_cluster_id=face_cluster_id, + min_confidence=min_confidence, + limit=limit + 1, + ) + + # Determine has_more and trim results + has_more = len(results) > limit + results = results[:limit] + + # Convert domain models to response schemas + response_results = [ + GlobalJumpResultSchema( + video_id=r.video_id, + video_filename=r.video_filename, + file_created_at=r.file_created_at, + jump_to=JumpToSchema( + start_ms=r.jump_to.start_ms, + end_ms=r.jump_to.end_ms, + ), + artifact_id=r.artifact_id, + preview=r.preview, + ) + for r in results + ] + + return GlobalJumpResponseSchema( + results=response_results, + has_more=has_more, + ) + + except VideoNotFoundError as e: + logger.warning(f"Video not found: {e.video_id}") + return create_error_response( + status_code=status.HTTP_404_NOT_FOUND, + detail="Video not found", + error_code=ERROR_CODES["VIDEO_NOT_FOUND"], + ) + + except InvalidParameterError as e: + logger.warning(f"Invalid parameter '{e.parameter}': {e.message}") + return create_error_response( + status_code=status.HTTP_400_BAD_REQUEST, + detail=e.message, + error_code=ERROR_CODES["INVALID_KIND"], + ) + + except Exception as e: + logger.error(f"Unexpected error in global_jump: {e}", exc_info=True) + return create_error_response( + status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, + detail="An unexpected error occurred", + error_code=ERROR_CODES["INTERNAL_ERROR"], + ) diff --git a/backend/src/api/schemas.py b/backend/src/api/schemas.py index 0b7c109..4b03352 100644 --- a/backend/src/api/schemas.py +++ b/backend/src/api/schemas.py @@ -3,11 +3,81 @@ from pydantic import BaseModel, Field +class ErrorResponseSchema(BaseModel): + """Schema for error responses with consistent format. + + All error responses include detail, error_code, and timestamp + for debugging and client-side error handling. + """ + + detail: str = Field( + ..., + description="Human-readable error message describing what went wrong", + examples=["Video not found"], + ) + error_code: str = Field( + ..., + description="Machine-readable error code for programmatic handling", + examples=["VIDEO_NOT_FOUND"], + ) + timestamp: datetime = Field( + ..., + description="UTC timestamp when the error occurred", + examples=["2025-05-19T02:22:21Z"], + ) + + class Config: + json_schema_extra = { + "example": { + "detail": "Video not found", + "error_code": "VIDEO_NOT_FOUND", + "timestamp": "2025-05-19T02:22:21Z", + } + } + + class JumpToSchema(BaseModel): - """Schema for jump target timestamp.""" + """Schema for jump target timestamp boundaries. + + Defines the temporal boundaries (start and end) for navigating to an + artifact within a video. The start_ms indicates where to seek, and + end_ms indicates the artifact's end boundary. + + For result chaining in global jump navigation, use end_ms as the + from_ms parameter in subsequent requests to continue navigation. + + Example: + ```json + {"start_ms": 15000, "end_ms": 15500} + ``` + + Attributes: + start_ms: Start timestamp in milliseconds - seek to this position. + end_ms: End timestamp in milliseconds - artifact ends here. + """ + + start_ms: int = Field( + ..., + description=( + "Start timestamp in milliseconds. This is the position to seek to " + "in the video player to view the artifact." + ), + examples=[0, 15000, 120000], + ge=0, + ) + end_ms: int = Field( + ..., + description=( + "End timestamp in milliseconds. This marks the end of the artifact's " + "temporal boundary. Use this value as from_ms in subsequent global " + "jump requests to continue navigation from after this artifact." + ), + examples=[500, 15500, 125000], + ge=0, + ) - start_ms: int = Field(..., description="Start timestamp in milliseconds") - end_ms: int = Field(..., description="End timestamp in milliseconds") + class Config: + json_schema_extra = {"example": {"start_ms": 15000, "end_ms": 15500}} class JumpResponseSchema(BaseModel): @@ -159,3 +229,213 @@ class LocationInfoSchema(BaseModel): class Config: from_attributes = True + + +# Global Jump Navigation Schemas + + +class GlobalJumpResultSchema(BaseModel): + """Schema for a single global jump navigation result. + + Represents a single artifact occurrence across the video library, + including video metadata and temporal boundaries for navigation. + + This schema is returned as part of the GlobalJumpResponseSchema and + contains all information needed to navigate to and display the artifact. + + Example: + ```json + { + "video_id": "abc-123", + "video_filename": "beach_trip.mp4", + "file_created_at": "2025-05-19T02:22:21Z", + "jump_to": {"start_ms": 15000, "end_ms": 15500}, + "artifact_id": "obj_xyz_001", + "preview": {"label": "dog", "confidence": 0.95} + } + ``` + + Attributes: + video_id: Unique identifier of the video containing the artifact. + video_filename: Human-readable filename for display in UI. + file_created_at: EXIF/filesystem creation date used for timeline ordering. + jump_to: Temporal boundaries for seeking to the artifact. + artifact_id: Unique identifier for this specific artifact occurrence. + preview: Kind-specific preview data for displaying result information. + """ + + video_id: str = Field( + ..., + description="Unique identifier of the video containing the artifact", + examples=["abc-123", "video_001", "550e8400-e29b-41d4-a716-446655440000"], + ) + video_filename: str = Field( + ..., + description="Filename of the video for display purposes in the UI", + examples=["beach_trip.mp4", "meeting_2025-01-15.mp4", "family_reunion.mov"], + ) + file_created_at: datetime | None = Field( + None, + description=( + "EXIF/filesystem creation date of the video, used as the primary " + "sort key for global timeline ordering. May be None if the creation " + "date could not be determined from EXIF metadata or filesystem." + ), + examples=["2025-05-19T02:22:21Z", "2024-12-25T10:30:00Z"], + ) + jump_to: JumpToSchema = Field( + ..., + description=( + "Temporal boundaries (start_ms, end_ms) defining where to seek in " + "the video to view this artifact. Use start_ms for initial seek " + "position and end_ms as the starting point for subsequent searches." + ), + ) + artifact_id: str = Field( + ..., + description=( + "Unique identifier of the specific artifact occurrence. Can be used " + "to fetch additional details about the artifact if needed." + ), + examples=["obj_xyz_001", "face_abc_002", "trans_def_003"], + ) + preview: dict = Field( + ..., + description=( + "Kind-specific preview data for displaying result information. " + "Contents vary by artifact kind:\n" + '- **object**: `{"label": "dog", "confidence": 0.95}`\n' + '- **face**: `{"cluster_id": "person_001", "confidence": 0.89}`\n' + '- **transcript**: `{"text": "...matched text snippet..."}`\n' + '- **ocr**: `{"text": "...detected text..."}`\n' + '- **scene**: `{"scene_index": 5}`\n' + '- **place**: `{"label": "beach", "confidence": 0.87}`\n' + '- **location**: `{"latitude": 37.7749, "longitude": -122.4194}`' + ), + examples=[ + {"label": "dog", "confidence": 0.95}, + {"cluster_id": "person_001", "confidence": 0.89}, + {"text": "...discussed the project timeline..."}, + ], + ) + + class Config: + from_attributes = True + json_schema_extra = { + "example": { + "video_id": "abc-123", + "video_filename": "beach_trip.mp4", + "file_created_at": "2025-05-19T02:22:21Z", + "jump_to": {"start_ms": 15000, "end_ms": 15500}, + "artifact_id": "obj_xyz_001", + "preview": {"label": "dog", "confidence": 0.95}, + } + } + + +class GlobalJumpResponseSchema(BaseModel): + """Schema for the global jump navigation API response. + + Contains the list of matching results and pagination information. + Results are ordered by the global timeline using a deterministic + three-level sort: file_created_at, video_id, start_ms. + + **Pagination:** + The `has_more` field is crucial for implementing continuous navigation. + When `has_more` is True, additional results exist beyond the requested + limit. To fetch the next page: + 1. Take the last result from the current response + 2. Use its `video_id` as `from_video_id` + 3. Use its `jump_to.end_ms` as `from_ms` + 4. Make another request with the same filters + + **Empty Results:** + When no matching artifacts are found, the response will have an empty + `results` array and `has_more` will be False. This is not an error + condition - it simply means no artifacts match the search criteria + from the specified position in the requested direction. + + Example (with results): + ```json + { + "results": [ + { + "video_id": "abc-123", + "video_filename": "beach_trip.mp4", + "file_created_at": "2025-05-19T02:22:21Z", + "jump_to": {"start_ms": 15000, "end_ms": 15500}, + "artifact_id": "obj_xyz_001", + "preview": {"label": "dog", "confidence": 0.95} + } + ], + "has_more": true + } + ``` + + Example (no results): + ```json + { + "results": [], + "has_more": false + } + ``` + + Attributes: + results: List of matching artifacts ordered by global timeline. + has_more: Pagination flag indicating if more results exist. + """ + + results: list[GlobalJumpResultSchema] = Field( + ..., + description=( + "List of matching artifacts ordered by global timeline. " + "The ordering is deterministic using three sort keys:\n" + "1. **file_created_at** (primary): EXIF/filesystem creation date\n" + "2. **video_id** (secondary): For deterministic ordering when dates match\n" + "3. **start_ms** (tertiary): Artifact timestamp within the video\n\n" + "For 'next' direction, results are in ascending order. " + "For 'prev' direction, results are in descending order." + ), + ) + has_more: bool = Field( + ..., + description=( + "Pagination indicator. True if additional results exist beyond the " + "requested limit, False otherwise. Use this to implement continuous " + "navigation:\n\n" + "- **has_more=true**: More results available. Use the last result's " + "`video_id` and `jump_to.end_ms` as starting point for next request.\n" + "- **has_more=false**: No more results in this direction. User has " + "reached the end (for 'next') or beginning (for 'prev') of matching " + "artifacts in the global timeline." + ), + ) + + class Config: + json_schema_extra = { + "examples": [ + { + "summary": "Results found with more available", + "value": { + "results": [ + { + "video_id": "abc-123", + "video_filename": "beach_trip.mp4", + "file_created_at": "2025-05-19T02:22:21Z", + "jump_to": {"start_ms": 15000, "end_ms": 15500}, + "artifact_id": "obj_xyz_001", + "preview": {"label": "dog", "confidence": 0.95}, + } + ], + "has_more": True, + }, + }, + { + "summary": "No results found", + "value": { + "results": [], + "has_more": False, + }, + }, + ] + } diff --git a/backend/src/api/video_controller.py b/backend/src/api/video_controller.py index 6124a33..4d806b5 100644 --- a/backend/src/api/video_controller.py +++ b/backend/src/api/video_controller.py @@ -1,7 +1,4 @@ -from pathlib import Path - -from fastapi import APIRouter, Depends, HTTPException, Request, status -from fastapi.responses import StreamingResponse +from fastapi import APIRouter, Depends, HTTPException, status from sqlalchemy.orm import Session from ..api.schemas import VideoCreateSchema, VideoResponseSchema, VideoUpdateSchema diff --git a/backend/src/domain/exceptions.py b/backend/src/domain/exceptions.py new file mode 100644 index 0000000..a369b86 --- /dev/null +++ b/backend/src/domain/exceptions.py @@ -0,0 +1,33 @@ +"""Domain exceptions for the application.""" + + +class GlobalJumpError(Exception): + """Base exception for global jump navigation errors.""" + + pass + + +class VideoNotFoundError(GlobalJumpError): + """Raised when a requested video does not exist. + + Attributes: + video_id: The video ID that was not found + """ + + def __init__(self, video_id: str): + self.video_id = video_id + super().__init__(f"Video not found: {video_id}") + + +class InvalidParameterError(GlobalJumpError): + """Raised when an invalid parameter is provided to a global jump query. + + Attributes: + parameter: Name of the invalid parameter + message: Description of the validation error + """ + + def __init__(self, parameter: str, message: str): + self.parameter = parameter + self.message = message + super().__init__(f"Invalid parameter '{parameter}': {message}") diff --git a/backend/src/domain/models.py b/backend/src/domain/models.py index a090e58..9ba0b8e 100644 --- a/backend/src/domain/models.py +++ b/backend/src/domain/models.py @@ -1,3 +1,4 @@ +from dataclasses import dataclass from datetime import datetime @@ -130,3 +131,43 @@ def fail(self, error: str) -> None: self.status = "failed" self.completed_at = datetime.utcnow() self.error = error + + +# Global Jump Navigation Models + + +@dataclass +class JumpTo: + """Temporal boundaries for an artifact occurrence. + + Attributes: + start_ms: Start timestamp in milliseconds + end_ms: End timestamp in milliseconds + """ + + start_ms: int + end_ms: int + + +@dataclass +class GlobalJumpResult: + """Result from a global jump navigation query. + + Represents a single artifact occurrence across the video library, + including video metadata and temporal boundaries for navigation. + + Attributes: + video_id: Unique identifier of the video containing the artifact + video_filename: Filename of the video + file_created_at: EXIF/filesystem creation date of the video (may be None) + jump_to: Temporal boundaries (start_ms, end_ms) for the artifact + artifact_id: Unique identifier of the specific artifact occurrence + preview: Kind-specific preview data (label, confidence, text snippet, etc.) + """ + + video_id: str + video_filename: str + file_created_at: datetime | None + jump_to: JumpTo + artifact_id: str + preview: dict diff --git a/backend/src/main_api.py b/backend/src/main_api.py index b5860d7..3b41176 100644 --- a/backend/src/main_api.py +++ b/backend/src/main_api.py @@ -83,6 +83,7 @@ def setup_logging(): from fastapi import FastAPI # noqa: E402 from src.api.artifact_controller import router as artifact_router # noqa: E402 +from src.api.global_jump_controller import router as global_jump_router # noqa: E402 from src.api.path_controller_full import router as path_router # noqa: E402 from src.api.task_routes import router as task_router # noqa: E402 from src.api.video_controller import router as video_router # noqa: E402 @@ -243,6 +244,7 @@ def create_app(config_path: str | None = None) -> FastAPI: app.include_router(artifact_router, prefix="/v1") app.include_router(path_router, prefix="/v1") app.include_router(task_router, prefix="/v1") + app.include_router(global_jump_router, prefix="/v1") logger.info("Routers included successfully") return app diff --git a/backend/src/services/global_jump_service.py b/backend/src/services/global_jump_service.py new file mode 100644 index 0000000..60bf937 --- /dev/null +++ b/backend/src/services/global_jump_service.py @@ -0,0 +1,1614 @@ +"""Service layer for Global Jump Navigation. + +This module provides the GlobalJumpService class which orchestrates cross-video +artifact search and navigation. It enables users to search for objects, faces, +text, and scenes across their entire video library in chronological order. + +The service uses existing projection tables (object_labels, face_clusters, +transcript_fts, ocr_fts, scene_ranges, video_locations) to provide fast queries +without requiring new data structures. +""" + +import logging +from typing import Literal + +from sqlalchemy import and_, or_, text +from sqlalchemy.orm import Session + +from ..database.models import ObjectLabel, SceneRange +from ..database.models import Video as VideoEntity +from ..domain.exceptions import InvalidParameterError, VideoNotFoundError +from ..domain.models import GlobalJumpResult, JumpTo +from ..repositories.interfaces import ArtifactRepository + +logger = logging.getLogger(__name__) + + +class GlobalJumpService: + """Service for cross-video artifact search and navigation. + + GlobalJumpService provides methods to navigate across videos in chronological + order based on artifact searches. It supports searching by object labels, + face clusters, transcript text, OCR text, scenes, places, and locations. + + The service uses a deterministic global timeline based on: + 1. file_created_at (EXIF/filesystem date) as primary sort key + 2. video_id as secondary sort key (for deterministic ordering) + 3. start_ms as tertiary sort key (for ordering within a video) + + Attributes: + session: SQLAlchemy database session for executing queries + artifact_repo: Repository for accessing artifact data + """ + + def __init__(self, session: Session, artifact_repo: ArtifactRepository): + """Initialize GlobalJumpService. + + Args: + session: SQLAlchemy database session for executing queries. + artifact_repo: Repository for accessing artifact envelope data. + """ + self.session = session + self.artifact_repo = artifact_repo + + def _get_video(self, video_id: str) -> VideoEntity: + """Fetch a video by ID from the database. + + Args: + video_id: Unique identifier of the video to fetch. + + Returns: + VideoEntity: The SQLAlchemy video entity with all metadata. + + Raises: + VideoNotFoundError: If no video exists with the given ID. + """ + video = ( + self.session.query(VideoEntity) + .filter(VideoEntity.video_id == video_id) + .first() + ) + + if video is None: + raise VideoNotFoundError(video_id) + + return video + + def _to_global_result( + self, + video_id: str, + video_filename: str, + file_created_at, + start_ms: int, + end_ms: int, + artifact_id: str, + preview: dict, + ) -> GlobalJumpResult: + """Convert database row data to a GlobalJumpResult object. + + Args: + video_id: Unique identifier of the video containing the artifact. + video_filename: Filename of the video for display purposes. + file_created_at: EXIF/filesystem creation date of the video. + start_ms: Start timestamp in milliseconds. + end_ms: End timestamp in milliseconds. + artifact_id: Unique identifier of the specific artifact occurrence. + preview: Kind-specific preview data. + + Returns: + GlobalJumpResult: A formatted result object ready for API response. + """ + return GlobalJumpResult( + video_id=video_id, + video_filename=video_filename, + file_created_at=file_created_at, + jump_to=JumpTo(start_ms=start_ms, end_ms=end_ms), + artifact_id=artifact_id, + preview=preview, + ) + + def _search_objects_global( + self, + direction: Literal["next", "prev"], + from_video_id: str, + from_ms: int, + label: str | None = None, + min_confidence: float | None = None, + limit: int = 1, + ) -> list[GlobalJumpResult]: + """Search for object labels across all videos in global timeline order. + + Queries the object_labels projection table joined with videos to find + matching objects in chronological order based on the global timeline. + + Args: + direction: Navigation direction ("next" for forward, "prev" for backward). + from_video_id: Starting video ID for the search. + from_ms: Starting timestamp in milliseconds within the video. + label: Optional label filter to match specific object types. + min_confidence: Optional minimum confidence threshold (0-1). + limit: Maximum number of results to return (default 1). + + Returns: + List of GlobalJumpResult objects ordered by global timeline. + For "next": ascending order (chronologically after current position). + For "prev": descending order (chronologically before current position). + Empty list if no matching objects are found. + + Raises: + VideoNotFoundError: If from_video_id does not exist. + """ + # Get the current video to determine its position in the global timeline + current_video = self._get_video(from_video_id) + current_file_created_at = current_video.file_created_at + + # Build base query joining object_labels with videos + query = self.session.query( + ObjectLabel.artifact_id, + ObjectLabel.asset_id, + ObjectLabel.label, + ObjectLabel.confidence, + ObjectLabel.start_ms, + ObjectLabel.end_ms, + VideoEntity.filename, + VideoEntity.file_created_at, + ).join(VideoEntity, VideoEntity.video_id == ObjectLabel.asset_id) + + # Apply label filter if specified + if label is not None: + query = query.filter(ObjectLabel.label == label) + + # Apply min_confidence filter if specified + if min_confidence is not None: + query = query.filter(ObjectLabel.confidence >= min_confidence) + + # Apply direction-specific WHERE clause for "next" direction + # Results must be chronologically after the current position + # Global timeline ordering: file_created_at > video_id > start_ms + if direction == "next": + # Handle NULL file_created_at values + # NULLs are treated as "unknown" and sorted after non-NULL values + if current_file_created_at is not None: + query = query.filter( + or_( + # Case 1: Videos with later file_created_at + VideoEntity.file_created_at > current_file_created_at, + # Case 2: Videos with NULL file_created_at (sorted after) + VideoEntity.file_created_at.is_(None), + # Case 3: Same file_created_at, later video_id + and_( + VideoEntity.file_created_at == current_file_created_at, + VideoEntity.video_id > from_video_id, + ), + # Case 4: Same video, later start_ms + and_( + VideoEntity.file_created_at == current_file_created_at, + VideoEntity.video_id == from_video_id, + ObjectLabel.start_ms > from_ms, + ), + ) + ) + else: + # Current video has NULL file_created_at + # Only consider videos with NULL file_created_at + query = query.filter( + or_( + # Case 1: Same NULL file_created_at, later video_id + and_( + VideoEntity.file_created_at.is_(None), + VideoEntity.video_id > from_video_id, + ), + # Case 2: Same video, later start_ms + and_( + VideoEntity.file_created_at.is_(None), + VideoEntity.video_id == from_video_id, + ObjectLabel.start_ms > from_ms, + ), + ) + ) + + # Order by global timeline (ascending for "next") + # NULLS LAST ensures NULL file_created_at values come after non-NULL + query = query.order_by( + VideoEntity.file_created_at.asc().nulls_last(), + VideoEntity.video_id.asc(), + ObjectLabel.start_ms.asc(), + ) + + elif direction == "prev": + # Apply direction-specific WHERE clause for "prev" direction + # Results must be chronologically before the current position + # Global timeline ordering: file_created_at > video_id > start_ms + if current_file_created_at is not None: + query = query.filter( + or_( + # Case 1: Videos with earlier file_created_at + and_( + VideoEntity.file_created_at.is_not(None), + VideoEntity.file_created_at < current_file_created_at, + ), + # Case 2: Same file_created_at, earlier video_id + and_( + VideoEntity.file_created_at == current_file_created_at, + VideoEntity.video_id < from_video_id, + ), + # Case 3: Same video, earlier start_ms + and_( + VideoEntity.file_created_at == current_file_created_at, + VideoEntity.video_id == from_video_id, + ObjectLabel.start_ms < from_ms, + ), + ) + ) + else: + # Current video has NULL file_created_at + # Consider all videos with non-NULL file_created_at (they come + # before) and videos with NULL file_created_at that are earlier + # in video_id order + query = query.filter( + or_( + # Case 1: Videos with non-NULL file_created_at + # (come before NULLs) + VideoEntity.file_created_at.is_not(None), + # Case 2: Same NULL file_created_at, earlier video_id + and_( + VideoEntity.file_created_at.is_(None), + VideoEntity.video_id < from_video_id, + ), + # Case 3: Same video, earlier start_ms + and_( + VideoEntity.file_created_at.is_(None), + VideoEntity.video_id == from_video_id, + ObjectLabel.start_ms < from_ms, + ), + ) + ) + + # Order by global timeline (descending for "prev") + # NULLS LAST ensures NULL file_created_at values come last in + # descending order (which means they were originally last in + # ascending order) + query = query.order_by( + VideoEntity.file_created_at.desc().nulls_last(), + VideoEntity.video_id.desc(), + ObjectLabel.start_ms.desc(), + ) + + # Apply limit + query = query.limit(limit) + + # Execute query and convert results to GlobalJumpResult objects + results = [] + for row in query.all(): + result = self._to_global_result( + video_id=row.asset_id, + video_filename=row.filename, + file_created_at=row.file_created_at, + start_ms=row.start_ms, + end_ms=row.end_ms, + artifact_id=row.artifact_id, + preview={ + "label": row.label, + "confidence": row.confidence, + }, + ) + results.append(result) + + logger.debug( + f"_search_objects_global: direction={direction}, " + f"from_video_id={from_video_id}, from_ms={from_ms}, " + f"label={label}, min_confidence={min_confidence}, " + f"found {len(results)} results" + ) + + return results + + def _search_transcript_global( + self, + direction: Literal["next", "prev"], + from_video_id: str, + from_ms: int, + query: str, + limit: int = 1, + ) -> list[GlobalJumpResult]: + """Search for transcript text across all videos in global timeline order. + + Queries the transcript_fts projection table joined with videos to find + matching transcript segments using PostgreSQL full-text search. Results + are ordered by the global timeline. + + Args: + direction: Navigation direction ("next" or "prev"). + from_video_id: Starting video ID for the search. + from_ms: Starting timestamp in milliseconds within the video. + query: Text query to search for in transcripts. + limit: Maximum number of results to return (default 1). + + Returns: + List of GlobalJumpResult objects ordered by global timeline. + For "next": ascending order (chronologically after current position). + For "prev": descending order (chronologically before current position). + Empty list if no matching transcript segments are found. + + Raises: + VideoNotFoundError: If from_video_id does not exist. + """ + # Get the current video to determine its position in the global timeline + current_video = self._get_video(from_video_id) + current_file_created_at = current_video.file_created_at + + # Determine database dialect + bind = self.session.bind + is_postgresql = bind.dialect.name == "postgresql" + + if is_postgresql: + return self._search_transcript_global_postgresql( + direction, from_video_id, from_ms, query, limit, current_file_created_at + ) + else: + return self._search_transcript_global_sqlite( + direction, from_video_id, from_ms, query, limit, current_file_created_at + ) + + def _search_transcript_global_postgresql( + self, + direction: Literal["next", "prev"], + from_video_id: str, + from_ms: int, + query: str, + limit: int, + current_file_created_at, + ) -> list[GlobalJumpResult]: + """PostgreSQL implementation of transcript global search.""" + # Build direction-specific SQL components + if direction == "next": + order_clause = """ + ORDER BY v.file_created_at ASC NULLS LAST, + v.video_id ASC, + t.start_ms ASC + """ + if current_file_created_at is not None: + direction_clause = """ + AND ( + v.file_created_at > :current_file_created_at + OR v.file_created_at IS NULL + OR (v.file_created_at = :current_file_created_at + AND v.video_id > :from_video_id) + OR (v.file_created_at = :current_file_created_at + AND v.video_id = :from_video_id + AND t.start_ms > :from_ms) + ) + """ + else: + direction_clause = """ + AND ( + (v.file_created_at IS NULL + AND v.video_id > :from_video_id) + OR (v.file_created_at IS NULL + AND v.video_id = :from_video_id + AND t.start_ms > :from_ms) + ) + """ + else: # direction == "prev" + order_clause = """ + ORDER BY v.file_created_at DESC NULLS LAST, + v.video_id DESC, + t.start_ms DESC + """ + if current_file_created_at is not None: + direction_clause = """ + AND ( + (v.file_created_at IS NOT NULL + AND v.file_created_at < :current_file_created_at) + OR (v.file_created_at = :current_file_created_at + AND v.video_id < :from_video_id) + OR (v.file_created_at = :current_file_created_at + AND v.video_id = :from_video_id + AND t.start_ms < :from_ms) + ) + """ + else: + direction_clause = """ + AND ( + v.file_created_at IS NOT NULL + OR (v.file_created_at IS NULL + AND v.video_id < :from_video_id) + OR (v.file_created_at IS NULL + AND v.video_id = :from_video_id + AND t.start_ms < :from_ms) + ) + """ + + # PostgreSQL: Use tsvector and tsquery with plainto_tsquery + sql = text( + f""" + SELECT + t.artifact_id, + t.asset_id, + t.start_ms, + t.end_ms, + t.text, + v.filename, + v.file_created_at + FROM transcript_fts t + JOIN videos v ON v.video_id = t.asset_id + WHERE t.text_tsv @@ plainto_tsquery('english', :query) + {direction_clause} + {order_clause} + LIMIT :limit + """ + ) + + params = { + "query": query, + "from_video_id": from_video_id, + "from_ms": from_ms, + "limit": limit, + } + if current_file_created_at is not None: + params["current_file_created_at"] = current_file_created_at + + rows = self.session.execute(sql, params).fetchall() + + # If FTS returned no results, try case-insensitive LIKE search + if not rows: + sql_fallback = text( + f""" + SELECT + t.artifact_id, + t.asset_id, + t.start_ms, + t.end_ms, + t.text, + v.filename, + v.file_created_at + FROM transcript_fts t + JOIN videos v ON v.video_id = t.asset_id + WHERE t.text ILIKE :query_like + {direction_clause} + {order_clause} + LIMIT :limit + """ + ) + + params["query_like"] = f"%{query}%" + rows = self.session.execute(sql_fallback, params).fetchall() + + # Convert results to GlobalJumpResult objects + results = [] + for row in rows: + result = self._to_global_result( + video_id=row.asset_id, + video_filename=row.filename, + file_created_at=row.file_created_at, + start_ms=row.start_ms, + end_ms=row.end_ms, + artifact_id=row.artifact_id, + preview={"text": row.text}, + ) + results.append(result) + + logger.debug( + f"_search_transcript_global: direction={direction}, " + f"from_video_id={from_video_id}, from_ms={from_ms}, " + f"query={query}, found {len(results)} results" + ) + + return results + + def _search_transcript_global_sqlite( + self, + direction: Literal["next", "prev"], + from_video_id: str, + from_ms: int, + query: str, + limit: int, + current_file_created_at, + ) -> list[GlobalJumpResult]: + """SQLite implementation of transcript global search using FTS5.""" + # SQLite: Use FTS5 MATCH syntax + # First get matching artifact_ids from FTS5 table + fts_sql = text( + """ + SELECT artifact_id, text + FROM transcript_fts + WHERE transcript_fts MATCH :query + """ + ) + + fts_rows = self.session.execute(fts_sql, {"query": query}).fetchall() + + if not fts_rows: + return [] + + # Get artifact_ids and text snippets + artifact_ids = [row.artifact_id for row in fts_rows] + text_map = {row.artifact_id: row.text for row in fts_rows} + + # Build placeholders for IN clause + placeholders = ",".join([f":id{i}" for i in range(len(artifact_ids))]) + + # Convert datetime to string for SQLite comparison + current_file_created_at_str = None + if current_file_created_at is not None: + current_file_created_at_str = current_file_created_at.strftime( + "%Y-%m-%d %H:%M:%S.%f" + ) + + # Build direction-specific SQL for SQLite (no NULLS LAST support) + if direction == "next": + # SQLite: Use CASE to handle NULL ordering (NULLs last) + order_clause = """ + ORDER BY CASE WHEN v.file_created_at IS NULL THEN 1 ELSE 0 END, + v.file_created_at ASC, + v.video_id ASC, + m.start_ms ASC + """ + if current_file_created_at_str is not None: + direction_clause = """ + AND ( + v.file_created_at > :current_file_created_at + OR v.file_created_at IS NULL + OR (v.file_created_at = :current_file_created_at + AND v.video_id > :from_video_id) + OR (v.file_created_at = :current_file_created_at + AND v.video_id = :from_video_id + AND m.start_ms > :from_ms) + ) + """ + else: + direction_clause = """ + AND ( + (v.file_created_at IS NULL + AND v.video_id > :from_video_id) + OR (v.file_created_at IS NULL + AND v.video_id = :from_video_id + AND m.start_ms > :from_ms) + ) + """ + else: # direction == "prev" + # SQLite: Use CASE to handle NULL ordering (NULLs last in DESC) + order_clause = """ + ORDER BY CASE WHEN v.file_created_at IS NULL THEN 1 ELSE 0 END, + v.file_created_at DESC, + v.video_id DESC, + m.start_ms DESC + """ + if current_file_created_at_str is not None: + direction_clause = """ + AND ( + (v.file_created_at IS NOT NULL + AND v.file_created_at < :current_file_created_at) + OR (v.file_created_at = :current_file_created_at + AND v.video_id < :from_video_id) + OR (v.file_created_at = :current_file_created_at + AND v.video_id = :from_video_id + AND m.start_ms < :from_ms) + ) + """ + else: + direction_clause = """ + AND ( + v.file_created_at IS NOT NULL + OR (v.file_created_at IS NULL + AND v.video_id < :from_video_id) + OR (v.file_created_at IS NULL + AND v.video_id = :from_video_id + AND m.start_ms < :from_ms) + ) + """ + + # Get metadata with video info and apply direction filter + metadata_sql = text( + f""" + SELECT + m.artifact_id, + m.asset_id, + m.start_ms, + m.end_ms, + v.filename, + v.file_created_at + FROM transcript_fts_metadata m + JOIN videos v ON v.video_id = m.asset_id + WHERE m.artifact_id IN ({placeholders}) + {direction_clause} + {order_clause} + LIMIT :limit + """ + ) + + params = {f"id{i}": aid for i, aid in enumerate(artifact_ids)} + params["from_video_id"] = from_video_id + params["from_ms"] = from_ms + params["limit"] = limit + if current_file_created_at_str is not None: + params["current_file_created_at"] = current_file_created_at_str + + rows = self.session.execute(metadata_sql, params).fetchall() + + # Convert to results with text from FTS + results = [] + for row in rows: + result = self._to_global_result( + video_id=row.asset_id, + video_filename=row.filename, + file_created_at=row.file_created_at, + start_ms=row.start_ms, + end_ms=row.end_ms, + artifact_id=row.artifact_id, + preview={"text": text_map.get(row.artifact_id, "")}, + ) + results.append(result) + + logger.debug( + f"_search_transcript_global: direction={direction}, " + f"from_video_id={from_video_id}, from_ms={from_ms}, " + f"query={query}, found {len(results)} results" + ) + + return results + + def _search_ocr_global( + self, + direction: Literal["next", "prev"], + from_video_id: str, + from_ms: int, + query: str, + limit: int = 1, + ) -> list[GlobalJumpResult]: + """Search for OCR text across all videos in global timeline order. + + Queries the ocr_fts projection table joined with videos to find + matching OCR text segments using PostgreSQL full-text search. Results + are ordered by the global timeline. + + Args: + direction: Navigation direction ("next" or "prev"). + from_video_id: Starting video ID for the search. + from_ms: Starting timestamp in milliseconds within the video. + query: Text query to search for in OCR text. + limit: Maximum number of results to return (default 1). + + Returns: + List of GlobalJumpResult objects ordered by global timeline. + For "next": ascending order (chronologically after current position). + For "prev": descending order (chronologically before current position). + Empty list if no matching OCR text segments are found. + + Raises: + VideoNotFoundError: If from_video_id does not exist. + """ + # Get the current video to determine its position in the global timeline + current_video = self._get_video(from_video_id) + current_file_created_at = current_video.file_created_at + + # Determine database dialect + bind = self.session.bind + is_postgresql = bind.dialect.name == "postgresql" + + if is_postgresql: + return self._search_ocr_global_postgresql( + direction, + from_video_id, + from_ms, + query, + limit, + current_file_created_at, + ) + else: + return self._search_ocr_global_sqlite( + direction, + from_video_id, + from_ms, + query, + limit, + current_file_created_at, + ) + + def _search_ocr_global_postgresql( + self, + direction: Literal["next", "prev"], + from_video_id: str, + from_ms: int, + query: str, + limit: int, + current_file_created_at, + ) -> list[GlobalJumpResult]: + """PostgreSQL implementation of OCR global search.""" + # Build direction-specific SQL components + if direction == "next": + order_clause = """ + ORDER BY v.file_created_at ASC NULLS LAST, + v.video_id ASC, + o.start_ms ASC + """ + if current_file_created_at is not None: + direction_clause = """ + AND ( + v.file_created_at > :current_file_created_at + OR v.file_created_at IS NULL + OR (v.file_created_at = :current_file_created_at + AND v.video_id > :from_video_id) + OR (v.file_created_at = :current_file_created_at + AND v.video_id = :from_video_id + AND o.start_ms > :from_ms) + ) + """ + else: + direction_clause = """ + AND ( + (v.file_created_at IS NULL + AND v.video_id > :from_video_id) + OR (v.file_created_at IS NULL + AND v.video_id = :from_video_id + AND o.start_ms > :from_ms) + ) + """ + else: # direction == "prev" + order_clause = """ + ORDER BY v.file_created_at DESC NULLS LAST, + v.video_id DESC, + o.start_ms DESC + """ + if current_file_created_at is not None: + direction_clause = """ + AND ( + (v.file_created_at IS NOT NULL + AND v.file_created_at < :current_file_created_at) + OR (v.file_created_at = :current_file_created_at + AND v.video_id < :from_video_id) + OR (v.file_created_at = :current_file_created_at + AND v.video_id = :from_video_id + AND o.start_ms < :from_ms) + ) + """ + else: + direction_clause = """ + AND ( + v.file_created_at IS NOT NULL + OR (v.file_created_at IS NULL + AND v.video_id < :from_video_id) + OR (v.file_created_at IS NULL + AND v.video_id = :from_video_id + AND o.start_ms < :from_ms) + ) + """ + + # PostgreSQL: Use tsvector and tsquery with plainto_tsquery + sql = text( + f""" + SELECT + o.artifact_id, + o.asset_id, + o.start_ms, + o.end_ms, + o.text, + v.filename, + v.file_created_at + FROM ocr_fts o + JOIN videos v ON v.video_id = o.asset_id + WHERE o.text_tsv @@ plainto_tsquery('english', :query) + {direction_clause} + {order_clause} + LIMIT :limit + """ + ) + + params = { + "query": query, + "from_video_id": from_video_id, + "from_ms": from_ms, + "limit": limit, + } + if current_file_created_at is not None: + params["current_file_created_at"] = current_file_created_at + + rows = self.session.execute(sql, params).fetchall() + + # If FTS returned no results, try case-insensitive LIKE search + if not rows: + sql_fallback = text( + f""" + SELECT + o.artifact_id, + o.asset_id, + o.start_ms, + o.end_ms, + o.text, + v.filename, + v.file_created_at + FROM ocr_fts o + JOIN videos v ON v.video_id = o.asset_id + WHERE o.text ILIKE :query_like + {direction_clause} + {order_clause} + LIMIT :limit + """ + ) + + params["query_like"] = f"%{query}%" + rows = self.session.execute(sql_fallback, params).fetchall() + + # Convert results to GlobalJumpResult objects + results = [] + for row in rows: + result = self._to_global_result( + video_id=row.asset_id, + video_filename=row.filename, + file_created_at=row.file_created_at, + start_ms=row.start_ms, + end_ms=row.end_ms, + artifact_id=row.artifact_id, + preview={"text": row.text}, + ) + results.append(result) + + logger.debug( + f"_search_ocr_global: direction={direction}, " + f"from_video_id={from_video_id}, from_ms={from_ms}, " + f"query={query}, found {len(results)} results" + ) + + return results + + def _search_ocr_global_sqlite( + self, + direction: Literal["next", "prev"], + from_video_id: str, + from_ms: int, + query: str, + limit: int, + current_file_created_at, + ) -> list[GlobalJumpResult]: + """SQLite implementation of OCR global search using FTS5.""" + # SQLite: Use FTS5 MATCH syntax + # First get matching artifact_ids from FTS5 table + fts_sql = text( + """ + SELECT artifact_id, text + FROM ocr_fts + WHERE ocr_fts MATCH :query + """ + ) + + fts_rows = self.session.execute(fts_sql, {"query": query}).fetchall() + + if not fts_rows: + return [] + + # Get artifact_ids and text snippets + artifact_ids = [row.artifact_id for row in fts_rows] + text_map = {row.artifact_id: row.text for row in fts_rows} + + # Build placeholders for IN clause + placeholders = ",".join([f":id{i}" for i in range(len(artifact_ids))]) + + # Convert datetime to string for SQLite comparison + current_file_created_at_str = None + if current_file_created_at is not None: + current_file_created_at_str = current_file_created_at.strftime( + "%Y-%m-%d %H:%M:%S.%f" + ) + + # Build direction-specific SQL for SQLite (no NULLS LAST support) + if direction == "next": + # SQLite: Use CASE to handle NULL ordering (NULLs last) + order_clause = """ + ORDER BY CASE WHEN v.file_created_at IS NULL THEN 1 ELSE 0 END, + v.file_created_at ASC, + v.video_id ASC, + m.start_ms ASC + """ + if current_file_created_at_str is not None: + direction_clause = """ + AND ( + v.file_created_at > :current_file_created_at + OR v.file_created_at IS NULL + OR (v.file_created_at = :current_file_created_at + AND v.video_id > :from_video_id) + OR (v.file_created_at = :current_file_created_at + AND v.video_id = :from_video_id + AND m.start_ms > :from_ms) + ) + """ + else: + direction_clause = """ + AND ( + (v.file_created_at IS NULL + AND v.video_id > :from_video_id) + OR (v.file_created_at IS NULL + AND v.video_id = :from_video_id + AND m.start_ms > :from_ms) + ) + """ + else: # direction == "prev" + # SQLite: Use CASE to handle NULL ordering (NULLs last in DESC) + order_clause = """ + ORDER BY CASE WHEN v.file_created_at IS NULL THEN 1 ELSE 0 END, + v.file_created_at DESC, + v.video_id DESC, + m.start_ms DESC + """ + if current_file_created_at_str is not None: + direction_clause = """ + AND ( + (v.file_created_at IS NOT NULL + AND v.file_created_at < :current_file_created_at) + OR (v.file_created_at = :current_file_created_at + AND v.video_id < :from_video_id) + OR (v.file_created_at = :current_file_created_at + AND v.video_id = :from_video_id + AND m.start_ms < :from_ms) + ) + """ + else: + direction_clause = """ + AND ( + v.file_created_at IS NOT NULL + OR (v.file_created_at IS NULL + AND v.video_id < :from_video_id) + OR (v.file_created_at IS NULL + AND v.video_id = :from_video_id + AND m.start_ms < :from_ms) + ) + """ + + # Get metadata with video info and apply direction filter + metadata_sql = text( + f""" + SELECT + m.artifact_id, + m.asset_id, + m.start_ms, + m.end_ms, + v.filename, + v.file_created_at + FROM ocr_fts_metadata m + JOIN videos v ON v.video_id = m.asset_id + WHERE m.artifact_id IN ({placeholders}) + {direction_clause} + {order_clause} + LIMIT :limit + """ + ) + + params = {f"id{i}": aid for i, aid in enumerate(artifact_ids)} + params["from_video_id"] = from_video_id + params["from_ms"] = from_ms + params["limit"] = limit + if current_file_created_at_str is not None: + params["current_file_created_at"] = current_file_created_at_str + + rows = self.session.execute(metadata_sql, params).fetchall() + + # Convert to results with text from FTS + results = [] + for row in rows: + result = self._to_global_result( + video_id=row.asset_id, + video_filename=row.filename, + file_created_at=row.file_created_at, + start_ms=row.start_ms, + end_ms=row.end_ms, + artifact_id=row.artifact_id, + preview={"text": text_map.get(row.artifact_id, "")}, + ) + results.append(result) + + logger.debug( + f"_search_ocr_global: direction={direction}, " + f"from_video_id={from_video_id}, from_ms={from_ms}, " + f"query={query}, found {len(results)} results" + ) + + return results + + def _search_scenes_global( + self, + direction: Literal["next", "prev"], + from_video_id: str, + from_ms: int, + limit: int = 1, + ) -> list[GlobalJumpResult]: + """Search for scene boundaries across all videos in global timeline order. + + Queries the scene_ranges projection table joined with videos to find + scene boundaries in chronological order based on the global timeline. + + Args: + direction: Navigation direction ("next" for forward, "prev" for backward). + from_video_id: Starting video ID for the search. + from_ms: Starting timestamp in milliseconds within the video. + limit: Maximum number of results to return (default 1). + + Returns: + List of GlobalJumpResult objects ordered by global timeline. + For "next": ascending order (chronologically after current position). + For "prev": descending order (chronologically before current position). + Empty list if no scene boundaries are found. + + Raises: + VideoNotFoundError: If from_video_id does not exist. + """ + # Get the current video to determine its position in the global timeline + current_video = self._get_video(from_video_id) + current_file_created_at = current_video.file_created_at + + # Build base query joining scene_ranges with videos + query = self.session.query( + SceneRange.artifact_id, + SceneRange.asset_id, + SceneRange.scene_index, + SceneRange.start_ms, + SceneRange.end_ms, + VideoEntity.filename, + VideoEntity.file_created_at, + ).join(VideoEntity, VideoEntity.video_id == SceneRange.asset_id) + + # Apply direction-specific WHERE clause + if direction == "next": + if current_file_created_at is not None: + query = query.filter( + or_( + # Case 1: Videos with later file_created_at + VideoEntity.file_created_at > current_file_created_at, + # Case 2: Videos with NULL file_created_at (sorted after) + VideoEntity.file_created_at.is_(None), + # Case 3: Same file_created_at, later video_id + and_( + VideoEntity.file_created_at == current_file_created_at, + VideoEntity.video_id > from_video_id, + ), + # Case 4: Same video, later start_ms + and_( + VideoEntity.file_created_at == current_file_created_at, + VideoEntity.video_id == from_video_id, + SceneRange.start_ms > from_ms, + ), + ) + ) + else: + query = query.filter( + or_( + and_( + VideoEntity.file_created_at.is_(None), + VideoEntity.video_id > from_video_id, + ), + and_( + VideoEntity.file_created_at.is_(None), + VideoEntity.video_id == from_video_id, + SceneRange.start_ms > from_ms, + ), + ) + ) + + # Order by global timeline (ascending for "next") + query = query.order_by( + VideoEntity.file_created_at.asc().nulls_last(), + VideoEntity.video_id.asc(), + SceneRange.start_ms.asc(), + ) + + elif direction == "prev": + if current_file_created_at is not None: + query = query.filter( + or_( + # Case 1: Videos with earlier file_created_at + and_( + VideoEntity.file_created_at.is_not(None), + VideoEntity.file_created_at < current_file_created_at, + ), + # Case 2: Same file_created_at, earlier video_id + and_( + VideoEntity.file_created_at == current_file_created_at, + VideoEntity.video_id < from_video_id, + ), + # Case 3: Same video, earlier start_ms + and_( + VideoEntity.file_created_at == current_file_created_at, + VideoEntity.video_id == from_video_id, + SceneRange.start_ms < from_ms, + ), + ) + ) + else: + query = query.filter( + or_( + VideoEntity.file_created_at.is_not(None), + and_( + VideoEntity.file_created_at.is_(None), + VideoEntity.video_id < from_video_id, + ), + and_( + VideoEntity.file_created_at.is_(None), + VideoEntity.video_id == from_video_id, + SceneRange.start_ms < from_ms, + ), + ) + ) + + # Order by global timeline (descending for "prev") + query = query.order_by( + VideoEntity.file_created_at.desc().nulls_last(), + VideoEntity.video_id.desc(), + SceneRange.start_ms.desc(), + ) + + # Apply limit + query = query.limit(limit) + + # Execute query and convert results to GlobalJumpResult objects + results = [] + for row in query.all(): + result = self._to_global_result( + video_id=row.asset_id, + video_filename=row.filename, + file_created_at=row.file_created_at, + start_ms=row.start_ms, + end_ms=row.end_ms, + artifact_id=row.artifact_id, + preview={ + "scene_index": row.scene_index, + }, + ) + results.append(result) + + logger.debug( + f"_search_scenes_global: direction={direction}, " + f"from_video_id={from_video_id}, from_ms={from_ms}, " + f"found {len(results)} results" + ) + + return results + + def _search_places_global( + self, + direction: Literal["next", "prev"], + from_video_id: str, + from_ms: int, + label: str | None = None, + min_confidence: float | None = None, + limit: int = 1, + ) -> list[GlobalJumpResult]: + """Search for place classifications across all videos in global timeline order. + + Place classifications are stored in the object_labels projection table + with place-specific labels (e.g., "kitchen", "beach", "office"). + This method queries object_labels with place-related filters. + + Args: + direction: Navigation direction ("next" for forward, "prev" for backward). + from_video_id: Starting video ID for the search. + from_ms: Starting timestamp in milliseconds within the video. + label: Optional place label filter (e.g., "kitchen", "beach"). + min_confidence: Optional minimum confidence threshold (0-1). + limit: Maximum number of results to return (default 1). + + Returns: + List of GlobalJumpResult objects ordered by global timeline. + For "next": ascending order (chronologically after current position). + For "prev": descending order (chronologically before current position). + Empty list if no matching places are found. + + Raises: + VideoNotFoundError: If from_video_id does not exist. + + Note: + Place classifications are stored in object_labels with labels from + the Places365 dataset. This method reuses the object search logic + but is semantically distinct for place-based navigation. + """ + # Place search uses the same underlying table as object search + # but is semantically distinct for place-based navigation + return self._search_objects_global( + direction=direction, + from_video_id=from_video_id, + from_ms=from_ms, + label=label, + min_confidence=min_confidence, + limit=limit, + ) + + def _search_locations_global( + self, + direction: Literal["next", "prev"], + from_video_id: str, + from_ms: int, + geo_bounds: dict | None = None, + limit: int = 1, + ) -> list[GlobalJumpResult]: + """Search for videos with GPS location data in global timeline order. + + Queries the video_locations projection table joined with videos to find + videos that have GPS coordinates. Unlike other artifact searches, location + data is per-video (not per-timestamp), so results point to the start of + each video. + + Args: + direction: Navigation direction ("next" for forward, "prev" for backward). + from_video_id: Starting video ID for the search. + from_ms: Starting timestamp in milliseconds (used for timeline position). + geo_bounds: Optional geographic bounds filter with keys: + min_lat, max_lat, min_lon, max_lon. + limit: Maximum number of results to return (default 1). + + Returns: + List of GlobalJumpResult objects ordered by global timeline. + For "next": ascending order (chronologically after current position). + For "prev": descending order (chronologically before current position). + Each result points to start_ms=0 since location is per-video. + Empty list if no videos with location data are found. + + Raises: + VideoNotFoundError: If from_video_id does not exist. + """ + # Get the current video to determine its position in the global timeline + current_video = self._get_video(from_video_id) + current_file_created_at = current_video.file_created_at + + # Convert datetime to string for SQLite comparison + current_file_created_at_param = None + if current_file_created_at is not None: + bind = self.session.bind + is_sqlite = bind.dialect.name == "sqlite" + if is_sqlite: + current_file_created_at_param = current_file_created_at.strftime( + "%Y-%m-%d %H:%M:%S.%f" + ) + else: + current_file_created_at_param = current_file_created_at + + # Build direction-specific SQL components + # Note: Location is per-video, so we exclude the current video entirely + # and only look at videos chronologically after/before it + if direction == "next": + order_clause = """ + ORDER BY v.file_created_at ASC NULLS LAST, + v.video_id ASC + """ + if current_file_created_at is not None: + direction_clause = """ + AND ( + v.file_created_at > :current_file_created_at + OR v.file_created_at IS NULL + OR (v.file_created_at = :current_file_created_at + AND v.video_id > :from_video_id) + ) + """ + else: + direction_clause = """ + AND ( + v.file_created_at IS NULL + AND v.video_id > :from_video_id + ) + """ + else: # direction == "prev" + order_clause = """ + ORDER BY v.file_created_at DESC NULLS LAST, + v.video_id DESC + """ + if current_file_created_at is not None: + direction_clause = """ + AND ( + (v.file_created_at IS NOT NULL + AND v.file_created_at < :current_file_created_at) + OR (v.file_created_at = :current_file_created_at + AND v.video_id < :from_video_id) + ) + """ + else: + direction_clause = """ + AND ( + v.file_created_at IS NOT NULL + OR (v.file_created_at IS NULL + AND v.video_id < :from_video_id) + ) + """ + + # Build geo_bounds filter if provided + geo_filter = "" + if geo_bounds: + geo_conditions = [] + if "min_lat" in geo_bounds: + geo_conditions.append("l.latitude >= :min_lat") + if "max_lat" in geo_bounds: + geo_conditions.append("l.latitude <= :max_lat") + if "min_lon" in geo_bounds: + geo_conditions.append("l.longitude >= :min_lon") + if "max_lon" in geo_bounds: + geo_conditions.append("l.longitude <= :max_lon") + if geo_conditions: + geo_filter = "AND " + " AND ".join(geo_conditions) + + sql = text( + f""" + SELECT + l.artifact_id, + l.video_id, + l.latitude, + l.longitude, + l.altitude, + l.country, + l.state, + l.city, + v.filename, + v.file_created_at + FROM video_locations l + JOIN videos v ON v.video_id = l.video_id + WHERE 1=1 + {direction_clause} + {geo_filter} + {order_clause} + LIMIT :limit + """ + ) + + params = { + "from_video_id": from_video_id, + "limit": limit, + } + if current_file_created_at_param is not None: + params["current_file_created_at"] = current_file_created_at_param + if geo_bounds: + params.update(geo_bounds) + + rows = self.session.execute(sql, params).fetchall() + + # Convert results to GlobalJumpResult objects + results = [] + for row in rows: + result = self._to_global_result( + video_id=row.video_id, + video_filename=row.filename, + file_created_at=row.file_created_at, + start_ms=0, # Location is per-video, so start at beginning + end_ms=0, + artifact_id=row.artifact_id, + preview={ + "latitude": row.latitude, + "longitude": row.longitude, + "altitude": row.altitude, + "country": row.country, + "state": row.state, + "city": row.city, + }, + ) + results.append(result) + + logger.debug( + f"_search_locations_global: direction={direction}, " + f"from_video_id={from_video_id}, from_ms={from_ms}, " + f"geo_bounds={geo_bounds}, found {len(results)} results" + ) + + return results + + # Valid artifact kinds for global jump navigation + VALID_KINDS = {"object", "face", "transcript", "ocr", "scene", "place", "location"} + + def jump_next( + self, + kind: str, + from_video_id: str, + from_ms: int | None = None, + label: str | None = None, + query: str | None = None, + face_cluster_id: str | None = None, + min_confidence: float | None = None, + limit: int = 1, + ) -> list[GlobalJumpResult]: + """Navigate forward in the global timeline to find matching artifacts. + + Routes to the appropriate search method based on the artifact kind. + Results are ordered chronologically after the current position. + + Args: + kind: Type of artifact to search for. Must be one of: + object, face, transcript, ocr, scene, place, location. + from_video_id: Starting video ID for the search. + from_ms: Starting timestamp in milliseconds (default: 0). + label: Filter by label (for object and place kinds). + query: Text search query (for transcript and ocr kinds). + face_cluster_id: Filter by face cluster ID (for face kind). + min_confidence: Minimum confidence threshold (0-1). + limit: Maximum number of results to return (default 1). + + Returns: + List of GlobalJumpResult objects ordered by global timeline. + Empty list if no matching artifacts are found. + + Raises: + InvalidParameterError: If kind is not a valid artifact type. + VideoNotFoundError: If from_video_id does not exist. + """ + if kind not in self.VALID_KINDS: + valid_kinds = ", ".join(sorted(self.VALID_KINDS)) + raise InvalidParameterError( + "kind", + f"Invalid artifact kind. Must be one of: {valid_kinds}", + ) + + # Default from_ms to 0 for "next" direction + if from_ms is None: + from_ms = 0 + + if kind == "object": + return self._search_objects_global( + direction="next", + from_video_id=from_video_id, + from_ms=from_ms, + label=label, + min_confidence=min_confidence, + limit=limit, + ) + elif kind == "face": + # Face cluster search not yet implemented + # Will be implemented in task 6 + raise InvalidParameterError( + "kind", "Face cluster search is not yet implemented" + ) + elif kind == "transcript": + if query is None: + raise InvalidParameterError( + "query", "Query parameter is required for transcript search" + ) + return self._search_transcript_global( + direction="next", + from_video_id=from_video_id, + from_ms=from_ms, + query=query, + limit=limit, + ) + elif kind == "ocr": + if query is None: + raise InvalidParameterError( + "query", "Query parameter is required for OCR search" + ) + return self._search_ocr_global( + direction="next", + from_video_id=from_video_id, + from_ms=from_ms, + query=query, + limit=limit, + ) + elif kind == "scene": + return self._search_scenes_global( + direction="next", + from_video_id=from_video_id, + from_ms=from_ms, + limit=limit, + ) + elif kind == "place": + return self._search_places_global( + direction="next", + from_video_id=from_video_id, + from_ms=from_ms, + label=label, + min_confidence=min_confidence, + limit=limit, + ) + elif kind == "location": + return self._search_locations_global( + direction="next", + from_video_id=from_video_id, + from_ms=from_ms, + limit=limit, + ) + + # This should never be reached due to the validation above + raise InvalidParameterError("kind", f"Unknown artifact kind: {kind}") + + def jump_prev( + self, + kind: str, + from_video_id: str, + from_ms: int | None = None, + label: str | None = None, + query: str | None = None, + face_cluster_id: str | None = None, + min_confidence: float | None = None, + limit: int = 1, + ) -> list[GlobalJumpResult]: + """Navigate backward in the global timeline to find matching artifacts. + + Routes to the appropriate search method based on the artifact kind. + Results are ordered chronologically before the current position. + + Args: + kind: Type of artifact to search for. Must be one of: + object, face, transcript, ocr, scene, place, location. + from_video_id: Starting video ID for the search. + from_ms: Starting timestamp in milliseconds. If None, defaults to + a large value representing the end of the video. + label: Filter by label (for object and place kinds). + query: Text search query (for transcript and ocr kinds). + face_cluster_id: Filter by face cluster ID (for face kind). + min_confidence: Minimum confidence threshold (0-1). + limit: Maximum number of results to return (default 1). + + Returns: + List of GlobalJumpResult objects ordered by global timeline + (descending - most recent first). + Empty list if no matching artifacts are found. + + Raises: + InvalidParameterError: If kind is not a valid artifact type. + VideoNotFoundError: If from_video_id does not exist. + """ + if kind not in self.VALID_KINDS: + valid_kinds = ", ".join(sorted(self.VALID_KINDS)) + raise InvalidParameterError( + "kind", + f"Invalid artifact kind. Must be one of: {valid_kinds}", + ) + + # Default from_ms to a large value for "prev" direction + # This represents "end of video" - searching backward from the end + if from_ms is None: + from_ms = 2**31 - 1 # Max 32-bit signed integer + + if kind == "object": + return self._search_objects_global( + direction="prev", + from_video_id=from_video_id, + from_ms=from_ms, + label=label, + min_confidence=min_confidence, + limit=limit, + ) + elif kind == "face": + # Face cluster search not yet implemented + # Will be implemented in task 6 + raise InvalidParameterError( + "kind", "Face cluster search is not yet implemented" + ) + elif kind == "transcript": + if query is None: + raise InvalidParameterError( + "query", "Query parameter is required for transcript search" + ) + return self._search_transcript_global( + direction="prev", + from_video_id=from_video_id, + from_ms=from_ms, + query=query, + limit=limit, + ) + elif kind == "ocr": + if query is None: + raise InvalidParameterError( + "query", "Query parameter is required for OCR search" + ) + return self._search_ocr_global( + direction="prev", + from_video_id=from_video_id, + from_ms=from_ms, + query=query, + limit=limit, + ) + elif kind == "scene": + return self._search_scenes_global( + direction="prev", + from_video_id=from_video_id, + from_ms=from_ms, + limit=limit, + ) + elif kind == "place": + return self._search_places_global( + direction="prev", + from_video_id=from_video_id, + from_ms=from_ms, + label=label, + min_confidence=min_confidence, + limit=limit, + ) + elif kind == "location": + return self._search_locations_global( + direction="prev", + from_video_id=from_video_id, + from_ms=from_ms, + limit=limit, + ) + + # This should never be reached due to the validation above + raise InvalidParameterError("kind", f"Unknown artifact kind: {kind}") diff --git a/backend/tests/test_backward_compatibility.py b/backend/tests/test_backward_compatibility.py new file mode 100644 index 0000000..0c95a4f --- /dev/null +++ b/backend/tests/test_backward_compatibility.py @@ -0,0 +1,562 @@ +"""Backward compatibility tests for single-video jump endpoint. + +This module verifies that the new global jump feature does not affect +the existing single-video jump functionality. + +**Property 19: Backward Compatibility** +*For any* request to the existing GET /videos/{video_id}/jump endpoint, +the system should continue to return results scoped to that video only, +without being affected by the new global jump feature. + +**Property 20: Global Jump Independence** +*For any* request to the new GET /jump/global endpoint, the system should +return results across all videos without affecting the behavior of the +existing single-video jump endpoint. + +**Validates: Requirements 10.1, 10.2, 10.3, 10.4** +""" + +import json +from datetime import datetime +from unittest.mock import MagicMock + +import pytest +from sqlalchemy import create_engine +from sqlalchemy.orm import sessionmaker + +from src.database.models import Base, ObjectLabel +from src.database.models import Video as VideoEntity +from src.domain.artifacts import ArtifactEnvelope +from src.domain.schema_initialization import register_all_schemas +from src.domain.schema_registry import SchemaRegistry +from src.repositories.artifact_repository import SqlArtifactRepository +from src.repositories.selection_policy_manager import SelectionPolicyManager +from src.services.global_jump_service import GlobalJumpService +from src.services.jump_navigation_service import JumpNavigationService + + +@pytest.fixture +def engine(): + """Create in-memory SQLite engine for testing.""" + engine = create_engine("sqlite:///:memory:") + Base.metadata.create_all(engine) + return engine + + +@pytest.fixture +def session(engine): + """Create database session for testing.""" + session_factory = sessionmaker(bind=engine) + session = session_factory() + yield session + session.close() + + +@pytest.fixture(scope="session") +def schema_registry(): + """Create and initialize schema registry once for all tests.""" + register_all_schemas() + return SchemaRegistry + + +@pytest.fixture +def artifact_repo(session, schema_registry): + """Create artifact repository instance with mocked projection sync.""" + mock_projection_sync = MagicMock() + mock_projection_sync.sync_artifact = MagicMock() + return SqlArtifactRepository(session, schema_registry, mock_projection_sync) + + +@pytest.fixture +def policy_manager(session): + """Create selection policy manager instance.""" + return SelectionPolicyManager(session) + + +@pytest.fixture +def single_video_jump_service(artifact_repo, policy_manager): + """Create single-video jump navigation service instance.""" + return JumpNavigationService(artifact_repo, policy_manager) + + +@pytest.fixture +def global_jump_service(session): + """Create global jump service instance.""" + return GlobalJumpService(session, artifact_repo=None) + + +def create_test_video( + session, + video_id: str, + filename: str, + file_created_at: datetime | None = None, +) -> VideoEntity: + """Helper to create a test video.""" + video = VideoEntity( + video_id=video_id, + file_path=f"/test/{filename}", + filename=filename, + last_modified=datetime.now(), + file_created_at=file_created_at, + status="completed", + ) + session.add(video) + session.commit() + return video + + +def create_object_artifact( + artifact_id: str, + asset_id: str, + start_ms: int, + end_ms: int, + label: str, + confidence: float = 0.9, + run_id: str = "run_1", +) -> ArtifactEnvelope: + """Helper to create object detection artifact.""" + payload = { + "label": label, + "confidence": confidence, + "bounding_box": {"x": 0.1, "y": 0.2, "width": 0.3, "height": 0.4}, + "frame_number": 0, + } + return ArtifactEnvelope( + artifact_id=artifact_id, + asset_id=asset_id, + artifact_type="object.detection", + schema_version=1, + span_start_ms=start_ms, + span_end_ms=end_ms, + payload_json=json.dumps(payload), + producer="yolo", + producer_version="8.0.0", + model_profile="balanced", + config_hash="xyz789", + input_hash="uvw012", + run_id=run_id, + created_at=datetime.now(), + ) + + +def create_object_label( + session, + artifact_id: str, + asset_id: str, + label: str, + confidence: float, + start_ms: int, + end_ms: int, +) -> ObjectLabel: + """Helper to create an object label in the projection table.""" + obj = ObjectLabel( + artifact_id=artifact_id, + asset_id=asset_id, + label=label, + confidence=confidence, + start_ms=start_ms, + end_ms=end_ms, + ) + session.add(obj) + session.commit() + return obj + + +class TestBackwardCompatibility: + """Tests for backward compatibility of single-video jump endpoint. + + **Property 19: Backward Compatibility** + **Validates: Requirements 10.1, 10.3** + """ + + def test_single_video_jump_still_works( + self, session, single_video_jump_service, artifact_repo + ): + """Test that existing single-video jump endpoint still works. + + Requirement 10.1: THE existing GET /videos/{video_id}/jump endpoint + SHALL remain unchanged and functional. + """ + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + # Create artifacts for the video + artifact1 = create_object_artifact("obj_1", video.video_id, 0, 100, "dog", 0.9) + artifact2 = create_object_artifact( + "obj_2", video.video_id, 500, 600, "dog", 0.85 + ) + artifact3 = create_object_artifact( + "obj_3", video.video_id, 1000, 1100, "dog", 0.95 + ) + + artifact_repo.create(artifact1) + artifact_repo.create(artifact2) + artifact_repo.create(artifact3) + + # Test jump_next + result = single_video_jump_service.jump_next( + asset_id=video.video_id, + artifact_type="object.detection", + from_ms=200, + label="dog", + ) + + assert result is not None + assert result["jump_to"]["start_ms"] == 500 + assert result["jump_to"]["end_ms"] == 600 + assert "obj_2" in result["artifact_ids"] + + def test_single_video_jump_prev_still_works( + self, session, single_video_jump_service, artifact_repo + ): + """Test that single-video jump_prev still works correctly.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + artifact1 = create_object_artifact("obj_1", video.video_id, 0, 100, "cat", 0.9) + artifact2 = create_object_artifact( + "obj_2", video.video_id, 500, 600, "cat", 0.85 + ) + artifact3 = create_object_artifact( + "obj_3", video.video_id, 1000, 1100, "cat", 0.95 + ) + + artifact_repo.create(artifact1) + artifact_repo.create(artifact2) + artifact_repo.create(artifact3) + + # Test jump_prev from 800ms - should find artifact2 (ends at 600ms < 800ms) + # The service finds artifacts that END before from_ms + result = single_video_jump_service.jump_prev( + asset_id=video.video_id, + artifact_type="object.detection", + from_ms=800, + label="cat", + ) + + assert result is not None + # Should find artifact2 (500-600ms) as it's the latest artifact + # that ends before 800ms + assert result["jump_to"]["start_ms"] == 500 + assert result["jump_to"]["end_ms"] == 600 + assert "obj_2" in result["artifact_ids"] + + def test_single_video_jump_returns_results_scoped_to_video_only( + self, session, single_video_jump_service, artifact_repo + ): + """Test that single-video jump returns results scoped to that video only. + + Requirement 10.3: WHEN a user uses the existing single-video jump endpoint + THEN THE System SHALL continue to return results scoped to that video only. + """ + # Create two videos with artifacts + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + # Create artifacts in both videos + artifact1 = create_object_artifact("obj_1", video1.video_id, 0, 100, "dog", 0.9) + artifact2 = create_object_artifact( + "obj_2", video2.video_id, 0, 100, "dog", 0.95 + ) + + artifact_repo.create(artifact1) + artifact_repo.create(artifact2) + + # Jump from video1 - should NOT find artifacts from video2 + result = single_video_jump_service.jump_next( + asset_id=video1.video_id, + artifact_type="object.detection", + from_ms=500, # After all artifacts in video1 + label="dog", + ) + + # Should return None because there are no more artifacts in video1 + # (even though video2 has a matching artifact) + assert result is None + + def test_single_video_jump_with_confidence_filter( + self, session, single_video_jump_service, artifact_repo + ): + """Test that single-video jump confidence filtering still works.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + artifact1 = create_object_artifact("obj_1", video.video_id, 0, 100, "dog", 0.5) + artifact2 = create_object_artifact( + "obj_2", video.video_id, 500, 600, "dog", 0.7 + ) + artifact3 = create_object_artifact( + "obj_3", video.video_id, 1000, 1100, "dog", 0.9 + ) + + artifact_repo.create(artifact1) + artifact_repo.create(artifact2) + artifact_repo.create(artifact3) + + # Jump with min_confidence=0.8 + result = single_video_jump_service.jump_next( + asset_id=video.video_id, + artifact_type="object.detection", + from_ms=0, + label="dog", + min_confidence=0.8, + ) + + assert result is not None + assert result["jump_to"]["start_ms"] == 1000 + assert "obj_3" in result["artifact_ids"] + + +class TestGlobalJumpIndependence: + """Tests for global jump independence from single-video jump. + + **Property 20: Global Jump Independence** + **Validates: Requirements 10.2, 10.4** + """ + + def test_global_jump_returns_results_across_all_videos( + self, session, global_jump_service + ): + """Test that global jump returns results across all videos. + + Requirement 10.2: THE new GET /jump/global endpoint SHALL be additive + and not modify existing video jump behavior. + """ + # Create multiple videos with different file_created_at + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + video3 = create_test_video( + session, "video_3", "video3.mp4", datetime(2025, 1, 3, 12, 0, 0) + ) + + # Create object labels in projection table + create_object_label(session, "obj_1", video1.video_id, "dog", 0.9, 0, 100) + create_object_label(session, "obj_2", video2.video_id, "dog", 0.85, 0, 100) + create_object_label(session, "obj_3", video3.video_id, "dog", 0.95, 0, 100) + + # Global jump from video1 should find artifacts in video2 and video3 + results = global_jump_service._search_objects_global( + direction="next", + from_video_id=video1.video_id, + from_ms=500, # After all artifacts in video1 + label="dog", + limit=10, + ) + + assert len(results) == 2 + assert results[0].video_id == "video_2" + assert results[1].video_id == "video_3" + + def test_global_jump_does_not_affect_single_video_jump( + self, session, single_video_jump_service, global_jump_service, artifact_repo + ): + """Test that using global jump doesn't affect single-video jump behavior. + + Requirement 10.4: WHEN a user uses the new global jump endpoint THEN + THE System SHALL return results across all videos without affecting + single-video jump functionality. + """ + # Create videos + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + # Create artifacts for single-video jump service + artifact1 = create_object_artifact("obj_1", video1.video_id, 0, 100, "cat", 0.9) + artifact2 = create_object_artifact( + "obj_2", video1.video_id, 500, 600, "cat", 0.85 + ) + artifact_repo.create(artifact1) + artifact_repo.create(artifact2) + + # Create object labels for global jump service + create_object_label(session, "obj_3", video1.video_id, "cat", 0.9, 0, 100) + create_object_label(session, "obj_4", video2.video_id, "cat", 0.85, 0, 100) + + # First, use global jump + global_results = global_jump_service._search_objects_global( + direction="next", + from_video_id=video1.video_id, + from_ms=500, + label="cat", + ) + + # Global jump should find artifact in video2 + assert len(global_results) == 1 + assert global_results[0].video_id == "video_2" + + # Now use single-video jump - it should still work correctly + # and return results scoped to video1 only + single_result = single_video_jump_service.jump_next( + asset_id=video1.video_id, + artifact_type="object.detection", + from_ms=200, + label="cat", + ) + + # Single-video jump should find artifact in video1 only + assert single_result is not None + assert single_result["jump_to"]["start_ms"] == 500 + assert "obj_2" in single_result["artifact_ids"] + + def test_services_are_independent( + self, session, single_video_jump_service, global_jump_service, artifact_repo + ): + """Test that single-video and global jump services are independent.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + # Create artifact for single-video jump + artifact = create_object_artifact( + "obj_1", video.video_id, 100, 200, "bird", 0.9 + ) + artifact_repo.create(artifact) + + # Create object label for global jump + create_object_label(session, "obj_2", video.video_id, "bird", 0.9, 300, 400) + + # Single-video jump uses artifact repository + single_result = single_video_jump_service.jump_next( + asset_id=video.video_id, + artifact_type="object.detection", + from_ms=0, + label="bird", + ) + + # Global jump uses projection tables + global_results = global_jump_service._search_objects_global( + direction="next", + from_video_id=video.video_id, + from_ms=0, + label="bird", + ) + + # Both should work independently + assert single_result is not None + assert single_result["jump_to"]["start_ms"] == 100 + + assert len(global_results) == 1 + assert global_results[0].jump_to.start_ms == 300 + + +class TestConcurrentUsage: + """Tests for concurrent usage of both jump endpoints.""" + + def test_alternating_between_single_and_global_jump( + self, session, single_video_jump_service, global_jump_service, artifact_repo + ): + """Test alternating between single-video and global jump.""" + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + # Create artifacts for single-video jump + artifact1 = create_object_artifact( + "obj_1", video1.video_id, 0, 100, "fish", 0.9 + ) + artifact2 = create_object_artifact( + "obj_2", video1.video_id, 500, 600, "fish", 0.85 + ) + artifact_repo.create(artifact1) + artifact_repo.create(artifact2) + + # Create object labels for global jump + create_object_label(session, "obj_3", video1.video_id, "fish", 0.9, 0, 100) + create_object_label(session, "obj_4", video1.video_id, "fish", 0.85, 500, 600) + create_object_label(session, "obj_5", video2.video_id, "fish", 0.95, 0, 100) + + # Alternate between single and global jump + # 1. Single-video jump - from 0ms should find first artifact at 0ms + single_result1 = single_video_jump_service.jump_next( + asset_id=video1.video_id, + artifact_type="object.detection", + from_ms=0, + label="fish", + ) + assert single_result1 is not None + # jump_next finds artifacts starting AT or AFTER from_ms + assert single_result1["jump_to"]["start_ms"] == 0 + + # 2. Global jump - from 0ms should find first artifact at 500ms (after 0) + global_result1 = global_jump_service._search_objects_global( + direction="next", + from_video_id=video1.video_id, + from_ms=0, + label="fish", + ) + assert len(global_result1) == 1 + # Global jump finds artifacts AFTER from_ms (strictly greater) + assert global_result1[0].jump_to.start_ms == 500 + + # 3. Single-video jump prev - from 1000ms should find artifact at 500ms + single_result2 = single_video_jump_service.jump_prev( + asset_id=video1.video_id, + artifact_type="object.detection", + from_ms=1000, + label="fish", + ) + assert single_result2 is not None + assert single_result2["jump_to"]["start_ms"] == 500 + + # 4. Global jump - from 600ms should find artifact in video2 + global_result2 = global_jump_service._search_objects_global( + direction="next", + from_video_id=video1.video_id, + from_ms=600, + label="fish", + ) + assert len(global_result2) == 1 + assert global_result2[0].video_id == "video_2" + + def test_no_state_leakage_between_services( + self, session, single_video_jump_service, global_jump_service, artifact_repo + ): + """Test that there's no state leakage between services.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + # Create artifacts + artifact = create_object_artifact( + "obj_1", video.video_id, 100, 200, "elephant", 0.9 + ) + artifact_repo.create(artifact) + create_object_label(session, "obj_2", video.video_id, "elephant", 0.9, 300, 400) + + # Use global jump multiple times + for _ in range(3): + global_jump_service._search_objects_global( + direction="next", + from_video_id=video.video_id, + from_ms=0, + label="elephant", + ) + + # Single-video jump should still work correctly + result = single_video_jump_service.jump_next( + asset_id=video.video_id, + artifact_type="object.detection", + from_ms=0, + label="elephant", + ) + + assert result is not None + assert result["jump_to"]["start_ms"] == 100 diff --git a/backend/tests/test_face_detection.py b/backend/tests/test_face_detection.py deleted file mode 100644 index bdbd11f..0000000 --- a/backend/tests/test_face_detection.py +++ /dev/null @@ -1,676 +0,0 @@ -"""Tests for face detection service and task handler.""" - -import uuid -from unittest.mock import Mock, patch - -import pytest - -from src.domain.models import Face, Task, Video -from src.services.face_detection_service import ( - FaceDetectionError, - FaceDetectionService, -) -from src.services.face_detection_task_handler import FaceDetectionTaskHandler -from src.services.task_orchestration import TaskType - -# ============================================================================ -# Fixtures -# ============================================================================ - - -@pytest.fixture -def mock_yolo_face_model(): - """Create a mock YOLO face model.""" - model = Mock() - model.names = {0: "face"} # Face model typically has single class - return model - - -@pytest.fixture -def mock_av_container(): - """Create a mock PyAV container.""" - container = Mock() - stream = Mock() - stream.average_rate = 30.0 - stream.frames = 90 - stream.codec_context.name = "h264" - container.streams.video = [stream] - return container - - -@pytest.fixture -def mock_frame(): - """Create a mock PyAV frame.""" - frame = Mock() - frame.to_ndarray.return_value = Mock() # numpy array - return frame - - -@pytest.fixture -def mock_face_detection_box(): - """Create a mock YOLO face detection box.""" - box = Mock() - box.cls = [0] # face class - box.xyxy = [Mock(tolist=Mock(return_value=[50.0, 60.0, 150.0, 200.0]))] - box.conf = [0.92] - return box - - -# ============================================================================ -# FaceDetectionService Initialization Tests -# ============================================================================ - - -class TestFaceDetectionServiceInit: - """Tests for FaceDetectionService initialization.""" - - @patch("ultralytics.YOLO") - def test_initialization_success(self, mock_yolo_class): - """Test successful service initialization.""" - mock_model = Mock() - mock_yolo_class.return_value = mock_model - - service = FaceDetectionService(model_name="yolov8n-face.pt") - - assert service.model_name == "yolov8n-face.pt" - assert service.model == mock_model - mock_yolo_class.assert_called_once_with("yolov8n-face.pt") - - @patch("ultralytics.YOLO") - def test_initialization_with_different_models(self, mock_yolo_class): - """Test initialization with different face model variants.""" - model_variants = [ - "yolov8n-face.pt", - "yolov8s-face.pt", - "yolov8m-face.pt", - ] - - for model_name in model_variants: - mock_yolo_class.reset_mock() - service = FaceDetectionService(model_name=model_name) - assert service.model_name == model_name - mock_yolo_class.assert_called_with(model_name) - - @patch("ultralytics.YOLO") - def test_initialization_model_load_failure(self, mock_yolo_class): - """Test initialization when model fails to load.""" - mock_yolo_class.side_effect = Exception("Model file not found") - - with pytest.raises(FaceDetectionError) as exc_info: - FaceDetectionService(model_name="nonexistent.pt") - - assert "Failed to load YOLO face model" in str(exc_info.value) - - -# ============================================================================ -# FaceDetectionService Detection Tests -# ============================================================================ - - -class TestFaceDetectionService: - """Tests for FaceDetectionService face detection.""" - - @patch("ultralytics.YOLO") - @patch("av.open") - @patch("pathlib.Path.exists") - def test_detect_faces_success( - self, - mock_path_exists, - mock_av_open, - mock_yolo_class, - mock_av_container, - mock_frame, - ): - """Test successful face detection.""" - # Setup mocks - mock_path_exists.return_value = True - mock_model = Mock() - mock_yolo_class.return_value = mock_model - mock_av_open.return_value = mock_av_container - - # Create mock detection - mock_box = Mock() - mock_box.xyxy = [Mock(tolist=Mock(return_value=[50.0, 60.0, 150.0, 200.0]))] - mock_box.conf = [0.92] - - mock_result = Mock() - mock_result.boxes = [mock_box] - mock_model.return_value = [mock_result] - - # Setup frame iteration (3 frames, sample every 30) - mock_av_container.decode.return_value = [mock_frame] * 90 - - service = FaceDetectionService() - faces = service.detect_faces_in_video( - video_path="/path/to/video.mp4", - video_id="test-video-id", - sample_rate=30, - ) - - # Should return single Face object with all detections - assert len(faces) == 1 - face = faces[0] - assert face.video_id == "test-video-id" - assert face.person_id is None # No clustering yet - assert len(face.timestamps) == 3 # 3 sampled frames - assert len(face.bounding_boxes) == 3 - assert face.confidence > 0 # Average confidence - - @patch("ultralytics.YOLO") - def test_detect_faces_video_not_found(self, mock_yolo_class): - """Test detection with non-existent video file.""" - mock_yolo_class.return_value = Mock() - service = FaceDetectionService() - - with pytest.raises(FaceDetectionError) as exc_info: - service.detect_faces_in_video( - video_path="/nonexistent/video.mp4", - video_id="test-id", - sample_rate=30, - ) - - assert "Video file not found" in str(exc_info.value) - - @patch("ultralytics.YOLO") - @patch("av.open") - @patch("pathlib.Path.exists") - def test_detect_faces_no_faces_found( - self, - mock_path_exists, - mock_av_open, - mock_yolo_class, - mock_av_container, - mock_frame, - ): - """Test detection when no faces are found.""" - mock_path_exists.return_value = True - mock_model = Mock() - mock_yolo_class.return_value = mock_model - mock_av_open.return_value = mock_av_container - - # No detections - mock_result = Mock() - mock_result.boxes = [] - mock_model.return_value = [mock_result] - - mock_av_container.decode.return_value = [mock_frame] * 90 - - service = FaceDetectionService() - faces = service.detect_faces_in_video( - video_path="/path/to/video.mp4", - video_id="test-video-id", - sample_rate=30, - ) - - # Should return empty list when no faces detected - assert len(faces) == 0 - - @patch("ultralytics.YOLO") - @patch("av.open") - @patch("pathlib.Path.exists") - def test_detect_faces_multiple_faces_per_frame( - self, - mock_path_exists, - mock_av_open, - mock_yolo_class, - mock_av_container, - mock_frame, - ): - """Test detection with multiple faces in single frame.""" - mock_path_exists.return_value = True - mock_model = Mock() - mock_yolo_class.return_value = mock_model - mock_av_open.return_value = mock_av_container - - # Create 3 face detections in one frame - mock_boxes = [] - for i in range(3): - box = Mock() - box.xyxy = [ - Mock( - tolist=Mock( - return_value=[ - 50.0 + i * 100, - 60.0, - 150.0 + i * 100, - 200.0, - ] - ) - ) - ] - box.conf = [0.90 + i * 0.02] - mock_boxes.append(box) - - mock_result = Mock() - mock_result.boxes = mock_boxes - mock_model.return_value = [mock_result] - - mock_av_container.decode.return_value = [mock_frame] * 30 - - service = FaceDetectionService() - faces = service.detect_faces_in_video( - video_path="/path/to/video.mp4", - video_id="test-video-id", - sample_rate=30, - ) - - # All faces grouped into single Face object - assert len(faces) == 1 - face = faces[0] - assert len(face.timestamps) == 3 # 3 detections from 1 sampled frame - assert len(face.bounding_boxes) == 3 - - @patch("ultralytics.YOLO") - @patch("av.open") - @patch("pathlib.Path.exists") - def test_detect_faces_confidence_calculation( - self, - mock_path_exists, - mock_av_open, - mock_yolo_class, - mock_av_container, - mock_frame, - ): - """Test average confidence calculation.""" - mock_path_exists.return_value = True - mock_model = Mock() - mock_yolo_class.return_value = mock_model - mock_av_open.return_value = mock_av_container - - # Create detections with known confidences - confidences = [0.90, 0.85, 0.95] - - # Create separate mock results for each call - mock_results = [] - for conf in confidences: - box = Mock() - box.xyxy = [Mock(tolist=Mock(return_value=[50.0, 60.0, 150.0, 200.0]))] - box.conf = [conf] - - result = Mock() - result.boxes = [box] - mock_results.append([result]) - - # Set side_effect to return different results for each call - mock_model.side_effect = mock_results - - # Create frames - frames = [] - for _ in confidences: - frame = Mock() - frame.to_ndarray.return_value = Mock() - frames.append(frame) - - mock_av_container.decode.return_value = frames - - service = FaceDetectionService() - faces = service.detect_faces_in_video( - video_path="/path/to/video.mp4", - video_id="test-video-id", - sample_rate=1, # Sample every frame - ) - - # Check average confidence - assert len(faces) == 1 - expected_avg = sum(confidences) / len(confidences) - assert abs(faces[0].confidence - expected_avg) < 0.01 - - @patch("ultralytics.YOLO") - @patch("av.open") - @patch("pathlib.Path.exists") - def test_detect_faces_person_id_is_null( - self, - mock_path_exists, - mock_av_open, - mock_yolo_class, - mock_av_container, - mock_frame, - ): - """Test that person_id is None (no clustering in Phase 1).""" - mock_path_exists.return_value = True - mock_model = Mock() - mock_yolo_class.return_value = mock_model - mock_av_open.return_value = mock_av_container - - mock_box = Mock() - mock_box.xyxy = [Mock(tolist=Mock(return_value=[50.0, 60.0, 150.0, 200.0]))] - mock_box.conf = [0.92] - - mock_result = Mock() - mock_result.boxes = [mock_box] - mock_model.return_value = [mock_result] - - mock_av_container.decode.return_value = [mock_frame] * 30 - - service = FaceDetectionService() - faces = service.detect_faces_in_video( - video_path="/path/to/video.mp4", - video_id="test-video-id", - sample_rate=30, - ) - - # person_id should be None (Phase 1 - no clustering) - assert len(faces) == 1 - assert faces[0].person_id is None - - @patch("ultralytics.YOLO") - @patch("av.open") - @patch("pathlib.Path.exists") - def test_detect_faces_timestamps_sorted( - self, - mock_path_exists, - mock_av_open, - mock_yolo_class, - mock_av_container, - mock_frame, - ): - """Test that timestamps are in chronological order.""" - mock_path_exists.return_value = True - mock_model = Mock() - mock_yolo_class.return_value = mock_model - mock_av_open.return_value = mock_av_container - - mock_box = Mock() - mock_box.xyxy = [Mock(tolist=Mock(return_value=[50.0, 60.0, 150.0, 200.0]))] - mock_box.conf = [0.92] - - mock_result = Mock() - mock_result.boxes = [mock_box] - mock_model.return_value = [mock_result] - - mock_av_container.decode.return_value = [mock_frame] * 90 - - service = FaceDetectionService() - faces = service.detect_faces_in_video( - video_path="/path/to/video.mp4", - video_id="test-video-id", - sample_rate=30, - ) - - # Timestamps should be in order - assert len(faces) == 1 - timestamps = faces[0].timestamps - assert timestamps == sorted(timestamps) - - @patch("ultralytics.YOLO") - @patch("av.open") - @patch("pathlib.Path.exists") - def test_detect_faces_bounding_boxes_format( - self, - mock_path_exists, - mock_av_open, - mock_yolo_class, - mock_av_container, - mock_frame, - ): - """Test bounding box format.""" - mock_path_exists.return_value = True - mock_model = Mock() - mock_yolo_class.return_value = mock_model - mock_av_open.return_value = mock_av_container - - mock_box = Mock() - mock_box.xyxy = [Mock(tolist=Mock(return_value=[50.0, 60.0, 150.0, 200.0]))] - mock_box.conf = [0.92] - - mock_result = Mock() - mock_result.boxes = [mock_box] - mock_model.return_value = [mock_result] - - mock_av_container.decode.return_value = [mock_frame] * 30 - - service = FaceDetectionService() - faces = service.detect_faces_in_video( - video_path="/path/to/video.mp4", - video_id="test-video-id", - sample_rate=30, - ) - - # Check bounding box format - assert len(faces) == 1 - bbox = faces[0].bounding_boxes[0] - assert "frame" in bbox - assert "timestamp" in bbox - assert "bbox" in bbox - assert "confidence" in bbox - assert len(bbox["bbox"]) == 4 # [x1, y1, x2, y2] - - @patch("ultralytics.YOLO") - @patch("av.open") - @patch("pathlib.Path.exists") - def test_detect_faces_frame_sampling( - self, - mock_path_exists, - mock_av_open, - mock_yolo_class, - mock_av_container, - mock_frame, - ): - """Test frame sampling with different sample rates.""" - mock_path_exists.return_value = True - mock_model = Mock() - mock_yolo_class.return_value = mock_model - mock_av_open.return_value = mock_av_container - - mock_box = Mock() - mock_box.xyxy = [Mock(tolist=Mock(return_value=[50.0, 60.0, 150.0, 200.0]))] - mock_box.conf = [0.92] - - mock_result = Mock() - mock_result.boxes = [mock_box] - mock_model.return_value = [mock_result] - - # 90 frames at 30fps = 3 seconds - mock_av_container.decode.return_value = [mock_frame] * 90 - - service = FaceDetectionService() - - # Sample every 30 frames (1 per second at 30fps) - faces = service.detect_faces_in_video( - video_path="/path/to/video.mp4", - video_id="test-video-id", - sample_rate=30, - ) - - # Should process 3 frames (0, 30, 60) - assert len(faces) == 1 - assert len(faces[0].timestamps) == 3 - - -# ============================================================================ -# FaceDetectionTaskHandler Tests -# ============================================================================ - - -class TestFaceDetectionTaskHandler: - """Tests for FaceDetectionTaskHandler.""" - - def test_handler_initialization(self): - """Test handler initialization.""" - mock_repo = Mock() - mock_service = Mock() - - handler = FaceDetectionTaskHandler( - face_repository=mock_repo, - detection_service=mock_service, - model_name="yolov8n-face.pt", - sample_rate=30, - ) - - assert handler.face_repository == mock_repo - assert handler.detection_service == mock_service - assert handler.model_name == "yolov8n-face.pt" - assert handler.sample_rate == 30 - - def test_process_face_detection_task_success(self): - """Test successful face detection task processing.""" - mock_repo = Mock() - mock_service = Mock() - - # Create mock face - mock_face = Face( - face_id=str(uuid.uuid4()), - video_id="test-video-id", - person_id=None, - timestamps=[1.0, 2.0, 3.0], - bounding_boxes=[ - { - "frame": 30, - "timestamp": 1.0, - "bbox": [50, 60, 150, 200], - "confidence": 0.92, - } - ], - confidence=0.92, - ) - - mock_service.detect_faces_in_video.return_value = [mock_face] - - handler = FaceDetectionTaskHandler( - face_repository=mock_repo, - detection_service=mock_service, - sample_rate=30, - ) - - task = Task( - task_id=str(uuid.uuid4()), - video_id="test-video-id", - task_type=TaskType.FACE_DETECTION.value, - ) - - video = Video( - video_id="test-video-id", - file_path="/path/to/video.mp4", - filename="video.mp4", - last_modified=Mock(), - ) - - result = handler.process_face_detection_task(task, video) - - assert result is True - mock_service.detect_faces_in_video.assert_called_once() - mock_repo.save.assert_called_once_with(mock_face) - - def test_process_face_detection_task_failure(self): - """Test face detection task processing failure.""" - mock_repo = Mock() - mock_service = Mock() - mock_service.detect_faces_in_video.side_effect = Exception("Detection failed") - - handler = FaceDetectionTaskHandler( - face_repository=mock_repo, - detection_service=mock_service, - ) - - task = Task( - task_id=str(uuid.uuid4()), - video_id="test-video-id", - task_type=TaskType.FACE_DETECTION.value, - ) - - video = Video( - video_id="test-video-id", - file_path="/path/to/video.mp4", - filename="video.mp4", - last_modified=Mock(), - ) - - result = handler.process_face_detection_task(task, video) - - assert result is False - mock_repo.save.assert_not_called() - - def test_get_detected_faces(self): - """Test getting detected faces for a video.""" - mock_repo = Mock() - mock_faces = [ - Face( - face_id=str(uuid.uuid4()), - video_id="test-video-id", - person_id=None, - timestamps=[1.0, 2.0], - bounding_boxes=[], - confidence=0.90, - ) - ] - mock_repo.find_by_video_id.return_value = mock_faces - - handler = FaceDetectionTaskHandler( - face_repository=mock_repo, - detection_service=Mock(), - ) - - faces = handler.get_detected_faces("test-video-id") - - assert faces == mock_faces - mock_repo.find_by_video_id.assert_called_once_with("test-video-id") - - def test_get_detected_faces_empty(self): - """Test getting faces when none exist.""" - mock_repo = Mock() - mock_repo.find_by_video_id.return_value = [] - - handler = FaceDetectionTaskHandler( - face_repository=mock_repo, - detection_service=Mock(), - ) - - faces = handler.get_detected_faces("test-video-id") - - assert faces == [] - - def test_get_faces_by_person(self): - """Test getting faces filtered by person ID.""" - mock_repo = Mock() - mock_faces = [ - Face( - face_id=str(uuid.uuid4()), - video_id="test-video-id", - person_id="person-1", - timestamps=[1.0, 2.0], - bounding_boxes=[], - confidence=0.90, - ) - ] - mock_repo.find_by_person_id.return_value = mock_faces - - handler = FaceDetectionTaskHandler( - face_repository=mock_repo, - detection_service=Mock(), - ) - - faces = handler.get_faces_by_person("test-video-id", "person-1") - - assert faces == mock_faces - mock_repo.find_by_person_id.assert_called_once_with("test-video-id", "person-1") - - def test_face_detection_task_with_custom_sample_rate(self): - """Test task processing with custom sample rate.""" - mock_repo = Mock() - mock_service = Mock() - mock_service.detect_faces_in_video.return_value = [] - - handler = FaceDetectionTaskHandler( - face_repository=mock_repo, - detection_service=mock_service, - sample_rate=60, # Custom sample rate - ) - - task = Task( - task_id=str(uuid.uuid4()), - video_id="test-video-id", - task_type=TaskType.FACE_DETECTION.value, - ) - - video = Video( - video_id="test-video-id", - file_path="/path/to/video.mp4", - filename="video.mp4", - last_modified=Mock(), - ) - - handler.process_face_detection_task(task, video) - - # Verify sample_rate was passed - call_args = mock_service.detect_faces_in_video.call_args - assert call_args[1]["sample_rate"] == 60 diff --git a/backend/tests/test_global_jump_router.py b/backend/tests/test_global_jump_router.py new file mode 100644 index 0000000..cc020ca --- /dev/null +++ b/backend/tests/test_global_jump_router.py @@ -0,0 +1,150 @@ +"""Tests for Global Jump Router error handling and response formatting. + +Tests verify: +- Error responses include detail, error_code, and timestamp (Requirements 8.1-8.5) +- Proper HTTP status codes (200, 400, 404, 500) +- Empty results return 200 with empty array and has_more=false + +Note: Integration tests for the /jump/global endpoint require the router to be +registered in main_api.py (Task 25). These tests focus on the error response +formatting functions. +""" + +import json +from datetime import datetime + +from fastapi import status + +from src.api.global_jump_controller import ERROR_CODES, create_error_response + + +class TestErrorResponseFormat: + """Tests for error response format with detail, error_code, and timestamp.""" + + def test_create_error_response_includes_all_fields(self): + """Test create_error_response includes detail, error_code, timestamp.""" + response = create_error_response( + status_code=400, + detail="Test error message", + error_code="TEST_ERROR", + ) + + assert response.status_code == 400 + body = json.loads(response.body.decode()) + assert "detail" in body + assert "error_code" in body + assert "timestamp" in body + assert body["detail"] == "Test error message" + assert body["error_code"] == "TEST_ERROR" + + def test_create_error_response_400_status(self): + """Test 400 status code for validation errors.""" + response = create_error_response( + status_code=status.HTTP_400_BAD_REQUEST, + detail="Invalid parameter", + error_code="INVALID_KIND", + ) + assert response.status_code == 400 + + def test_create_error_response_404_status(self): + """Test 404 status code for not found errors.""" + response = create_error_response( + status_code=status.HTTP_404_NOT_FOUND, + detail="Video not found", + error_code="VIDEO_NOT_FOUND", + ) + assert response.status_code == 404 + + def test_create_error_response_500_status(self): + """Test 500 status code for internal errors.""" + response = create_error_response( + status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, + detail="An unexpected error occurred", + error_code="INTERNAL_ERROR", + ) + assert response.status_code == 500 + + def test_error_response_timestamp_is_valid_iso_format(self): + """Test that timestamp is a valid ISO format datetime.""" + response = create_error_response( + status_code=400, + detail="Test error", + error_code="TEST_ERROR", + ) + body = json.loads(response.body.decode()) + # Should be parseable as datetime + timestamp = body["timestamp"] + assert isinstance(timestamp, str) + # Parse the ISO format timestamp + datetime.fromisoformat(timestamp.replace("Z", "+00:00")) + + +class TestErrorCodes: + """Tests for error code constants.""" + + def test_all_error_codes_defined(self): + """Test that all expected error codes are defined.""" + expected_codes = [ + "INVALID_VIDEO_ID", + "INVALID_KIND", + "INVALID_DIRECTION", + "CONFLICTING_FILTERS", + "INVALID_FROM_MS", + "INVALID_CONFIDENCE", + "INVALID_LIMIT", + "VIDEO_NOT_FOUND", + "INTERNAL_ERROR", + ] + for code in expected_codes: + assert code in ERROR_CODES + assert ERROR_CODES[code] == code + + def test_error_codes_match_design_document(self): + """Test that error codes match the design document specification. + + Design document specifies: + - INVALID_KIND for invalid artifact kind + - INVALID_DIRECTION for invalid direction + - CONFLICTING_FILTERS for both label and query specified + - VIDEO_NOT_FOUND for non-existent video + - INVALID_CONFIDENCE for confidence out of range + - INVALID_LIMIT for limit out of range + """ + assert ERROR_CODES["INVALID_KIND"] == "INVALID_KIND" + assert ERROR_CODES["INVALID_DIRECTION"] == "INVALID_DIRECTION" + assert ERROR_CODES["CONFLICTING_FILTERS"] == "CONFLICTING_FILTERS" + assert ERROR_CODES["VIDEO_NOT_FOUND"] == "VIDEO_NOT_FOUND" + assert ERROR_CODES["INVALID_CONFIDENCE"] == "INVALID_CONFIDENCE" + assert ERROR_CODES["INVALID_LIMIT"] == "INVALID_LIMIT" + + +class TestErrorResponseSchema: + """Tests for ErrorResponseSchema in api/schemas.py.""" + + def test_error_response_schema_has_required_fields(self): + """Test ErrorResponseSchema has detail, error_code, timestamp fields.""" + from src.api.schemas import ErrorResponseSchema + + # Create an instance to verify fields + error = ErrorResponseSchema( + detail="Test error", + error_code="TEST_CODE", + timestamp=datetime.now(), + ) + assert error.detail == "Test error" + assert error.error_code == "TEST_CODE" + assert error.timestamp is not None + + def test_error_response_schema_serialization(self): + """Test that ErrorResponseSchema serializes correctly to JSON.""" + from src.api.schemas import ErrorResponseSchema + + error = ErrorResponseSchema( + detail="Video not found", + error_code="VIDEO_NOT_FOUND", + timestamp=datetime(2025, 5, 19, 2, 22, 21), + ) + json_data = error.model_dump(mode="json") + assert json_data["detail"] == "Video not found" + assert json_data["error_code"] == "VIDEO_NOT_FOUND" + assert "timestamp" in json_data diff --git a/backend/tests/test_global_jump_service.py b/backend/tests/test_global_jump_service.py new file mode 100644 index 0000000..c68c666 --- /dev/null +++ b/backend/tests/test_global_jump_service.py @@ -0,0 +1,4699 @@ +"""Tests for GlobalJumpService.""" + +from datetime import datetime + +import pytest +from sqlalchemy import create_engine, text +from sqlalchemy.orm import sessionmaker + +from src.database.models import Base, ObjectLabel +from src.database.models import Video as VideoEntity +from src.domain.exceptions import InvalidParameterError, VideoNotFoundError +from src.services.global_jump_service import GlobalJumpService + + +@pytest.fixture +def engine(): + """Create in-memory SQLite engine for testing.""" + engine = create_engine("sqlite:///:memory:") + Base.metadata.create_all(engine) + return engine + + +@pytest.fixture +def session(engine): + """Create database session for testing.""" + session_factory = sessionmaker(bind=engine) + session = session_factory() + yield session + session.close() + + +@pytest.fixture +def global_jump_service(session): + """Create GlobalJumpService instance.""" + # artifact_repo is not used by _search_objects_global, so we pass None + return GlobalJumpService(session, artifact_repo=None) + + +def create_test_video( + session, + video_id: str, + filename: str, + file_created_at: datetime | None = None, +) -> VideoEntity: + """Helper to create a test video.""" + video = VideoEntity( + video_id=video_id, + file_path=f"/test/{filename}", + filename=filename, + last_modified=datetime.now(), + file_created_at=file_created_at, + status="completed", + ) + session.add(video) + session.commit() + return video + + +def create_object_label( + session, + artifact_id: str, + asset_id: str, + label: str, + confidence: float, + start_ms: int, + end_ms: int, +) -> ObjectLabel: + """Helper to create an object label in the projection table.""" + obj = ObjectLabel( + artifact_id=artifact_id, + asset_id=asset_id, + label=label, + confidence=confidence, + start_ms=start_ms, + end_ms=end_ms, + ) + session.add(obj) + session.commit() + return obj + + +class TestSearchObjectsGlobalNext: + """Tests for _search_objects_global with direction='next'.""" + + def test_search_objects_next_single_video(self, session, global_jump_service): + """Test searching for next object within the same video.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + create_object_label(session, "obj_1", video.video_id, "dog", 0.9, 0, 100) + create_object_label(session, "obj_2", video.video_id, "dog", 0.85, 500, 600) + create_object_label(session, "obj_3", video.video_id, "dog", 0.95, 1000, 1100) + + results = global_jump_service._search_objects_global( + direction="next", + from_video_id=video.video_id, + from_ms=200, + label="dog", + ) + + assert len(results) == 1 + assert results[0].artifact_id == "obj_2" + assert results[0].jump_to.start_ms == 500 + assert results[0].preview["label"] == "dog" + + def test_search_objects_next_cross_video(self, session, global_jump_service): + """Test searching for next object across multiple videos.""" + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + create_object_label(session, "obj_1", video1.video_id, "cat", 0.9, 0, 100) + create_object_label(session, "obj_2", video2.video_id, "cat", 0.85, 500, 600) + + # Search from end of video1 + results = global_jump_service._search_objects_global( + direction="next", + from_video_id=video1.video_id, + from_ms=5000, + label="cat", + ) + + assert len(results) == 1 + assert results[0].video_id == "video_2" + assert results[0].artifact_id == "obj_2" + + def test_search_objects_next_with_label_filter(self, session, global_jump_service): + """Test that label filter correctly filters results.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + create_object_label(session, "obj_1", video.video_id, "dog", 0.9, 100, 200) + create_object_label(session, "obj_2", video.video_id, "cat", 0.9, 200, 300) + create_object_label(session, "obj_3", video.video_id, "dog", 0.9, 300, 400) + + results = global_jump_service._search_objects_global( + direction="next", + from_video_id=video.video_id, + from_ms=150, + label="dog", + ) + + assert len(results) == 1 + assert results[0].artifact_id == "obj_3" + assert results[0].preview["label"] == "dog" + + def test_search_objects_next_with_confidence_filter( + self, session, global_jump_service + ): + """Test that min_confidence filter correctly filters results.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + create_object_label(session, "obj_1", video.video_id, "dog", 0.5, 100, 200) + create_object_label(session, "obj_2", video.video_id, "dog", 0.7, 200, 300) + create_object_label(session, "obj_3", video.video_id, "dog", 0.9, 300, 400) + + results = global_jump_service._search_objects_global( + direction="next", + from_video_id=video.video_id, + from_ms=0, + label="dog", + min_confidence=0.8, + ) + + assert len(results) == 1 + assert results[0].artifact_id == "obj_3" + assert results[0].preview["confidence"] == 0.9 + + def test_search_objects_next_ordering(self, session, global_jump_service): + """Test that results are ordered by global timeline.""" + # Create videos with different file_created_at + video1 = create_test_video( + session, "video_a", "video_a.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_b", "video_b.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + video3 = create_test_video( + session, "video_c", "video_c.mp4", datetime(2025, 1, 3, 12, 0, 0) + ) + + create_object_label(session, "obj_3", video3.video_id, "dog", 0.9, 0, 100) + create_object_label(session, "obj_1", video1.video_id, "dog", 0.9, 0, 100) + create_object_label(session, "obj_2", video2.video_id, "dog", 0.9, 0, 100) + + # Search from before all videos + results = global_jump_service._search_objects_global( + direction="next", + from_video_id=video1.video_id, + from_ms=500, + label="dog", + limit=3, + ) + + assert len(results) == 2 + # Should be ordered by file_created_at + assert results[0].video_id == "video_b" + assert results[1].video_id == "video_c" + + def test_search_objects_next_limit(self, session, global_jump_service): + """Test that limit parameter restricts results.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + for i in range(5): + create_object_label( + session, f"obj_{i}", video.video_id, "dog", 0.9, i * 100, i * 100 + 50 + ) + + results = global_jump_service._search_objects_global( + direction="next", + from_video_id=video.video_id, + from_ms=0, + label="dog", + limit=2, + ) + + assert len(results) == 2 + + def test_search_objects_next_no_results(self, session, global_jump_service): + """Test that empty list is returned when no matching objects found.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + create_object_label(session, "obj_1", video.video_id, "dog", 0.9, 0, 100) + + results = global_jump_service._search_objects_global( + direction="next", + from_video_id=video.video_id, + from_ms=500, + label="dog", + ) + + assert len(results) == 0 + + def test_search_objects_next_video_not_found(self, session, global_jump_service): + """Test that VideoNotFoundError is raised for non-existent video.""" + with pytest.raises(VideoNotFoundError) as exc_info: + global_jump_service._search_objects_global( + direction="next", + from_video_id="non_existent_video", + from_ms=0, + ) + + assert exc_info.value.video_id == "non_existent_video" + + def test_search_objects_next_null_file_created_at( + self, session, global_jump_service + ): + """Test handling of videos with NULL file_created_at.""" + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, + "video_2", + "video2.mp4", + None, # NULL file_created_at + ) + + create_object_label(session, "obj_1", video1.video_id, "dog", 0.9, 0, 100) + create_object_label(session, "obj_2", video2.video_id, "dog", 0.9, 0, 100) + + # Search from video1 - should find video2 (NULL sorted after) + results = global_jump_service._search_objects_global( + direction="next", + from_video_id=video1.video_id, + from_ms=500, + label="dog", + ) + + assert len(results) == 1 + assert results[0].video_id == "video_2" + + def test_search_objects_next_same_file_created_at_different_video_id( + self, session, global_jump_service + ): + """Test ordering when videos have same file_created_at.""" + same_time = datetime(2025, 1, 1, 12, 0, 0) + video1 = create_test_video(session, "video_a", "video_a.mp4", same_time) + video2 = create_test_video(session, "video_b", "video_b.mp4", same_time) + + create_object_label(session, "obj_1", video1.video_id, "dog", 0.9, 0, 100) + create_object_label(session, "obj_2", video2.video_id, "dog", 0.9, 0, 100) + + # Search from video_a - should find video_b (alphabetically later) + results = global_jump_service._search_objects_global( + direction="next", + from_video_id=video1.video_id, + from_ms=500, + label="dog", + ) + + assert len(results) == 1 + assert results[0].video_id == "video_b" + + def test_search_objects_next_result_contains_all_fields( + self, session, global_jump_service + ): + """Test that results contain all required fields.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + create_object_label(session, "obj_1", video.video_id, "dog", 0.95, 100, 200) + + results = global_jump_service._search_objects_global( + direction="next", + from_video_id=video.video_id, + from_ms=0, + label="dog", + ) + + assert len(results) == 1 + result = results[0] + assert result.video_id == "video_1" + assert result.video_filename == "video1.mp4" + assert result.file_created_at == datetime(2025, 1, 1, 12, 0, 0) + assert result.jump_to.start_ms == 100 + assert result.jump_to.end_ms == 200 + assert result.artifact_id == "obj_1" + assert result.preview == {"label": "dog", "confidence": 0.95} + + +class TestSearchObjectsGlobalPrev: + """Tests for _search_objects_global with direction='prev'.""" + + def test_search_objects_prev_single_video(self, session, global_jump_service): + """Test searching for previous object within the same video.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + create_object_label(session, "obj_1", video.video_id, "dog", 0.9, 0, 100) + create_object_label(session, "obj_2", video.video_id, "dog", 0.85, 500, 600) + create_object_label(session, "obj_3", video.video_id, "dog", 0.95, 1000, 1100) + + results = global_jump_service._search_objects_global( + direction="prev", + from_video_id=video.video_id, + from_ms=800, + label="dog", + ) + + assert len(results) == 1 + assert results[0].artifact_id == "obj_2" + assert results[0].jump_to.start_ms == 500 + assert results[0].preview["label"] == "dog" + + def test_search_objects_prev_cross_video(self, session, global_jump_service): + """Test searching for previous object across multiple videos.""" + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + create_object_label(session, "obj_1", video1.video_id, "cat", 0.9, 500, 600) + create_object_label(session, "obj_2", video2.video_id, "cat", 0.85, 500, 600) + + # Search from beginning of video2 + results = global_jump_service._search_objects_global( + direction="prev", + from_video_id=video2.video_id, + from_ms=0, + label="cat", + ) + + assert len(results) == 1 + assert results[0].video_id == "video_1" + assert results[0].artifact_id == "obj_1" + + def test_search_objects_prev_with_label_filter(self, session, global_jump_service): + """Test that label filter correctly filters results for prev direction.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + create_object_label(session, "obj_1", video.video_id, "dog", 0.9, 100, 200) + create_object_label(session, "obj_2", video.video_id, "cat", 0.9, 200, 300) + create_object_label(session, "obj_3", video.video_id, "dog", 0.9, 300, 400) + + results = global_jump_service._search_objects_global( + direction="prev", + from_video_id=video.video_id, + from_ms=250, + label="dog", + ) + + assert len(results) == 1 + assert results[0].artifact_id == "obj_1" + assert results[0].preview["label"] == "dog" + + def test_search_objects_prev_with_confidence_filter( + self, session, global_jump_service + ): + """Test that min_confidence filter correctly filters results for prev.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + create_object_label(session, "obj_1", video.video_id, "dog", 0.9, 100, 200) + create_object_label(session, "obj_2", video.video_id, "dog", 0.7, 200, 300) + create_object_label(session, "obj_3", video.video_id, "dog", 0.5, 300, 400) + + results = global_jump_service._search_objects_global( + direction="prev", + from_video_id=video.video_id, + from_ms=500, + label="dog", + min_confidence=0.8, + ) + + assert len(results) == 1 + assert results[0].artifact_id == "obj_1" + assert results[0].preview["confidence"] == 0.9 + + def test_search_objects_prev_ordering(self, session, global_jump_service): + """Test that results are ordered by global timeline (descending for prev).""" + # Create videos with different file_created_at + video1 = create_test_video( + session, "video_a", "video_a.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_b", "video_b.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + video3 = create_test_video( + session, "video_c", "video_c.mp4", datetime(2025, 1, 3, 12, 0, 0) + ) + + create_object_label(session, "obj_1", video1.video_id, "dog", 0.9, 0, 100) + create_object_label(session, "obj_2", video2.video_id, "dog", 0.9, 0, 100) + create_object_label(session, "obj_3", video3.video_id, "dog", 0.9, 0, 100) + + # Search from video3 - should find video2 first (descending order) + results = global_jump_service._search_objects_global( + direction="prev", + from_video_id=video3.video_id, + from_ms=0, + label="dog", + limit=3, + ) + + assert len(results) == 2 + # Should be ordered by file_created_at descending + assert results[0].video_id == "video_b" + assert results[1].video_id == "video_a" + + def test_search_objects_prev_limit(self, session, global_jump_service): + """Test that limit parameter restricts results for prev direction.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + for i in range(5): + create_object_label( + session, f"obj_{i}", video.video_id, "dog", 0.9, i * 100, i * 100 + 50 + ) + + results = global_jump_service._search_objects_global( + direction="prev", + from_video_id=video.video_id, + from_ms=500, + label="dog", + limit=2, + ) + + assert len(results) == 2 + + def test_search_objects_prev_no_results(self, session, global_jump_service): + """Test that empty list is returned when no matching objects found.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + create_object_label(session, "obj_1", video.video_id, "dog", 0.9, 500, 600) + + results = global_jump_service._search_objects_global( + direction="prev", + from_video_id=video.video_id, + from_ms=100, + label="dog", + ) + + assert len(results) == 0 + + def test_search_objects_prev_null_file_created_at( + self, session, global_jump_service + ): + """Test handling of videos with NULL file_created_at for prev direction.""" + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, + "video_2", + "video2.mp4", + None, # NULL file_created_at + ) + + create_object_label(session, "obj_1", video1.video_id, "dog", 0.9, 0, 100) + create_object_label(session, "obj_2", video2.video_id, "dog", 0.9, 0, 100) + + # Search from video2 (NULL) - should find video1 (non-NULL comes before) + results = global_jump_service._search_objects_global( + direction="prev", + from_video_id=video2.video_id, + from_ms=0, + label="dog", + ) + + assert len(results) == 1 + assert results[0].video_id == "video_1" + + def test_search_objects_prev_same_file_created_at_different_video_id( + self, session, global_jump_service + ): + """Test ordering when videos have same file_created_at for prev direction.""" + same_time = datetime(2025, 1, 1, 12, 0, 0) + video1 = create_test_video(session, "video_a", "video_a.mp4", same_time) + video2 = create_test_video(session, "video_b", "video_b.mp4", same_time) + + create_object_label(session, "obj_1", video1.video_id, "dog", 0.9, 0, 100) + create_object_label(session, "obj_2", video2.video_id, "dog", 0.9, 0, 100) + + # Search from video_b - should find video_a (alphabetically earlier) + results = global_jump_service._search_objects_global( + direction="prev", + from_video_id=video2.video_id, + from_ms=0, + label="dog", + ) + + assert len(results) == 1 + assert results[0].video_id == "video_a" + + def test_search_objects_prev_result_contains_all_fields( + self, session, global_jump_service + ): + """Test that results contain all required fields for prev direction.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + create_object_label(session, "obj_1", video.video_id, "dog", 0.95, 100, 200) + + results = global_jump_service._search_objects_global( + direction="prev", + from_video_id=video.video_id, + from_ms=500, + label="dog", + ) + + assert len(results) == 1 + result = results[0] + assert result.video_id == "video_1" + assert result.video_filename == "video1.mp4" + assert result.file_created_at == datetime(2025, 1, 1, 12, 0, 0) + assert result.jump_to.start_ms == 100 + assert result.jump_to.end_ms == 200 + assert result.artifact_id == "obj_1" + assert result.preview == {"label": "dog", "confidence": 0.95} + + +class TestSearchTranscriptGlobalNext: + """Tests for _search_transcript_global with direction='next'.""" + + @pytest.fixture + def setup_transcript_fts(self, session): + """Set up transcript_fts table for SQLite testing.""" + # Create FTS5 virtual table for SQLite + session.execute( + text( + """ + CREATE VIRTUAL TABLE IF NOT EXISTS transcript_fts USING fts5( + artifact_id UNINDEXED, + asset_id UNINDEXED, + start_ms UNINDEXED, + end_ms UNINDEXED, + text + ) + """ + ) + ) + # Create metadata table for SQLite + session.execute( + text( + """ + CREATE TABLE IF NOT EXISTS transcript_fts_metadata ( + artifact_id TEXT PRIMARY KEY, + asset_id TEXT NOT NULL, + start_ms INTEGER NOT NULL, + end_ms INTEGER NOT NULL + ) + """ + ) + ) + session.commit() + yield + # Cleanup + session.execute(text("DROP TABLE IF EXISTS transcript_fts_metadata")) + session.execute(text("DROP TABLE IF EXISTS transcript_fts")) + session.commit() + + def _insert_transcript( + self, + session, + artifact_id: str, + asset_id: str, + start_ms: int, + end_ms: int, + text_content: str, + ): + """Helper to insert transcript into FTS tables.""" + session.execute( + text( + """ + INSERT INTO transcript_fts + (artifact_id, asset_id, start_ms, end_ms, text) + VALUES (:artifact_id, :asset_id, :start_ms, :end_ms, :text) + """ + ), + { + "artifact_id": artifact_id, + "asset_id": asset_id, + "start_ms": start_ms, + "end_ms": end_ms, + "text": text_content, + }, + ) + session.execute( + text( + """ + INSERT INTO transcript_fts_metadata + (artifact_id, asset_id, start_ms, end_ms) + VALUES (:artifact_id, :asset_id, :start_ms, :end_ms) + """ + ), + { + "artifact_id": artifact_id, + "asset_id": asset_id, + "start_ms": start_ms, + "end_ms": end_ms, + }, + ) + session.commit() + + def test_search_transcript_next_single_video( + self, session, global_jump_service, setup_transcript_fts + ): + """Test searching for next transcript within the same video.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + self._insert_transcript( + session, "trans_1", video.video_id, 0, 100, "hello world" + ) + self._insert_transcript( + session, "trans_2", video.video_id, 500, 600, "hello again" + ) + self._insert_transcript( + session, "trans_3", video.video_id, 1000, 1100, "goodbye world" + ) + + results = global_jump_service._search_transcript_global( + direction="next", + from_video_id=video.video_id, + from_ms=200, + query="hello", + ) + + assert len(results) == 1 + assert results[0].artifact_id == "trans_2" + assert results[0].jump_to.start_ms == 500 + assert "hello" in results[0].preview["text"].lower() + + def test_search_transcript_next_cross_video( + self, session, global_jump_service, setup_transcript_fts + ): + """Test searching for next transcript across multiple videos.""" + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + self._insert_transcript( + session, "trans_1", video1.video_id, 0, 100, "kubernetes tutorial" + ) + self._insert_transcript( + session, "trans_2", video2.video_id, 500, 600, "kubernetes explained" + ) + + # Search from end of video1 + results = global_jump_service._search_transcript_global( + direction="next", + from_video_id=video1.video_id, + from_ms=5000, + query="kubernetes", + ) + + assert len(results) == 1 + assert results[0].video_id == "video_2" + assert results[0].artifact_id == "trans_2" + + def test_search_transcript_next_ordering( + self, session, global_jump_service, setup_transcript_fts + ): + """Test that results are ordered by global timeline.""" + video1 = create_test_video( + session, "video_a", "video_a.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_b", "video_b.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + video3 = create_test_video( + session, "video_c", "video_c.mp4", datetime(2025, 1, 3, 12, 0, 0) + ) + + self._insert_transcript( + session, "trans_3", video3.video_id, 0, 100, "python programming" + ) + self._insert_transcript( + session, "trans_1", video1.video_id, 0, 100, "python basics" + ) + self._insert_transcript( + session, "trans_2", video2.video_id, 0, 100, "python advanced" + ) + + results = global_jump_service._search_transcript_global( + direction="next", + from_video_id=video1.video_id, + from_ms=500, + query="python", + limit=3, + ) + + assert len(results) == 2 + assert results[0].video_id == "video_b" + assert results[1].video_id == "video_c" + + def test_search_transcript_next_no_results( + self, session, global_jump_service, setup_transcript_fts + ): + """Test that empty list is returned when no matching transcripts found.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + self._insert_transcript( + session, "trans_1", video.video_id, 0, 100, "hello world" + ) + + results = global_jump_service._search_transcript_global( + direction="next", + from_video_id=video.video_id, + from_ms=0, + query="nonexistent", + ) + + assert len(results) == 0 + + def test_search_transcript_next_video_not_found( + self, session, global_jump_service, setup_transcript_fts + ): + """Test that VideoNotFoundError is raised for non-existent video.""" + with pytest.raises(VideoNotFoundError) as exc_info: + global_jump_service._search_transcript_global( + direction="next", + from_video_id="non_existent_video", + from_ms=0, + query="test", + ) + + assert exc_info.value.video_id == "non_existent_video" + + def test_search_transcript_next_result_contains_all_fields( + self, session, global_jump_service, setup_transcript_fts + ): + """Test that results contain all required fields.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + self._insert_transcript( + session, "trans_1", video.video_id, 100, 200, "test content here" + ) + + results = global_jump_service._search_transcript_global( + direction="next", + from_video_id=video.video_id, + from_ms=0, + query="test", + ) + + assert len(results) == 1 + result = results[0] + assert result.video_id == "video_1" + assert result.video_filename == "video1.mp4" + # SQLite returns datetime as string, PostgreSQL returns datetime object + # Check that file_created_at is present and contains expected date + assert result.file_created_at is not None + if isinstance(result.file_created_at, str): + assert "2025-01-01" in result.file_created_at + else: + assert result.file_created_at == datetime(2025, 1, 1, 12, 0, 0) + assert result.jump_to.start_ms == 100 + assert result.jump_to.end_ms == 200 + assert result.artifact_id == "trans_1" + assert "text" in result.preview + + +class TestSearchTranscriptGlobalPrev: + """Tests for _search_transcript_global with direction='prev'.""" + + @pytest.fixture + def setup_transcript_fts(self, session): + """Set up transcript_fts table for SQLite testing.""" + session.execute( + text( + """ + CREATE VIRTUAL TABLE IF NOT EXISTS transcript_fts USING fts5( + artifact_id UNINDEXED, + asset_id UNINDEXED, + start_ms UNINDEXED, + end_ms UNINDEXED, + text + ) + """ + ) + ) + session.execute( + text( + """ + CREATE TABLE IF NOT EXISTS transcript_fts_metadata ( + artifact_id TEXT PRIMARY KEY, + asset_id TEXT NOT NULL, + start_ms INTEGER NOT NULL, + end_ms INTEGER NOT NULL + ) + """ + ) + ) + session.commit() + yield + session.execute(text("DROP TABLE IF EXISTS transcript_fts_metadata")) + session.execute(text("DROP TABLE IF EXISTS transcript_fts")) + session.commit() + + def _insert_transcript( + self, + session, + artifact_id: str, + asset_id: str, + start_ms: int, + end_ms: int, + text_content: str, + ): + """Helper to insert transcript into FTS tables.""" + session.execute( + text( + """ + INSERT INTO transcript_fts + (artifact_id, asset_id, start_ms, end_ms, text) + VALUES (:artifact_id, :asset_id, :start_ms, :end_ms, :text) + """ + ), + { + "artifact_id": artifact_id, + "asset_id": asset_id, + "start_ms": start_ms, + "end_ms": end_ms, + "text": text_content, + }, + ) + session.execute( + text( + """ + INSERT INTO transcript_fts_metadata + (artifact_id, asset_id, start_ms, end_ms) + VALUES (:artifact_id, :asset_id, :start_ms, :end_ms) + """ + ), + { + "artifact_id": artifact_id, + "asset_id": asset_id, + "start_ms": start_ms, + "end_ms": end_ms, + }, + ) + session.commit() + + def test_search_transcript_prev_single_video( + self, session, global_jump_service, setup_transcript_fts + ): + """Test searching for previous transcript within the same video.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + self._insert_transcript( + session, "trans_1", video.video_id, 0, 100, "hello world" + ) + self._insert_transcript( + session, "trans_2", video.video_id, 500, 600, "hello again" + ) + self._insert_transcript( + session, "trans_3", video.video_id, 1000, 1100, "goodbye world" + ) + + results = global_jump_service._search_transcript_global( + direction="prev", + from_video_id=video.video_id, + from_ms=800, + query="hello", + ) + + assert len(results) == 1 + assert results[0].artifact_id == "trans_2" + assert results[0].jump_to.start_ms == 500 + + def test_search_transcript_prev_cross_video( + self, session, global_jump_service, setup_transcript_fts + ): + """Test searching for previous transcript across multiple videos.""" + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + self._insert_transcript( + session, "trans_1", video1.video_id, 500, 600, "docker container" + ) + self._insert_transcript( + session, "trans_2", video2.video_id, 500, 600, "docker image" + ) + + # Search from beginning of video2 + results = global_jump_service._search_transcript_global( + direction="prev", + from_video_id=video2.video_id, + from_ms=0, + query="docker", + ) + + assert len(results) == 1 + assert results[0].video_id == "video_1" + assert results[0].artifact_id == "trans_1" + + def test_search_transcript_prev_ordering( + self, session, global_jump_service, setup_transcript_fts + ): + """Test that results are ordered by global timeline (descending).""" + video1 = create_test_video( + session, "video_a", "video_a.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_b", "video_b.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + video3 = create_test_video( + session, "video_c", "video_c.mp4", datetime(2025, 1, 3, 12, 0, 0) + ) + + self._insert_transcript( + session, "trans_1", video1.video_id, 0, 100, "react tutorial" + ) + self._insert_transcript( + session, "trans_2", video2.video_id, 0, 100, "react hooks" + ) + self._insert_transcript( + session, "trans_3", video3.video_id, 0, 100, "react components" + ) + + results = global_jump_service._search_transcript_global( + direction="prev", + from_video_id=video3.video_id, + from_ms=0, + query="react", + limit=3, + ) + + assert len(results) == 2 + # Should be ordered by file_created_at descending + assert results[0].video_id == "video_b" + assert results[1].video_id == "video_a" + + def test_search_transcript_prev_no_results( + self, session, global_jump_service, setup_transcript_fts + ): + """Test that empty list is returned when no matching transcripts found.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + self._insert_transcript( + session, "trans_1", video.video_id, 500, 600, "hello world" + ) + + results = global_jump_service._search_transcript_global( + direction="prev", + from_video_id=video.video_id, + from_ms=100, + query="hello", + ) + + assert len(results) == 0 + + +class TestSearchOcrGlobalNext: + """Tests for _search_ocr_global with direction='next'.""" + + @pytest.fixture + def setup_ocr_fts(self, session): + """Set up ocr_fts table for SQLite testing.""" + # Create FTS5 virtual table for SQLite + session.execute( + text( + """ + CREATE VIRTUAL TABLE IF NOT EXISTS ocr_fts USING fts5( + artifact_id UNINDEXED, + asset_id UNINDEXED, + start_ms UNINDEXED, + end_ms UNINDEXED, + text + ) + """ + ) + ) + # Create metadata table for SQLite + session.execute( + text( + """ + CREATE TABLE IF NOT EXISTS ocr_fts_metadata ( + artifact_id TEXT PRIMARY KEY, + asset_id TEXT NOT NULL, + start_ms INTEGER NOT NULL, + end_ms INTEGER NOT NULL + ) + """ + ) + ) + session.commit() + yield + # Cleanup + session.execute(text("DROP TABLE IF EXISTS ocr_fts_metadata")) + session.execute(text("DROP TABLE IF EXISTS ocr_fts")) + session.commit() + + def _insert_ocr( + self, + session, + artifact_id: str, + asset_id: str, + start_ms: int, + end_ms: int, + text_content: str, + ): + """Helper to insert OCR text into FTS tables.""" + session.execute( + text( + """ + INSERT INTO ocr_fts + (artifact_id, asset_id, start_ms, end_ms, text) + VALUES (:artifact_id, :asset_id, :start_ms, :end_ms, :text) + """ + ), + { + "artifact_id": artifact_id, + "asset_id": asset_id, + "start_ms": start_ms, + "end_ms": end_ms, + "text": text_content, + }, + ) + session.execute( + text( + """ + INSERT INTO ocr_fts_metadata + (artifact_id, asset_id, start_ms, end_ms) + VALUES (:artifact_id, :asset_id, :start_ms, :end_ms) + """ + ), + { + "artifact_id": artifact_id, + "asset_id": asset_id, + "start_ms": start_ms, + "end_ms": end_ms, + }, + ) + session.commit() + + def test_search_ocr_next_single_video( + self, session, global_jump_service, setup_ocr_fts + ): + """Test searching for next OCR text within the same video.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + self._insert_ocr(session, "ocr_1", video.video_id, 0, 100, "Welcome Screen") + self._insert_ocr(session, "ocr_2", video.video_id, 500, 600, "Welcome Back") + self._insert_ocr(session, "ocr_3", video.video_id, 1000, 1100, "Goodbye Screen") + + results = global_jump_service._search_ocr_global( + direction="next", + from_video_id=video.video_id, + from_ms=200, + query="Welcome", + ) + + assert len(results) == 1 + assert results[0].artifact_id == "ocr_2" + assert results[0].jump_to.start_ms == 500 + assert "Welcome" in results[0].preview["text"] + + def test_search_ocr_next_cross_video( + self, session, global_jump_service, setup_ocr_fts + ): + """Test searching for next OCR text across multiple videos.""" + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + self._insert_ocr(session, "ocr_1", video1.video_id, 0, 100, "Error Message") + self._insert_ocr(session, "ocr_2", video2.video_id, 500, 600, "Error Code 404") + + # Search from end of video1 + results = global_jump_service._search_ocr_global( + direction="next", + from_video_id=video1.video_id, + from_ms=5000, + query="Error", + ) + + assert len(results) == 1 + assert results[0].video_id == "video_2" + assert results[0].artifact_id == "ocr_2" + + def test_search_ocr_next_ordering( + self, session, global_jump_service, setup_ocr_fts + ): + """Test that results are ordered by global timeline.""" + video1 = create_test_video( + session, "video_a", "video_a.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_b", "video_b.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + video3 = create_test_video( + session, "video_c", "video_c.mp4", datetime(2025, 1, 3, 12, 0, 0) + ) + + self._insert_ocr(session, "ocr_3", video3.video_id, 0, 100, "Login Button") + self._insert_ocr(session, "ocr_1", video1.video_id, 0, 100, "Login Form") + self._insert_ocr(session, "ocr_2", video2.video_id, 0, 100, "Login Page") + + results = global_jump_service._search_ocr_global( + direction="next", + from_video_id=video1.video_id, + from_ms=500, + query="Login", + limit=3, + ) + + assert len(results) == 2 + assert results[0].video_id == "video_b" + assert results[1].video_id == "video_c" + + def test_search_ocr_next_no_results( + self, session, global_jump_service, setup_ocr_fts + ): + """Test that empty list is returned when no matching OCR text found.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + self._insert_ocr(session, "ocr_1", video.video_id, 0, 100, "Hello World") + + results = global_jump_service._search_ocr_global( + direction="next", + from_video_id=video.video_id, + from_ms=0, + query="nonexistent", + ) + + assert len(results) == 0 + + def test_search_ocr_next_video_not_found( + self, session, global_jump_service, setup_ocr_fts + ): + """Test that VideoNotFoundError is raised for non-existent video.""" + with pytest.raises(VideoNotFoundError) as exc_info: + global_jump_service._search_ocr_global( + direction="next", + from_video_id="non_existent_video", + from_ms=0, + query="test", + ) + + assert exc_info.value.video_id == "non_existent_video" + + def test_search_ocr_next_result_contains_all_fields( + self, session, global_jump_service, setup_ocr_fts + ): + """Test that results contain all required fields.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + self._insert_ocr(session, "ocr_1", video.video_id, 100, 200, "Test Label") + + results = global_jump_service._search_ocr_global( + direction="next", + from_video_id=video.video_id, + from_ms=0, + query="Test", + ) + + assert len(results) == 1 + result = results[0] + assert result.video_id == "video_1" + assert result.video_filename == "video1.mp4" + # SQLite returns datetime as string, PostgreSQL returns datetime object + assert result.file_created_at is not None + if isinstance(result.file_created_at, str): + assert "2025-01-01" in result.file_created_at + else: + assert result.file_created_at == datetime(2025, 1, 1, 12, 0, 0) + assert result.jump_to.start_ms == 100 + assert result.jump_to.end_ms == 200 + assert result.artifact_id == "ocr_1" + assert "text" in result.preview + + +class TestSearchOcrGlobalPrev: + """Tests for _search_ocr_global with direction='prev'.""" + + @pytest.fixture + def setup_ocr_fts(self, session): + """Set up ocr_fts table for SQLite testing.""" + session.execute( + text( + """ + CREATE VIRTUAL TABLE IF NOT EXISTS ocr_fts USING fts5( + artifact_id UNINDEXED, + asset_id UNINDEXED, + start_ms UNINDEXED, + end_ms UNINDEXED, + text + ) + """ + ) + ) + session.execute( + text( + """ + CREATE TABLE IF NOT EXISTS ocr_fts_metadata ( + artifact_id TEXT PRIMARY KEY, + asset_id TEXT NOT NULL, + start_ms INTEGER NOT NULL, + end_ms INTEGER NOT NULL + ) + """ + ) + ) + session.commit() + yield + session.execute(text("DROP TABLE IF EXISTS ocr_fts_metadata")) + session.execute(text("DROP TABLE IF EXISTS ocr_fts")) + session.commit() + + def _insert_ocr( + self, + session, + artifact_id: str, + asset_id: str, + start_ms: int, + end_ms: int, + text_content: str, + ): + """Helper to insert OCR text into FTS tables.""" + session.execute( + text( + """ + INSERT INTO ocr_fts + (artifact_id, asset_id, start_ms, end_ms, text) + VALUES (:artifact_id, :asset_id, :start_ms, :end_ms, :text) + """ + ), + { + "artifact_id": artifact_id, + "asset_id": asset_id, + "start_ms": start_ms, + "end_ms": end_ms, + "text": text_content, + }, + ) + session.execute( + text( + """ + INSERT INTO ocr_fts_metadata + (artifact_id, asset_id, start_ms, end_ms) + VALUES (:artifact_id, :asset_id, :start_ms, :end_ms) + """ + ), + { + "artifact_id": artifact_id, + "asset_id": asset_id, + "start_ms": start_ms, + "end_ms": end_ms, + }, + ) + session.commit() + + def test_search_ocr_prev_single_video( + self, session, global_jump_service, setup_ocr_fts + ): + """Test searching for previous OCR text within the same video.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + self._insert_ocr(session, "ocr_1", video.video_id, 0, 100, "Submit Button") + self._insert_ocr(session, "ocr_2", video.video_id, 500, 600, "Submit Form") + self._insert_ocr(session, "ocr_3", video.video_id, 1000, 1100, "Cancel Button") + + results = global_jump_service._search_ocr_global( + direction="prev", + from_video_id=video.video_id, + from_ms=800, + query="Submit", + ) + + assert len(results) == 1 + assert results[0].artifact_id == "ocr_2" + assert results[0].jump_to.start_ms == 500 + + def test_search_ocr_prev_cross_video( + self, session, global_jump_service, setup_ocr_fts + ): + """Test searching for previous OCR text across multiple videos.""" + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + self._insert_ocr(session, "ocr_1", video1.video_id, 500, 600, "Settings Menu") + self._insert_ocr(session, "ocr_2", video2.video_id, 500, 600, "Settings Page") + + # Search from beginning of video2 + results = global_jump_service._search_ocr_global( + direction="prev", + from_video_id=video2.video_id, + from_ms=0, + query="Settings", + ) + + assert len(results) == 1 + assert results[0].video_id == "video_1" + assert results[0].artifact_id == "ocr_1" + + def test_search_ocr_prev_ordering( + self, session, global_jump_service, setup_ocr_fts + ): + """Test that results are ordered by global timeline (descending).""" + video1 = create_test_video( + session, "video_a", "video_a.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_b", "video_b.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + video3 = create_test_video( + session, "video_c", "video_c.mp4", datetime(2025, 1, 3, 12, 0, 0) + ) + + self._insert_ocr(session, "ocr_1", video1.video_id, 0, 100, "Dashboard View") + self._insert_ocr(session, "ocr_2", video2.video_id, 0, 100, "Dashboard Stats") + self._insert_ocr(session, "ocr_3", video3.video_id, 0, 100, "Dashboard Home") + + results = global_jump_service._search_ocr_global( + direction="prev", + from_video_id=video3.video_id, + from_ms=0, + query="Dashboard", + limit=3, + ) + + assert len(results) == 2 + # Should be ordered by file_created_at descending + assert results[0].video_id == "video_b" + assert results[1].video_id == "video_a" + + def test_search_ocr_prev_no_results( + self, session, global_jump_service, setup_ocr_fts + ): + """Test that empty list is returned when no matching OCR text found.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + self._insert_ocr(session, "ocr_1", video.video_id, 500, 600, "Hello World") + + results = global_jump_service._search_ocr_global( + direction="prev", + from_video_id=video.video_id, + from_ms=100, + query="Hello", + ) + + assert len(results) == 0 + + +class TestJumpNext: + """Tests for jump_next() public method.""" + + def test_jump_next_object_routes_correctly(self, session, global_jump_service): + """Test that kind='object' routes to object search.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + create_object_label(session, "obj_1", video.video_id, "dog", 0.9, 100, 200) + + results = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + from_ms=0, + label="dog", + ) + + assert len(results) == 1 + assert results[0].artifact_id == "obj_1" + assert results[0].preview["label"] == "dog" + + def test_jump_next_invalid_kind_raises_error(self, session, global_jump_service): + """Test that invalid kind raises InvalidParameterError.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + with pytest.raises(InvalidParameterError) as exc_info: + global_jump_service.jump_next( + kind="invalid_kind", + from_video_id=video.video_id, + from_ms=0, + ) + + assert exc_info.value.parameter == "kind" + assert "Invalid artifact kind" in exc_info.value.message + + def test_jump_next_transcript_requires_query(self, session, global_jump_service): + """Test that transcript search requires query parameter.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + with pytest.raises(InvalidParameterError) as exc_info: + global_jump_service.jump_next( + kind="transcript", + from_video_id=video.video_id, + from_ms=0, + ) + + assert exc_info.value.parameter == "query" + + def test_jump_next_ocr_requires_query(self, session, global_jump_service): + """Test that OCR search requires query parameter.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + with pytest.raises(InvalidParameterError) as exc_info: + global_jump_service.jump_next( + kind="ocr", + from_video_id=video.video_id, + from_ms=0, + ) + + assert exc_info.value.parameter == "query" + + def test_jump_next_default_from_ms(self, session, global_jump_service): + """Test that from_ms defaults to 0 when not provided.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + create_object_label(session, "obj_1", video.video_id, "cat", 0.9, 100, 200) + + # Call without from_ms - should default to 0 and find the object + results = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + label="cat", + ) + + assert len(results) == 1 + assert results[0].artifact_id == "obj_1" + + def test_jump_next_video_not_found(self, session, global_jump_service): + """Test that VideoNotFoundError is raised for non-existent video.""" + with pytest.raises(VideoNotFoundError) as exc_info: + global_jump_service.jump_next( + kind="object", + from_video_id="non_existent_video", + from_ms=0, + ) + + assert exc_info.value.video_id == "non_existent_video" + + def test_jump_next_with_limit(self, session, global_jump_service): + """Test that limit parameter is respected.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + for i in range(5): + create_object_label( + session, f"obj_{i}", video.video_id, "bird", 0.9, i * 100, i * 100 + 50 + ) + + results = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + from_ms=0, + label="bird", + limit=3, + ) + + assert len(results) == 3 + + def test_jump_next_with_min_confidence(self, session, global_jump_service): + """Test that min_confidence filter is applied.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + create_object_label(session, "obj_1", video.video_id, "car", 0.5, 100, 200) + create_object_label(session, "obj_2", video.video_id, "car", 0.9, 200, 300) + + results = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + from_ms=0, + label="car", + min_confidence=0.8, + ) + + assert len(results) == 1 + assert results[0].artifact_id == "obj_2" + + +class TestJumpPrev: + """Tests for jump_prev() public method.""" + + def test_jump_prev_object_routes_correctly(self, session, global_jump_service): + """Test that kind='object' routes to object search with prev direction.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + create_object_label(session, "obj_1", video.video_id, "dog", 0.9, 100, 200) + create_object_label(session, "obj_2", video.video_id, "dog", 0.9, 500, 600) + + results = global_jump_service.jump_prev( + kind="object", + from_video_id=video.video_id, + from_ms=400, + label="dog", + ) + + assert len(results) == 1 + assert results[0].artifact_id == "obj_1" + assert results[0].preview["label"] == "dog" + + def test_jump_prev_invalid_kind_raises_error(self, session, global_jump_service): + """Test that invalid kind raises InvalidParameterError.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + with pytest.raises(InvalidParameterError) as exc_info: + global_jump_service.jump_prev( + kind="invalid_kind", + from_video_id=video.video_id, + from_ms=0, + ) + + assert exc_info.value.parameter == "kind" + assert "Invalid artifact kind" in exc_info.value.message + + def test_jump_prev_transcript_requires_query(self, session, global_jump_service): + """Test that transcript search requires query parameter.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + with pytest.raises(InvalidParameterError) as exc_info: + global_jump_service.jump_prev( + kind="transcript", + from_video_id=video.video_id, + from_ms=0, + ) + + assert exc_info.value.parameter == "query" + + def test_jump_prev_ocr_requires_query(self, session, global_jump_service): + """Test that OCR search requires query parameter.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + with pytest.raises(InvalidParameterError) as exc_info: + global_jump_service.jump_prev( + kind="ocr", + from_video_id=video.video_id, + from_ms=0, + ) + + assert exc_info.value.parameter == "query" + + def test_jump_prev_default_from_ms(self, session, global_jump_service): + """Test that from_ms defaults to max value when not provided.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + create_object_label(session, "obj_1", video.video_id, "cat", 0.9, 100, 200) + create_object_label(session, "obj_2", video.video_id, "cat", 0.9, 500, 600) + + # Call without from_ms - should default to max and find the last object + results = global_jump_service.jump_prev( + kind="object", + from_video_id=video.video_id, + label="cat", + ) + + assert len(results) == 1 + # Should find the last object (obj_2) since we're searching from the end + assert results[0].artifact_id == "obj_2" + + def test_jump_prev_video_not_found(self, session, global_jump_service): + """Test that VideoNotFoundError is raised for non-existent video.""" + with pytest.raises(VideoNotFoundError) as exc_info: + global_jump_service.jump_prev( + kind="object", + from_video_id="non_existent_video", + from_ms=0, + ) + + assert exc_info.value.video_id == "non_existent_video" + + def test_jump_prev_with_limit(self, session, global_jump_service): + """Test that limit parameter is respected.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + for i in range(5): + create_object_label( + session, f"obj_{i}", video.video_id, "bird", 0.9, i * 100, i * 100 + 50 + ) + + results = global_jump_service.jump_prev( + kind="object", + from_video_id=video.video_id, + from_ms=1000, + label="bird", + limit=3, + ) + + assert len(results) == 3 + + def test_jump_prev_with_min_confidence(self, session, global_jump_service): + """Test that min_confidence filter is applied.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + create_object_label(session, "obj_1", video.video_id, "car", 0.9, 100, 200) + create_object_label(session, "obj_2", video.video_id, "car", 0.5, 200, 300) + + results = global_jump_service.jump_prev( + kind="object", + from_video_id=video.video_id, + from_ms=500, + label="car", + min_confidence=0.8, + ) + + assert len(results) == 1 + assert results[0].artifact_id == "obj_1" + + +class TestSearchScenesGlobalNext: + """Tests for _search_scenes_global with direction='next'.""" + + @pytest.fixture + def setup_scene_ranges(self, session): + """Set up scene_ranges table for testing.""" + session.execute( + text( + """ + CREATE TABLE IF NOT EXISTS scene_ranges ( + artifact_id TEXT PRIMARY KEY, + asset_id TEXT NOT NULL, + scene_index INTEGER NOT NULL, + start_ms INTEGER NOT NULL, + end_ms INTEGER NOT NULL + ) + """ + ) + ) + session.commit() + yield + session.execute(text("DROP TABLE IF EXISTS scene_ranges")) + session.commit() + + def _insert_scene( + self, + session, + artifact_id: str, + asset_id: str, + scene_index: int, + start_ms: int, + end_ms: int, + ): + """Helper to insert scene into scene_ranges table.""" + session.execute( + text( + """ + INSERT INTO scene_ranges + (artifact_id, asset_id, scene_index, start_ms, end_ms) + VALUES (:artifact_id, :asset_id, :scene_index, :start_ms, :end_ms) + """ + ), + { + "artifact_id": artifact_id, + "asset_id": asset_id, + "scene_index": scene_index, + "start_ms": start_ms, + "end_ms": end_ms, + }, + ) + session.commit() + + def test_search_scenes_next_single_video( + self, session, global_jump_service, setup_scene_ranges + ): + """Test searching for next scene within the same video.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + self._insert_scene(session, "scene_1", video.video_id, 0, 0, 5000) + self._insert_scene(session, "scene_2", video.video_id, 1, 5000, 10000) + self._insert_scene(session, "scene_3", video.video_id, 2, 10000, 15000) + + results = global_jump_service._search_scenes_global( + direction="next", + from_video_id=video.video_id, + from_ms=3000, + ) + + assert len(results) == 1 + assert results[0].artifact_id == "scene_2" + assert results[0].jump_to.start_ms == 5000 + assert results[0].preview["scene_index"] == 1 + + def test_search_scenes_next_cross_video( + self, session, global_jump_service, setup_scene_ranges + ): + """Test searching for next scene across multiple videos.""" + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + self._insert_scene(session, "scene_1", video1.video_id, 0, 0, 5000) + self._insert_scene(session, "scene_2", video2.video_id, 0, 0, 5000) + + # Search from end of video1 + results = global_jump_service._search_scenes_global( + direction="next", + from_video_id=video1.video_id, + from_ms=10000, + ) + + assert len(results) == 1 + assert results[0].video_id == "video_2" + assert results[0].artifact_id == "scene_2" + + def test_search_scenes_next_ordering( + self, session, global_jump_service, setup_scene_ranges + ): + """Test that results are ordered by global timeline.""" + video1 = create_test_video( + session, "video_a", "video_a.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_b", "video_b.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + video3 = create_test_video( + session, "video_c", "video_c.mp4", datetime(2025, 1, 3, 12, 0, 0) + ) + + self._insert_scene(session, "scene_3", video3.video_id, 0, 0, 5000) + self._insert_scene(session, "scene_1", video1.video_id, 0, 0, 5000) + self._insert_scene(session, "scene_2", video2.video_id, 0, 0, 5000) + + results = global_jump_service._search_scenes_global( + direction="next", + from_video_id=video1.video_id, + from_ms=10000, + limit=3, + ) + + assert len(results) == 2 + assert results[0].video_id == "video_b" + assert results[1].video_id == "video_c" + + def test_search_scenes_next_no_results( + self, session, global_jump_service, setup_scene_ranges + ): + """Test that empty list is returned when no scenes found.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + self._insert_scene(session, "scene_1", video.video_id, 0, 0, 5000) + + results = global_jump_service._search_scenes_global( + direction="next", + from_video_id=video.video_id, + from_ms=10000, + ) + + assert len(results) == 0 + + def test_search_scenes_next_video_not_found( + self, session, global_jump_service, setup_scene_ranges + ): + """Test that VideoNotFoundError is raised for non-existent video.""" + with pytest.raises(VideoNotFoundError) as exc_info: + global_jump_service._search_scenes_global( + direction="next", + from_video_id="non_existent_video", + from_ms=0, + ) + + assert exc_info.value.video_id == "non_existent_video" + + +class TestSearchScenesGlobalPrev: + """Tests for _search_scenes_global with direction='prev'.""" + + @pytest.fixture + def setup_scene_ranges(self, session): + """Set up scene_ranges table for testing.""" + session.execute( + text( + """ + CREATE TABLE IF NOT EXISTS scene_ranges ( + artifact_id TEXT PRIMARY KEY, + asset_id TEXT NOT NULL, + scene_index INTEGER NOT NULL, + start_ms INTEGER NOT NULL, + end_ms INTEGER NOT NULL + ) + """ + ) + ) + session.commit() + yield + session.execute(text("DROP TABLE IF EXISTS scene_ranges")) + session.commit() + + def _insert_scene( + self, + session, + artifact_id: str, + asset_id: str, + scene_index: int, + start_ms: int, + end_ms: int, + ): + """Helper to insert scene into scene_ranges table.""" + session.execute( + text( + """ + INSERT INTO scene_ranges + (artifact_id, asset_id, scene_index, start_ms, end_ms) + VALUES (:artifact_id, :asset_id, :scene_index, :start_ms, :end_ms) + """ + ), + { + "artifact_id": artifact_id, + "asset_id": asset_id, + "scene_index": scene_index, + "start_ms": start_ms, + "end_ms": end_ms, + }, + ) + session.commit() + + def test_search_scenes_prev_single_video( + self, session, global_jump_service, setup_scene_ranges + ): + """Test searching for previous scene within the same video.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + self._insert_scene(session, "scene_1", video.video_id, 0, 0, 5000) + self._insert_scene(session, "scene_2", video.video_id, 1, 5000, 10000) + self._insert_scene(session, "scene_3", video.video_id, 2, 10000, 15000) + + results = global_jump_service._search_scenes_global( + direction="prev", + from_video_id=video.video_id, + from_ms=8000, + ) + + assert len(results) == 1 + assert results[0].artifact_id == "scene_2" + assert results[0].jump_to.start_ms == 5000 + + def test_search_scenes_prev_cross_video( + self, session, global_jump_service, setup_scene_ranges + ): + """Test searching for previous scene across multiple videos.""" + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + self._insert_scene(session, "scene_1", video1.video_id, 0, 5000, 10000) + self._insert_scene(session, "scene_2", video2.video_id, 0, 5000, 10000) + + # Search from beginning of video2 + results = global_jump_service._search_scenes_global( + direction="prev", + from_video_id=video2.video_id, + from_ms=0, + ) + + assert len(results) == 1 + assert results[0].video_id == "video_1" + assert results[0].artifact_id == "scene_1" + + def test_search_scenes_prev_ordering( + self, session, global_jump_service, setup_scene_ranges + ): + """Test that results are ordered by global timeline (descending).""" + video1 = create_test_video( + session, "video_a", "video_a.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_b", "video_b.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + video3 = create_test_video( + session, "video_c", "video_c.mp4", datetime(2025, 1, 3, 12, 0, 0) + ) + + self._insert_scene(session, "scene_1", video1.video_id, 0, 0, 5000) + self._insert_scene(session, "scene_2", video2.video_id, 0, 0, 5000) + self._insert_scene(session, "scene_3", video3.video_id, 0, 0, 5000) + + results = global_jump_service._search_scenes_global( + direction="prev", + from_video_id=video3.video_id, + from_ms=0, + limit=3, + ) + + assert len(results) == 2 + # Should be ordered by file_created_at descending + assert results[0].video_id == "video_b" + assert results[1].video_id == "video_a" + + def test_search_scenes_prev_no_results( + self, session, global_jump_service, setup_scene_ranges + ): + """Test that empty list is returned when no scenes found.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + self._insert_scene(session, "scene_1", video.video_id, 0, 5000, 10000) + + results = global_jump_service._search_scenes_global( + direction="prev", + from_video_id=video.video_id, + from_ms=1000, + ) + + assert len(results) == 0 + + +class TestSearchPlacesGlobal: + """Tests for _search_places_global method.""" + + def test_search_places_next_routes_to_objects(self, session, global_jump_service): + """Test that place search uses object_labels table.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + # Places are stored as object labels + create_object_label( + session, "place_1", video.video_id, "kitchen", 0.9, 100, 200 + ) + create_object_label(session, "place_2", video.video_id, "beach", 0.8, 500, 600) + + results = global_jump_service._search_places_global( + direction="next", + from_video_id=video.video_id, + from_ms=0, + label="kitchen", + ) + + assert len(results) == 1 + assert results[0].artifact_id == "place_1" + assert results[0].preview["label"] == "kitchen" + + def test_search_places_prev_routes_to_objects(self, session, global_jump_service): + """Test that place search prev direction works.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + create_object_label(session, "place_1", video.video_id, "office", 0.9, 100, 200) + create_object_label(session, "place_2", video.video_id, "office", 0.8, 500, 600) + + results = global_jump_service._search_places_global( + direction="prev", + from_video_id=video.video_id, + from_ms=400, + label="office", + ) + + assert len(results) == 1 + assert results[0].artifact_id == "place_1" + + def test_search_places_with_confidence_filter(self, session, global_jump_service): + """Test that confidence filter is applied to place search.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + create_object_label(session, "place_1", video.video_id, "park", 0.5, 100, 200) + create_object_label(session, "place_2", video.video_id, "park", 0.9, 500, 600) + + results = global_jump_service._search_places_global( + direction="next", + from_video_id=video.video_id, + from_ms=0, + label="park", + min_confidence=0.8, + ) + + assert len(results) == 1 + assert results[0].artifact_id == "place_2" + + +class TestJumpNextScene: + """Tests for jump_next() with kind='scene'.""" + + @pytest.fixture + def setup_scene_ranges(self, session): + """Set up scene_ranges table for testing.""" + session.execute( + text( + """ + CREATE TABLE IF NOT EXISTS scene_ranges ( + artifact_id TEXT PRIMARY KEY, + asset_id TEXT NOT NULL, + scene_index INTEGER NOT NULL, + start_ms INTEGER NOT NULL, + end_ms INTEGER NOT NULL + ) + """ + ) + ) + session.commit() + yield + session.execute(text("DROP TABLE IF EXISTS scene_ranges")) + session.commit() + + def _insert_scene( + self, + session, + artifact_id: str, + asset_id: str, + scene_index: int, + start_ms: int, + end_ms: int, + ): + """Helper to insert scene into scene_ranges table.""" + session.execute( + text( + """ + INSERT INTO scene_ranges + (artifact_id, asset_id, scene_index, start_ms, end_ms) + VALUES (:artifact_id, :asset_id, :scene_index, :start_ms, :end_ms) + """ + ), + { + "artifact_id": artifact_id, + "asset_id": asset_id, + "scene_index": scene_index, + "start_ms": start_ms, + "end_ms": end_ms, + }, + ) + session.commit() + + def test_jump_next_scene_routes_correctly( + self, session, global_jump_service, setup_scene_ranges + ): + """Test that kind='scene' routes to scene search.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + self._insert_scene(session, "scene_1", video.video_id, 0, 100, 5000) + + results = global_jump_service.jump_next( + kind="scene", + from_video_id=video.video_id, + from_ms=0, + ) + + assert len(results) == 1 + assert results[0].artifact_id == "scene_1" + assert "scene_index" in results[0].preview + + +class TestJumpNextPlace: + """Tests for jump_next() with kind='place'.""" + + def test_jump_next_place_routes_correctly(self, session, global_jump_service): + """Test that kind='place' routes to place search.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + create_object_label( + session, "place_1", video.video_id, "restaurant", 0.9, 100, 200 + ) + + results = global_jump_service.jump_next( + kind="place", + from_video_id=video.video_id, + from_ms=0, + label="restaurant", + ) + + assert len(results) == 1 + assert results[0].artifact_id == "place_1" + assert results[0].preview["label"] == "restaurant" + + +class TestJumpPrevScene: + """Tests for jump_prev() with kind='scene'.""" + + @pytest.fixture + def setup_scene_ranges(self, session): + """Set up scene_ranges table for testing.""" + session.execute( + text( + """ + CREATE TABLE IF NOT EXISTS scene_ranges ( + artifact_id TEXT PRIMARY KEY, + asset_id TEXT NOT NULL, + scene_index INTEGER NOT NULL, + start_ms INTEGER NOT NULL, + end_ms INTEGER NOT NULL + ) + """ + ) + ) + session.commit() + yield + session.execute(text("DROP TABLE IF EXISTS scene_ranges")) + session.commit() + + def _insert_scene( + self, + session, + artifact_id: str, + asset_id: str, + scene_index: int, + start_ms: int, + end_ms: int, + ): + """Helper to insert scene into scene_ranges table.""" + session.execute( + text( + """ + INSERT INTO scene_ranges + (artifact_id, asset_id, scene_index, start_ms, end_ms) + VALUES (:artifact_id, :asset_id, :scene_index, :start_ms, :end_ms) + """ + ), + { + "artifact_id": artifact_id, + "asset_id": asset_id, + "scene_index": scene_index, + "start_ms": start_ms, + "end_ms": end_ms, + }, + ) + session.commit() + + def test_jump_prev_scene_routes_correctly( + self, session, global_jump_service, setup_scene_ranges + ): + """Test that kind='scene' routes to scene search with prev direction.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + self._insert_scene(session, "scene_1", video.video_id, 0, 100, 5000) + self._insert_scene(session, "scene_2", video.video_id, 1, 5000, 10000) + + results = global_jump_service.jump_prev( + kind="scene", + from_video_id=video.video_id, + from_ms=8000, + ) + + assert len(results) == 1 + assert results[0].artifact_id == "scene_2" + + +class TestJumpPrevPlace: + """Tests for jump_prev() with kind='place'.""" + + def test_jump_prev_place_routes_correctly(self, session, global_jump_service): + """Test that kind='place' routes to place search with prev direction.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + create_object_label(session, "place_1", video.video_id, "gym", 0.9, 100, 200) + create_object_label(session, "place_2", video.video_id, "gym", 0.8, 500, 600) + + results = global_jump_service.jump_prev( + kind="place", + from_video_id=video.video_id, + from_ms=400, + label="gym", + ) + + assert len(results) == 1 + assert results[0].artifact_id == "place_1" + + +class TestSearchLocationsGlobalNext: + """Tests for _search_locations_global with direction='next'.""" + + @pytest.fixture + def setup_video_locations(self, session): + """Set up video_locations table for testing.""" + session.execute( + text( + """ + CREATE TABLE IF NOT EXISTS video_locations ( + id INTEGER PRIMARY KEY, + video_id TEXT NOT NULL UNIQUE, + artifact_id TEXT NOT NULL, + latitude REAL NOT NULL, + longitude REAL NOT NULL, + altitude REAL, + country TEXT, + state TEXT, + city TEXT + ) + """ + ) + ) + session.commit() + yield + session.execute(text("DROP TABLE IF EXISTS video_locations")) + session.commit() + + def _insert_location( + self, + session, + video_id: str, + artifact_id: str, + latitude: float, + longitude: float, + altitude: float | None = None, + country: str | None = None, + state: str | None = None, + city: str | None = None, + ): + """Helper to insert location into video_locations table.""" + session.execute( + text( + """ + INSERT INTO video_locations + (video_id, artifact_id, latitude, longitude, altitude, + country, state, city) + VALUES (:video_id, :artifact_id, :latitude, :longitude, :altitude, + :country, :state, :city) + """ + ), + { + "video_id": video_id, + "artifact_id": artifact_id, + "latitude": latitude, + "longitude": longitude, + "altitude": altitude, + "country": country, + "state": state, + "city": city, + }, + ) + session.commit() + + def test_search_locations_next_single_video( + self, session, global_jump_service, setup_video_locations + ): + """Test searching for next video with location.""" + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + self._insert_location( + session, + video1.video_id, + "loc_1", + 35.6762, + 139.6503, + country="Japan", + city="Tokyo", + ) + self._insert_location( + session, + video2.video_id, + "loc_2", + 40.7128, + -74.0060, + country="USA", + city="New York", + ) + + results = global_jump_service._search_locations_global( + direction="next", + from_video_id=video1.video_id, + from_ms=0, + ) + + assert len(results) == 1 + assert results[0].video_id == "video_2" + assert results[0].artifact_id == "loc_2" + assert results[0].preview["city"] == "New York" + + def test_search_locations_next_ordering( + self, session, global_jump_service, setup_video_locations + ): + """Test that results are ordered by global timeline.""" + video1 = create_test_video( + session, "video_a", "video_a.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_b", "video_b.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + video3 = create_test_video( + session, "video_c", "video_c.mp4", datetime(2025, 1, 3, 12, 0, 0) + ) + + self._insert_location(session, video3.video_id, "loc_3", 51.5074, -0.1278) + self._insert_location(session, video1.video_id, "loc_1", 35.6762, 139.6503) + self._insert_location(session, video2.video_id, "loc_2", 40.7128, -74.0060) + + results = global_jump_service._search_locations_global( + direction="next", + from_video_id=video1.video_id, + from_ms=0, + limit=3, + ) + + assert len(results) == 2 + assert results[0].video_id == "video_b" + assert results[1].video_id == "video_c" + + def test_search_locations_next_no_results( + self, session, global_jump_service, setup_video_locations + ): + """Test that empty list is returned when no locations found.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + self._insert_location(session, video.video_id, "loc_1", 35.6762, 139.6503) + + results = global_jump_service._search_locations_global( + direction="next", + from_video_id=video.video_id, + from_ms=0, + ) + + assert len(results) == 0 + + def test_search_locations_next_video_not_found( + self, session, global_jump_service, setup_video_locations + ): + """Test that VideoNotFoundError is raised for non-existent video.""" + with pytest.raises(VideoNotFoundError) as exc_info: + global_jump_service._search_locations_global( + direction="next", + from_video_id="non_existent_video", + from_ms=0, + ) + + assert exc_info.value.video_id == "non_existent_video" + + def test_search_locations_next_result_contains_all_fields( + self, session, global_jump_service, setup_video_locations + ): + """Test that results contain all required fields.""" + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + self._insert_location( + session, + video2.video_id, + "loc_2", + 35.6762, + 139.6503, + altitude=10.5, + country="Japan", + state="Tokyo", + city="Shibuya", + ) + + results = global_jump_service._search_locations_global( + direction="next", + from_video_id=video1.video_id, + from_ms=0, + ) + + assert len(results) == 1 + result = results[0] + assert result.video_id == "video_2" + assert result.video_filename == "video2.mp4" + assert result.jump_to.start_ms == 0 + assert result.jump_to.end_ms == 0 + assert result.artifact_id == "loc_2" + assert result.preview["latitude"] == 35.6762 + assert result.preview["longitude"] == 139.6503 + assert result.preview["altitude"] == 10.5 + assert result.preview["country"] == "Japan" + assert result.preview["state"] == "Tokyo" + assert result.preview["city"] == "Shibuya" + + +class TestSearchLocationsGlobalPrev: + """Tests for _search_locations_global with direction='prev'.""" + + @pytest.fixture + def setup_video_locations(self, session): + """Set up video_locations table for testing.""" + session.execute( + text( + """ + CREATE TABLE IF NOT EXISTS video_locations ( + id INTEGER PRIMARY KEY, + video_id TEXT NOT NULL UNIQUE, + artifact_id TEXT NOT NULL, + latitude REAL NOT NULL, + longitude REAL NOT NULL, + altitude REAL, + country TEXT, + state TEXT, + city TEXT + ) + """ + ) + ) + session.commit() + yield + session.execute(text("DROP TABLE IF EXISTS video_locations")) + session.commit() + + def _insert_location( + self, + session, + video_id: str, + artifact_id: str, + latitude: float, + longitude: float, + altitude: float | None = None, + country: str | None = None, + state: str | None = None, + city: str | None = None, + ): + """Helper to insert location into video_locations table.""" + session.execute( + text( + """ + INSERT INTO video_locations + (video_id, artifact_id, latitude, longitude, altitude, + country, state, city) + VALUES (:video_id, :artifact_id, :latitude, :longitude, :altitude, + :country, :state, :city) + """ + ), + { + "video_id": video_id, + "artifact_id": artifact_id, + "latitude": latitude, + "longitude": longitude, + "altitude": altitude, + "country": country, + "state": state, + "city": city, + }, + ) + session.commit() + + def test_search_locations_prev_single_video( + self, session, global_jump_service, setup_video_locations + ): + """Test searching for previous video with location.""" + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + self._insert_location( + session, + video1.video_id, + "loc_1", + 35.6762, + 139.6503, + country="Japan", + city="Tokyo", + ) + self._insert_location( + session, + video2.video_id, + "loc_2", + 40.7128, + -74.0060, + country="USA", + city="New York", + ) + + results = global_jump_service._search_locations_global( + direction="prev", + from_video_id=video2.video_id, + from_ms=0, + ) + + assert len(results) == 1 + assert results[0].video_id == "video_1" + assert results[0].artifact_id == "loc_1" + assert results[0].preview["city"] == "Tokyo" + + def test_search_locations_prev_ordering( + self, session, global_jump_service, setup_video_locations + ): + """Test that results are ordered by global timeline (descending).""" + video1 = create_test_video( + session, "video_a", "video_a.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_b", "video_b.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + video3 = create_test_video( + session, "video_c", "video_c.mp4", datetime(2025, 1, 3, 12, 0, 0) + ) + + self._insert_location(session, video1.video_id, "loc_1", 35.6762, 139.6503) + self._insert_location(session, video2.video_id, "loc_2", 40.7128, -74.0060) + self._insert_location(session, video3.video_id, "loc_3", 51.5074, -0.1278) + + results = global_jump_service._search_locations_global( + direction="prev", + from_video_id=video3.video_id, + from_ms=0, + limit=3, + ) + + assert len(results) == 2 + # Should be ordered by file_created_at descending + assert results[0].video_id == "video_b" + assert results[1].video_id == "video_a" + + def test_search_locations_prev_no_results( + self, session, global_jump_service, setup_video_locations + ): + """Test that empty list is returned when no locations found.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + self._insert_location(session, video.video_id, "loc_1", 35.6762, 139.6503) + + results = global_jump_service._search_locations_global( + direction="prev", + from_video_id=video.video_id, + from_ms=0, + ) + + assert len(results) == 0 + + +class TestJumpNextLocation: + """Tests for jump_next() with kind='location'.""" + + @pytest.fixture + def setup_video_locations(self, session): + """Set up video_locations table for testing.""" + session.execute( + text( + """ + CREATE TABLE IF NOT EXISTS video_locations ( + id INTEGER PRIMARY KEY, + video_id TEXT NOT NULL UNIQUE, + artifact_id TEXT NOT NULL, + latitude REAL NOT NULL, + longitude REAL NOT NULL, + altitude REAL, + country TEXT, + state TEXT, + city TEXT + ) + """ + ) + ) + session.commit() + yield + session.execute(text("DROP TABLE IF EXISTS video_locations")) + session.commit() + + def _insert_location( + self, + session, + video_id: str, + artifact_id: str, + latitude: float, + longitude: float, + ): + """Helper to insert location into video_locations table.""" + session.execute( + text( + """ + INSERT INTO video_locations + (video_id, artifact_id, latitude, longitude) + VALUES (:video_id, :artifact_id, :latitude, :longitude) + """ + ), + { + "video_id": video_id, + "artifact_id": artifact_id, + "latitude": latitude, + "longitude": longitude, + }, + ) + session.commit() + + def test_jump_next_location_routes_correctly( + self, session, global_jump_service, setup_video_locations + ): + """Test that kind='location' routes to location search.""" + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + self._insert_location(session, video2.video_id, "loc_2", 35.6762, 139.6503) + + results = global_jump_service.jump_next( + kind="location", + from_video_id=video1.video_id, + from_ms=0, + ) + + assert len(results) == 1 + assert results[0].video_id == "video_2" + assert "latitude" in results[0].preview + assert "longitude" in results[0].preview + + +class TestJumpPrevLocation: + """Tests for jump_prev() with kind='location'.""" + + @pytest.fixture + def setup_video_locations(self, session): + """Set up video_locations table for testing.""" + session.execute( + text( + """ + CREATE TABLE IF NOT EXISTS video_locations ( + id INTEGER PRIMARY KEY, + video_id TEXT NOT NULL UNIQUE, + artifact_id TEXT NOT NULL, + latitude REAL NOT NULL, + longitude REAL NOT NULL, + altitude REAL, + country TEXT, + state TEXT, + city TEXT + ) + """ + ) + ) + session.commit() + yield + session.execute(text("DROP TABLE IF EXISTS video_locations")) + session.commit() + + def _insert_location( + self, + session, + video_id: str, + artifact_id: str, + latitude: float, + longitude: float, + ): + """Helper to insert location into video_locations table.""" + session.execute( + text( + """ + INSERT INTO video_locations + (video_id, artifact_id, latitude, longitude) + VALUES (:video_id, :artifact_id, :latitude, :longitude) + """ + ), + { + "video_id": video_id, + "artifact_id": artifact_id, + "latitude": latitude, + "longitude": longitude, + }, + ) + session.commit() + + def test_jump_prev_location_routes_correctly( + self, session, global_jump_service, setup_video_locations + ): + """Test that kind='location' routes to location search with prev direction.""" + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + self._insert_location(session, video1.video_id, "loc_1", 35.6762, 139.6503) + + results = global_jump_service.jump_prev( + kind="location", + from_video_id=video2.video_id, + from_ms=0, + ) + + assert len(results) == 1 + assert results[0].video_id == "video_1" + assert "latitude" in results[0].preview + + +class TestBoundaryConditionFromMsBeyondDuration: + """Tests for boundary condition handling when from_ms exceeds video duration. + + Property 15: Boundary Condition - from_ms Beyond Duration + Validates: Requirements 11.4 + + When from_ms is beyond the video duration, the system should treat it as + the end of that video and search forward (for "next") or backward (for "prev") + accordingly without error. + """ + + def test_from_ms_beyond_duration_next_moves_to_next_video( + self, session, global_jump_service + ): + """Test that from_ms beyond duration moves to next video for 'next' direction. + + When from_ms is far beyond any artifact in the current video, + the search should naturally find artifacts in the next video. + """ + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + # Create artifacts - video1 has artifacts up to 5000ms + create_object_label(session, "obj_1", video1.video_id, "dog", 0.9, 1000, 2000) + create_object_label(session, "obj_2", video1.video_id, "dog", 0.9, 3000, 4000) + create_object_label(session, "obj_3", video2.video_id, "dog", 0.9, 500, 1000) + + # Search with from_ms far beyond video1's content (simulating beyond duration) + results = global_jump_service.jump_next( + kind="object", + from_video_id=video1.video_id, + from_ms=999999999, # Way beyond any video duration + label="dog", + ) + + # Should find artifact in video2, not raise an error + assert len(results) == 1 + assert results[0].video_id == "video_2" + assert results[0].artifact_id == "obj_3" + + def test_from_ms_beyond_duration_prev_finds_last_artifact( + self, session, global_jump_service + ): + """Test that from_ms beyond duration finds last artifact for 'prev' direction. + + When from_ms is far beyond any artifact in the current video, + the search should find the last artifact in that video. + """ + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + # Create artifacts at various timestamps + create_object_label(session, "obj_1", video.video_id, "cat", 0.9, 1000, 2000) + create_object_label(session, "obj_2", video.video_id, "cat", 0.9, 3000, 4000) + create_object_label(session, "obj_3", video.video_id, "cat", 0.9, 5000, 6000) + + # Search with from_ms far beyond video's content + results = global_jump_service.jump_prev( + kind="object", + from_video_id=video.video_id, + from_ms=999999999, # Way beyond any video duration + label="cat", + ) + + # Should find the last artifact (obj_3), not raise an error + assert len(results) == 1 + assert results[0].artifact_id == "obj_3" + assert results[0].jump_to.start_ms == 5000 + + def test_from_ms_beyond_duration_no_error_raised( + self, session, global_jump_service + ): + """Test that no error is raised when from_ms exceeds video duration.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + create_object_label(session, "obj_1", video.video_id, "dog", 0.9, 1000, 2000) + + # These should not raise any exceptions + results_next = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + from_ms=2**31 - 1, # Max 32-bit signed integer + label="dog", + ) + + results_prev = global_jump_service.jump_prev( + kind="object", + from_video_id=video.video_id, + from_ms=2**31 - 1, # Max 32-bit signed integer + label="dog", + ) + + # No errors raised, results are valid (may be empty for next) + assert isinstance(results_next, list) + assert isinstance(results_prev, list) + # prev should find the artifact since it's before the large from_ms + assert len(results_prev) == 1 + assert results_prev[0].artifact_id == "obj_1" + + def test_from_ms_beyond_duration_empty_results_when_no_next_video( + self, session, global_jump_service + ): + """Test empty results when from_ms beyond duration and no next video exists.""" + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + create_object_label(session, "obj_1", video.video_id, "dog", 0.9, 1000, 2000) + + # Search next with from_ms beyond duration, no other videos exist + results = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + from_ms=999999999, + label="dog", + ) + + # Should return empty list, not raise an error + assert len(results) == 0 + + def test_from_ms_beyond_duration_prev_crosses_to_previous_video( + self, session, global_jump_service + ): + """Test prev direction crosses to previous video when from_ms > duration.""" + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + create_object_label(session, "obj_1", video1.video_id, "bird", 0.9, 1000, 2000) + # No artifacts in video2 + + # Search prev from video2 with large from_ms + results = global_jump_service.jump_prev( + kind="object", + from_video_id=video2.video_id, + from_ms=999999999, + label="bird", + ) + + # Should find artifact in video1 + assert len(results) == 1 + assert results[0].video_id == "video_1" + assert results[0].artifact_id == "obj_1" + + +class TestArbitraryPositionNavigation: + """Tests for arbitrary position navigation. + + Property 13: Arbitrary Position Navigation + Validates: Requirements 11.1, 11.2, 11.3 + + For any global jump query with from_video_id and from_ms parameters, + the search should start from that position in the global timeline. + Results should be chronologically after (for "next") or before (for "prev") + that position. + """ + + def test_arbitrary_position_next_within_same_video( + self, session, global_jump_service + ): + """Test next from arbitrary position within the same video. + + Validates: Requirement 11.1 - from_video_id and from_ms as starting point + """ + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + # Create artifacts at various timestamps + create_object_label(session, "obj_1", video.video_id, "dog", 0.9, 100, 200) + create_object_label(session, "obj_2", video.video_id, "dog", 0.9, 500, 600) + create_object_label(session, "obj_3", video.video_id, "dog", 0.9, 1000, 1100) + create_object_label(session, "obj_4", video.video_id, "dog", 0.9, 1500, 1600) + + # Search from arbitrary position at 700ms + results = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + from_ms=700, + label="dog", + ) + + # Should find obj_3 (first artifact after 700ms) + assert len(results) == 1 + assert results[0].artifact_id == "obj_3" + assert results[0].jump_to.start_ms == 1000 + assert results[0].jump_to.start_ms > 700 # Chronologically after + + def test_arbitrary_position_prev_within_same_video( + self, session, global_jump_service + ): + """Test prev from arbitrary position within the same video. + + Validates: Requirement 11.3 - prev returns results before position + """ + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + # Create artifacts at various timestamps + create_object_label(session, "obj_1", video.video_id, "cat", 0.9, 100, 200) + create_object_label(session, "obj_2", video.video_id, "cat", 0.9, 500, 600) + create_object_label(session, "obj_3", video.video_id, "cat", 0.9, 1000, 1100) + create_object_label(session, "obj_4", video.video_id, "cat", 0.9, 1500, 1600) + + # Search from arbitrary position at 1200ms + results = global_jump_service.jump_prev( + kind="object", + from_video_id=video.video_id, + from_ms=1200, + label="cat", + ) + + # Should find obj_3 (first artifact before 1200ms in descending order) + assert len(results) == 1 + assert results[0].artifact_id == "obj_3" + assert results[0].jump_to.start_ms == 1000 + assert results[0].jump_to.start_ms < 1200 # Chronologically before + + def test_arbitrary_position_next_crosses_video_boundary( + self, session, global_jump_service + ): + """Test next from arbitrary position crosses to next video. + + Validates: Requirement 11.2 - next returns results after position + """ + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + # Create artifacts in both videos + create_object_label(session, "obj_1", video1.video_id, "bird", 0.9, 100, 200) + create_object_label(session, "obj_2", video1.video_id, "bird", 0.9, 500, 600) + create_object_label(session, "obj_3", video2.video_id, "bird", 0.9, 100, 200) + + # Search from arbitrary position at end of video1 + results = global_jump_service.jump_next( + kind="object", + from_video_id=video1.video_id, + from_ms=800, # After all artifacts in video1 + label="bird", + ) + + # Should find obj_3 in video2 (chronologically after video1) + assert len(results) == 1 + assert results[0].video_id == "video_2" + assert results[0].artifact_id == "obj_3" + + def test_arbitrary_position_prev_crosses_video_boundary( + self, session, global_jump_service + ): + """Test prev from arbitrary position crosses to previous video. + + Validates: Requirement 11.3 - prev returns results before position + """ + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + # Create artifacts in both videos + create_object_label(session, "obj_1", video1.video_id, "fish", 0.9, 500, 600) + create_object_label(session, "obj_2", video2.video_id, "fish", 0.9, 1000, 1100) + + # Search from arbitrary position at beginning of video2 + results = global_jump_service.jump_prev( + kind="object", + from_video_id=video2.video_id, + from_ms=50, # Before all artifacts in video2 + label="fish", + ) + + # Should find obj_1 in video1 (chronologically before video2) + assert len(results) == 1 + assert results[0].video_id == "video_1" + assert results[0].artifact_id == "obj_1" + + def test_arbitrary_position_with_various_video_orderings( + self, session, global_jump_service + ): + """Test arbitrary position with videos in various chronological orders. + + Validates: Requirements 11.1, 11.2, 11.3 + """ + # Create videos with non-sequential IDs but sequential dates + video_c = create_test_video( + session, "video_c", "video_c.mp4", datetime(2025, 1, 3, 12, 0, 0) + ) + video_a = create_test_video( + session, "video_a", "video_a.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video_b = create_test_video( + session, "video_b", "video_b.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + # Create artifacts in each video + create_object_label(session, "obj_a", video_a.video_id, "car", 0.9, 500, 600) + create_object_label(session, "obj_b", video_b.video_id, "car", 0.9, 500, 600) + create_object_label(session, "obj_c", video_c.video_id, "car", 0.9, 500, 600) + + # Search next from video_a - should find video_b (chronologically next) + results_next = global_jump_service.jump_next( + kind="object", + from_video_id=video_a.video_id, + from_ms=700, + label="car", + ) + + assert len(results_next) == 1 + assert results_next[0].video_id == "video_b" + + # Search prev from video_c - should find video_b (chronologically previous) + results_prev = global_jump_service.jump_prev( + kind="object", + from_video_id=video_c.video_id, + from_ms=0, + label="car", + ) + + assert len(results_prev) == 1 + assert results_prev[0].video_id == "video_b" + + def test_arbitrary_position_with_same_file_created_at( + self, session, global_jump_service + ): + """Test arbitrary position when videos have same file_created_at. + + When file_created_at is the same, video_id is used as secondary sort key. + Validates: Requirements 11.1, 11.2, 11.3 + """ + same_time = datetime(2025, 1, 1, 12, 0, 0) + + # Create videos with same timestamp but different IDs + video_x = create_test_video(session, "video_x", "video_x.mp4", same_time) + video_y = create_test_video(session, "video_y", "video_y.mp4", same_time) + video_z = create_test_video(session, "video_z", "video_z.mp4", same_time) + + # Create artifacts + create_object_label(session, "obj_x", video_x.video_id, "tree", 0.9, 500, 600) + create_object_label(session, "obj_y", video_y.video_id, "tree", 0.9, 500, 600) + create_object_label(session, "obj_z", video_z.video_id, "tree", 0.9, 500, 600) + + # Search next from video_x - should find video_y (alphabetically next) + results_next = global_jump_service.jump_next( + kind="object", + from_video_id=video_x.video_id, + from_ms=700, + label="tree", + ) + + assert len(results_next) == 1 + assert results_next[0].video_id == "video_y" + + # Search prev from video_z - should find video_y (alphabetically previous) + results_prev = global_jump_service.jump_prev( + kind="object", + from_video_id=video_z.video_id, + from_ms=0, + label="tree", + ) + + assert len(results_prev) == 1 + assert results_prev[0].video_id == "video_y" + + def test_arbitrary_position_exact_timestamp_match( + self, session, global_jump_service + ): + """Test arbitrary position when from_ms exactly matches an artifact start_ms. + + The artifact at the exact position should NOT be included in results. + Validates: Requirements 11.2, 11.3 + """ + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + # Create artifacts at specific timestamps + create_object_label(session, "obj_1", video.video_id, "dog", 0.9, 100, 200) + create_object_label(session, "obj_2", video.video_id, "dog", 0.9, 500, 600) + create_object_label(session, "obj_3", video.video_id, "dog", 0.9, 1000, 1100) + + # Search next from exact position of obj_2 (500ms) + results_next = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + from_ms=500, # Exact match with obj_2 + label="dog", + ) + + # Should find obj_3, NOT obj_2 (which is at the exact position) + assert len(results_next) == 1 + assert results_next[0].artifact_id == "obj_3" + assert results_next[0].jump_to.start_ms > 500 + + # Search prev from exact position of obj_2 (500ms) + results_prev = global_jump_service.jump_prev( + kind="object", + from_video_id=video.video_id, + from_ms=500, # Exact match with obj_2 + label="dog", + ) + + # Should find obj_1, NOT obj_2 (which is at the exact position) + assert len(results_prev) == 1 + assert results_prev[0].artifact_id == "obj_1" + assert results_prev[0].jump_to.start_ms < 500 + + def test_arbitrary_position_with_multiple_results( + self, session, global_jump_service + ): + """Test arbitrary position returns multiple results in correct order. + + Validates: Requirements 11.1, 11.2, 11.3 + """ + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + # Create multiple artifacts + create_object_label(session, "obj_1", video1.video_id, "dog", 0.9, 100, 200) + create_object_label(session, "obj_2", video1.video_id, "dog", 0.9, 500, 600) + create_object_label(session, "obj_3", video1.video_id, "dog", 0.9, 1000, 1100) + create_object_label(session, "obj_4", video2.video_id, "dog", 0.9, 100, 200) + create_object_label(session, "obj_5", video2.video_id, "dog", 0.9, 500, 600) + + # Search next from arbitrary position with limit=3 + results = global_jump_service.jump_next( + kind="object", + from_video_id=video1.video_id, + from_ms=300, + label="dog", + limit=3, + ) + + # Should return 3 results in chronological order + assert len(results) == 3 + assert results[0].artifact_id == "obj_2" # video1, 500ms + assert results[1].artifact_id == "obj_3" # video1, 1000ms + assert results[2].artifact_id == "obj_4" # video2, 100ms + + # All results should be chronologically after 300ms in video1 + assert results[0].jump_to.start_ms > 300 + + def test_arbitrary_position_from_middle_of_timeline( + self, session, global_jump_service + ): + """Test arbitrary position from middle of global timeline. + + Validates: Requirements 11.1, 11.2, 11.3 + """ + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + video3 = create_test_video( + session, "video_3", "video3.mp4", datetime(2025, 1, 3, 12, 0, 0) + ) + + # Create artifacts in all videos + create_object_label(session, "obj_1", video1.video_id, "cat", 0.9, 500, 600) + create_object_label(session, "obj_2", video2.video_id, "cat", 0.9, 500, 600) + create_object_label(session, "obj_3", video3.video_id, "cat", 0.9, 500, 600) + + # Search next from middle video (video2) + results_next = global_jump_service.jump_next( + kind="object", + from_video_id=video2.video_id, + from_ms=700, + label="cat", + ) + + # Should find video3 (chronologically after video2) + assert len(results_next) == 1 + assert results_next[0].video_id == "video_3" + + # Search prev from middle video (video2) + results_prev = global_jump_service.jump_prev( + kind="object", + from_video_id=video2.video_id, + from_ms=0, + label="cat", + ) + + # Should find video1 (chronologically before video2) + assert len(results_prev) == 1 + assert results_prev[0].video_id == "video_1" + + def test_arbitrary_position_with_null_file_created_at( + self, session, global_jump_service + ): + """Test arbitrary position with NULL file_created_at values. + + NULL file_created_at values are sorted after non-NULL values. + Validates: Requirements 11.1, 11.2, 11.3 + """ + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, + "video_2", + "video2.mp4", + None, # NULL file_created_at + ) + video3 = create_test_video( + session, "video_3", "video3.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + # Create artifacts + create_object_label(session, "obj_1", video1.video_id, "dog", 0.9, 500, 600) + create_object_label(session, "obj_2", video2.video_id, "dog", 0.9, 500, 600) + create_object_label(session, "obj_3", video3.video_id, "dog", 0.9, 500, 600) + + # Search next from video3 - should find video2 (NULL sorted after) + results_next = global_jump_service.jump_next( + kind="object", + from_video_id=video3.video_id, + from_ms=700, + label="dog", + ) + + assert len(results_next) == 1 + assert results_next[0].video_id == "video_2" + + # Search prev from video2 (NULL) - should find video3 (non-NULL before NULL) + results_prev = global_jump_service.jump_prev( + kind="object", + from_video_id=video2.video_id, + from_ms=0, + label="dog", + ) + + assert len(results_prev) == 1 + assert results_prev[0].video_id == "video_3" + + def test_arbitrary_position_no_results_after_position( + self, session, global_jump_service + ): + """Test arbitrary position when no results exist after the position. + + Validates: Requirements 11.2 + """ + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + # Create artifacts only at early timestamps + create_object_label(session, "obj_1", video.video_id, "dog", 0.9, 100, 200) + create_object_label(session, "obj_2", video.video_id, "dog", 0.9, 300, 400) + + # Search next from position after all artifacts + results = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + from_ms=500, + label="dog", + ) + + # Should return empty list + assert len(results) == 0 + + def test_arbitrary_position_no_results_before_position( + self, session, global_jump_service + ): + """Test arbitrary position when no results exist before the position. + + Validates: Requirements 11.3 + """ + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + # Create artifacts only at late timestamps + create_object_label(session, "obj_1", video.video_id, "cat", 0.9, 500, 600) + create_object_label(session, "obj_2", video.video_id, "cat", 0.9, 700, 800) + + # Search prev from position before all artifacts + results = global_jump_service.jump_prev( + kind="object", + from_video_id=video.video_id, + from_ms=100, + label="cat", + ) + + # Should return empty list + assert len(results) == 0 + + def test_arbitrary_position_with_transcript_search( + self, session, global_jump_service + ): + """Test arbitrary position with transcript full-text search. + + Validates: Requirements 11.1, 11.2, 11.3 + """ + # Set up transcript FTS table for SQLite + session.execute( + text( + """ + CREATE VIRTUAL TABLE IF NOT EXISTS transcript_fts USING fts5( + artifact_id UNINDEXED, + asset_id UNINDEXED, + start_ms UNINDEXED, + end_ms UNINDEXED, + text + ) + """ + ) + ) + session.execute( + text( + """ + CREATE TABLE IF NOT EXISTS transcript_fts_metadata ( + artifact_id TEXT PRIMARY KEY, + asset_id TEXT NOT NULL, + start_ms INTEGER NOT NULL, + end_ms INTEGER NOT NULL + ) + """ + ) + ) + session.commit() + + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + # Insert transcripts + for artifact_id, start_ms, text_content in [ + ("trans_1", 100, "hello world"), + ("trans_2", 500, "hello again"), + ("trans_3", 1000, "goodbye world"), + ]: + session.execute( + text( + """ + INSERT INTO transcript_fts + (artifact_id, asset_id, start_ms, end_ms, text) + VALUES (:artifact_id, :asset_id, :start_ms, :end_ms, :text) + """ + ), + { + "artifact_id": artifact_id, + "asset_id": video.video_id, + "start_ms": start_ms, + "end_ms": start_ms + 100, + "text": text_content, + }, + ) + session.execute( + text( + """ + INSERT INTO transcript_fts_metadata + (artifact_id, asset_id, start_ms, end_ms) + VALUES (:artifact_id, :asset_id, :start_ms, :end_ms) + """ + ), + { + "artifact_id": artifact_id, + "asset_id": video.video_id, + "start_ms": start_ms, + "end_ms": start_ms + 100, + }, + ) + session.commit() + + # Search next from arbitrary position + results = global_jump_service.jump_next( + kind="transcript", + from_video_id=video.video_id, + from_ms=300, + query="hello", + ) + + # Should find trans_2 (first "hello" after 300ms) + assert len(results) == 1 + assert results[0].artifact_id == "trans_2" + assert results[0].jump_to.start_ms == 500 + + # Cleanup + session.execute(text("DROP TABLE IF EXISTS transcript_fts_metadata")) + session.execute(text("DROP TABLE IF EXISTS transcript_fts")) + session.commit() + + def test_arbitrary_position_with_scene_search(self, session, global_jump_service): + """Test arbitrary position with scene search. + + Validates: Requirements 11.1, 11.2, 11.3 + """ + from src.database.models import SceneRange + + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + # Create scene ranges + for artifact_id, scene_index, start_ms in [ + ("scene_1", 0, 0), + ("scene_2", 1, 5000), + ("scene_3", 2, 10000), + ]: + scene = SceneRange( + artifact_id=artifact_id, + asset_id=video.video_id, + scene_index=scene_index, + start_ms=start_ms, + end_ms=start_ms + 4000, + ) + session.add(scene) + session.commit() + + # Search next from arbitrary position + results = global_jump_service.jump_next( + kind="scene", + from_video_id=video.video_id, + from_ms=6000, + ) + + # Should find scene_3 (first scene after 6000ms) + assert len(results) == 1 + assert results[0].artifact_id == "scene_3" + assert results[0].jump_to.start_ms == 10000 + + +class TestFilterChangeIndependence: + """Tests for filter change independence. + + Verifies that changing filters (label, query, kind) doesn't affect timeline + position. Each query is independent and doesn't carry state from previous + queries. + + Validates: Requirements 12.1, 12.2, 12.3, 12.4 + """ + + def test_consecutive_queries_with_different_labels( + self, session, global_jump_service + ): + """Test that consecutive queries with different labels are independent. + + Validates: Requirements 12.1, 12.2 + """ + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + # Create objects with different labels at different timestamps + create_object_label(session, "obj_dog_1", video.video_id, "dog", 0.9, 100, 200) + create_object_label(session, "obj_cat_1", video.video_id, "cat", 0.9, 150, 250) + create_object_label(session, "obj_dog_2", video.video_id, "dog", 0.9, 300, 400) + create_object_label(session, "obj_cat_2", video.video_id, "cat", 0.9, 350, 450) + create_object_label(session, "obj_dog_3", video.video_id, "dog", 0.9, 500, 600) + create_object_label(session, "obj_cat_3", video.video_id, "cat", 0.9, 550, 650) + + # First query: search for "dog" from position 0 + results_dog = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + from_ms=0, + label="dog", + ) + assert len(results_dog) == 1 + assert results_dog[0].artifact_id == "obj_dog_1" + assert results_dog[0].preview["label"] == "dog" + + # Second query: search for "cat" from the SAME position (0) + # This should NOT be affected by the previous "dog" query + results_cat = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + from_ms=0, + label="cat", + ) + assert len(results_cat) == 1 + assert results_cat[0].artifact_id == "obj_cat_1" + assert results_cat[0].preview["label"] == "cat" + + # Third query: search for "dog" again from position 0 + # Should return the same result as the first query + results_dog_again = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + from_ms=0, + label="dog", + ) + assert len(results_dog_again) == 1 + assert results_dog_again[0].artifact_id == "obj_dog_1" + assert results_dog_again[0].preview["label"] == "dog" + + def test_filter_change_maintains_timeline_position( + self, session, global_jump_service + ): + """Test that changing filters maintains the same timeline position. + + Validates: Requirements 12.2, 12.4 + """ + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + # Create objects at various timestamps + create_object_label(session, "obj_dog_1", video.video_id, "dog", 0.9, 100, 200) + create_object_label(session, "obj_cat_1", video.video_id, "cat", 0.9, 200, 300) + create_object_label(session, "obj_dog_2", video.video_id, "dog", 0.9, 400, 500) + create_object_label(session, "obj_cat_2", video.video_id, "cat", 0.9, 500, 600) + + # Search for "dog" from position 250 + results_dog = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + from_ms=250, + label="dog", + ) + assert len(results_dog) == 1 + assert results_dog[0].artifact_id == "obj_dog_2" + assert results_dog[0].jump_to.start_ms == 400 + + # Change filter to "cat" but keep same position (250) + # Should find cat at 500ms, not be affected by previous dog search + results_cat = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + from_ms=250, + label="cat", + ) + assert len(results_cat) == 1 + assert results_cat[0].artifact_id == "obj_cat_2" + assert results_cat[0].jump_to.start_ms == 500 + + def test_kind_change_routes_to_different_projection_table( + self, session, global_jump_service + ): + """Test that changing kind routes to appropriate projection table. + + Validates: Requirements 12.3 + """ + from src.database.models import SceneRange + + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + # Create object labels + create_object_label(session, "obj_1", video.video_id, "dog", 0.9, 100, 200) + create_object_label(session, "obj_2", video.video_id, "dog", 0.9, 500, 600) + + # Create scene ranges + scene1 = SceneRange( + artifact_id="scene_1", + asset_id=video.video_id, + scene_index=0, + start_ms=0, + end_ms=300, + ) + scene2 = SceneRange( + artifact_id="scene_2", + asset_id=video.video_id, + scene_index=1, + start_ms=300, + end_ms=700, + ) + session.add(scene1) + session.add(scene2) + session.commit() + + # Search for objects from position 0 + results_object = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + from_ms=0, + label="dog", + ) + assert len(results_object) == 1 + assert results_object[0].artifact_id == "obj_1" + assert "label" in results_object[0].preview + + # Change kind to "scene" from same position + # Should search scene_ranges table, not object_labels + results_scene = global_jump_service.jump_next( + kind="scene", + from_video_id=video.video_id, + from_ms=0, + ) + assert len(results_scene) == 1 + assert results_scene[0].artifact_id == "scene_2" + assert "scene_index" in results_scene[0].preview + + def test_multiple_consecutive_queries_different_filters( + self, session, global_jump_service + ): + """Test multiple consecutive queries with different filters. + + Validates: Requirements 12.1, 12.2, 12.3, 12.4 + """ + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + # Create various objects + create_object_label(session, "obj_dog", video.video_id, "dog", 0.9, 100, 200) + create_object_label(session, "obj_cat", video.video_id, "cat", 0.8, 200, 300) + create_object_label(session, "obj_bird", video.video_id, "bird", 0.7, 300, 400) + create_object_label(session, "obj_car", video.video_id, "car", 0.95, 400, 500) + + # Run multiple queries from the same position + position = 50 + + # Query 1: dog + r1 = global_jump_service.jump_next( + kind="object", from_video_id=video.video_id, from_ms=position, label="dog" + ) + assert r1[0].artifact_id == "obj_dog" + + # Query 2: cat + r2 = global_jump_service.jump_next( + kind="object", from_video_id=video.video_id, from_ms=position, label="cat" + ) + assert r2[0].artifact_id == "obj_cat" + + # Query 3: bird + r3 = global_jump_service.jump_next( + kind="object", from_video_id=video.video_id, from_ms=position, label="bird" + ) + assert r3[0].artifact_id == "obj_bird" + + # Query 4: car + r4 = global_jump_service.jump_next( + kind="object", from_video_id=video.video_id, from_ms=position, label="car" + ) + assert r4[0].artifact_id == "obj_car" + + # Query 5: back to dog - should still return same result + r5 = global_jump_service.jump_next( + kind="object", from_video_id=video.video_id, from_ms=position, label="dog" + ) + assert r5[0].artifact_id == "obj_dog" + + def test_filter_change_with_no_results_for_new_filter( + self, session, global_jump_service + ): + """Test that changing to a filter with no results returns empty array. + + Validates: Requirements 12.5 + """ + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + # Create only dog objects + create_object_label(session, "obj_dog_1", video.video_id, "dog", 0.9, 100, 200) + create_object_label(session, "obj_dog_2", video.video_id, "dog", 0.9, 300, 400) + + # First query: search for "dog" - should find results + results_dog = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + from_ms=0, + label="dog", + ) + assert len(results_dog) == 1 + assert results_dog[0].artifact_id == "obj_dog_1" + + # Second query: search for "elephant" - no such objects exist + results_elephant = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + from_ms=0, + label="elephant", + ) + assert len(results_elephant) == 0 + + # Third query: back to "dog" - should still work + results_dog_again = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + from_ms=0, + label="dog", + ) + assert len(results_dog_again) == 1 + assert results_dog_again[0].artifact_id == "obj_dog_1" + + def test_filter_change_with_confidence_threshold( + self, session, global_jump_service + ): + """Test filter changes with varying confidence thresholds. + + Validates: Requirements 12.1, 12.2 + """ + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + # Create objects with different confidence levels + create_object_label( + session, "obj_dog_low", video.video_id, "dog", 0.5, 100, 200 + ) + create_object_label( + session, "obj_dog_high", video.video_id, "dog", 0.9, 300, 400 + ) + create_object_label( + session, "obj_cat_low", video.video_id, "cat", 0.4, 150, 250 + ) + create_object_label( + session, "obj_cat_high", video.video_id, "cat", 0.95, 350, 450 + ) + + # Query 1: dog with high confidence threshold + r1 = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + from_ms=0, + label="dog", + min_confidence=0.8, + ) + assert len(r1) == 1 + assert r1[0].artifact_id == "obj_dog_high" + + # Query 2: cat with high confidence threshold + r2 = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + from_ms=0, + label="cat", + min_confidence=0.8, + ) + assert len(r2) == 1 + assert r2[0].artifact_id == "obj_cat_high" + + # Query 3: dog with low confidence threshold + r3 = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + from_ms=0, + label="dog", + min_confidence=0.3, + ) + assert len(r3) == 1 + assert r3[0].artifact_id == "obj_dog_low" + + def test_filter_change_across_videos(self, session, global_jump_service): + """Test filter changes work correctly across multiple videos. + + Validates: Requirements 12.1, 12.2, 12.4 + """ + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + # Create objects in both videos + create_object_label(session, "v1_dog", video1.video_id, "dog", 0.9, 100, 200) + create_object_label(session, "v1_cat", video1.video_id, "cat", 0.9, 200, 300) + create_object_label(session, "v2_dog", video2.video_id, "dog", 0.9, 100, 200) + create_object_label(session, "v2_cat", video2.video_id, "cat", 0.9, 200, 300) + + # Search for dog from video1 + r1 = global_jump_service.jump_next( + kind="object", + from_video_id=video1.video_id, + from_ms=150, + label="dog", + ) + assert len(r1) == 1 + assert r1[0].video_id == "video_2" + assert r1[0].artifact_id == "v2_dog" + + # Change filter to cat from same position + r2 = global_jump_service.jump_next( + kind="object", + from_video_id=video1.video_id, + from_ms=150, + label="cat", + ) + assert len(r2) == 1 + assert r2[0].video_id == "video_1" + assert r2[0].artifact_id == "v1_cat" + + def test_prev_direction_filter_independence(self, session, global_jump_service): + """Test filter independence works for prev direction too. + + Validates: Requirements 12.1, 12.2 + """ + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + # Create objects + create_object_label(session, "obj_dog_1", video.video_id, "dog", 0.9, 100, 200) + create_object_label(session, "obj_cat_1", video.video_id, "cat", 0.9, 200, 300) + create_object_label(session, "obj_dog_2", video.video_id, "dog", 0.9, 400, 500) + create_object_label(session, "obj_cat_2", video.video_id, "cat", 0.9, 500, 600) + + # Search prev for dog from position 600 + r1 = global_jump_service.jump_prev( + kind="object", + from_video_id=video.video_id, + from_ms=600, + label="dog", + ) + assert len(r1) == 1 + assert r1[0].artifact_id == "obj_dog_2" + + # Change filter to cat from same position + r2 = global_jump_service.jump_prev( + kind="object", + from_video_id=video.video_id, + from_ms=600, + label="cat", + ) + assert len(r2) == 1 + assert r2[0].artifact_id == "obj_cat_2" + + # Back to dog - should return same result + r3 = global_jump_service.jump_prev( + kind="object", + from_video_id=video.video_id, + from_ms=600, + label="dog", + ) + assert len(r3) == 1 + assert r3[0].artifact_id == "obj_dog_2" + + def test_service_has_no_query_state(self, session, global_jump_service): + """Verify that the service doesn't store any query state. + + This test verifies the stateless design of GlobalJumpService. + Validates: Requirements 12.1, 12.2, 12.3, 12.4 + """ + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + create_object_label(session, "obj_1", video.video_id, "dog", 0.9, 100, 200) + + # Verify service only has session and artifact_repo attributes + # No query state should be stored + # The service should only have methods and the two injected dependencies + assert "session" in dir(global_jump_service) + assert "artifact_repo" in dir(global_jump_service) + + # Verify no state-related attributes exist + state_related_attrs = [ + "last_query", + "last_filter", + "last_label", + "last_kind", + "query_cache", + "filter_state", + "current_position", + ] + for attr in state_related_attrs: + assert not hasattr( + global_jump_service, attr + ), f"Service should not have {attr} attribute" + + +class TestResultChaining: + """Tests for result chaining capability. + + Verifies that using a result's video_id and end_ms as the next starting + point works correctly. Results should be chronologically after the + previous result. + + Property 17: Result Chaining + Validates: Requirements 13.5, 14.2, 14.3 + """ + + def test_result_chaining_within_same_video(self, session, global_jump_service): + """Test chaining results within the same video. + + When using result.end_ms as the next from_ms, the next result + should be chronologically after the previous result. + + Validates: Requirements 13.5 + """ + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + # Create objects at sequential timestamps + create_object_label(session, "obj_1", video.video_id, "dog", 0.9, 100, 200) + create_object_label(session, "obj_2", video.video_id, "dog", 0.9, 300, 400) + create_object_label(session, "obj_3", video.video_id, "dog", 0.9, 500, 600) + create_object_label(session, "obj_4", video.video_id, "dog", 0.9, 700, 800) + + # First query: get first result + result1 = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + from_ms=0, + label="dog", + ) + assert len(result1) == 1 + assert result1[0].artifact_id == "obj_1" + assert result1[0].jump_to.start_ms == 100 + assert result1[0].jump_to.end_ms == 200 + + # Chain: use result1's end_ms as next from_ms + result2 = global_jump_service.jump_next( + kind="object", + from_video_id=result1[0].video_id, + from_ms=result1[0].jump_to.end_ms, + label="dog", + ) + assert len(result2) == 1 + assert result2[0].artifact_id == "obj_2" + assert result2[0].jump_to.start_ms == 300 + # Verify result2 is chronologically after result1 + assert result2[0].jump_to.start_ms > result1[0].jump_to.end_ms + + # Chain again: use result2's end_ms as next from_ms + result3 = global_jump_service.jump_next( + kind="object", + from_video_id=result2[0].video_id, + from_ms=result2[0].jump_to.end_ms, + label="dog", + ) + assert len(result3) == 1 + assert result3[0].artifact_id == "obj_3" + assert result3[0].jump_to.start_ms == 500 + # Verify result3 is chronologically after result2 + assert result3[0].jump_to.start_ms > result2[0].jump_to.end_ms + + # Chain once more + result4 = global_jump_service.jump_next( + kind="object", + from_video_id=result3[0].video_id, + from_ms=result3[0].jump_to.end_ms, + label="dog", + ) + assert len(result4) == 1 + assert result4[0].artifact_id == "obj_4" + assert result4[0].jump_to.start_ms > result3[0].jump_to.end_ms + + def test_result_chaining_across_videos(self, session, global_jump_service): + """Test chaining results across multiple videos. + + When the current video has no more results, chaining should + continue to the next video in the global timeline. + + Validates: Requirements 14.2, 14.3 + """ + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + video3 = create_test_video( + session, "video_3", "video3.mp4", datetime(2025, 1, 3, 12, 0, 0) + ) + + # Create objects in each video + create_object_label(session, "v1_obj_1", video1.video_id, "cat", 0.9, 100, 200) + create_object_label(session, "v1_obj_2", video1.video_id, "cat", 0.9, 500, 600) + create_object_label(session, "v2_obj_1", video2.video_id, "cat", 0.9, 100, 200) + create_object_label(session, "v3_obj_1", video3.video_id, "cat", 0.9, 100, 200) + + # Start from video1 + result1 = global_jump_service.jump_next( + kind="object", + from_video_id=video1.video_id, + from_ms=0, + label="cat", + ) + assert result1[0].video_id == "video_1" + assert result1[0].artifact_id == "v1_obj_1" + + # Chain to next result in video1 + result2 = global_jump_service.jump_next( + kind="object", + from_video_id=result1[0].video_id, + from_ms=result1[0].jump_to.end_ms, + label="cat", + ) + assert result2[0].video_id == "video_1" + assert result2[0].artifact_id == "v1_obj_2" + + # Chain to video2 (no more results in video1 after end_ms=600) + result3 = global_jump_service.jump_next( + kind="object", + from_video_id=result2[0].video_id, + from_ms=result2[0].jump_to.end_ms, + label="cat", + ) + assert result3[0].video_id == "video_2" + assert result3[0].artifact_id == "v2_obj_1" + + # Chain to video3 + result4 = global_jump_service.jump_next( + kind="object", + from_video_id=result3[0].video_id, + from_ms=result3[0].jump_to.end_ms, + label="cat", + ) + assert result4[0].video_id == "video_3" + assert result4[0].artifact_id == "v3_obj_1" + + def test_result_chaining_no_more_results(self, session, global_jump_service): + """Test chaining when no more results exist. + + Validates: Requirements 13.5 + """ + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + # Create only one object + create_object_label(session, "obj_1", video.video_id, "dog", 0.9, 100, 200) + + # Get first result + result1 = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + from_ms=0, + label="dog", + ) + assert len(result1) == 1 + assert result1[0].artifact_id == "obj_1" + + # Chain: should return empty list (no more results) + result2 = global_jump_service.jump_next( + kind="object", + from_video_id=result1[0].video_id, + from_ms=result1[0].jump_to.end_ms, + label="dog", + ) + assert len(result2) == 0 + + def test_result_chaining_with_overlapping_artifacts( + self, session, global_jump_service + ): + """Test chaining with overlapping artifact time ranges. + + When artifacts overlap, using end_ms should skip to the next + artifact that starts after end_ms. + + Validates: Requirements 13.5 + """ + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + # Create overlapping objects + create_object_label(session, "obj_1", video.video_id, "dog", 0.9, 100, 300) + create_object_label( + session, "obj_2", video.video_id, "dog", 0.9, 200, 400 + ) # Overlaps with obj_1 + create_object_label(session, "obj_3", video.video_id, "dog", 0.9, 500, 600) + + # Get first result + result1 = global_jump_service.jump_next( + kind="object", + from_video_id=video.video_id, + from_ms=0, + label="dog", + ) + assert result1[0].artifact_id == "obj_1" + assert result1[0].jump_to.end_ms == 300 + + # Chain using end_ms=300 should skip obj_2 (starts at 200 < 300) + # and return obj_3 (starts at 500 > 300) + result2 = global_jump_service.jump_next( + kind="object", + from_video_id=result1[0].video_id, + from_ms=result1[0].jump_to.end_ms, + label="dog", + ) + assert len(result2) == 1 + assert result2[0].artifact_id == "obj_3" + assert result2[0].jump_to.start_ms == 500 + + def test_result_chaining_continuous_navigation(self, session, global_jump_service): + """Test continuous navigation through multiple results using chaining. + + Simulates a user clicking "next" repeatedly to navigate through + all occurrences of an object. + + Validates: Requirements 13.5, 14.2, 14.3 + """ + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 12, 0, 0) + ) + + # Create multiple objects across videos + create_object_label(session, "obj_1", video1.video_id, "bird", 0.9, 100, 200) + create_object_label(session, "obj_2", video1.video_id, "bird", 0.9, 400, 500) + create_object_label(session, "obj_3", video1.video_id, "bird", 0.9, 800, 900) + create_object_label(session, "obj_4", video2.video_id, "bird", 0.9, 100, 200) + create_object_label(session, "obj_5", video2.video_id, "bird", 0.9, 500, 600) + + # Simulate continuous navigation + current_video_id = video1.video_id + current_ms = 0 + visited_artifacts = [] + + for _ in range(10): # Max iterations to prevent infinite loop + results = global_jump_service.jump_next( + kind="object", + from_video_id=current_video_id, + from_ms=current_ms, + label="bird", + ) + + if not results: + break + + result = results[0] + visited_artifacts.append(result.artifact_id) + + # Chain to next position + current_video_id = result.video_id + current_ms = result.jump_to.end_ms + + # Should have visited all 5 artifacts in order + assert visited_artifacts == ["obj_1", "obj_2", "obj_3", "obj_4", "obj_5"] + + def test_result_chaining_prev_direction(self, session, global_jump_service): + """Test result chaining in the prev direction. + + When navigating backward, using start_ms as from_ms should + return the previous result. + + Validates: Requirements 13.5 + """ + video = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + + # Create objects at sequential timestamps + create_object_label(session, "obj_1", video.video_id, "dog", 0.9, 100, 200) + create_object_label(session, "obj_2", video.video_id, "dog", 0.9, 300, 400) + create_object_label(session, "obj_3", video.video_id, "dog", 0.9, 500, 600) + + # Start from end of video + result1 = global_jump_service.jump_prev( + kind="object", + from_video_id=video.video_id, + from_ms=1000, + label="dog", + ) + assert result1[0].artifact_id == "obj_3" + + # Chain backward using start_ms + result2 = global_jump_service.jump_prev( + kind="object", + from_video_id=result1[0].video_id, + from_ms=result1[0].jump_to.start_ms, + label="dog", + ) + assert result2[0].artifact_id == "obj_2" + assert result2[0].jump_to.start_ms < result1[0].jump_to.start_ms + + # Chain backward again + result3 = global_jump_service.jump_prev( + kind="object", + from_video_id=result2[0].video_id, + from_ms=result2[0].jump_to.start_ms, + label="dog", + ) + assert result3[0].artifact_id == "obj_1" + assert result3[0].jump_to.start_ms < result2[0].jump_to.start_ms + + def test_result_chaining_preserves_chronological_order( + self, session, global_jump_service + ): + """Test that chained results maintain strict chronological order. + + Each subsequent result must be strictly after the previous one + in the global timeline. + + Validates: Requirements 13.5, 14.2, 14.3 + """ + # Create videos with specific timestamps + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 10, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 1, 12, 0, 0) + ) + video3 = create_test_video( + session, "video_3", "video3.mp4", datetime(2025, 1, 1, 14, 0, 0) + ) + + # Create objects + create_object_label(session, "obj_1", video1.video_id, "car", 0.9, 1000, 1100) + create_object_label(session, "obj_2", video2.video_id, "car", 0.9, 500, 600) + create_object_label(session, "obj_3", video3.video_id, "car", 0.9, 200, 300) + + # Navigate through all results + results = [] + current_video_id = video1.video_id + current_ms = 0 + + for _ in range(5): + r = global_jump_service.jump_next( + kind="object", + from_video_id=current_video_id, + from_ms=current_ms, + label="car", + ) + if not r: + break + results.append(r[0]) + current_video_id = r[0].video_id + current_ms = r[0].jump_to.end_ms + + # Verify chronological order + assert len(results) == 3 + assert results[0].video_id == "video_1" + assert results[1].video_id == "video_2" + assert results[2].video_id == "video_3" + + # Verify file_created_at ordering + for i in range(len(results) - 1): + current = results[i] + next_result = results[i + 1] + # Next result should be in a video with same or later file_created_at + if current.file_created_at and next_result.file_created_at: + assert next_result.file_created_at >= current.file_created_at + + +class TestCrossVideoNavigationCorrectness: + """Tests for cross-video navigation correctness. + + Property 18: Cross-Video Navigation + For any global jump query that returns a result from a different video + than from_video_id, that result should be the first matching artifact + in the next/previous video in the global timeline (based on direction). + + Validates: Requirements 14.1, 14.5 + """ + + def test_next_returns_first_artifact_in_next_video( + self, session, global_jump_service + ): + """Test 'next' returns the first matching artifact in the next video. + + When navigating to a different video, the result should be the first + (earliest start_ms) matching artifact in that video, not any arbitrary + match. + + Validates: Requirements 14.1, 14.5 + """ + # Create videos in chronological order + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 10, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 10, 0, 0) + ) + + # Create objects in video1 + create_object_label(session, "v1_obj_1", video1.video_id, "dog", 0.9, 100, 200) + + # Create multiple objects in video2 at different timestamps + # The first one (earliest start_ms) should be returned + create_object_label( + session, "v2_obj_3", video2.video_id, "dog", 0.9, 3000, 3100 + ) # Latest + create_object_label( + session, "v2_obj_1", video2.video_id, "dog", 0.9, 500, 600 + ) # First (earliest) + create_object_label( + session, "v2_obj_2", video2.video_id, "dog", 0.9, 1500, 1600 + ) # Middle + + # Search from end of video1 - should get first artifact in video2 + results = global_jump_service.jump_next( + kind="object", + from_video_id=video1.video_id, + from_ms=5000, # After all objects in video1 + label="dog", + ) + + assert len(results) == 1 + assert results[0].video_id == "video_2" + # Should be the first artifact (earliest start_ms) in video2 + assert results[0].artifact_id == "v2_obj_1" + assert results[0].jump_to.start_ms == 500 + + def test_prev_returns_last_artifact_in_previous_video( + self, session, global_jump_service + ): + """Test 'prev' returns the last matching artifact in the previous video. + + When navigating backward to a different video, the result should be the + last (latest start_ms) matching artifact in that video. + + Validates: Requirements 14.1, 14.5 + """ + # Create videos in chronological order + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 10, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 10, 0, 0) + ) + + # Create multiple objects in video1 at different timestamps + # The last one (latest start_ms) should be returned when going backward + create_object_label( + session, "v1_obj_1", video1.video_id, "cat", 0.9, 100, 200 + ) # First + create_object_label( + session, "v1_obj_2", video1.video_id, "cat", 0.9, 1000, 1100 + ) # Middle + create_object_label( + session, "v1_obj_3", video1.video_id, "cat", 0.9, 2000, 2100 + ) # Last (latest) + + # Create object in video2 + create_object_label(session, "v2_obj_1", video2.video_id, "cat", 0.9, 500, 600) + + # Search from beginning of video2 - should get last artifact in video1 + results = global_jump_service.jump_prev( + kind="object", + from_video_id=video2.video_id, + from_ms=0, # Before all objects in video2 + label="cat", + ) + + assert len(results) == 1 + assert results[0].video_id == "video_1" + # Should be the last artifact (latest start_ms) in video1 + assert results[0].artifact_id == "v1_obj_3" + assert results[0].jump_to.start_ms == 2000 + + def test_cross_video_with_multiple_videos_different_orders( + self, session, global_jump_service + ): + """Test cross-video navigation with videos in different chronological orders. + + Videos are created in non-chronological order to verify that the service + correctly orders by file_created_at, not by insertion order. + + Validates: Requirements 14.1, 14.5 + """ + # Create videos in non-chronological order (by insertion) + video3 = create_test_video( + session, "video_c", "video_c.mp4", datetime(2025, 1, 3, 10, 0, 0) + ) + video1 = create_test_video( + session, "video_a", "video_a.mp4", datetime(2025, 1, 1, 10, 0, 0) + ) + video2 = create_test_video( + session, "video_b", "video_b.mp4", datetime(2025, 1, 2, 10, 0, 0) + ) + + # Create objects in each video + create_object_label( + session, "v3_obj", video3.video_id, "bird", 0.9, 100, 200 + ) # Jan 3 + create_object_label( + session, "v1_obj", video1.video_id, "bird", 0.9, 100, 200 + ) # Jan 1 + create_object_label( + session, "v2_obj", video2.video_id, "bird", 0.9, 100, 200 + ) # Jan 2 + + # Navigate from video1 (Jan 1) - should go to video2 (Jan 2), not video3 + results = global_jump_service.jump_next( + kind="object", + from_video_id=video1.video_id, + from_ms=500, # After the object in video1 + label="bird", + ) + + assert len(results) == 1 + assert results[0].video_id == "video_b" # Jan 2, not Jan 3 + assert results[0].artifact_id == "v2_obj" + + # Continue to video3 + results2 = global_jump_service.jump_next( + kind="object", + from_video_id=results[0].video_id, + from_ms=results[0].jump_to.end_ms, + label="bird", + ) + + assert len(results2) == 1 + assert results2[0].video_id == "video_c" # Jan 3 + assert results2[0].artifact_id == "v3_obj" + + def test_cross_video_video_id_matches_expected(self, session, global_jump_service): + """Test that video_id in results matches the expected video. + + Validates: Requirements 14.1, 14.5 + """ + # Create videos with distinct IDs + video_alpha = create_test_video( + session, "alpha_video", "alpha.mp4", datetime(2025, 1, 1, 10, 0, 0) + ) + video_beta = create_test_video( + session, "beta_video", "beta.mp4", datetime(2025, 1, 2, 10, 0, 0) + ) + video_gamma = create_test_video( + session, "gamma_video", "gamma.mp4", datetime(2025, 1, 3, 10, 0, 0) + ) + + # Create objects + create_object_label( + session, "alpha_obj", video_alpha.video_id, "car", 0.9, 100, 200 + ) + create_object_label( + session, "beta_obj", video_beta.video_id, "car", 0.9, 100, 200 + ) + create_object_label( + session, "gamma_obj", video_gamma.video_id, "car", 0.9, 100, 200 + ) + + # Navigate forward and verify video_id at each step + result1 = global_jump_service.jump_next( + kind="object", + from_video_id=video_alpha.video_id, + from_ms=500, + label="car", + ) + assert result1[0].video_id == "beta_video" + assert result1[0].video_filename == "beta.mp4" + + result2 = global_jump_service.jump_next( + kind="object", + from_video_id=result1[0].video_id, + from_ms=result1[0].jump_to.end_ms, + label="car", + ) + assert result2[0].video_id == "gamma_video" + assert result2[0].video_filename == "gamma.mp4" + + # Navigate backward and verify + result3 = global_jump_service.jump_prev( + kind="object", + from_video_id=video_gamma.video_id, + from_ms=0, + label="car", + ) + assert result3[0].video_id == "beta_video" + + result4 = global_jump_service.jump_prev( + kind="object", + from_video_id=result3[0].video_id, + from_ms=0, + label="car", + ) + assert result4[0].video_id == "alpha_video" + + def test_cross_video_maintains_filter_across_boundaries( + self, session, global_jump_service + ): + """Test that search filter is maintained when crossing video boundaries. + + When navigating to a different video, only artifacts matching the + original filter should be returned. + + Validates: Requirements 14.5 + """ + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 10, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 10, 0, 0) + ) + + # Create objects in video1 + create_object_label(session, "v1_dog", video1.video_id, "dog", 0.9, 100, 200) + + # Create objects in video2 - mix of dog and cat + create_object_label( + session, "v2_cat", video2.video_id, "cat", 0.9, 100, 200 + ) # First by time + create_object_label( + session, "v2_dog", video2.video_id, "dog", 0.9, 500, 600 + ) # Second by time + + # Search for "dog" from end of video1 + results = global_jump_service.jump_next( + kind="object", + from_video_id=video1.video_id, + from_ms=500, + label="dog", + ) + + assert len(results) == 1 + assert results[0].video_id == "video_2" + # Should return the dog, not the cat (even though cat is earlier) + assert results[0].artifact_id == "v2_dog" + assert results[0].preview["label"] == "dog" + + def test_cross_video_with_same_file_created_at(self, session, global_jump_service): + """Test cross-video navigation when videos have the same file_created_at. + + When videos have the same file_created_at, ordering should fall back + to video_id for deterministic results. + + Validates: Requirements 14.1, 14.5 + """ + same_time = datetime(2025, 1, 1, 12, 0, 0) + + # Create videos with same file_created_at but different video_ids + video_a = create_test_video(session, "aaa_video", "aaa.mp4", same_time) + video_b = create_test_video(session, "bbb_video", "bbb.mp4", same_time) + video_c = create_test_video(session, "ccc_video", "ccc.mp4", same_time) + + # Create objects in each video + create_object_label(session, "a_obj", video_a.video_id, "fish", 0.9, 100, 200) + create_object_label(session, "b_obj", video_b.video_id, "fish", 0.9, 100, 200) + create_object_label(session, "c_obj", video_c.video_id, "fish", 0.9, 100, 200) + + # Navigate from video_a - should go to video_b (alphabetically next) + results = global_jump_service.jump_next( + kind="object", + from_video_id=video_a.video_id, + from_ms=500, + label="fish", + ) + + assert len(results) == 1 + assert results[0].video_id == "bbb_video" + + # Continue to video_c + results2 = global_jump_service.jump_next( + kind="object", + from_video_id=results[0].video_id, + from_ms=results[0].jump_to.end_ms, + label="fish", + ) + + assert len(results2) == 1 + assert results2[0].video_id == "ccc_video" + + def test_cross_video_returns_first_artifact_with_multiple_matches( + self, session, global_jump_service + ): + """Test that with multiple matches in next video, the first one is returned. + + This is a more comprehensive test with many artifacts to ensure + the ordering is correct. + + Validates: Requirements 14.1, 14.5 + """ + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 10, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 10, 0, 0) + ) + + # Create one object in video1 + create_object_label(session, "v1_obj", video1.video_id, "person", 0.9, 100, 200) + + # Create many objects in video2 at various timestamps (inserted in random order) + timestamps = [5000, 1000, 3000, 500, 2000, 4000, 100] # 100 is the earliest + for i, ts in enumerate(timestamps): + create_object_label( + session, + f"v2_obj_{ts}", + video2.video_id, + "person", + 0.9, + ts, + ts + 100, + ) + + # Search from end of video1 + results = global_jump_service.jump_next( + kind="object", + from_video_id=video1.video_id, + from_ms=500, + label="person", + ) + + assert len(results) == 1 + assert results[0].video_id == "video_2" + # Should return the artifact with the earliest start_ms (100) + assert results[0].artifact_id == "v2_obj_100" + assert results[0].jump_to.start_ms == 100 + + def test_cross_video_prev_returns_last_artifact_with_multiple_matches( + self, session, global_jump_service + ): + """Test backward navigation returns the last artifact in the previous video. + + Validates: Requirements 14.1, 14.5 + """ + video1 = create_test_video( + session, "video_1", "video1.mp4", datetime(2025, 1, 1, 10, 0, 0) + ) + video2 = create_test_video( + session, "video_2", "video2.mp4", datetime(2025, 1, 2, 10, 0, 0) + ) + + # Create many objects in video1 at various timestamps + timestamps = [100, 500, 1000, 2000, 3000, 4000, 5000] # 5000 is the latest + for ts in timestamps: + create_object_label( + session, + f"v1_obj_{ts}", + video1.video_id, + "tree", + 0.9, + ts, + ts + 100, + ) + + # Create one object in video2 + create_object_label(session, "v2_obj", video2.video_id, "tree", 0.9, 100, 200) + + # Search backward from beginning of video2 + results = global_jump_service.jump_prev( + kind="object", + from_video_id=video2.video_id, + from_ms=0, + label="tree", + ) + + assert len(results) == 1 + assert results[0].video_id == "video_1" + # Should return the artifact with the latest start_ms (5000) + assert results[0].artifact_id == "v1_obj_5000" + assert results[0].jump_to.start_ms == 5000 diff --git a/frontend/src/components/MetadataViewer.test.tsx b/frontend/src/components/MetadataViewer.test.tsx index 6b84b8b..c22eeb7 100644 --- a/frontend/src/components/MetadataViewer.test.tsx +++ b/frontend/src/components/MetadataViewer.test.tsx @@ -3,7 +3,8 @@ import { describe, it, expect, beforeEach, vi } from 'vitest'; import MetadataViewer from './MetadataViewer'; // Mock fetch -global.fetch = vi.fn(); +const mockFetch = vi.fn(); +globalThis.fetch = mockFetch; describe('MetadataViewer', () => { beforeEach(() => { @@ -11,7 +12,7 @@ describe('MetadataViewer', () => { }); it('displays loading state initially', () => { - (global.fetch as any).mockImplementation( + mockFetch.mockImplementation( () => new Promise(() => {}) // Never resolves ); @@ -20,7 +21,7 @@ describe('MetadataViewer', () => { }); it('displays error message on fetch failure', async () => { - (global.fetch as any).mockRejectedValue(new Error('Network error')); + mockFetch.mockRejectedValue(new Error('Network error')); render(); @@ -30,7 +31,7 @@ describe('MetadataViewer', () => { }); it('displays no metadata message when empty', async () => { - (global.fetch as any).mockResolvedValue({ + mockFetch.mockResolvedValue({ json: async () => [], }); @@ -42,7 +43,7 @@ describe('MetadataViewer', () => { }); it('displays GPS coordinates in user-friendly format', async () => { - (global.fetch as any).mockResolvedValue({ + mockFetch.mockResolvedValue({ json: async () => [ { artifact_id: 'artifact_001', @@ -64,7 +65,7 @@ describe('MetadataViewer', () => { }); it('displays camera information', async () => { - (global.fetch as any).mockResolvedValue({ + mockFetch.mockResolvedValue({ json: async () => [ { artifact_id: 'artifact_001', @@ -87,7 +88,7 @@ describe('MetadataViewer', () => { }); it('displays file information', async () => { - (global.fetch as any).mockResolvedValue({ + mockFetch.mockResolvedValue({ json: async () => [ { artifact_id: 'artifact_001', @@ -116,7 +117,7 @@ describe('MetadataViewer', () => { }); it('displays temporal information', async () => { - (global.fetch as any).mockResolvedValue({ + mockFetch.mockResolvedValue({ json: async () => [ { artifact_id: 'artifact_001', @@ -141,7 +142,7 @@ describe('MetadataViewer', () => { }); it('displays image information', async () => { - (global.fetch as any).mockResolvedValue({ + mockFetch.mockResolvedValue({ json: async () => [ { artifact_id: 'artifact_001', @@ -167,7 +168,7 @@ describe('MetadataViewer', () => { }); it('displays bitrate information', async () => { - (global.fetch as any).mockResolvedValue({ + mockFetch.mockResolvedValue({ json: async () => [ { artifact_id: 'artifact_001', @@ -187,7 +188,7 @@ describe('MetadataViewer', () => { }); it('handles missing fields gracefully', async () => { - (global.fetch as any).mockResolvedValue({ + mockFetch.mockResolvedValue({ json: async () => [ { artifact_id: 'artifact_001', @@ -212,7 +213,7 @@ describe('MetadataViewer', () => { }); it('uses custom API URL', async () => { - (global.fetch as any).mockResolvedValue({ + mockFetch.mockResolvedValue({ json: async () => [ { artifact_id: 'artifact_001', @@ -226,14 +227,14 @@ describe('MetadataViewer', () => { render(); await waitFor(() => { - expect(global.fetch).toHaveBeenCalledWith( + expect(mockFetch).toHaveBeenCalledWith( 'http://custom-api:8080/api/v1/videos/video_001/artifacts?type=video.metadata' ); }); }); it('displays all metadata sections when all fields are present', async () => { - (global.fetch as any).mockResolvedValue({ + mockFetch.mockResolvedValue({ json: async () => [ { artifact_id: 'artifact_001',