Auto-frame suggestions on video load #1
Description
Summary
On video load, automatically sample frames (evenly and via scene-change detection) and suggest 6–12 “interesting” candidates for the user to pick from.
User Story
As a creator, I want the app to propose interesting frames from my video so I can quickly select good candidates without manual scrubbing.
Scope
- Sample frames evenly across the video duration
- Detect scene-change boundaries using a lightweight heuristic (e.g., HSV histogram delta or SSIM) and include boundary frames
- Produce 6–12 suggested frames (limit configurable)
- Display suggestions as thumbnails in the “Selected frames” area with a distinct “Suggested” badge and an Add button (does not auto-add to the main frames list until user confirms)
- Respect the existing limit of frames sent to the model (currently 3 in API)
Acceptance Criteria
- Given a loaded video, within ~1–3s the UI displays a panel of 6–12 suggested frames
- Suggestions remain stable for the same video on re-open, unless user toggles a different sampling density
- Clicking “Add” moves that frame into the main Selected frames list used by generation
- Performance: memory footprint remains acceptable (no crashes on ~5–10min 1080p videos); revoke Blob URLs when not needed
- Works in latest Chrome/Edge/Safari
Technical Notes
- Implement client-side frame extraction using HTMLVideoElement + OffscreenCanvas/Canvas; seek to timestamps to capture frames (throttle to avoid rapid seeks)
- Scene change heuristic: compute HSV histogram (e.g., 16×16×16 bins or simplified R/G/B) and select frames with high delta from preceding frame; cap to prevent clustering
- Store and display suggestions separately from
framesstate; add a simple cache keyed by file name + size + duration - Code touchpoints: app/page.tsx (frame sampling hooks, state), UI for suggestions grid
- Follow-up: expose a “Show more” to generate additional suggestions on demand