Skip to content

[Refactor] Overhaul Test Suite: Remove Over-Mocking, Integrate Real MediaPipe, Achieve Meaningful Coverage #35

@ambmt

Description

@ambmt

Goal

Objective:
Comprehensively refactor the test suite to eliminate excessive over-mocking (257+ instances), integrate real MediaPipe processing and actual video assets, and achieve 80%+ meaningful coverage. Address all aspects outlined in the executive summary and improvement roadmap, shifting to a progressive, business-logic validating test strategy.

Scope of Changes

  • Remove/replace all instances of over-mocking in tests (Mock, patch, MagicMock) except for true external services (AWS, DB, etc.).
  • Integrate real MediaPipe in all e2e/integration tests. Use assets/test_video.mp4 in place of stubs/synthetics.
  • Replace stubbed/synthetic MediaPipe pose data in:
    • tests/e2e/test_pipeline_synthetic_stubbed.py
    • tests/integration/analyzers/test_analyzer_integration.py
    • tests/worker/analysis/test_video_processing.py
  • Create new tests:
    • tests/e2e/test_real_mediapipe_pipeline.py (real E2E)
    • tests/integration/metrics/test_real_pose_calculations.py (real pose data)
    • tests/performance/test_real_mediapipe_performance.py (real perf tests)
  • Replace mocked OpenCV in tests/worker/test_processor.py with real video processing
  • Update/expand tests for all core calculator/analyzer classes (SideViewAnalyzer, SwingArcCalculator, KinematicSequenceAnalyzer, CentralizedHandicapScoringSystem, etc.) to use real video/MediaPipe data
  • Update pytest.ini with new markers (real_mediapipe, real_video, synthetic_only)
  • Expand performance tests to use real MediaPipe workloads and validate memory/resource usage
  • Add/expand edge case and configuration system tests (lighting, occlusion, dynamic configs)
  • Update documentation to reflect new testing strategy and real usage examples

Acceptance Criteria

  • All over-mocking removed (except for true third-party service boundaries)
  • All e2e/integration tests use real MediaPipe and assets/test_video.mp4
  • New real MediaPipe and real video tests created and passing
  • Core business logic (calculators, analyzers, scoring) validated with real data
  • "Integration" tests truly integrate system components
  • Performance tests cover actual MediaPipe workloads and memory/resource usage
  • Pytest markers added for real_mediapipe, real_video, synthetic_only
  • Coverage quality metrics tracked (real vs. mock)
  • 80%+ coverage achieved including real MediaPipe paths
  • Documentation updated with real test usage examples
  • All tests green in CI

Rollout

  • Refactor tests in manageable phases (see roadmap)
  • Run CI with coverage gating
  • Monitor for uncovered edge cases and integration bugs
  • Update all relevant documentation (README, ARCHITECTURE, CONTRIBUTING)

Metadata

Metadata

Assignees

Labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions