Video Analysis and Object Tracking Techniques

Problem Statement and Goal of Project

This project serves as a practical exploration of various fundamental and advanced techniques in video analysis and object tracking. The primary goal is not to achieve state-of-the-art metrics, but to implement, visualize, and understand the mechanics of different algorithms available in computer vision, particularly within the OpenCV library. It showcases the implementation of different methods for motion estimation, foreground detection, and single/multi-object tracking.

---

Solution Approach

The project is a Jupyter Notebook containing several distinct modules, each demonstrating a different CV concept:

• Optical Flow:
  • Lucas-Kanade (Sparse): Implements sparse optical flow to track a set of detected Shi-Tomasi corner features between frames in a traffic video (videos/slow_traffic_small.mp4).
  • Farneback (Dense): Implements dense optical flow to calculate the motion vectors for all pixels in a scene, visualizing the flow field using an HSV color map. This is applied to both a pre-recorded video (videos/vtest.avi) and a live webcam feed.
• Background Subtraction:
  • Demonstrates foreground-background segmentation using cv2.createBackgroundSubtractorMOG2.
  • Includes an enhanced example that applies morphological operations (erosion and dilation) to reduce noise and refine the resulting foreground mask.
• Single Object Tracking:
  • Meanshift & Camshift: Implements histogram-based tracking to follow a specific object (a car) based on its color profile in the HSV space. This demonstrates tracking that adapts (Camshift) and doesn't adapt (Meanshift) its search window.
  • OpenCV Tracker Class (GOTURN): Utilizes the dedicated cv2.TrackerGOTURN_create() class, a Deep Learning-based tracker, to follow a race car (videos/race_car.mp4), demonstrating a more robust tracking method.
• Multi-Object Tracking:
  • Implements cv2.MultiTracker_create() to track several objects (soccer players in videos/soccer_posession.mp4) simultaneously, using cv2.legacy_TrackerCSRT.create() as the underlying algorithm for each object.

---

Technologies & Libraries

• Python
• OpenCV (cv2): The core library used for all computer vision tasks (cv2.VideoCapture, cv2.calcOpticalFlowPyrLK, cv2.calcOpticalFlowFarneback, cv2.createBackgroundSubtractorMOG2, cv2.meanShift, cv2.CamShift, cv2.TrackerGOTURN_create, cv2.MultiTracker_create, cv2.legacy_TrackerCSRT.create).
• OpenCV-Contrib: Required for the advanced tracker modules (like GOTURN and CSRT).
• NumPy: For numerical operations and array manipulation.
• Matplotlib: Used for displaying a sample frame with a bounding box.
• Jupyter Notebook: For organizing and presenting the code and analysis.

---

Description about Dataset

This project uses several video files for demonstration, sourced from online CV resources:

• videos/slow_traffic_small.mp4: Used for Lucas-Kanade, Meanshift, and Camshift.
• videos/vtest.avi: Used for Dense Optical Flow and Background Subtraction.
• videos/race_car.mp4: Used for the GOTURN single-object tracker.
• videos/soccer_posession.mp4: Used for the multi-object tracker.
• The notebook also includes implementations that can run on a live webcam feed (cv2.VideoCapture(0)).

---

Installation & Execution Guide

1. Clone the repository.
2. Ensure you have Python, NumPy, and Matplotlib installed.
3. This project requires the main OpenCV library and the contrib modules. Install them using pip:

   pip install opencv-python
   pip install opencv-contrib-python

4. Run the Jupyter Notebook:

   jupyter notebook "video analyze and tracking.ipynb"

5. Execute the cells sequentially to see each implementation. Some code blocks are set to use a webcam; others read from the included video files.

---

Key Results / Performance

As the project's focus is on implementation and conceptual understanding, formal performance metrics (e.g., accuracy, MOTA) are not calculated. The key result is the successful, real-time visualization of each tracking and analysis algorithm, demonstrating a practical understanding of their application and behavior.

---

Sample Output

The notebook generates live video windows displaying the following outputs:

• Sparse Optical Flow: A video feed with feature points (dots) and motion trails (lines) drawn on moving objects.
• Dense Optical Flow: A color-coded (HSV) video feed where color and brightness represent the direction and magnitude of motion, respectively.
• Background Subtraction: A black-and-white video showing only the moving foreground objects (e.g., people walking) as white silhouettes.
• Meanshift/Camshift: A video feed with a bounding box that follows a designated car.
• GOTURN/Multi-Tracker: Video output showing bounding boxes successfully tracking single (race car) or multiple (soccer players) objects.

---

Additional Learnings / Reflections

This project provided a deep dive into the practical differences between various tracking methodologies:

• Sparse vs. Dense Optical Flow: A key takeaway was the difference in output and use-case between cv2.calcOpticalFlowPyrLK and cv2.calcOpticalFlowFarneback.
  • PyrLK (Sparse) tracks a list of specific points, returning their new coordinates and a status (success/failure) for each.
  • Farneback (Dense) calculates a motion vector ($dx$, $dy$) for every pixel, returning a full vector field that describes the entire scene's movement.
• Tracker Selection: The project implemented a range of trackers, from simple color-based methods (Meanshift) to advanced Deep Learning models (GOTURN) and robust modern algorithms (CSRT), highlighting the trade-offs between speed, accuracy, and robustness to occlusion.

---

💡 Some interactive outputs (e.g., plots, widgets) may not display correctly on GitHub. If so, please view this notebook via nbviewer.org for full rendering.

🙏 Acknowledgments

This project represents my initial steps into the practical application of computer vision. The foundational knowledge and guidance for this work were derived from the outstanding OpenCV course taught by Alireza Akhavanpour on the Maktabkhooneh platform. His ability to deconstruct complex topics into clear, actionable steps was instrumental in the successful implementation of this project.

👤 Author

Mehran Asgari

Email: imehranasgari@gmail.com.

GitHub: https://github.com/imehranasgari.

📄 License

This project is licensed under the Apache 2.0 License – see the LICENSE file for details.

---