Crowd Behavior Recognition

Sam Adams, Matt Keller, and Tom McClenahan

Directory Structure

• src: Contains all code files
• results: Contains plots and output generated by the code
• data:

  • images/categorized

    • activity: Contains the original dataset, sorted into two subdirectories according to the activity (idle or cheering)

    • density: Contains the original dataset, sorted into two subdirectories according to the density level (dense or sparse)

    • activity_density: Contains the original dataset, sorted into three subdirectories according to both the activity and density.

  • images/sampleImages: Contains the large, general dataset of images. All images are cropped and scaled to a resolution of 640x480.

  • images/hand_classifier: Contains the image samples and classifiers used for identifying hands.

  • images/hand_sampleImages: Contains the images that were used to create the samples for the hand classifier. These images are unscaled from their original sizes.

  • videos: Contains a single video that can be analyzed with video.py.

Dependencies

A working installation of OpenCV for Python is required. To run the video analysis script 'video.py', the scikit-video Python module is requred.

Matlab is required to run the .m files. (Command line or GUI)

Python is required to run all .py files.

Running

First, make sure the python files are executable by navigating to 'src/' and running 'chmod +x *.py'

Run './main.py' to execute the main script. This script uses the more simplistic methods of edge detection and skin detection to identify the density and behavior in the image. The script will analyze all images in data/images/sampleImages and print estimates for each image, and overall results.

To run the bag-of-features analysis, run the three Matlab files: bof_behavior.m, bof_density.m, and bof_behavior_density.m. All three of these scripts behave identically, except for the features the use for classification. Upon running the script, it will train a bag of features classifier and identifies the features to distinguish between categories. After the classifier is trained, it will test against the entire image set, and print the confusion matrix. Afterwards, it will also display one image from each category (idle/cheering for bof_behavior, sparse/dense for bof_density, or sparse/idle/cheering for bof_behavior_density).

Run './edit_samples.py' to edit the positive and negative samples for the hand classifier. To identify a positive feature, left-click and drag a box around a feature. To identify a negative feature, right-click and drag a box. To advance forward and backward through images, use the left and right arrow keys. Pressing the 'R' key will reset the features in the current image.

Run './face.py' to run the attempted face detection script.

Run './video.py' to analyze the video in data/video/sample.avi. This script analyzes each frame of the video using the methods of the main script (edge detection and skin detection). It also generates a plot of activity and density over time.