Skip to content

Latest commit

 

History

History
156 lines (113 loc) · 4.98 KB

README.md

File metadata and controls

156 lines (113 loc) · 4.98 KB

Preparing AVA

Introduction

[DATASET]

@inproceedings{gu2018ava,
  title={Ava: A video dataset of spatio-temporally localized atomic visual actions},
  author={Gu, Chunhui and Sun, Chen and Ross, David A and Vondrick, Carl and Pantofaru, Caroline and Li, Yeqing and Vijayanarasimhan, Sudheendra and Toderici, George and Ricco, Susanna and Sukthankar, Rahul and others},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  pages={6047--6056},
  year={2018}
}

For basic dataset information, please refer to the official website. Before we start, please make sure that the directory is located at $MMACTION2/tools/data/ava/.

Step 1. Prepare Annotations

First of all, you can run the following script to prepare annotations.

bash download_annotations.sh

This command will download ava_v2.1.zip for AVA v2.1 annotation. If you need the AVA v2.2 annotation, you can try the following script.

VERSION=2.2 bash download_annotations.sh

Step 2. Prepare Videos

Then, use the following script to prepare videos. The codes are adapted from the official crawler. Note that this might take a long time.

bash download_videos.sh

Or you can use the following command to downloading AVA videos in parallel using a python script.

bash download_videos_parallel.sh

Note that if you happen to have sudoer or have GNU parallel on your machine, you can speed up the procedure by downloading in parallel.

# sudo apt-get install parallel
bash download_videos_gnu_parallel.sh

Step 3. Cut Videos

Cut each video from its 15th to 30th minute and make them at 30 fps.

bash cut_videos.sh

Step 4. Extract RGB and Flow

This part is optional if you only want to use the video loader.

Before extracting, please refer to install.md for installing denseflow.

If you have plenty of SSD space, then we recommend extracting frames there for better I/O performance. And you can run the following script to soft link the extracted frames.

# execute these two line (Assume the SSD is mounted at "/mnt/SSD/")
mkdir /mnt/SSD/ava_extracted/
ln -s /mnt/SSD/ava_extracted/ ../data/ava/rawframes/

If you only want to play with RGB frames (since extracting optical flow can be time-consuming), consider running the following script to extract RGB-only frames using denseflow.

bash extract_rgb_frames.sh

If you didn't install denseflow, you can still extract RGB frames using OpenCV by the following script, but it will keep the original size of the images.

bash extract_rgb_frames_opencv.sh

Or using ffmpeg to extract RGB frames by the following script.

bash extract_rgb_frames_ffmpeg.sh

If both are required, run the following script to extract frames.

bash extract_frames.sh

Step 5. Fetching Proposal Files

The scripts are adapted from FAIR's Long-Term Feature Banks.

Run the follow scripts to fetch pre-computed proposal list.

bash fetch_ava_proposals.sh

Step 6. Folder Structure

After the whole data pipeline for AVA preparation. you can get the rawframes (RGB + Flow), videos and annotation files for AVA.

In the context of the whole project (for AVA only), the minimal folder structure will look like: (minimal means that some data are not necessary: for example, you may want to evaluate AVA using the original video format.)

mmaction2
├── mmaction
├── tools
├── configs
├── data
│   ├── ava
│   │   ├── annotations
│   │   |   ├── ava_dense_proposals_train.FAIR.recall_93.9.pkl
│   │   |   ├── ava_dense_proposals_val.FAIR.recall_93.9.pkl
│   │   |   ├── ava_dense_proposals_test.FAIR.recall_93.9.pkl
│   │   |   ├── ava_train_v2.1.csv
│   │   |   ├── ava_val_v2.1.csv
│   │   |   ├── ava_train_excluded_timestamps_v2.1.csv
│   │   |   ├── ava_val_excluded_timestamps_v2.1.csv
│   │   |   ├── ava_action_list_v2.1_for_activitynet_2018.pbtxt
│   │   ├── videos
│   │   │   ├── 053oq2xB3oU.mkv
│   │   │   ├── 0f39OWEqJ24.mp4
│   │   │   ├── ...
│   │   ├── videos_15min
│   │   │   ├── 053oq2xB3oU.mkv
│   │   │   ├── 0f39OWEqJ24.mp4
│   │   │   ├── ...
│   │   ├── rawframes
│   │   │   ├── 053oq2xB3oU
|   │   │   │   ├── img_00001.jpg
|   │   │   │   ├── img_00002.jpg
|   │   │   │   ├── ...

For training and evaluating on AVA, please refer to getting_started.

Reference

  1. O. Tange (2018): GNU Parallel 2018, March 2018, https://doi.org/10.5281/zenodo.1146014