Skip to content

Commit

Permalink
[Feature] Support mim download odl datasets (#2436)
Browse files Browse the repository at this point in the history
  • Loading branch information
Tau-J authored Jun 29, 2023
1 parent fe4dec7 commit db34546
Show file tree
Hide file tree
Showing 19 changed files with 702 additions and 254 deletions.
1 change: 1 addition & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
include requirements/*.txt
include mmpose/.mim/model-index.yml
include mmpose/.mim/dataset-index.yml
recursive-include mmpose/.mim/configs *.py *.yml
recursive-include mmpose/.mim/tools *.py *.sh
recursive-include mmpose/.mim/demo *.py
71 changes: 71 additions & 0 deletions dataset-index.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
coco2017:
dataset: COCO_2017
download_root: data
data_root: data/pose
script: tools/dataset_converters/scripts/preprocess_coco2017.sh

mpii:
dataset: MPII_Human_Pose
download_root: data
data_root: data/pose
script: tools/dataset_converters/scripts/preprocess_mpii.sh

aic:
dataset: AI_Challenger
download_root: data
data_root: data/pose
script: tools/dataset_converters/scripts/preprocess_aic.sh

crowdpose:
dataset: CrowdPose
download_root: data
data_root: data/pose
script: tools/dataset_converters/scripts/preprocess_crowdpose.sh

halpe:
dataset: Halpe
download_root: data
data_root: data/pose
script: tools/dataset_converters/scripts/preprocess_halpe.sh

lapa:
dataset: LaPa
download_root: data
data_root: data/pose
script: tools/dataset_converters/scripts/preprocess_lapa.sh

300w:
dataset: 300w
download_root: data
data_root: data/pose
script: tools/dataset_converters/scripts/preprocess_300w.sh

wflw:
dataset: WFLW
download_root: data
data_root: data/pose
script: tools/dataset_converters/scripts/preprocess_wflw.sh

onehand10k:
dataset: OneHand10K
download_root: data
data_root: data/pose
script: tools/dataset_converters/scripts/preprocess_onehand10k.sh

freihand:
dataset: FreiHAND
download_root: data
data_root: data/pose
script: tools/dataset_converters/scripts/preprocess_freihand.sh

ap10k:
dataset: AP-10K
download_root: data
data_root: data/pose
script: tools/dataset_converters/scripts/preprocess_ap10k.sh

hagrid:
dataset: HaGRID
download_root: data
data_root: data/pose
script: tools/dataset_converters/scripts/preprocess_hagrid.sh
2 changes: 1 addition & 1 deletion docs/en/advanced_guides/customize_datasets.md
Original file line number Diff line number Diff line change
Expand Up @@ -256,4 +256,4 @@ combined_dataset = dict(

- **Converter transforms of sub-datasets** are applied when there exist mismatches of annotation format between sub-datasets and the combined dataset. For example, the number and order of keypoints might be different in the combined dataset and the sub-datasets. Then `KeypointConverter` can be used to unify the keypoints number and order.

- More details about `CombinedDataset` and `KeypointConverter` can be found in Advanced Guides-[Training with Mixed Datasets](../advanced_guides/mixed_datasets.md).
- More details about `CombinedDataset` and `KeypointConverter` can be found in Advanced Guides-[Training with Mixed Datasets](../user_guides/mixed_datasets.md).
66 changes: 65 additions & 1 deletion docs/en/user_guides/prepare_datasets.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Prepare Datasets

In this document, we will give a guide on the process of preparing datasets for the MMPose. Various aspects of dataset preparation will be discussed, including using built-in datasets, creating custom datasets, combining datasets for training, and browsing the dataset.
In this document, we will give a guide on the process of preparing datasets for the MMPose. Various aspects of dataset preparation will be discussed, including using built-in datasets, creating custom datasets, combining datasets for training, browsing and downloading the datasets.

## Use built-in datasets

Expand Down Expand Up @@ -155,3 +155,67 @@ Here is a processed sample
![transformed_coco](https://user-images.githubusercontent.com/26127467/187386652-bd47335d-797c-4e8c-b823-2a4915f9812f.jpg)

The heatmap target will be visualized together if it is generated in the pipeline.

## Download dataset via MIM

By using [OpenDataLab](https://opendatalab.com/), you can obtain free formatted datasets in various fields. Through the search function of the platform, you may address the dataset they look for quickly and easily. Using the formatted datasets from the platform, you can efficiently conduct tasks across datasets.

If you use MIM to download, make sure that the version is greater than v0.3.8. You can use the following command to update, install, login and download the dataset:

```shell
# upgrade your MIM
pip install -U openmim

# install OpenDataLab CLI tools
pip install -U opendatalab
# log in OpenDataLab, registry
odl login

# download coco2017 and preprocess by MIM
mim download mmpose --dataset coco2017
```

### Supported datasets

Here is the list of supported datasets, we will continue to update it in the future.

#### Body

| Dataset name | Download command |
| ------------- | ----------------------------------------- |
| COCO 2017 | `mim download mmpose --dataset coco2017` |
| MPII | `mim download mmpose --dataset mpii` |
| AI Challenger | `mim download mmpose --dataset aic` |
| CrowdPose | `mim download mmpose --dataset crowdpose` |

#### Face

| Dataset name | Download command |
| ------------ | ------------------------------------ |
| LaPa | `mim download mmpose --dataset lapa` |
| 300W | `mim download mmpose --dataset 300w` |
| WFLW | `mim download mmpose --dataset wflw` |

#### Hand

| Dataset name | Download command |
| ------------ | ------------------------------------------ |
| OneHand10K | `mim download mmpose --dataset onehand10k` |
| FreiHand | `mim download mmpose --dataset freihand` |
| HaGRID | `mim download mmpose --dataset hagrid` |

#### Whole Body

| Dataset name | Download command |
| ------------ | ------------------------------------- |
| Halpe | `mim download mmpose --dataset halpe` |

#### Animal

| Dataset name | Download command |
| ------------ | ------------------------------------- |
| AP-10K | `mim download mmpose --dataset ap10k` |

#### Fashion

Coming Soon
Loading

0 comments on commit db34546

Please sign in to comment.