MoLEF

Moment Localization Evaluation Framework
Explore the docs »

View Demo · Report Bug · Request Feature

Table of Contents

1. About The Project
   • Built With
2. Getting Started
   • Prerequisites
   • Installation
3. Contact
4. Acknowledgments

About The Project

MoLEF is a moment localization evaluation framework for video and text multimodal research. MoLEF contains reference implementations of state-of-the-art video and language models. See full list of project inside or built on MoLEF.

MoLEF is powered by PyTorch, allows distributed parallel training and flexible experiments by embedding custom datasets and models.

MoLEF also acts starter codebase for challenges around video and text datasets (ActivityNet-captions, Charades-STA, DiDeMo, TACoS, YouCook2, MSR-VTT, TVR).

The below image is illustrated our framework, MoLEF.

(back to top)

Built With

• Python 3.8
• PyTorch 1.10.0 with CUDA 11.3
• torch-geometric 2.0.3
• fairseq 0.10.2

(back to top)

Getting Started

This is an example of how you may give instructions on setting up your project locally. To get a local copy up and running follow these simple example steps.

Prerequisites - Data

See details in data/README.md.

Installation

Below is an example of how you can install an environment and MoLEF.

1. Clone the repo

   git clone https://github.com/snuviplab/MoLEF.git

2. Install environment

   conda create -n molef python=3.8
   pip install six
   pip install torch==1.10.0+cu113 torchvision==0.11.1+cu113 torchaudio==0.10.0+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html
   pip install torch-scatter torch-sparse torch-cluster torch-spline-conv torch-geometric -f https://data.pyg.org/whl/torch-1.10.0+cu113.html
   pip install -r requirements.txt
   python -m pip install -e. 

Train command

python code/main.py --model model_name --mode train --word2vec-path data/glove.840B.300d.bin --dataset Tacos \
              --feature-path data/tacos/org  --train-data data/tacos/train_data.json \
              --val-data data/tacos/val_data.json  --test-data data/tacos/test_data.json \
              --max-num-epochs 20  --warmup-updates 300 --warmup-init-lr 1e-06 --lr 8e-4 \
              --weight-decay 1e-7 --model-saved-path results/ --cfg code/configs/model_name.yml 

Evaluation command

python code/main.py --model model_name --mode evaluation --word2vec-path  data/glove.840B.300d.bin \
                --dataset Tacos --feature-path  data/tacos/org  --train-data data/tacos/train_data.json \
                --val-data  data/tacos/val_data.json  --test-data  data/tacos/test_data.json --max-num-epochs 20 \
                --warmup-updates 300 --warmup-init-lr 1e-06 --lr 8e-4 --weight-decay 1e-7 --model-saved-path results/ \
                --cfg code/configs/model_name.yml 

(back to top)

Contact

Jinyeong Chae - jiny491@gmail.com

Project Link: https://github.com/snuviplab/MoLEF

(back to top)

Acknowledgments

This project is licenced under the MIT Licence - see the LICENSE.md file for details.

(back to top)