Rethinking Gradient Sparsification as Total error minimization

Deep Neural Network experiments

Our DNN experiments consist of three tasks: Image Classification using CNNs, Language Modelling using LSTMs, and Recommendation using NCF. Image Classification and Language Modelling experiments are in the cnn-lstm directory, and Recommendation experiment is in the ncf directory.

Logistic Regression experiments

Our logistic regression experiment is implemented in the logistic-regression directory.

Create the Conda environment

To install the necessary dependencies, use the provided environment.yml to create a Conda envrironment by running the following command.

$ conda env create --prefix ./env --file environment.yml

Once the new environment has been created you can activate the environment with the following command.

$ conda activate ./env

Reference

If you use this code, please cite the following paper

@inproceedings{sda+2021rethinking-sparsification,
  author = {Sahu, Atal Narayan and Dutta, Aritra and Abdelmoniem, Ahmed M. and Banerjee, Trambak and Canini, Marco and Kalnis, Panos},
  title = "{Rethinking gradient sparsification as total error minimization}",
  booktitle = {NeurIPS 2021 - Advances in Neural Information Processing Systems},
  year = 2021,
  url = {https://arxiv.org/abs/2108.00951}
}