Towards Robust Federated Analytics via Differentially Private Measurements of Statistical Heterogeneity: Experimental Evaluation

This repository contains all the supporting files for the experimental section of the paper Towards Robust Federated Analytics via Differentially Private Measurements of Statistical Heterogeneity, including all the Python files necessary for anyone to repeat all of the experiments given as evidence for the results in the paper.

Environment

• Install the latest version of Python 3, and install the additional packages idx2numpy, matplotlib, mpmath, numpy and prettytable using PIP.
• Download the Python file agm_balle_all_datasets.py.
• Download the archives cifar-10-python.tar.gz and cifar-100-python.tar.gz using the instructions from Alex Krizhevsky's home page, and keep the files data_batch_x (each ~30GB, where x ranges from 1 to 5), and the file train (~150GB) respectively.
• Download the archives train-images-idx3-ubyte.gz and train-labels-idx1-ubyte.gz using the instructions from Zalando, and keep the files train-images-idx3-ubyte (~45GB) and train-labels-idx1-ubyte (~60KB) respectively.

Instructions

• After setting up the Python environment and downloading the required files as outlined above, open and run agm_balle_all_datasets.py.
• After each dataset has been analyzed, the tables are printed in the terminal and the plots are saved in the same folder as the Python files. This should happen within 1 hour.
• These tables and graphs should be exactly those found in the experimental section of Towards Robust Federated Analytics via Differentially Private Measurements of Statistical Heterogeneity.

Authors

• Mary Scott, Department of Computer Science, University of Warwick
• Graham Cormode, Department of Computer Science, University of Warwick
• Carsten Maple, WMG, University of Warwick