Trained Models of 27 Techniques (16 run each)

The attached files are one 7z archive, which is divided into 1.5GB volumes due to the file size limitation of Github.

The directory structure is: <Technique>/run_<id>.pth
<Technique> follows the abbreviation in Table 1 of the paper, such as "S-Base" and "A-ALM"
<id> is the index of each run. With 16 FIT runs, the indices range from 00 to 15.

All the models are saved by PyTorch.

Raw Prediction Results of 27 Techniques (16 run each)

The attached file contains the raw prediction results of 27 techniques in the paper, while each technique has the result for 16 fixed-seed identical runs.

The directory structure is: <Technique>/run_<id>.csv
<Technique> follows the abbreviation in Table 1 of the paper, such as "S-Base" and "A-ALM".
<id> is the index of each run. With 16 FIT runs, the indices range from 00 to 15.

Each csv file has four columns:

• idx: The index of the sample.
• ground_truth: The ground truth (task) label.
• prediction_result: The predicted (task) label of the model.
• protected_label: The protected label of the sample.

For binary and multi-class classifications, each row of the ground_truth and the prediction_result column only contain one number, which is the index of the task label.

For multi-label classification such as MS-COCO, each row of the ground_truth and the prediction_result column is the string literal of a Python list, which is the one-hot encoding of the labels.

For example, string literal [0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] means task label 2 and 6 are associated with the corresponding sample.

Docker Images for Reproducing the Results

The attached files are two 7z archives of docker images, which are used to reproduce our study.

The repo and the tag of the images are: dlfairness:balanced_dataset_not_enough_coco and dlfairness:nifr.

Dataset

Datasets used for the experiments