Visit "The Robust Deep Learning Library" (our newest work) to quickly use this loss and much more:

The Robust Deep Learning Library

---

Distinction Maximization Loss (DisMax)

Efficiently Improving Out-of-Distribution Detection and Uncertainty Estimation by Replacing the Loss and Calibrating

We keep single network inference efficiency. No hyperparameter tuning. We need to train only once. SOTA.

Read the full paper: Distinction Maximization Loss: Efficiently Improving Out-of-Distribution Detection and Uncertainty Estimation by Replacing the Loss and Calibrating.

Train on CIFAR10, CIFAR100, and ImageNet.

Results

Dataset=ImageNet, Model=ResNet18, Near OOD=ImageNet-O

Loss [Score]	Class (ACC)	Near OOD (AUROC)
Cross-Entropy [MPS]	69.9	52.4
DisMax [MMLES]	69.6	75.8

Dataset=CIFAR

Use DisMax in your project!!!

Replace the SoftMax loss with the DisMax loss changing few lines of code!

Replace the model classifier last layer with the DisMax loss first part:

class Model(nn.Module):
    def __init__(self):
    (...)
    #self.classifier = nn.Linear(num_features, num_classes)
    self.classifier = losses.DisMaxLossFirstPart(num_features, num_classes)

Replace the criterion by the DisMax loss second part:

model = Model()
#criterion = nn.CrossEntropyLoss()
criterion = losses.DisMaxLossSecondPart(model.classifier)

Preprocess before forwarding in the training loop:

# In the training loop, add the line of code below for preprocessing before forwarding.
inputs, targets = criterion.preprocess(inputs, targets) 
(...)
# The code below is preexistent. Just keep the following lines unchanged!
outputs = model(inputs)
loss = criterion(outputs, targets)

Detect during inference:

# Return the score values during inference.
scores = model.classifier.scores(outputs) 

Run the example:

python example.py

Code

Software requirements

Much code reused from deep_Mahalanobis_detector, odin-pytorch, and entropic-out-of-distribution-detection.

Please, install all package requirments runing the command bellow:

pip install -r requirements.txt

Preparing the data

Please, move to the data directory and run all the prepare data bash scripts:

# Download and prepare out-of-distrbution data for CIFAR10 and CIFAR100 datasets.
./prepare_cifar.sh
# Download and prepare out-of-distrbution data for ImageNet.
./prepare_imagenet.sh

Reproducing the experiments

Train and evaluate the classification, uncertainty estimation, and out-of-distribution detection performances:

./run_cifar100_densenetbc100.sh*
./run_cifar100_resnet34.sh*
./run_cifar100_wideresnet2810.sh*
./run_cifar10_densenetbc100.sh*
./run_cifar10_resnet34.sh*
./run_cifar10_wideresnet2810.sh*
./run_imagenet1k_resnet18.sh*

Analyzing the results

Print the experiment results:

./analyze.sh

Citation

Please, cite our papers if you use our loss in your works:

@article{DBLP:journals/corr/abs-2205-05874,
  author    = {David Mac{\^{e}}do and
               Cleber Zanchettin and
               Teresa Bernarda Ludermir},
  title     = {Distinction Maximization Loss:
  Efficiently Improving Out-of-Distribution Detection and Uncertainty Estimation
  Simply Replacing the Loss and Calibrating},
  journal   = {CoRR},
  volume    = {abs/2205.05874},
  year      = {2022},
  url       = {https://doi.org/10.48550/arXiv.2205.05874},
  doi       = {10.48550/arXiv.2205.05874},
  eprinttype = {arXiv},
  eprint    = {2205.05874},
  timestamp = {Tue, 17 May 2022 17:31:03 +0200},
  biburl    = {https://dblp.org/rec/journals/corr/abs-2205-05874.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}

@article{DBLP:journals/corr/abs-2208-03566,
  author    = {David Mac{\^{e}}do},
  title     = {Towards Robust Deep Learning using Entropic Losses},
  journal   = {CoRR},
  volume    = {abs/2208.03566},
  year      = {2022},
  url       = {https://doi.org/10.48550/arXiv.2208.03566},
  doi       = {10.48550/arXiv.2208.03566},
  eprinttype = {arXiv},
  eprint    = {2208.03566},
  timestamp = {Wed, 10 Aug 2022 14:49:54 +0200},
  biburl    = {https://dblp.org/rec/journals/corr/abs-2208-03566.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}