Original Paper: Dropout as a Bayesian Approximation
Epistemic Uncertainty: accounts for uncertainty in the model -- uncertainty which can be explained away given enough data （e.g., in the problem of dangerous creatures predicton, if only given the picture of lion and giraffe in the training phase, the model will have high uncertainty of classify zombie included test dataset.
Aleatoric uncertainty: captures noise inherent in the observations. (e.g, in the example above, probably not dangerous if they are not hungry).
MC Dropout
- Don’t change structure of NN
- Set weight = 0 when drop out
- Probability = drop weight/ total weight
- Open drop out during prediction
- Result: Deviation between confidence and hit probabilty = 0.1656