Links to some important research papers or links. I plan to add notes as I go through each topic one by one.
- Invariant Information Clustering
- Mutual Information Neural Estimation
- Deep Infomax
- Learning Representations by Maximizing Mutual Information Across Views
- How Google decoupled MI maximization and representation learning: On Mutual Information Maximization for Representation Learning
Decided not to delve deeper into this topic. It is not mature yet.
* [Disentangling Disentanglement in Variational Autoencoders](https://arxiv.org/abs/1812.02833)
* [Isolating Sources of Disentanglement in Variational Autoencoders](https://arxiv.org/abs/1802.04942)
* [InfoGAN-CR: Disentangling Generative Adversarial Networks with Contrastive Regularizers](https://arxiv.org/abs/1906.06034)
* [Disentangling by Factorising, pdf](https://www.cs.toronto.edu/~amnih/papers/disentangling_nips_ws.pdf)
- Representation Learning with Contrastive Predictive Coding
- Data-Efficient Image Recognition with Contrastive Predictive Coding
- Contrastive Multiview Coding
- Momentum Contrast for Unsupervised Visual Representation Learning
- SimCLR
- MoCo v2: Improved Baselines with Momentum Contrastive Learning
- The shortest paper I have seen. The authors modified MoCo based on some ideas borrowed from SimCLR.
- Notes
- Emerging Properties in Self-Supervised Vision Transformers PDF | Official Code
- Automatic differentiation in machine learning: a survey
- Automatic Reverse-Mode Differentiation: Lecture Notes
- Reverse mode automatic differentiation
- Neural Ordinary Differential Equations
- Augmented Neural ODEs
- Invertible ResNets
- Universal Differential Equations for Scientific Machine Learning
- Probabilistic models of cognition
- The Design and Implementation of Probabilistic Programming Languages
- Composition in Probabilistic Language Understanding
- Lottery ticket hypothesis
- Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask
- Rigging the Lottery: Making All Tickets Winners
- What's Hidden in a Randomly Weighted Neural Network?
- Topological properties of the set of functions generated by neural networks of fixed size
- YOUR CLASSIFIER IS SECRETLY AN ENERGY BASED MODEL AND YOU SHOULD TREAT IT LIKE ONE
- Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topology
- Amortized Population Gibbs Samplers with Neural Sufficient Statistics
- Evaluating Combinatorial Generalization in Variational Autoencoders