Skip to content

Latest commit

 

History

History
10 lines (6 loc) · 736 Bytes

README.md

File metadata and controls

10 lines (6 loc) · 736 Bytes

Gradient Starvation 🍴💀

Welcome to my STMO project!

I wanted to explore the findings of Pezeshki et al. presented in their publication Gradient Starvation: A Learning Proclivity in Neural Networks. The goal was to explain the problem of gradient starvation and to investigate its effects and a proposed remedy on a simple 2D classification task.

The notebook and the code of my project are separate! You can find them in their respective subfolders notebook and src.

All experiments conducted, including the related figures, can be simply repeated and re-created by running include("src\\gradientstarvation.jl") from a Julia REPL after checking out the project on your local machine.