Gradient-Descent-Algos

This repo contains an implementation of famous Gradient Descent Algorithms in Matlab such as :

1. Classical Gradient Descent
2. Momentum Method
3. Nesterov Momentum Method
4. AdaGrad
5. RMSprop
6. Adam

The code also alows to make visualization of the algos for famous functions such as:

1. 3humpcamel
2. Ackley
3. Matyas
4. Easom
5. quadratic
6. Rastrigin

An example of code use can be seen in run.m.

The folders with the name of multidim functions contain the visualization results of the algos for 2dim case.

Hist contains the histograms of the results of the algos for different initialization points and params.

Giphs contain visualization giph results for several functions

Example for Periodic function Adam and Nesterov Momentum Method: