This simulation visualizes gradient descent methods in different situations. The main motivation behind that is to better understand optimizers which are being used by deep learning frameworks.
All optimizers use the gradient for calculating their next step. In this simulation the gradient is calculated numerically (symmetric derivative).
The optimizers use the following formulas to calculate their next step:
3D View
Top-down view
3D View
Top-down view
3D View
Top-down view
3D View
Top-down view
3D View
Top-down view
3D View
Top-down view
