Neural Networks Learning The Basics: Gradient Descent and Stochastic Gradient Descent

Overview In the previous post we looked at backpropagation. In the example we calculated the gradient of the loss with respect to each weight parameter. The gradient determined which direction to move the weight parameter and how much to move it by. The new weight parameter was then updated using the formula below: where is … Continue reading Neural Networks Learning The Basics: Gradient Descent and Stochastic Gradient Descent