This post continues from neural network basics part 1: Layers Matrix Multiplication. This post therefore assumes that you have basic knowledge on what a neuron is. Why Add an Activation Function? In neural network basics part 1: Layers Matrix Multiplication I covered matrix multiplication and defined a simple neural network as the weighted sum of inputs with the equation: [...]