Activation Function

An activation function is a small math function applied to a neuron's output in a neural network.