Deep learning
Activation function
An activation function transforms a neuron’s pre-activation value.
How it works
Without nonlinear activations, many stacked linear layers collapse into one linear transformation. ReLU, GELU, and sigmoid create different gradient and output behavior.
A concrete example
Illustrative: ReLU maps [-2, 0.5, 3] to [0, 0.5, 3].
Do not confuse it with Loss function
activation shapes internal signals; loss scores final prediction error.