Sigmoid Activation From Scratch

Difficulty: Easy · Concept: Activation Functions — Historical / Saturating Functions

$\sigma(z) = \frac{1}{1+e^{-z}}$

Sigmoid squashes any real number into $(0, 1)$ — rarely used in hidden layers today (see the concept page for why), but still standard for a binary classifier's output layer, where the output needs to read as a probability. It's also the same function Logistic Regression is built on.

Your task: implement sigmoid(z) for a single real number z, using math.exp (no NumPy).

Implement it yourself

assert abs(sigmoid(0) - 0.5) < 1e-9
assert abs(sigmoid(2) - 0.8807970779778823) < 1e-9
assert abs(sigmoid(-2) - 0.11920292202211755) < 1e-9
assert 0 < sigmoid(20) < 1
assert 0 < sigmoid(-20) < 1e-6

Next: Sigmoid Derivative (Vectorized, Numerically Stable) (a harder variant), or skip ahead to Numerically Stable Softmax From Scratch