Deep learning

Dropout

Dropout randomly masks activations during training to reduce co-adaptation.

Dropout randomly masks activations during training to reduce co-adaptation. Values shown in the worked visual are illustrative unless the labels state otherwise.

How it works

Each training step samples a different thinned network. At inference, dropout is disabled and scaling preserves the expected activation.

A concrete example

Illustrative: dropout p = 0.2 masks about 200 of 1,000 activations per training pass in expectation.

Do not confuse it with Pruning

dropout is temporary training noise; pruning permanently removes parameters or structures.

Study the full lesson