Neural Mastery

Training

Batch size

Batch size is the number of examples used to estimate one gradient update.

batch 1batch 2batch 3batch 4epoch32323232One update per batch, here 32 examples each.
Batch size is the number of examples used to estimate one gradient update. Values shown in the worked visual are illustrative unless the labels state otherwise.

How it works

Larger batches reduce gradient noise and increase memory demand. Smaller batches produce noisier updates and more optimizer steps per epoch.

A concrete example

Illustrative: 96,000 examples with batch size 192 produce 500 steps per epoch.

Do not confuse it with Epoch

batch size counts examples per update; an epoch covers the training set once.

Study the full lesson