Training
Learning rate
The learning rate scales each optimizer update.
How it works
A rate that is too large can overshoot useful regions, while a rate that is too small wastes steps. Schedules often change it over the run.
A concrete example
Illustrative: gradient 0.04 with learning rate 0.001 changes a parameter by 0.00004.
Do not confuse it with Batch size
learning rate controls step magnitude; batch size controls examples per gradient estimate.