Overview
Dropout is a simple but highly effective way to prevent overfitting in neural networks. During each training step, a certain percentage of neurons (e.g., 20-50%) are 'dropped out' or set to zero.
Why it Works
It forces the network to learn redundant representations and prevents it from relying on any single neuron or specific combination of neurons. It's like training an ensemble of many smaller networks.
Inference
During inference (when using the model), dropout is turned off, and all neurons are used, usually with their weights scaled down to account for the missing neurons during training.