Overview
Hyperparameters are the 'external' settings that control how a model learns. They are distinct from 'parameters' (like weights and biases), which the model learns automatically during training.
Examples
- Learning Rate: How much to adjust weights at each step.
- Batch Size: How many examples to look at before updating weights.
- Number of Epochs: How many times to go through the entire dataset.
- Dropout Rate: The percentage of neurons to ignore for regularization.
Tuning
Finding the best hyperparameters is a critical part of model development, often done via Grid Search, Random Search, or Bayesian Optimization.