Overview
Hyperparameters are the 'knobs' you turn before training starts, such as the learning rate, batch size, number of layers, or dropout rate. They significantly impact the model's performance.
Methods
- Grid Search: Trying every possible combination from a predefined list.
- Random Search: Trying random combinations (often more efficient than grid search).
- Bayesian Optimization: Using a probabilistic model to find the best settings more quickly.
Goal
To maximize the model's accuracy and generalization while minimizing training time and resource use.