Overview
Grid search is a 'brute force' approach. You define a list of values for each hyperparameter, and the algorithm trains and evaluates a model for every possible combination.
Example
If you want to tune 'learning rate' [0.01, 0.1] and 'batch size' [16, 32], grid search will try four combinations: (0.01, 16), (0.01, 32), (0.1, 16), and (0.1, 32).
Pros and Cons
- Pros: Simple to implement; guaranteed to find the best combination within the specified grid.
- Cons: Extremely slow and computationally expensive as the number of hyperparameters increases (the 'combinatorial explosion').