Overview
KNN is a 'lazy learner' because it doesn't build a model during training. Instead, it stores the entire training dataset and performs calculations only when a new prediction is needed.
How it Works
- Choose the number of neighbors (K).
- Calculate the distance (e.g., Euclidean distance) between the new point and all points in the training set.
- Find the K closest points.
- For classification, take a majority vote of the neighbors' labels; for regression, take the average of their values.
Considerations
- Choice of K: A small K is sensitive to noise; a large K can smooth out important boundaries.
- Feature Scaling: Since it relies on distance, all features should be on the same scale (e.g., 0 to 1).