Back to glossary

Bias-Variance Tradeoff

The fundamental tension in machine learning between a model's ability to minimize errors from erroneous assumptions (bias) and its sensitivity to small fluctuations in the training set (variance).

Categoryai

Overview

Finding the right balance between bias and variance is the key to building models that generalize well to new data.

Key Concepts

  • High Bias (Underfitting): The model is too simple and misses the underlying patterns (e.g., using a straight line for a curved relationship).
  • High Variance (Overfitting): The model is too complex and 'memorizes' the noise in the training data.

The Goal

To find the 'sweet spot' where the total error (Bias² + Variance + Irreducible Noise) is minimized.

Related Terms

Cross-Validation