Overview
XGBoost is a specific implementation of Gradient Boosting that became famous for winning numerous Kaggle competitions. It is optimized for speed and performance.
Key Features
- Regularization: Includes L1 and L2 regularization to prevent overfitting.
- Parallel Processing: Can use multiple CPU cores for faster training.
- Handling Missing Values: Has an internal strategy for dealing with missing data.
- Tree Pruning: Uses a 'depth-first' approach and prunes trees backward to improve efficiency.
Use Case
It is the go-to algorithm for structured/tabular data in many industrial applications.