Overview
Cross-entropy measures the 'distance' between the predicted probability distribution and the actual distribution (the true labels).
Why use it?
It penalizes confident but wrong predictions more heavily than less confident ones, which helps the model converge faster during training.
Binary vs. Categorical
- Binary Cross-Entropy: Used for two-class problems.
- Categorical Cross-Entropy: Used for multi-class problems.