Overfitting means a model has fit the training data so well that it has also learned its accidental details and noise. As a result, it performs excellently on that same data but poorly when it meets new data.
An analogy#
Imagine a student who, instead of understanding the lesson, memorized the answers to last year's exam questions. If the same questions appear, they get full marks, but a new question confuses them. An overfitted model is the same: it memorized, it did not learn.
How to detect it#
The model's accuracy on training data is very high, but on test data it has never seen, it is much lower. This large gap is the main sign.
Why it happens#
- The model is too complex or too large.
- The training data is small or not diverse.
- Training has gone on too long.
- The data has a lot of noise or errors.
Ways to prevent it#
- Collect more and more varied data.
- Simplify the model.
- Use regularizers such as randomly dropping neurons.
- Stop training earlier.
- Always evaluate on a separate test set.
The opposite condition#
"Underfitting" is when the model is too simple and fails to learn even the main pattern of the training data.
Common mistake#
High accuracy on training data is not a sign of a good model; only accuracy on unseen data matters.