What is Underfitting?

Underfitting means when our model can't fit or touch enough/much/maximum data points on the best-fitted line. For this reason, our model gives bad or very bad predictions on both training and test dataset and this is called underfitting. If our model has high bias and low variance then we will say it is underfitted.

Suppose we have five data points in the training dataset and we will apply linear regression. After applying linear regression we see that our best-fitted line only fits or touches one data. If it happens then we can say that our model will give a very bad prediction on the training dataset. But when a test dataset comes or new data points come the same things happen. So in this case we will say that our dataset is under-fitted. In underfitting, the model gives bad prediction to both training and testing datasets.

What is Overfitting?

Overfitting means when our model gives very good accuracy on training data and gives very poor accuracy on test data. If our model has low bias and high variance then we will say it is overfitted.

Suppose we have some data points and we will apply polynomial regression. After applying polynomial regression the best fitted line goes over all the data points means our line is touching all the data points. It means that now our model will give 100% or near 100% accuracy on those data points. But when we test our model on a training dataset or when new data points come from training data set or from any where then our model can't do a good prediction. It happens, because our model is overtrained and can't take new data points which are out of the line. For this reason, it will ignore all those data points which are little far or too much far from the line. For this reason, our model does a very good prediction on the training dataset but do a very bad prediction on the test dataset and this is called overfitting.

Now there can be a question that is when we can say that our model will be a good model?

So the answer is, when our model gives good accuracy and both training and test dataset. We can also say when our model has low bias and low variance, then we can say it is a good model. So we should train our model like it should not touch or fit every data point but should fit maximum numbers of data points.