When is a Model Underfitted? Logic often states that by underfitting a model, it's capacity to generalize is increased. That said, clearly at some point underfitting a model cause models to become worse regardless of the complexity of data.

How do you know when your model has struck the right balance and is not underfitting the data it seeks to model?



Note: This is a followup to my question, "Why Is Overfitting Bad?"
A model underfits when it is too simple with regards to the data it is trying to model.

One way to detect such situation is to use the bias–variance approach, which can represented like this:



Your model is underfitted when you have a high bias.



To know whether you have a too high bias or a too high variance, you view the phenomenon in terms of training and test errors:

High bias: This learning curve shows high error on both the training and test sets, so the algorithm is suffering from high bias:



High variance: This learning curve shows a large gap between training and test set errors, so the algorithm is suffering from high variance.



If an algorithm is suffering from high variance:


more data will probably help
otherwise reduce the model complexity


If an algorithm is suffering from high bias:


increase the model complexity


I would advise to watch Coursera' Machine Learning course, section "10: Advice for applying Machine Learning", from which I took the above graphs.