modify the feature spaceit may be that we do not have the right set of features we couldfor examplechange the scale of our current features or design even more new features or ratherwe could remove some of our current features in case some features are aliasing others change the modelit may be that knn is generally not good fit for our use casesuch that it will never be capable of achieving good prediction performance no matter how complex we allow it to be and how sophisticated the feature space will become in real lifeat this pointpeople often try to improve the current performance by randomly picking one of the preceding options and trying them out in no particular orderhoping to find the golden configuration by chance we could do the same herebut it will surely take longer than making informed decisions let' take the informed routefor which we need to introduce the bias-variance tradeoff bias-variance and its trade-off in getting started with python machine learningwe tried to fit polynomials of different complexities controlled by the dimensionality parameterdto fit the data we realized that two-dimensional polynomiala straight linedid not fit the example data very well because the data was not of linear nature no matter how elaborate our fitting procedure would have beenour two-dimensional model will see everything as straight line we say that it is too biased for the data at handit is under-fitting we played bit with the dimensions and found out that the -dimensional polynomial was actually fitting very well into the data on which it was trained (we did not know about train-test splits at the timehoweverwe quickly found that it was fitting too well we realized that it was over-fitting so badly that with different samples of the data pointswe would have gotten totally different -dimensional polynomials we say that the model has too high variance for the given data or that it is over-fitting these are the extremes between which most of our machine learning problems reside ideallywe want to have both low bias and low variance butwe are in bad world and have to trade off between them if we improve on onewe will likely get worse on the other fixing high bias let us assume that we are suffering from high bias in this caseadding more training data clearly will not help alsoremoving features surely will not help as our model is probably already overly simplistic