Dealing with training set of questionable quality
Welcome to the real world of data science. Here, the data sets are not as clean as you thought while doing those courses/tutorials online.  Those are super polished and refined.  But, the real world data is not so.

The step where one does the cleaning and scrubbing is called the data pre-processing step.

So, some nice data cleaning techniques, in addition to @jknappen's excellent answer are:


Elimination of zero variance columns/predictors: These columns are not important, and they cause the model and the fit to crash and leak errors. So, eliminating them would make complete sense.
Correlated Predictors: Reducing the level of correlation between the predictors would be a very nice step in the pre-processing process.
Scaling: You must be knowing why scaling is important during pre-process. 
Predictor Transformations


A nice reference from Kaggle forums where the pre-processing and cleaning of data sets is discussed.