Mean Absolute Error in Random Forest Regression I am new to the whole ML scene and am trying to resolve the Allstate Kaggle challenge to get a better feeling for the Random Forest Regression technique. 

The challenge is evaluated based on the MAE for each row. 

I've run the sklearn RandomForrestRegressor on my validation set, using the criterion=mae attribute. To my understanding this will run the Forest algorithm calculating the mae instead of the mse for each node.

After that I've used this: metrics.mean_absolute_error(Y_valid, m.predict(X_valid)) in order to calculate the MAE for each row of data.

What I would like to know is if the logic I'm following is sound. Am I making a fundamental mistake or missing something here? Should I have used the default MSE based Regressor and then calculate the MAE of each row using the mean_absolute_error function?
Let me clarify few fundamental things:


In sklearn, RandomForrest Regressor criterion is:


  The function to measure the quality of a split


It's a performance measure (by default, MSE) which helps the algorithm to decide on a rule for an optimum split on a node in a tree.
Kaggle is giving you a metric, i.e. MAE (again a performance/ quality measure) but to evaluate the performance of your ML model, once finalized.


To come back to your question: while both MAE/ MSE are performance measures, they are being used at two different stages of a modeling process and might not be related. So, while it makes sense to evaluate your final model on MAE as you would be judged on it, you can choose any of MAE/ MSE for criterion (i.e. for RandomForest) depending on performance at validation stage.

While the above being said, keep in mind that you might want to evaluate the validation errors (i.e. for finalizing a model) on the same metric (i.e. MAE in this case), to keep error measure consistent with the test set evaluation.