High error machine learning regressor algorithm in Python - XGBOOST Regressor
It is not necessary to one-hot-encode your categorical features when using tree-based methods. Basically the idea is that the tree has to make many splits to figure out the category. Instead, you can use ordinal encoder (even if categories are not ordered).
I also would not have set 'YEAR_BUILT' as a categorical variable, even though it is discrete. By one-hot encoding this variable, you end up with a very splitted dataset where the tree has to consider every YEAR_BUILT as a subset and it probably has too few observations to do so. By considering it as continuous it enables the algorithm to determine 2 or 3 pivot years that are relevant (changed currency, economic crisis, ...).


I think the second point can really improve performance.