What are the factors to consider when setting the depth of a decision tree? In scikit learn, one of the parameters to set when instantiating a decision tree is the maximum depth. What are the factors to consider when setting the depth of a decision tree? Does larger depth usually lead to higher accuracy?
Yes, but it also means you're likely to overfit to the training data, so you need to find the value that strikes a balance between accuracy and properly fitting the data. Deciding on the proper setting of the max_depth parameter is the task of the tuning process, via either Grid Search or Randomised Search with cross-validation. 

This page from the scikit-learn documentation explains the process well: https://scikit-learn.org/stable/modules/grid_search.html