What are the factors to consider when setting the depth of a decision tree?
Yes, but it also means you're likely to overfit to the training data, so you need to find the value that strikes a balance between accuracy and properly fitting the data. Deciding on the proper setting of the max_depth parameter is the task of the tuning process, via either Grid Search or Randomised Search with cross-validation. 

This page from the scikit-learn documentation explains the process well: https://scikit-learn.org/stable/modules/grid_search.html