Can regression trees predict continuously? Suppose I have a smooth function like $f(x, y) = x^2+y^2$. I have a training set $D \subsetneq \{((x, y), f(x,y)) | (x,y) \in \mathbb{R}^2\}$ and, of course, I don't know $f$ although I can evaluate $f$ wherever I want.

Are regression trees capable of finding a smooth model of the function (hence a tiny change in the input should only give a tiny change in the output)?

From what I've read in Lecture 10: Regression Trees it seems to me that regression trees basically put the function values into bins:


  For classic regression trees, the model in each cell is just a constant estimate of
  Y .


As they write "classic" I guess there is a variant where the cells do something more interesting?
Regression trees, particularly gradient boosting (essentially many trees), tend to do very well on continuous predictions, often outperforming models that are truly continuous like linear regression when. This is especially true when there are variable interactions and when you have a sufficiently large dataset (over 10,000 records) so that overfitting is less likely. If your primary objective is simply predictive power, then whether the model is 100% continuous or pseudo continuous should be irrelevant. If making your regression trees more continuous enhances out of sample predictive power, then you can simply increase tree depth or add more trees.