Using a random forest, would a RandomForest performance be less if I drop the first or the last tree?
The two slightly-smaller models will perform exactly the same, on average.  There is no difference baked in to the different trees: &quot;the last tree will be the best trained&quot; is not true.  The only difference among the trees is the random subsample they work with and random effects while building the tree (feature subsetting, e.g.).
Gradient boosted trees are a different story.  If you drop the first tree after you finish training, the resulting model will be mostly garbage.  Every subsequent tree was trained to improve upon the fit of the previous trees, and removing any single tree will put all future trees out of context.  (To give an extreme example, suppose the first tree actually captures &quot;the correct&quot; model.  All future trees will just fit on the remaining noise.)  On the other hand, removing the final tree is equivalent to having just trained one fewer tree, which may be good or bad depending on your bias-variance tradeoff at that point.