Regression - How Random Forest and Gradient Boosting really works?
Random Forest:  Build a decision tree.  


Sample N examples from your population with replacement (meaning examples can appear multiple times).
At each node do the following


Select m predictors from all predictors
Split based on predictor that performs best via some objective function
Go to the next node, select another m predictors and repeat



Combine all trees as an average or weighted via some scheme.

Gradient Boosting

One key note is that random forest trees essentially indepedent of each other.  Boosting algorithms add a certain depedency to the model.


Initialize a model by finding the minimizer of a certain objective function
For each iteration


Compute the partial derivative of -$L(y_{i}, F(x_{i}))$ with respect to $F(x_{i})$ for all i to n
Fit a tree $h_m$ to the the result from above.
solve $\lambda_m =arg min \sum_{i=1}^{n}L(y_{i}, F_{m-1}(x_{i}) + \lambda h_{m}(x_{i})$

Update model via  $F_m(x) = F_{m-1} + \lambda_m h_{m} (x)$