How does xgboost work with linear booster? I know how to implement linear objective function and linear boosts in XGBoost. My concrete question is: when the algorithm it fits the residual (or the negative gradient) is it using one feature at each step (i.e. univariate model) or all features (multivariate model)?

Any reference to documentation about the linear boosts in XGBoost will be appreciated.

Thanks!

EDIT: Linear boosts can be implemented in XGBoost by setting the 'booster' parameter to 'gblinear'. See:
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3885826/
for useful information on linear boosting. Note that I am not speaking about the objective function (which can be also linear) but about the boosts themselve.
Refer to the authors' paper on xgboost here - XGBoost: A Scalable Tree Boosting System.

Equation 3 on page 2 mentions that in each step all the predictors are used to greedily fit the next additive tree.
However, since column sub-sampling is also employed to prevent over-fitting, not all features may actually be used for each step, see section 2.3 of the same paper.

Hence, the specific answer to your query is that the algorithm uses all features to fit the residual.