Does anyone have memory utilization benchmark for random forest and xgboost?
Try to use memory_profiler. 
All what you need to do is to decorate your training function:

@profile
def train_xgb():
   gb = xgb.XGBClassifier(learning_rate=0.1, n_estimators=100, subsample=0.8, max_depth=6)
   gb.fit(new_trainX,new_trainY)