What is meant by Distributed for a gradient boosting library?
It means that it can be run on a distributed system (i.e. on multiple networked computers).

From XGBoost's documentation:


  The same code runs on major distributed environment(Hadoop, SGE, MPI) and can solve problems beyond billions of examples. The most recent version integrates naturally with DataFlow frameworks(e.g. Flink and Spark).