Is there a good alternative to XGBoost for learn to rank? My problem with XGBoost is that when I load the train dataset into the XGBoost DMatrix, there is a memory spike that is unavoidable, and I can't get my dataset loaded into RAM without crashing first.
I am trying to predict rankings over time, similar to a search engine query problem. XGBoost has produced good results using the pairwise objective but I have reached my limits due to the memory spike issue.
You asked for alternatives - also consider: LightGBM, CatBoost or RankLib.
How big is your dataset?  If you are hitting memory problems just creating the DMatrix its possible the other libraries will have similar problems.
What are you loading your dataset from?  Perhaps creating the DMatrix from a numpy array is trying to fit two copies of your data in memory at the same time?  Perhaps you could divide your dataset into smaller chunks and append them to the DMatrix to avoid having them all in memory at once?