Why doesn't LightGBM perform better than SVC (linear kernel)? [sentiment analysis] I've read and heard about the mighty XGBoost which is one of the most famous models people are using today to solve Kaggle challenge. This makes me interest to develop my own intuition about the model so I decided to try XGBoost on my dataset, but unfortunately, I experienced a lot of dependency related problems when I tried to install XGBoost on mac :(

However, I did a bit more researches and found out another related library called LightGBM from Microsoft which they claim to achieve better result (or at least, equivalent) over XGBoost. Therefore, I decided to go with the LightGBM instead since I didn't encounter any problems during the library installation.

I expected that the model should outperform, or at least, perform similarly compared to other models on the same dataset. My dataset contains 14k of text documents and has 0 or 1 values which refer to positive and negative sentiment respectively as target variable.

As a result showing below, LightGBM with count vectorizer as input got the 6th place with an accuracy of 94.60%. I'm quite curious since the accuracy of LightGBM is not only lower than SVC but also lower than Extra Tree model (300 esitimators). 

I don't have enough foundation and solid understanding behind these models so I couldn't come up with an answer about these outcomes. Can someone please give me an idea or assumption according to the result?

PS. All models was performed with 5-fold cross validation 

model             score
--------------  -------
extraT           0.9528
svc              0.9514
sgd_elas         0.9481
extraT_tfidf     0.9476
svc_tfidf        0.9473
lightGBM         0.9460
sgd_elas_tfidf   0.9458
lightGBM_tfidf   0.9420
randomF          0.9409
randomF_tfidf    0.9345
mulNB            0.9307
berNB            0.9087
berNB_tfidf      0.9087
mulNB_tfidf      0.9036
Xgboost usually gives a higher accuracy(this is just an observation, not a fact). But the trick with xgboost is people dont know how to best use it. The most common mistake made using xgboost is not letting it train enough. Xgboost is an algorithm with overfits more than the other algorithms(well overfit is not the right term, it should be xgboost has high variance than the others). Do not tune the other parameters until you have found a converging nrounds for appropriate learning rate(0.01 or 0.1 or 0.3 etc). If you see that the cross val accuracy is much much lower than train accuracy then tune gamma(the second order derivative) first, then other parameters. If the cross val and train accuracy have a significant but not a lot difference(lets say train acc goes from 90 to 99 and cross val is stuck at 85 after significant iterations) then try reducing the max depth of trees. If the cross val and train accuracy are running side by side then just let the algorithm converge for a particular eta and then tune the other parameters(all except gamma).