title: Creating word embedings from bert and feeding them to random forest for classification
body: <p>I have used bert base pretrained model with 512 dimensions to generate contextual features. Feeding those vectors to random forest classifier is providing 83 percent accuracy but in various researches i have seen that bert minimal gives 90 percent.
  I have some other features too like word2vec, lexicon, TFIDF and punctuation features.
  Even when i merged all the features i got 83 percent accuracy. The research paper which i am using as base paper mentioned an accuracy score of 92 percent but they have used an ensemble based approach in which they classified through bert and trained random forest on weights.
  But i was willing to do some innovation thus didn't followed that approach.
  My dataset is biased to positive reviews so according to me the accuracy is less as model is also biased for positive labels but still I am looking for an expert advise</p>
  <p>Code implementation of bert</p>
  <p><code>https://github.com/Awais-mohammad/Sentiment-Analysis/blob/main/Bert_Features.ipynb</code></p>
  <p>Random forest on all features independently</p>
  <p><code>https://github.com/Awais-mohammad/Sentiment-Analysis/blob/main/RandomForestClassifier.ipynb</code></p>
  <p>Random forest on all features jointly</p>
  <p><code>https://github.com/Awais-mohammad/Sentiment-Analysis/blob/main/Merging_Feature.ipynb</code></p>