title: scikit RandomForestClassifier - real results not matching prediction score
body: <p>I am new to machine learning and I am trying to classify text using a scikit RandomForestClassifier. The problem I have is that my test data results don't match the sklearn classification report. The training set has about 25k samples, about 25% of which are tagged 1 and 75% 0. I have a an additional test set of 1k which I am using to test after training. </p>
  
  <p><strong># Train</strong></p>
  
  <pre><code>    vectorizer = TfidfVectorizer(max_features=40, stop_words=stopwords.words('english'))
      X = vectorizer.fit_transform(documents).toarray()
      X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.15, random_state=0)
  
      classifier = RandomForestClassifier(
      n_jobs=-1, bootstrap=False, n_estimators=200, random_state=0)
  
      classifier.fit(X_train, y_train)
      y_pred = classifier.predict(X_test)
  
      print(confusion_matrix(y_test, y_pred))
      print(classification_report(y_test, y_pred))
      print(accuracy_score(y_test, y_pred))
  
                 precision    recall  f1-score   support
  
         0       0.98      0.99      0.98      4231
         1       0.95      0.90      0.92       883
  
  accuracy                           0.97      5114
  </code></pre>
  
  <p><strong># Test</strong></p>
  
  <pre><code>df = pandas.read_csv(input_file, header=None)
  df.columns = ["data", "target"]
  
  df, y = df.data, df.target
  
  test_documents = []
  for body in range(0, len(df)):
      document = str(df[body])
      test_documents.append(document)
  X = vectorizer.fit_transform(test_documents).toarray()
  prediction = pd.DataFrame(classifier.predict(X))
  
                precision    recall  f1-score   support
  
         0       0.89      0.87      0.88       856
         1       0.38      0.42      0.40       154
  </code></pre>