title: Natural Language Processing - Features for Text Classification
body: <p>So I'm trying to classify texts using Weka SVM. So far, my feature vectors used for training the SVM are composed of TF-IDF statistics for unigrams and bigrams that appear in the training texts. But, the results I get from testing the trained SVM model haven't been accurate at all, so can someone give me feedback on my procedure? I am following these steps to classify texts:</p>
  
  <ol>
  <li>Construct a dictionary made up of extracted unigrams and bigrams from the training texts</li>
  <li>Count how many times each unigram/bigram appears in each training text, as well as how many training texts the unigram/bigram appears in</li>
  <li>Use the data from step 2 to calcuate the TF-IDF for each unigram/bigram </li>
  <li>For each document, construct a feature vector that is the length of the dictionary, and store the corresponding TF-IDF statistic in each element of the vector (so for example, the first element in the feature vector for document one would correspond to the TF-IDF for the first word in the dictionary relative to document one)</li>
  <li>Append class label to each feature vector to distinguish which text belongs to which author</li>
  <li>Train SVM using these feature vectors</li>
  <li>Feature vectors for the testing texts are constructed in the same way as the training texts, and are classified by the SVM</li>
  </ol>
  
  <p>Also, could it be that I need to train the SVM with more features? If so, what features are most effective in this case? Any help would be greatly appreciated, thanks.</p>