title: How to create Training data for Text classification on 4 categories
body: <p>My machine learning goal is to search for potential risks (will cost more money) and opportunities (will save money) from a Project Requirements document.</p>
  
  <p>My idea is to classify sentences from the data into one of these categories: Risk, Opportunity and Irrelevant (no risk, no opportunity, default categorie).</p>
  
  <p>I will use a multinomial Bayes classifier for this with tf-dif.</p>
  
  <p>Now I need to have data for my training set and test set. The way I will do this is label every sentence from requirement documents with 1 of the 3 categories. Is this a good approach?</p>
  
  <p>Or should I only label sentences which are obviously a risk/opportunity/irrelevant?</p>
  
  <p>Also, is the Irrelevant categorie a good idea? </p>