title: how to debug "factor has new levels" error for linear model and prediction
body: <p>I am trying to make and test a linear model as follows:</p>
  
  <pre><code>lm_model &lt;- lm(Purchase ~., data = train)
  lm_prediction &lt;- predict(lm_model, test)
  </code></pre>
  
  <p>This results in the following error, stating that the <code>Product_Category_1</code> column has values that exist in the <code>test</code> data frame but not the <code>train</code> data frame):</p>
  
  <blockquote>
    <p>factor Product_Category_1 has new levels 7, 9, 14, 16, 17, 18</p>
  </blockquote>
  
  <p>However, if I check these they definitely look to appear in both data frames:</p>
  
  <pre><code>&gt; nrow(subset(train, Product_Category_1 == "7"))
  [1] 2923
  &gt; nrow(subset(test, Product_Category_1 == "7"))
  [1] 745
  &gt; nrow(subset(train, Product_Category_1 == "9"))
  [1] 312
  &gt; nrow(subset(test, Product_Category_1 == "9"))
  [1] 92
  </code></pre>
  
  <p>Also showing the table for <code>train</code> and <code>test</code> show they have the same factors:</p>
  
  <pre><code>&gt; table(train$Product_Category_1)
  
       1      2      3      4      5      6      7      8      9     10     11     12     13     14     15     16     17     18 
  110820  18818  15820   9265 118955  16159   2923  89511    312   4030  19113   3108   4407   1201   4991   7730    467   2430 
  &gt; table(test$Product_Category_1)
  
      1     2     3     4     5     6     7     8     9    10    11    12    13    14    15    16    17    18 
  27533  4681  4029  2301 29637  4005   745 22621    92  1002  4847   767  1033   299  1212  1967   100   645 
  &gt; 
  </code></pre>