title: R tree-based methods like randomForest, adaboost: interpret result of same data with different format
body: <p>Suppose my dataset is a <code>100 x 3</code> matrix filled with categorical variables. I would like to do binary classification on the response variable. Let's make up a dataset with following code:</p>
  
  <pre><code>set.seed(2013)
  y &lt;- as.factor(round(runif(n=100,min=0,max=1),0))
  var1 &lt;- rep(c("red","blue","yellow","green"),each=25)
  var2 &lt;- rep(c("shortest","short","tall","tallest"),25)
  df &lt;- data.frame(y,var1,var2)
  </code></pre>
  
  <p>The data looks like this:</p>
  
  <pre><code>&gt; head(df)
    y var1     var2
  1 0  red shortest
  2 1  red    short
  3 1  red     tall
  4 1  red  tallest
  5 0  red shortest
  6 1  red    short
  </code></pre>
  
  <p>I tried to do random forest and adaboost on this data with two different approaches. The first approach is to use the data as it is:</p>
  
  <pre><code>&gt; library(randomForest)
  &gt; randomForest(y~var1+var2,data=df,ntrees=500)
  
  Call:
   randomForest(formula = y ~ var1 + var2, data = df, ntrees = 500) 
                 Type of random forest: classification
                       Number of trees: 500
  No. of variables tried at each split: 1
  
          OOB estimate of  error rate: 44%
  Confusion matrix:
     0  1 class.error
  0 29 22   0.4313725
  1 22 27   0.4489796
  
  ----------------------------------------------------
  &gt; library(ada)
  &gt; ada(y~var1+var2,data=df)
  
  Call:
  ada(y ~ var1 + var2, data = df)
  
  Loss: exponential Method: discrete   Iteration: 50 
  
  Final Confusion Matrix for Data:
            Final Prediction
  True value  0  1
           0 34 17
           1 16 33
  
  Train Error: 0.33 
  
  Out-Of-Bag Error:  0.33  iteration= 11 
  
  Additional Estimates of number of iterations:
  
  train.err1 train.kap1 
          10         16 
  </code></pre>
  
  <p>The second approach is to transform the dataset into wide format and treat each category as a variable. The reason I am doing this is because my actual dataset has 500+ factors in var1 and var2, and as a result, tree partitioning will always divide the 500 categories into 2 splits. A lot of information is lost by doing that. To transform the data:</p>
  
  <pre><code>id &lt;- 1:100
  library(reshape2)
  tmp1 &lt;- dcast(melt(cbind(id,df),id.vars=c("id","y")),id+y~var1,fun.aggregate=length)
  tmp2 &lt;- dcast(melt(cbind(id,df),id.vars=c("id","y")),id+y~var2,fun.aggregate=length)
  df2 &lt;- merge(tmp1,tmp2,by=c("id","y"))
  </code></pre>
  
  <p>The new data looks like this:</p>
  
  <pre><code>&gt; head(df2)
     id y blue green red yellow short shortest tall tallest
  1   1 0    0     0   2      0     0        2    0       0
  2  10 1    0     0   2      0     2        0    0       0
  3 100 0    0     2   0      0     0        0    0       2
  4  11 0    0     0   2      0     0        0    2       0
  5  12 0    0     0   2      0     0        0    0       2
  6  13 1    0     0   2      0     0        2    0       0
  </code></pre>
  
  <p>I apply random forest and adaboost to this new dataset:</p>
  
  <pre><code>&gt; library(randomForest)
  &gt; randomForest(y~blue+green+red+yellow+short+shortest+tall+tallest,data=df2,ntrees=500)
  
  Call:
   randomForest(formula = y ~ blue + green + red + yellow + short +      shortest + tall + tallest, data = df2, ntrees = 500) 
                 Type of random forest: classification
                       Number of trees: 500
  No. of variables tried at each split: 2
  
          OOB estimate of  error rate: 39%
  Confusion matrix:
     0  1 class.error
  0 32 19   0.3725490
  1 20 29   0.4081633
  
  ----------------------------------------------------
  &gt; library(ada)
  &gt; ada(y~blue+green+red+yellow+short+shortest+tall+tallest,data=df2)
  Call:
  ada(y ~ blue + green + red + yellow + short + shortest + tall + 
  tallest, data = df2)
  
  Loss: exponential Method: discrete   Iteration: 50 
  
  Final Confusion Matrix for Data:
            Final Prediction
  True value  0  1
           0 36 15
           1 20 29
  
  Train Error: 0.35 
  
  Out-Of-Bag Error:  0.33  iteration= 26 
  
  Additional Estimates of number of iterations:
  
  train.err1 train.kap1 
           5         10 
  </code></pre>
  
  <p>The results from two approaches are different. The difference is more obvious as we introduce more levels in each variable, i.e., <code>var1</code> and <code>var2</code>. My question is, since we are using exactly the same data, why is the result different? How should we interpret the results from both approaches? Which is more reliable?</p>