title: How do I improve performance when iterating through multiple conditions in R?
body: <p>Let's assume that I have a dataset with the following structure:</p>
  
  <ul>
  <li>I have N products</li>
  <li>I'm operating in N countries</li>
  <li>I have N payment partner</li>
  <li>May dataset contains of N days</li>
  <li>I have N different prices that customers can choose from</li>
  </ul>
  
  <p>For example:</p>
  
  <pre><code>customer.id &lt;- c(1,2,3,4,5,6,7,8)
  product &lt;- c("product1","product2","product1","product2","product1","product2","product1","product2")
  country &lt;- c("country1","country2","country1","country2","country1","country2","country1","country2")
  payment.partner &lt;- c("pp1","pp2","pp1","pp2","pp1","pp2","pp1","pp2")
  day &lt;- c("day1","day2","day1","day2","day1","day2","day1","day2")
  price &lt;- c("price1","price2","price1","price2","price1","price2","price1","price2")
  
  customer.data &lt;- data.frame(customer.id,product,country,payment.partner,day,price)
  customer.data &lt;- data.table(customer.data)
  </code></pre>
  
  <p>Suppose I want to generate an aggregate out of it that, for instance, performs a forecasting algorithm for each combination. In order to do so, I identify the unique items for each condition and iterate it as follows:</p>
  
  <pre><code>unique.products &lt;- droplevels(unique(customer.data[,product]))
  unique.countries &lt;- droplevels(unique(customer.data[,country]))
  unique.payment.partners &lt;- droplevels(unique(customer.data[,payment.partner]))
  unique.days &lt;- droplevels(unique(customer.data[,day]))
  unique.prices &lt;- droplevels(unique(customer.data[,price]))
  
  for(i in seq_along(unique.products)){
    temp.data1 &lt;- customer.data[product==unique.products[[i]]]
    for(j in seq_along(unique.countries)){
      temp.data2 &lt;- temp.data1[country==unique.countries[[j]]]
      for(k in seq_along(unique.payment.partners)){
        temp.data3 &lt;- temp.data2[payment.partner==unique.payment.partners[[k]]]
        for(l in seq_along(unique.days)){
          temp.data4 &lt;- temp.data3[day==unique.days[[l]]]
          for(m in seq_along(unique.prices)){
            temp.data5 &lt;- temp.data4[price==unique.prices[[m]]]
            if(nrow(temp.data5)!=0){
              # do your calculations here
              print(temp.data5)
            }
          }
        }
      }
    }
  }
  </code></pre>
  
  <p>In general, this code structure works fine, but it gets really annoying when applying real data with 5 million rows on it. I guess R is not the best language in terms of speed and performance. Of course, I have used multicore processing in the past or tried to get such an aggregate straight out of Hive or an MySQL DataWarehouse. Using another language like C++ or Python is also always an option. </p>
  
  <p>However, sometimes all these options are not possible, which then always leads me to that exact same processing structure. So I'm wondering for quite a while if there is a better, respectively faster solution from a rather architectural point of view since it is known (and also becomes VERY clear when benchmarking) that for loops and frequent data subselection is very, very slow.</p>
  
  <p>Grateful for all comments, hints and possible solutions!</p>