title: Speed up unpacking aggregated data
body: <p>I have received aggregated data of 2.7M lines with 13 columns of aggregated data. I am trying to unpack it quickly. A shortened example of potential data and code is below:</p>
  <p>columns types = sex, age, hair_color, eye_color, birth_state, survey_date, count</p>
  <pre><code>new_data &lt;- tibble(sex = c(&quot;M&quot;, &quot;M&quot;, &quot;F&quot;, &quot;F&quot;, &quot;F&quot;), 
           age = c(18, 27, 34, 21, 25),
           hair_color = c(&quot;Brown&quot;, &quot;Blonde&quot;,&quot;Black&quot;, &quot;Pink&quot;, &quot;Blonde&quot;),
           eye_color = c(&quot;Brown&quot;, &quot;Blue&quot;,&quot;Black&quot;, &quot;Green&quot;, &quot;Green&quot;),
           birth_state = c(&quot;AK&quot;, &quot;CO&quot;,&quot;CO&quot;, &quot;FL&quot;, &quot;CA&quot;),
           survey_date = as.POSIXct(c(&quot;1/1/2020&quot;, &quot;5/1/2020&quot;,&quot;2/2/2020&quot;, &quot;1/10/2020&quot;, &quot;1/1/2020&quot;),format = &quot;%d/%m/%Y&quot;),
           count = c(10,14,6,8,6))
  </code></pre>
  <p>The &quot;10&quot; in the last category of the first row, &quot;count&quot;, demonstrates that there were 10 individuals counted which matched the preceding column information. I am trying to unpack the data set to have 10 lines all with the same preceding information rather than a single line with a &quot;10&quot; at the end.</p>
  <p>Below is my current code:</p>
  <pre><code>i &lt;- as.numeric(nrow(new_data))
  check &lt;- new_data
  
  for (i in 1:i){
    k &lt;- new_data[i,]
    j &lt;- new_data[i,7]-1
    u &lt;- data.frame(t(replicate(j, k, simplify = TRUE)))
    l &lt;- list(check, u)
    check &lt;- do.call(&quot;rbind&quot;, l)
    print(i)
  }
  
  check$cnts &lt;- 1
  end &lt;- Sys.time()
  start-end
  
  Code thoughts:
  i - find out how many total lines i will have to duplicate
  k - pulling the line that I am going to duplicate
  j - identifying how many times I need to repeat the line
  u - establish a data.frame which uses replicate to generate multiple lines. I have to transpose it to get it into the right frame work.
  l - create two lists to be joined.
  check - to call &quot;rbind()&quot; to link the list together. This is providing me the best data.frame I have found yet.
  
  check$cnts &lt;- 1 just changes the count from whatever it was to 1. This indicates that it is a single survey point.
  </code></pre>
  <p>I have tried a handful of different methods and can't seem to find anything that runs quickly. I have been trying to unpack this for a couple days and it is still running. When I have tried other packages (rbind(), repeat(), etc) I end up with a data.frame which has lists embedded in it. Researching that problem, I seem to find that rbind() has problems at times.</p>
  <p>I read another post about rbind() and it looks like I can't use multiple processors to speed things up. Any help would be greatly appreciated!</p>