title: R : group by with custom functions
body: <p>I have managed to aggregate data successfully using the following pattern:</p>
  
  <pre><code>newdf &lt;- setDT(df)[, list(X=sum(x),Y=max(y)), by=Z]
  </code></pre>
  
  <p>However, the moment I try to do anything more complicated, although the code runs, it no longer aggregates by Z: it seems to create a dataframe with the same number of observations as the original df so I know that no grouping is actually occurring.</p>
  
  <p>The custom function I would like to apply is to find the n-quantile for the current list of values and then do some other stuff with it. I saw use of sdcols in another SO answer and tried something like:</p>
  
  <pre><code>customfunc &lt;- function(dt){
  q = unname(quantile(dt$column,0.25))
  n = nrow(dt[dt$column &lt;= q])
  return(n/dt$someOtherColumn)
  }
  #fails to group anything!!! also rather slow...
  newdf &lt;- setDT(df)[, customfunc(.SD), by=Z, .SDcols=c(column, someOtherColumn)]
  </code></pre>
  
  <p>Can someone please help me figure out what is wrong with the way I'm trying to use group by and custom functions? Thank you very much.</p>
  
  <p>Literal example as requested:</p>
  
  <pre><code>&gt; df &lt;- data.frame(Z=c("abc","abc","def","abc"), column=c(1,2,3,4), someOtherColumn=c(5,6,7,8))
  &gt; df
      Z column someOtherColumn
  1 abc      1               5
  2 abc      2               6
  3 def      3               7
  4 abc      4               8
  &gt; newdf &lt;- setDT(df)[, customfunc(.SD), by=Z, .SDcols=c("column", "someOtherColumn")]
  &gt; newdf
       Z        V1
  1: abc 0.2000000
  2: abc 0.1666667
  3: abc 0.1250000
  4: def 0.1428571
  &gt; 
  </code></pre>
  
  <p>As you can see, DF is not grouped. There should just be two rows, one for "abc", and another for "def" since I am trying to group by Z.</p>