More elegant filter script in R I'm very new both to programming and to R so please bear with me :-)
I've written the following bit of code, which works perfectly well and runs through a data file with 17446 rows in about 35 seconds. I don't really have a problem with this but am sure that this could be a lot more elegant with probably the use of the tapply function. I would love to see how you experts would rewrite this more efficiently and am sure the rewrite would teach me a lot. I've included the first few rows of the output file and hopefully the code should be fairly obvious (it must be if I managed to write it!!); it's a simple filter based on the standard deviation of a subset in one column dictating the output to another column which is set up at the start of the code. If a value in the NEE column is more than 2 stdevs of the preceeding three values then the value is taken from the mean of a subset in the NEE2 column else the value is copied from the first (if that makes sense). Please also note the "count" variable as I need to retrieve the amount of values replaced. Hope this piques someone's interest and thanks in advance for all of your time.
JonP

Dspke &lt;- read.csv(file.choose())
nr = nrow(Dspke)
count = 0

Dspke$NEE2 &lt;- (1:nr) #creates a new column ready for input of values

for (i in 4:nr) {

#standard deviation of the previous three values in NEE
stdev &lt;- sd(Dspke$NEE[(i-2):i]) 

#if stdev&gt;2 then NEE2 value is mean of the previous 3 values in NEE2 else copy value from NEE
if(stdev&gt;=2) {
  Dspke$NEE2[i] &lt;- mean(Dspke$NEE2[(i-2):i-1])
    count = count+1

    }else { Dspke$NEE2[i] &lt;- Dspke$NEE[i]
  }

 }   
  write.csv(Dspke,"Dspke.csv")

 Date_Time       NEE     NEE2
1 03/01/2012 13:00 -2.300000  1.00000
2 03/01/2012 13:30 -2.385610  2.00000
3 03/01/2012 14:00 -2.081935  3.00000
4 03/01/2012 14:30 -1.778260 -1.77826
5 03/01/2012 15:00 -2.409490 -2.40949
6 03/01/2012 15:30 -0.741030 -0.74103
The embed function is well-suited for constructing trailing vectors by row:

 Dspke[ , paste0("E3", 1:3)] &lt;- NA     # create columns to hold the trailing values
 Dspke[4:nrow(Dspke), 5:7] &lt;- embed(Dspke$NEE, 3)
 Dspke$NEE3 &lt;- ifelse( apply(Dspke[,5:7], 1, sd) &gt;2,   #test
                             rowMeans(Dspke[,5:7]),   # result if TRUE
                             Dspke[,"NEE"])           # result if FALSE


I see the count vector being created but cannot tell from your description what it is supposed to do. Perhaps you should store the value of:

&gt; apply(Dspke[,5:7], 1, sd) &gt; 2)
    1     2     3     4     5     6 
   NA    NA    NA FALSE FALSE FALSE 


(Not a very good test of the code presented by the data since all of the tests are FALSE.)