Title: clean up the dataset
Tags: r
Question: I have this dataset:
sample:

```x=rnorm(45)
std_d=sd(x)
```

Now x looks like:

``` [1] -0.08059702  0.90403763 -0.18618130 -0.48590834  1.23714656  1.02248570

  [7] -0.28970333 -0.19626563  0.89060697  0.87530362 
```

Let p=abs(x[i] - x[i+1]). I want to put NA in place of values of x if p> sd(x). It should be done in the way that it should check x[i] -x[i+1] initially. if this does not satisfy the condition check for next i.
Now if the condition satisfies, it should put NA for x[i+1].

Then next time p should be p= x[i] - x[i+2]. It should skip the NA value and keep the first term of p (x[i]) same until the condition is not satisfied. Once this happens, first term should become the term next to NA value and 2nd term becomes the term next to first term.

I think this can be done with combination of if else and for loop. But I am not able to figure out the algorithm even after trying hard. I require help on this.

Thank you for your consideration.

Comment: ok. Lets consider this this:

This is what should happen:
1. abs(x[1]-x[2])>sd then x[2]=NA.
2. abs(x[1]-x[3])sd then x[5]=NA
5. abs(x[4]-x[5]<sd..and so on.

Comment: What have you tried?  `p <- abs(diff(x, lag=1))` will get your values for p.  I don't understand much of what happens after that... maybe you could use `set.seed` and walk through a few iterations of how you expect the calculation to go?

Here is the accepted answer: There has got to be a better way... but in horrible c style:

```x <- c(-0.08059702, 0.90403763, -0.18618130, -0.48590834, 1.23714656, 1.02248570, 0.28970333, -0.19626563, 0.89060697, 0.87530362)

std_d <- sd(x)

for(i in seq_along(x)) {
  if(is.na(x[i])) next
  ctr <- i
  while(ctr < length(x)) {
    if(abs(x[i] - x[ctr+1]) &gt; std_d) {
      x[ctr+1] <- NA
      ctr <- ctr + 1
      std_d <- sd(x, na.rm=TRUE)
    } else {
      break
    }
  }
}
```

If you're setting things to ```NA```, ```sd(x)``` is changing so I included that too...

Comment for this answer: Thanks. this is what I was looking for.

Here is another answer: ```is.na(x) <- c(FALSE, abs(diff(x)) &gt; sd(x) )
#Pass two:  Here your description could use a set.seed and a desired result.
&gt; X1 <- x
&gt; is.na(X1) <- c(FALSE, abs(diff(X1)) &gt; sd(x) )
&gt; X1
 [1]          NA -0.21797491 -1.02600445 -0.72889123 -0.62503927          NA          NA  0.15337312
 [9]          NA          NA  0.42646422 -0.29507148          NA  0.87813349  0.82158108  0.68864025
386-652-4785 -0.06191171 -0.30596266 -0.38047100 -0.69470698 -0.20791728          NA          NA
[25]          NA          NA -0.40288484 -0.46665535          NA -0.08336907  0.25331851 -0.02854676
[33] -0.04287046          NA          NA          NA          NA          NA  0.12385424  0.21594157
[41]  0.37963948          NA -0.33320738 -1.01857538 -1.07179123
&gt; X2 <- X1
&gt; is.na(X2) <- c(FALSE, FALSE,  abs(diff(X2, lag=2)) &gt; sd(x) )
&gt; X2
 [1]          NA -0.21797491 -1.02600445 -0.72889123 -0.62503927          NA          NA  0.15337312
 [9]          NA          NA  0.42646422 -0.29507148          NA          NA  0.82158108  0.68864025
386-652-4785 -0.06191171 -0.30596266 -0.38047100 -0.69470698 -0.20791728          NA          NA
[25]          NA          NA -0.40288484 -0.46665535          NA -0.08336907  0.25331851 -0.02854676
[33] -0.04287046          NA          NA          NA          NA          NA  0.12385424  0.21594157
[41]  0.37963948          NA -0.33320738 -1.01857538 -1.07179123
```


{'source': 'StackExchangeDataset', 'subset': 'stackoverflow.com', 'metadata': {'AnswerCount': 2, 'CommentCount': 2, 'ContentLicense': 'CC BY-SA 3.0', 'CreationDate': '2012:00:05/30/12', 'FavoriteCount': 0.0, 'PostTypeId': 1, 'Tags': '<r>', 'Title': 'clean up the dataset', 'ViewCount': 612, 'id': 10823971, 'user_AboutMe': '<p>I am an engineering student in IIT Bhubaneswar, India.\nCurrently working on R software.</p>\n', 'user_AccountId': 1503231.0, 'user_DisplayName': "Swapnil 'Tux' Takle", 'user_Reputation': 1187.0, 'user_Views': 149.0, 'user_id': 1407223.0}}