Learning a perceptron from stream data I want to train a Perceptron using stochastic gradient rulefrom the stream data. I have very limited amount of memory and i can store only $N$ examples. 

Suppose my population consist of point as show in the following picture:


Now suppose my first $N$ examples come in following fashion and i can classify them correctly as show in the next picture: 



Now the problem is if the next $N$ examples come in this way:

and i have classified them as shown. The problem is that since i can't train perceptron for previous $N$ examples(because i can store only $N$ examples and previous $N$ examples need to be thrown away) and training on next $N$ examples contradicts the hyper-plane for previous $N$ examples. 

How to train a perceptron from stream data? Do i need to store all examples or there is an alternative way?
I think you're confusing yourself by worrying about a worst case that likely won't happen.  Assuming each element of the stream comes randomly and independently from some distribution, then it is exponentially unlikely that the case you mention happens.

How do you train from a stream?  Easy: each time you see an example, you update the weights of the neural network following the rules of gradient descent, then throw away that example.  In this way, the weights of the neural network change over time, and you don't need to store any past examples.  Over time, as you've seen enough examples, this should converge to a neural network that performs about as well as possible on the data, assuming that all of the data in the stream comes from the same distribution.