RNN unable to classify time series I have 400 time series of length 50.
200 of them have values between 1-10 and are considered of type A.
The rest 200 have values 1-10 with the exception that 3 from the total of 50 data points have value 20, and are considered of type B.

I am trying to use a RNN network to make it learn these differences using tensorflow.

I use GRU cell with input size is 50 (the whole time series), internal state is 100, 3 layers, feature length is 1 and dropout 0.8.
Batch size is 1 (1 whole time series of 50 values)
80% of the series are used for training and 20% for test, evenly distributed between type A and type B.

I use softmax as activation function and for gradient descent optimization i use RMSPropOptimizer. Also i am not changing the data at all before feeding it into the neural network, i tried normalizing them using (x-min()/max()-min()) but it didn't work.

When i am trying to train it, i always get 50% error. Maximum epochs i tried are 10 and it still was 50%.

The same network can classify correctly if i use it on data that have way more differences than just on these 3 points, even on small dataset of about 50 runs of each type.

Is it impossible for a RNN to learn these small differences on just those 3 data points out of the 50?
What would you suggest i can change in my configuration?
I also tried with 2000 time series and still the error is the same.

Also note that these 3 data points exist at the beginning of the time series. (in case this has something to do with my issue).
I let it run for more epochs, as suggested, and i noticed that after 23 epochs, rnn did get 0% error.

I then tried changing activation function of GRUCell from the default tanh to relu and it greatly speed up training. It took 1-4 epochs to get 0% error.