GRU/LSTM models - Train/Test split
LSTM is often used for Sequence Prediction problems, for example, when the dataset is a time series.

In that kind of dataset, you don't split the dataset the normal (a.k.a random) way to avoid look-ahead bias. We should train with "old observations" and test with "new observations" as you said.


  Should, in the case of LSTMs, the train test (in the toy example above) be 10 000 observations long (i.e. 9 000 old 'train' observations and 1 000 new 'test' ones)?


This sentence is a little bit unclear to me. If you have a dataset with 10 000 observations, you can train with 9000 old observations and test with 1000 new observations. The training set shouldn't contain the test data in any case.

Here's an example of how to split a time series dataset:
We have data from 01-01-2000 to 31-12-2004, we can choose 01-01-2004 as a split day:


Training set: data from 01-01-2000 to 31-12-2003
Test set: data from 01-01-2004 to 31-12-2004


In this case, no knowledge from the future is used in the training phase and we can use the model to predict the data in the test set.