Splitting Temporal Data I have a project where I'm required to compute (using some regression) how long a task will take. From the definition of the business problem it is clear that there is some temporal relationship in the data so I need to split into train/test by some cutoff date (rather than randomly sampling 70/30 split from the data).
The issue I'm running into is that no matter what date I split by, the distribution of my target variable is different in the testing &amp; training set. Earlier targets in the set are a lot larger, and as a result the model fit on the training set tends to way over-predict on the test set.
Any advice on how to tackle something like this?
It sounds likes you have non-stationary data which can be a challenge to model.
One option is to completely discard all older data and only train/test on newer data. This way the model will only capture the newer relationships. This approach assumes there is enough newer data to train a model.
Another option is heavily regularize the model. The goal of regularization  is to increase generalization.