Dealing with population instability I am working on using machine learning to correctly predict a binary classification using an input dataset that I receive about once a month. 

The idea is that I train, test and validate the classifier on one month of data, and use it to classify another month. Whether I repeatedly add the next month of data to the training dataset and upset my classifier is open for debate.
I've spent a little while on this and have something that works well but it works better on some months than others. I am using either RandomForest or LinearSVC(penalty=’l1’) because I have a lot of features, around 400,000 observations and limited computing power.

I do not have much insight into the processes that produce the input dataset and based on previous analysis, I do not trust their quality control. What worries me is that one month I might receive a dataset that is very different to the data I have been training on and I wouldn’t really know unless the distribution of classes was quite different.

I am sure that I cannot be the first person to have this worry but I can’t find much information about it, so maybe I am misguided here. I’ve found the Population Stability Index, which is the right kind of idea, but I am unsure about it because it is dependent on bin size.

So my question(s): 


How is a worry like this normally dealt with?
Can you point me to some ideas/case studies that will help me?
Is there anything I can do on the classification side that will make
my classifier more robust to population instability?
There could be significant problems from using one time chunk in a time series to predict the next chunk. For example, if you were using data from September and October to predict retail shopping expenditure in November and December, you would be very wrong. Instead, you need to note the annual trend of increased shopping between Thanksgiving and Christmas to make correct predictions.

So, while this sort of skirts the questions you asked, I think that if you are having difficulty using one month to predict the next month, you need to start looking at longer-term trends, especially annual ones. I would investigate this first before looking into population instability.