Classification: Dealing with unknown class balance in the prediction dataset
This phenomenon is called Covariate Shift and it can affect features,target, you name it.

This happens when the test data distribution is different from the training data. Or your problem is a time-series one where time is the biggest independent variable.

First case : No model can adapt to the target shifting in its behavior. You have to either wait until you collect enough data from the current times, and retrain a fairly satisfying model, or do online learning which means that every datapoint that you predict for , is fed into the model to be trained on. That way; if changes in the target occur, your model will get updated with every new datapoint.

Second case : Lets say your target variable has a tendency to go up every 2 years, and then goes down. This is an assumption i'm making. Your task is to detect that shift in time that affects your target and model it. You create a model that takes into account that every 2 years, your target goes down. and so on. It's what we call a time-series problem where time is the biggest contributor to the target's changes.

If the second case doesn't make sense to you, and doesn't relate to your problem, then look at first case.