Trade-off between variable stability and discriminatory model quality It is very normal for data scientists and modeling professionals to be concerned with the stability of the model. It basically means that if a variable is important today, it cannot change its importance over time because this instability would be a demonstration of the model's weakness.
It is difficult to argue that stability is good. But in the area of ​​credit modeling (which produces credit scores, for example) people are cracked by stability, to the point of preferring models with less discrimination to a more stable one.
My question is what is the true utility of stability and what is the correct way to make this trade-off with the discriminatory metric of the model (KS, Gini, AUC, etc.).
I would guess (personal guess) that stability somehow ties the model to a less than ideal metric because it does not allow behaviors to change over time. If they don't really change, there will be no problems, but if the effects of the variables are changing over time, it is normal to assume that this is reflected in less stability and analysts work around this by placing simpler models or removing the variables.
I think there might be a bit of confusion here: what is usually called a stable model is a model for which the performance doesn't vary (or not significantly) when sampling a different subset of training data or test data. In other words a model is stable if chance doesn't affect its performance. Typically one can use cross-validation in order to assess the stability of a model: if the variance is high between different splits then the model is unstable. Unstability is often a sign of overfitting, so one should be very cautious with an unstable model even if it seems to perform well: the risk is that the model might obtain good performance on the test set by chance, which means that it will actually perform poorly on new data but there will be no way to detect the problem. This is why people might favor a stable model to a high performance in some applications, because the consequences of deploying an unstable model in production would be costly.
But what you are talking about is not general model stability, it's stability of the performance across time for time-based data. That's a completely different question, it's about whether the model can represent time variations accurately. There are many types of models which can do that, but of course the more complex the model the more data it needs and the higher the risk of overfitting.