How to solve the "dangerous feedback loops" in machine learning?
I read the article you linked, and what you are missing are that the given conversion probabilities are assessed pre-callback - i.e. they include an assessment of whether you will even call them back or not. So of course the probabilities change if you change your behaviour. The writer of the article has created a bit of a straw man argument by defining a model and decision process that don't go well together. They should have used a model that predicted conversion rates after callback, then it could be used as they wanted.
So the sales example is pretty easy to solve. Make the model accept the lead source as a feature, and predict the conversion rate after callback. That will give you the &quot;action value&quot; of choosing to callback, which is much more useful value to have if you are deciding whether or not to callback.
To cover the possibility that probabilities change over time, you have to be willing to test that hypothesis by calling back at least some leads that the model predicts have a low probability of success, in order to update the model. This is related to exploration vs exploitation trade off in bandit problems or in reinforcement learning. The callback optimisation problem looks a lot like a bandit problem in fact.
Many problems of choosing how to act optimally, based on an updating statistical model of results, can be re-framed successfully as bandit problems or reinforcement learning problems. This is one way to try and address the feedback loops issue, because these include theory around decision making. It is not a magic fix for all these problems, but contextual bandits are very strong in advertising strategies that are very similar to the example you give.
To be able to address feedback loops like this well does require that source data is unbiased, so that adding conditional actions to your model helps obtain better ground truth. So the other examples that the linked article gives - recidivism and recruitment - are much harder to address, because biased data is inserted as ground truth even if you add the conditions (e.g. for exploring and tracking results in a statistically neutral way). For some, adding these conditions may be enough to help, it will make the model accurate enough that it starts to address bias. For others, there is no really good solution other than to be aware of weakness of the model.
You can attempt to de-bias the model by using some abstract ideal. A common technique used in recruiting for instance is to remove features that might be causing bias: Literally remove race, sex and related determiners from the CV on the grounds that these should not be part of any fair model, regardless of whether or not a ML model could make of use them to predict results. This might accidentally throw out data that would improve accuracy too, but can be used to achieve a decision process that observers agree should be free of contentious soruces of bias.
Which features to remove or adjust is an open and subjective question. Unfortunately ML models can often determine protected characteristics from other data, and with &quot;black box&quot; ones such as neural networks it is not clear whether or not they are doing so. It's still an open question, and the main defence against biased models is basically awareness that it is a problem and that feeding all your data into a computer to get a statistical model does not in any way &quot;purify&quot; it or achieve some higher level of objective truth.