How to show value of a classification model even though it doesn't get the desired performance? I developed a classification model for a telecom client. Where we classify between Dual-sim and non-Dual-Sim clients. After many iteration the best precision we can get is 60%. The contract says that the acceptance criteria is 75% precision. The client measures precision based on campaign results. In other words, they call sample data and ask them explicitly are a dual sim or not.

Facts we know:  


Learning curve shows that more training data won't do the trick.
We have got all variables that were used in the market before(and everything we could think of, but nothing improved the model).
Training data don't even get 1% of the whole population I am generalizing on(and we can't get more than that).
The random baseline is 35%, so we got a lift of around 1.9 (but they won't approve of it).
We have tried many iterations from the most simple models to the most complex, and this is the best we can get.
The dual sim and non-dual sim distribution across the variables, isn't that different.
The language used were SQL, R.


So the question is, what else can I do to prove that the model has a value but without getting the 75% precision?
Often customers don't understand the value behind the performance of machine learning models. 60% precision might sound not so good, since 50% is often related to nearly random for binary problems. However, it might well be a very difficult problem, and 60% might well be very good performance.

One way you can show your customer that your model is indeed performing well, is to compare its precision against a human classification.

You can:


Define a random baseline, and explain why it is 35% and not 50% for your problem.
Get a couple of humans from your client side to try the task manually.
Let your model perform the task and show the improvement.


More often than not, humans perform tasks more precisely than models (if in small scale). So, if your model performs better, hopefully, they will see the value your model is adding to their company.