Changing behaviour of an ML model I am trying to create a ranking system for recommending books to an user. Let's suppose we have some subjects of books like 'A', 'B', 'C', 'D' and from the past behaviour, it is observed that the user is more inclined towards subjects 'A' and 'B'. My dataset has a format of binary classification dataset i.e. it contains specifications of books and a labels specifying whether the user read it or not (1/0). Based on this, I assign probabilities of the user liking a book, using a model like RandomForestClassifier. The problem is, based on the past behaviour, the books with subject 'A' and 'B' always tend to get higher probability scores. Is there any way to add some 'discoverability' to the model so that it assigns high scores to even books of other subjects?
This is a kind of confirmation bias, it's often found in recommender systems and it's hard to beat: since the system is designed to find similar books, it's normal that it returns the most similar books but it can be disappointing for the user. This bias is especially strong if there are few past books to refer to, i.e. the reference sample used by the model is too small to represent the user's preferences.
As Nikos suggested in a comment, one way to diversify the recommendations is to broaden the set of similarity features. A more advanced way is to use some second-degree approach: the system doesn't only recommend books similar to a specific user's past, but finds other users who liked the same books and recommends books that they liked (similar to &quot;other users who like X also like Y&quot;).