Anomaly detection on sparse categorical data
No need for machine learning here.
After you've transposed the dataframe, just count the number of unique combinations in the new column, and then rank them by frequency. Set a suitable threshold of &quot;rareness&quot; (like freq=2 below) and you will have your list of strange combinations.
There's a tool in Pandas for this called df.values_count()
e.g.




combination
freq




a,b
1


a,c
1


a,d
1


a,b,c
2


a,b,c,d
10


b,d
10




Then just compare you new combinations with your &quot;bank of rare combinations&quot;, and update them if they are no longer rare.