Which metrics for evaluating a recommender system with implicit data?
There are two major types of evaluation - online and offline.
Online evaluation means showing the model's predictions to actual users. Since the goal of a recommender system to sell more products, the best overall best metric for a recommender system is increasing sales to actual users. This is best done by putting the model in production and A/B test if the model increase sales. This approach is not always possible given the limited resources (time or access to a production system).
Offline evaluation means simulating online evaluation by holding out existing data to evaluate the model.
If possible, split the data based on time. Train the model on the earlier data. Test the model on the later data. For a given pairing of product and user, the model will predict buy or not buy (binary classifier). The model can be evaluated as any binary classifier. For given domain, precision or recall may be more important.
However, this is not often times possible because time data many not be tracked or product-user pairs maybe sparse. If time is not tracked, the data may be split randomly to simulate time. If product-user pairs are sparse, product-user pairs are clustered along latent factors.