that' another simple modification you can make and play around with there are probably some outliers in our user rating datasetwhat if were to throw away people that rated some ridiculous number of moviesmaybe they're skewing everything you could actually try to identify those users and throw them outas another idea andif you really want big projectif you really want to sink your teeth into this stuffyou could actually evaluate the results of this recommender engine by using the techniques of train/test sowhat if instead of having an arbitrary recommendation score that sums up the correlation scores of each individual movieactually scale that down to predicted rating for each given movie if the output of my recommender system were movie and my predicted rating for that moviein train/test system could actually try to figure out how well do predict movies that the user has in fact watched and rated beforeokaysoi could set aside some of the ratings data and see how well my recommender system is able to predict the user' ratings for those movies andthat would be quantitative and principled way to measure the error of this recommender engine but againthere' little bit more of an art than science to this even though the netflix prize actually used that error metriccalled root-meansquare error is what they used in particularis that really measure of good recommender systembasicallyyou're measuring the ability of your recommender system to predict the ratings of movies that person already watched but isn' the purpose of recommender engine to recommend movies that person hasn' watchedthat they might enjoythose are two different things so unfortunatelyit' not very easy to measure the thing you really want to be measuring so sometimesyou do kind of have to go with your gut instinct andthe right way to measure the results of recommender engine is to measure the results that you're trying to promote through it maybe ' trying to get people to watch more moviesor rate new movies more highlyor buy more stuff running actual controlled experiments on real website would be the right way to optimize for thatas opposed to using train/test soyou knowi went into little bit more detail there than probably should havebut the lesson isyou can' always think about these things in black and white sometimesyou can' really measure things directly and quantitativelyand you have to use little bit of common senseand this is an example of that