how do measure if my fraud prediction is actually workingdo have the right data to evaluate an algorithmif am successfulwhat will be the business impact of my solutionas we discussed in it is best if you can measure the performance of your algorithm directly using business metriclike increased profit or decreased losses this is often hard to dothough question that can be easier to answer is "what if built the perfect model?if perfectly detecting any fraud will save your company $ monththese possible savings will probably not be enough to warrant the effort of you even starting to develop an algorithm on the other handif the model might save your company tens of thousands of dollars every monththe problem might be worth exploring say you've defined the problem to solveyou know solution might have significant impact for your projectand you've ensured that you have the right information to evaluate success the next steps are usually acquiring the data and building working prototype in this book we have talked about many models you can employand how to properly evaluate and tune these models while trying out modelsthoughkeep in mind that this is only small part of larger data science workflowand model building is often part of feedback circle of collecting new datacleaning databuilding modelsand analyzing the models analyzing the mistakes model makes can often be informative about what is missing in the datawhat additional data could be collectedor how the task could be reformulated to make machine learning more effective collecting more or different data or changing the task formulation slightly might provide much higher payoff than running endless grid searches to tune parameters humans in the loop you should also consider if and how you should have humans in the loop some processes (like pedestrian detection in self-driving carneed to make immediate decisions others might not need immediate responsesand so it can be possible to have humans confirm uncertain decisions medical applicationsfor examplemight need very high levels of precision that possibly cannot be achieved by machine learning algorithm alone but if an algorithm can make percent percentor maybe even just percent of decisions automaticallythat might already increase response time or reduce cost many applications are dominated by "simple cases,for which an algorithm can make decisionwith relatively few "complicated cases,which can be rerouted to human wrapping up