My company has recently engaged a consultant firm to develop a predictive model to detect defective works.I understand that there are many ways to validate the model, for example, using k-fold cross-validation and I believe that the consultant firm will carry out the validation before submitting the model to us.However, at the employer's side, how can I check the accuracy of the model developed by the consultant firm ?? Someone suggested that I can give 2000-2015 data to the consultant firm and keep 2016 data for our own checking. However, a model with good accuracy on 2016 data does not imply that it will have good predictive power in the future. In my view, keeping 2016 data for checking seems like adding one more test set for validation, which in my view, is unnecessary since I already hv k-fold cross validation.Could someone advise what the employer can do to check the consultant's model? 
Check Accuracy of Model Provided by Consultant
machine learning;predictive modeling;cross validation;accuracy
Cross validation can be used in parameter tuning or model selection, but it does not evaluate the performance of a model.When developing a model, you divide your data between train, validation and testing. In the best case scenario, testing is only used once at the end to score the model. You should definitely keep the 2016 data. If you give all your data, it is easy to have a model learning by heart your expected data but it will not generalize well to future years. This is overfiting. The only way to know is by testing it on unknown data, here, 2016Employed for measuring model performance, cross validation can measure more than just the average accuracy and you can select your features to answer the best accuracy score