14. The method of claim 10 wherein assigning the average score is computed as the arithmetic average among all scores assigned by each evaluators.
10. A method to evaluate user experience in a spoken dialog system comprising: selecting a set of recorded interactions for an interactive spoken dialog system, whereby the set represents a corpus of interactions handled by the spoken dialog interaction system; selecting a number of evaluators to analyze the set; having each evaluator score the recordings; accepting a score of the interactions that do not show any scoring difference across different evaluators; assigning an average score to the interactions that have an absolute predetermined score difference among the evaluators; performing an adjudication process for all the interactions that have a score difference among the evaluators that is larger than the predetermined score difference to obtain an reevaluated score; and averaging the assigned average score and the reevaluated score to obtain an estimated value of the user experience, wherein the interactive spoken dialog system comprises a computer including a processor, and non-transitory memory, including: a data store for storing recordings of interactions on the spoken dialog system; and an automated rater including a machine-learning algorithm configured to automatically estimate a user experience score for any set of interactions on the spoken dialog system.