1. A system for replaying performance of a predictive engine, comprising: a processor; an engine variant of the predictive engine stored in a digital working memory, wherein the engine variant is determined by an engine parameter set; and a non-transitory digital storage medium for storing program code, the program code executable by the processor, the storage medium comprising program code to: deploy the engine variant of the predictive engine based on the engine parameter set; receive a plurality of queries to the engine variant from an end-user device, wherein each of the plurality of queries contains a timestamp; apply a first query segment filter to the plurality of queries to generate a first replay group of queries; for each query in the first replay group, the engine variant generates a corresponding predicted result, receives a corresponding actual result, and computes a corresponding prediction score based on the corresponding predicted result and the corresponding actual result; group queries in the first replay group into one or more query segments of the first replay group according to the timestamps; and generate, as a replay of the performance of the engine variant, an accumulated prediction score over each of the one or more query segments of the first replay group by applying an accumulation function.