1. A method, comprising: receiving first performance scores for a ranker component that are indicative of performance of the ranker component for respective queries in training data, the training data comprising the queries and labeled search results that indicates a desired order of the search results for the queries; and optimizing the ranker component for a risk-oriented objective, the risk-oriented objective being a function of a variance in the performance scores for the ranker, wherein the optimizing of the ranker component is performed by a computer processor.