1. A method for improving a ranking model, the method comprising: obtaining a data set, wherein the data set includes queries, a set of documents associated with each query, and document labels, wherein the document label describes categories of documents; initializing active sets using the document labels, wherein two active sets are established for each query, a first active set and a second active set; optimizing an empirical loss function by the use of the first and second active set, wherein parameters of the ranking model are modified in accordance to the empirical loss function; updating the active sets with additional ranking data, wherein the updates are configured to work in conjunction with the optimized loss function; recalculating the active sets by the use of the ranking model, wherein the recalculated active sets provide an indication for ranking the documents in a way that is consistent with the training set.