1. A system effectuated on a machine that at least one of trains or evaluates ranking techniques by at least one of employing or obtaining relative preference judgments, comprising: a component that retrieves a set of documents from a storage device, combines the set of documents with a judgment task received via an interface to form a comparative selection panel and presents the comparative selection panel for evaluation by an assessor, the component requests the assessor to make a selection as to which document included in the set of documents most satisfies the query, the component produces a ranked comparative assessment of the set of documents based at least in part on the selection elicited from the assessor.