1. A system comprising: one or more computers configured to perform operations comprising: receiving a plurality of test queries; generating, for each test query, a first group of resources corresponding to a first automated resource selection process and a second group of resources corresponding to a second automated resource selection process, the generating comprising, for each test query: identifying a plurality of resources responsive to the test query, determining, for each resource in the plurality of resources, whether the first automated resource selection process would classify the resource as to-be-indexed or not-to-be-indexed, and then selecting all resources classified as to-be-indexed as the first group of resources, and determining, for each resource in the plurality of resources, whether the second automated resource selection process would classify the resource as to-be-indexed or not-to-be-indexed, and then selecting all resources classified as to-be-indexed as the second group of resources, wherein the determination whether the first automated resource selection process would classify a particular resource as to-be-indexed or not-to-be-indexed is made independently of the determination whether the second automated resource selection process would classify that particular resource as to-be-indexed or not-to-be-indexed.
1. A system comprising: one or more computers configured to perform operations comprising: receiving a plurality of test queries; generating, for each test query, a first group of resources corresponding to a first automated resource selection process and a second group of resources corresponding to a second automated resource selection process, the generating comprising, for each test query: identifying a plurality of resources responsive to the test query, determining, for each resource in the plurality of resources, whether the first automated resource selection process would classify the resource as to-be-indexed or not-to-be-indexed, and then selecting all resources classified as to-be-indexed as the first group of resources, and determining, for each resource in the plurality of resources, whether the second automated resource selection process would classify the resource as to-be-indexed or not-to-be-indexed, and then selecting all resources classified as to-be-indexed as the second group of resources, wherein the determination whether the first automated resource selection process would classify a particular resource as to-be-indexed or not-to-be-indexed is made independently of the determination whether the second automated resource selection process would classify that particular resource as to-be-indexed or not-to-be-indexed. 12. The system of claim 1 , wherein the operations further comprise: for one or more of the test queries: receiving the test query through a search engine user interface from each of a plurality of users; presenting search results corresponding to the first group of resources to one or more first users in the plurality of users in response to receiving the test query from each first user, and presenting search results corresponding to the second group of resources to one or more second users in the plurality of users in response to receiving the test query from each second user; and comparing user assessment of the search results corresponding to the first group of resources and user assessment of search results corresponding to the second group of resources, resulting in a comparison; and analyzing the comparison for each of the one or more test queries to select either the first automated resource selection process or the second automated resource selection process.