Sorting Big Data by Revealed Preference with Application to College Ranking
Xingwei Hu
stat.ML cs.LG econ.GN q-fin.EC 91A11, 91B08, 91B42, 91B68, 91B69, 91D30 E.4; F.2.2; G.3; J.4






	1417 Sorting Big Data by Revealed Preference with 
 Application to College Ranking[To cite this article: Xingwei Hu (2020): Sorting Big Data by Revealed Preference with Application to College Ranking, Journal of Big Data, DOI: 10.1186/s40537-020-00300-1][The views expressed herein are those of the author and should not be attributed to the IMF, its Executive Board, or its management.]
	
	
	
	

	Xingwei Hu[
		Email: xhu@imf.org; 		Phone: 202-623-8317; 		Fax: 202-589-8317.
	][
	The author acknowledges many helpful and detailed comments from anonymous referees.
	The author also thanks Jinpeng Ma, Kei Moriya, Bruce Moses, LLoyd S. Shapley, Wei Zhang, and seminar participants at International Monetary Fund and Stony Brook University for suggestions.
	] 
	
	.5cm
	International Monetary Fund 

	700 19th St NW, Washington, DC 20431, USA
	



Abstract:


When ranking big data observations such as colleges in the United States, diverse consumers reveal heterogeneous preferences. 
The objective of this paper is to sort out a linear ordering for these observations and to recommend strategies to improve their relative positions in the ranking.
A properly sorted solution could help consumers make the right choices, and governments make wise policy decisions.
Previous researchers have applied exogenous weighting or multivariate regression approaches to sort big data objects, ignoring their variety and variability.
By recognizing the diversity and heterogeneity among both the observations and the consumers, we instead apply endogenous weighting to these contradictory revealed preferences.
The outcome is a consistent steady-state solution to the counterbalance equilibrium within these contradictions.
The solution takes into consideration the spillover effects of multiple-step interactions among the observations.
When information from data is efficiently revealed in preferences, the revealed preferences greatly reduce the volume of the required data in the sorting process. 
The employed approach can be applied in many other areas, such as sports team ranking, academic journal ranking, voting, and real effective exchange rates.





	5.2in
		
			
			
  : 
			revealed preference;			authority distribution;			endogenous weighting; 			college ranking;			big data; 			matching game;			sort; 			counterbalance equilibrium
			
  :
			C68, C71, C78, D57, D58, D74
		
	












§ INTRODUCTION


The problem we address here relates to the following typical situation: 
millions of consumers face hundreds of alternatives when making decisions. 
Each alternative is of significant complexity and variety.
From an affordable and pre-selected shortlist, a consumer picks just one alternative, thus revealing some preference over others on the shortlist.
Our objective is to sort all of these alternatives according to revealed preferences. 
One concrete context is to rank the colleges in the United States, where each student selects only one college to attend when admitted by multiple institutions.

The results in this paper arise from two considerations. 
On the one hand, college ranking is of broad public interest. 
The consumers are not only students and their families but also governments that aim to rationalize their allocation of funding as well as college administrators who are interested
in comparing their ranking to peer institutions  on a national level. 
Alumni pay attention to similar comparisons, and  leading companies prefer to hire graduates from the best colleges.
As a consequence, there are dozens of college rankings in the U.S. market, including those published by mainstream media, such as U.S. News & World Report, 
Washington Monthly, the Wall Street Journal (WSJ), Time, and Forbes.
As indicated by many researchers (e.g., Bastedo and Bowman, 2010 and 2011), these rankings have demonstrable effects on potential students and administrators due to their perceived influence on resource providers.
On the other hand, however, almost all these rankings rely on some preset weighting system using merely several criteria, such as acceptance rates, standardized test scores, alumni donation rates, and class sizes.
The weights are then applied to each college to obtain a ranking score. 
The selection of criteria and the weights are subjective; a slight variation leads to a different ranking result.
A consequence of this methodology is an isomorphism: the diversity of the higher educational institutions are not valued. 
Many researchers have claimed such rankings often harm higher education (e.g., Craig, 2015; Moed, 2017; Perez-Pena and Slotnik, 2012).

Our solution to the problem is to apply authority distribution (Hu and Shapley, 2003) to the revealed preferences by the students.
We believe there  exists no exogenous weighting system applicable to all colleges. 
Each college is unique, and just a few measurements cannot accurately portray it.
For example, a liberal arts college and an engineering-focused college could value SAT scores and faculty publications quite differently.
The characteristics of a college are embodied in the applicants in general, and the admitted students in particular.
We remove the subjective weighting systems and ignore the criteria selection. 
However, we believe each consumer could have his or her specific weighting on a much broader set of criteria, including qualitative and latent ones.
 A student could have dozens of other considerations, such as the distance from home, tuition, sports, job and internship potential, personal connections, campus visit experience, and median salary after graduation —far beyond the few criteria listed in mainstream media.
Personal considerations and weighting are effectively revealed in his or her preference when selecting one college and rejecting others.
Identifying logical and consistent inferences from these personal preferences is the main challenge.
 Hu and Shapley (2003) partially overcomes the challenge and also includes a simple college ranking using artificial data. 
In this paper, we build a detailed roadmap to get around the obstacles in ranking colleges.

 The approach has three advantages.
First, we allow heterogeneity from the consumers as well as from the colleges. 
We use a massive endogenous weighting scheme to avoid any subjective weighting and criteria selection.
Secondly, our method is robust. 
We use revealed preferences  as a dimensionality reduction mechanism —each preference could have taken into consideration countless selection criteria, and the ranking uses preferences from millions of households.
Additionally, our ranking considers long-term and system-wide influences by aggregating spillover effects from direct bilateral influences.
Thirdly, the specific application to the game-theoretic authority distribution respects individual rationality and values collective welfare. 
It seamlessly links authority distribution with revealed preference, heterogeneity, big data, endogenous weighting, strategyproofness, and public interests in college ranking. 
Furthermore, we envision many potential applications in other fields.
As a consequence of these advantages, we believe our ranking methodology is more authoritative than those used in the popular rankings.

We organize the remainder of the paper as follows. 
Section <ref> summarizes the critical issues in other rankings and introduces the new sorting methodology, which integrates the revealed preference  with the authority distribution framework.
Section <ref> analyzes the data, calculates the ranking scores, and simulates their confidence intervals.
 Next, Section <ref> discusses a few properties of the ranking method.
Notably, we address strategies to improve ranking scores.
Section <ref> studies an empirical ranking using real data.
 In Sections <ref>, we discuss vulnerabilities, extensions, and other applications of the framework.
Finally, we conclude in Section <ref>.
Our exposition is self-contained, and the proofs are in the Appendix.





§ THE METHODOLOGY

Before our formal discussion, we introduce a few notations.
Let ℕ = {1, 2,⋯,n} denote the set of colleges in the United States, indexed as 1,2,..., n. 
In the time frame of one academic year, let A_i be the number of students admitted by college i. 
Out of the A_i admitted students, Ñ_ij students would attend college j ∈ℕ.
We use a multinomial distribution to model these random numbers Ñ_ij, using the parameters A_i and (P_i1,⋯, P_in).
The probability P_ij is the likelihood that anyone in A_i would enroll in college j.
Of course, the student is also admitted by college j.
We write matrix P =  [ P_ij ], i,j=1,2,⋯, n.
After the admission and acceptance process has been completed, we observe a Ñ_ii, which is the number of enrolled students, denoted by E_i.
For j≠ i, however, Ñ_ij are generally not observable at this time  or thereafter.
The data A_i and E_i are available  from many sources, including the National Center for Education Statistics (NCES) and the Common Data Set (CDS) for these colleges.





 §.§ Key Issues in Popular Rankings

Of the dozens of popular college rankings in the United States, each uses several criteria to weight the colleges and calculate a weighted score for each college. 
For example, the following seven criteria drove the U.S. News & World Report's college ranking ( U.S. News) in 2019:
Graduation and Retention Rates (22.5%),
Undergraduate Academic Reputation (22.5%),
Faculty Resources (20%),
Student Selectivity (12.5%),
Financial Resources (10%),
Graduation Rate Performance (7.5%),
and Alumni Giving Rate (5%).
This weighting system raises several questions: Why are other criteria not included? How were the weights determined? How do we measure qualitative variables such as selectivity and reputation? 
 Also, “giving rates” and “resources” data are vulnerable to manipulation.

A generic recipe for these rankings is as follows.
Let there be k criteria, w_j be the weight on the jth criterion, and x_ji be college i's score on the jth criterion.
These rankings use 

    (
    w_1, ⋯, w_k
     )
     [
    [ x_11    … x_1n;    ⋮    ⋮    ⋮; x_k1    … x_kn ] ]
    =
     (
    ∑_j=1^k w_j x_j1,
    ⋯,
    ∑_j=1^k w_j x_jn )

to calculate a ranking score ∑_j=1^k w_j x_ji for college i. The ranking scores on the right side of (<ref>) are then used to sort the colleges. 
Variants of criteria and their weights can produce quite different ranks for the same college. 
Table <ref> is an example: the lowest rank of 31 for UCLA (University of California, Los Angeles) is six times more than the highest rank of 5 for the same college. 




However, these popular rankings have been widely criticized (e.g., Bruni, 2016; Ehrenberg, 2005; Grewal, Dearden, and Lilien, 2008; Luca and Smith, 2011; Moed, 2017; Perez-Pena and Slotnik, 2012). 
Let us mention just a few reasons why.
First, the premise of these popular rankings —that a few criteria will produce a complete comparison for all colleges— is not universally shared. 
It does not offer a realistic scenario nor a good approximation of one. 
Each college has not only a unique location but also unique features.
Some , for example, focus on research while others on teaching. 
Some are science- and engineering-based, while others are arts- and social science-oriented.
Secondly, these rankings hardly align with the selection criteria of consumers.
In particular, they ignore several vital concerns of prospective students,
such as the distance between college and home, campus life, sports, median post-graduation starting salaries, location, and the number of graduate programs. 
 Of course, tuition and room and board are among the most significant concerns for students who apply for education loans.
All of these considerations are taken into account when choosing one college out of many.
Thirdly, these rankings inherit the subjectivity and bias from a single model designed by a small committee of educators and business people.
For example, the weights w_j in (<ref>) are very subjective; many scores x_ji —such as academic prestige— are also subjective,
and they are subject to manipulation or inflation by colleges.
Lastly, these criteria-based rankings discourage institutional diversity and reinforce similarities among universities.
If colleges were to base their development plans on the U.S. News criteria in order to raise their ranks, there might be little value in retaining programs such as sports.

A recent ranking by Avery et al. (2013) is worth highlighting. 
In their research, they study several econometric models that describe the decision process by colleges versus applicants. 
An extensive set of data was collected to estimate the unknown weights in these models. 
The result is three sets of rankings.
In contrast, we study college versus college comparisons, using rejection and acceptance by the students as the basis for comparison. 
We believe that sorting or ranking is about comparisons, and anyone's preference is already a ranking between two colleges by that individual.
Also, we argue that a constant coefficient in an econometric model cannot capture the marginal effect for all schools or all students. 
If a regression model is used, then its residual term accounts for the distinctive characteristics of a college or the unique considerations of a student.
Unfortunately, the residual term is generally treated as random noise.
Our new ranking method, in contrast, ingests every piece of recent public information about a college, including any scandals and victories by the football team, for example.

In contrast to (<ref>), we write a deterministic decision function F_is(⋯) for student s' choice y_is on college i,

    y_is = F_is(x_1, x_2, x_3, ⋯)

where (x_1, x_2, x_3, ⋯) is an indefinite list of all possible considerations for all of the students.
The big data (x_1, x_2, x_3, ⋯) could include thousands of explanatory variables about the colleges.
Being viewed as an observation of available data  on the Internet, a college could be as “big” as Harvard.
However, these functions F_is(⋯) are personal-specific and not identifiable, given the availability of the data. 
Similarly, we could also define an indefinite-dimensional decision function for each college.
  Because of the dimensionality of (x_1, x_2, x_3, ⋯) and the privacy of F_is(⋯), some level of data abstraction becomes indispensable to order the colleges from the best to the worst.
In the literature, however, there is hardly any research which targets sorting big data objects, though
many (e.g., Eeckhout, Pinheiro, and Schmidheiny, 2010 and 2014; Dohmen and Falk, 2011; Langville and Meyer, 2012) have sorted complicated objects using spatial, network, or multidimensional analyses.





 §.§ Revealed Preference

Revealed preference theory (cf. Samuelson, 1948) analyzes the actual choices by individual consumers who have heterogeneous utility functions. 
An individual may execute individual rationality or bounded individual rationality by maximizing his or her utility function.
Individual consumers may also demonstrate behavioral biases in making their choices.
Revealed preference, however, does not hold the axioms of a utility function or a multi-person utility function (Baucells and Shapley, 2008).
The focus is on what choice y_is individual s makes, not on how he or she makes the choice.
Thus, we ignore the personal decision function F_is(⋯) and the explanatory variables (x_1, x_2, x_3, ⋯) in (<ref>).
The choice y_is has considered them.
Personal considerations, however, play an essential role in making an individual choice. 
A typical practice for a student is to compare one college with another without knowing or using any utility function.
Thus, a universally accepted set of considerations and decision functions does not work for millions of high school seniors. 
 In the literature, revealed preference analysis had been used as an alternative to regression approaches (e.g., Chirinko and Schaller, 2004; Pritchard, 2018; Tieskens et al., 2018).


 By ignoring F_is(⋯) and (x_1, x_2, ⋯), we transform the big data into a high-dimensional space of y_is.
In the transformation, the underlying assumption is that personal preferences have adequately reflected the ignored data.
A significant reason for the assumption is the informational efficiency in the preference revelation.
The efficiency is driven by the strong motivation of parents and students to acquire relevant information.
 To gather that data, they might fly thousands of miles for a campus visit or spend weeks in examining slight differences between two candidate colleges.
The efficiency is also due to the collective power to assemble and assimilate the information by millions of consumers, 
though an  individual is influenced by emotional biases and could make cognitive errors in processing the information.

 Colleges vary significantly among the higher education system.
A college is also a rational decision-maker that admits the best students it can attract. 
Its preference is effectively revealed in its admitted students.
Accordingly, there exists no universally accepted set of criteria for all these colleges. 
For example, a research-focused college may have a different utility function, if there is any, from a teaching-oriented college.
A public university has to comply with specific enrollment mandates set by state authorities.
The mandates and public spending on higher education are also not acknowledged in the rankings by mainstream media.
Admission procedures vary among colleges.
Unlike the idealized selection process stated in Gale and Shapley (1962), a college cannot rank all of its applicants, and it generally offers more admissions than needed.
It may establish a waiting list, and it may also offer early admissions and scholarships in order to hold a repository of students who can no longer accept offers from other schools.

In the many-to-many matching game between the students and the colleges, preference is officially revealed at least three times.
First, a student files applications to a shortlist of pre-selected schools, ignoring any unfavorite or unfit ones.
A student could take years to observe and consider the shortlist before filing applications.
Preference is revealed a second time when a college admits potential students who satisfy the standards of the college.
 A college admission officer also considers enrollment size and diversity of the admitted students. 
In the last stage, the student decides on his or her favorite school and rejects all others.

Lastly, these revealed preferences are contradictory; they offer no utility function nor any linear ordering of the alternatives.
Indeed,  neither students  nor parents use a utility function or an econometric model to make decisions.
They make pairwise comparisons.
 Nevertheless, we cannot apply any comparison sorting algorithms for arrays, such as Quicksort or Heapsort; revealed preferences also show how much one alternative is preferred over another (cf. Table <ref>).
This paper applies authority distribution
to combine these conflicting preferences, which gradually spills over the direct influence among all colleges.
The evolution of the spillover effects eventually smooths out all conflicts among the preferences. 
The infinite-step evolution process can be simplified by solving a counterbalance equilibrium equation in (<ref>).




 §.§ Authority Distribution

In the context of authority distribution, there is a network of multiple players. 
These players have a direct influence, broadly called power, over each other. 
For each player, the network constitutes a “command game” by the direct influence on the player. 
 Together, there are n command games that describe the power-in-and-out dynamics in the network.
When the players are sufficiently connected, there exists a general equilibrium, called counterbalance equilibrium, which is the steady-state solution for the power flow dynamics.
A player derives authority from others, whom he directly influences. 
When we apply authority distribution to sort the U.S. higher education institutions, the players are the colleges.
The direct influence could be, for example, the acceptance and rejection by prospective students, sports,
faculty recruitment, research fund competition, and publication cross-citation. 
In this paper, we focus solely on the preferences of potential students who make rational choices after observing the other aspects of influence;
we argue that consumers' choices make the most comprehensive interactions when comparing two choices.
In the literature, researchers (e.g., Bastedo and Bowman, 2011; Correntea, Grecoa, and Lowinskic, 2018; Grewal, Dearden, and Lilien, 2008) have also recognized the interdependence of institutions when evaluating colleges.

For college i, for example, its pool of admitted students and their choices constitute a game of foot voting in which students' preferences are explicitly exercised through their actions of acceptance or rejection, i.e., y_is. 
In this student-college matching, dollar voting is part of the foot voting, as tuition is a significant concern for parents and student borrowers.
Ironically, by excluding tuition in their criteria, mainstream media ignore one of the most salient issues among  education consumers.
Besides, the pool itself represents college i's characteristics: it could be competitive or not; it could be small or large based on its enrollment size.
Statistics about GPAs, standardized test scores, and demographic data are already embedded in the pool of admitted students.

Colleges i and j become rivals to compete in recruiting the shared pool of students when their pools overlap.
Each college has its rivals, and the number of rivals varies across the higher education system. 
Additionally, two non-competing colleges may have one or more common rivals, and
three or more pools could also intersect (see Figure <ref>(a)).
Consequently, a competitive battlefield of such a shared pool could involve many players.



For all the pools, the power transition matrix P collectively specifies the likelihood of any student's final choice after admission.
As the proportion of students who decide to attend college j, 
P_ij measures college j's direct influence or power in college i's foot voting game.
Asymmetry generally exists when P_ij≠ P_ji; thus, power flows from one college to another unevenly.
 Figure <ref>(b) shows the bilateral power movement among three colleges; each has a direct influence on the other two. 
 Besides, when aggregating y_is to the matrix P, we further reduce the dimension of data.

Authority distribution associated with P is a row vector π = (π_1, π_2, ⋯, π_n) which satisfies the counterbalance equilibrium

    (π_1, π_2, ⋯, π_n) = (π_1, π_2, ⋯, π_n) P

subject to the normalization condition ∑_i=1^nπ_i = 1 and positivity condition π_i ≥ 0 for all i∈ℕ.
We can see a direct influence from two aspects.
For a specific i∈ℕ, it derives authority from others on whom it has a direct impact, i.e., by (<ref>),

    π_i = ∑_j=1^n π_j P_ji.

Equation (<ref>) describes how power flows from other players into i.
It derives more authority from influential players than from non-influential players, other things being equal. 
Moreover, it also derives more authority from players on which it has a large direct influence, other things being equal. 
In the second aspect, i also contributes to other colleges that have direct influences on i. This can be seen from the power outflow equation

    π_k =  π_i P_ik +  ∑_j≠ iπ_j P_jk.

The larger P_ik, the more college i contributes to college k's authority.
The inflow power in (<ref>) and outflow power in (<ref>) eventually reach an equilibrium described by (<ref>).
In essence, π is a power index in the interactive and yet controversial network.
The derivation of the counterbalance equilibrium and the properties of π were studied in Hu and Shapley (2003). 
This distribution has been mostly used in social networks, corporate networks, and controls (e.g., Crama and Leruth, 2007; Grabisch and Rusinowska, 2010).
 Compared to Google's PageRank (e.g., Langville and Meyer, 2012), authority distribution drops the damping factor and has a stochastic matrix with a non-zero diagonal.




§ ESTIMATION
 
This section estimates P, π, and its confidence interval, mean, and median.



 §.§ The Data

All the data used to estimate P come from the Internet.
With regard to sampling errors, official admission data provide high precision to the diagonal elements of P while online survey data cast more doubt on the off-diagonal ones.



  §.§.§ The Official Enrollment Rates

The probability P_ii is the likelihood that any student admitted by college i would enroll in college i.
We can estimate it by the enrollment rate

    P̂_ii = E_i/A_i.

The enrollment rates are available on many websites, including the CDS, the NCES, Wikipedia, U.S. News, and those for colleges.
Table <ref> is an example of admission statistics for UCLA between 2013 and 2018. 
In this table, UCLA admitted 15,988 of 113,779 applicants for fall 2018. Out of the admitted students, 6,240 enrolled at UCLA. 
Thus, the enrollment rate was 6,240/15,988 = 39.03% for 2018.







  §.§.§ The Survey Data of Preference

When i≠j, P_ij is the percentage of students who are admitted by college i but  decide to attend college j. 
 These data  are not wholly available, so we estimate them from the survey data of preference posted on Parchment, Niche, and similar websites.
Many students also post their decisions on online discussion boards.
Table <ref> lists a sample of revealed preferences, described as odds ratios, for twelve colleges. These include seven private universities and five public ones, often ranked highly by mainstream media. 
At the ith row and jth column, the numerator is the percentage for college i and the denominator for college j.
For example, of the students admitted by both Harvard and Stanford and deciding to attend either Harvard or Stanford, 44% choose Stanford, and 56% Harvard.
Also, the percentage 56% has the .95 confidence interval between 51.2% to 60.7%, listed on the Parchment website, and calculated by the Wilson score method (Wilson, 1927). 
Similarly, the percentage 44% has the .95 confidence interval between 39.3% to 48.8%.
Of course, these students may also be admitted by other schools.



 Some simple conclusions could be reached by using only the data in Table <ref>. 
First, column 1 shows that Harvard is preferred over all other colleges; a reasonable ranking would place it before the others, regardless of the confidence intervals of or the weights on the odds ratios.
Secondly, we would expect Stanford to place second since it is preferred over all others, except Harvard (cf. Column 2). 
It is also at the top when Harvard acts as the reference (cf. Column 1).
A further analysis implies that MIT and Yale would compete for third place.
They are preferred over all other schools except Harvard and Stanford (cf. Columns 3 and 5).
When either Harvard or Standford acts as the reference (cf. Columns 1 and 2), they place higher than all other schools except Stanford and Harvard, respectively.

However, there are a few data issues worth mentioning, as different odds ratios come from different shared pools of students.
First, the transitivity of preference does not hold. 
For example, we could find a circular chain of preference, Mich ≼ UVA ≼ UCLA ≼ Mich, when using the bilateral preferences only. 
Secondly, the ratios are not multiplicative. 
For instance, UCLA ≼ Mich by 43/57 and UCLA ≼ UCB by 47/53 does not imply UCB ≼ Mich by 53/4743/57 = 2,279/2,679. 
On the contrary, Mich ≼ UCB by 39/61.
Thirdly, contradictory orderings exist if we compare colleges using different benchmarks. 
For example, if we use Harvard as the reference benchmark (cf. Column 1), then Mich ≼ UCLA ≼ UVA. 
But if we use UCB as the reference school (cf. Column 8), the ordering is UVA ≼ Mich ≼ UCLA.
If Harvard has more authority than UCB, then the first ordering should be weighted more than the second one.
Also, spatial adjacency plays a crucial role in these odds ratios. 
In the seventh row, for example, Berkeley has the lowest odds ratio versus Stanford, and its distance from Stanford is also the shortest.
A similar case is the seventh column, where UCLA has the lowest odds ratio against CalTech, and the shortest distance from CalTech.
Lastly, the odds ratios alone are not enough to determine all P_ij; we are more interested in the number of students who are admitted by college i but decide to go to college j.






 §.§ Conditional Estimability of P
 
This subsection illustrates how to estimate P_ij, conditional on P_ii = E_i/A_i and other observations.
Clearly, the enrollment rate E_i/A_i is an unbiased estimate for P_ii. 
Besides, both E_i and A_i from the CDS and the NCES have an annual frequency.
But the preference survey data have a real-time frequency, and they likely come from multiple years. 
In order to fix the frequency divergence, we assume that both the odds ratios and the enrollment rates remain stable across multiple years.

 To link the odds ratios in Table <ref> to P, we capitalize on the confidence intervals of the odds ratios.
Out of the students who are admitted by both colleges i and j and decide to attend either i or j, we let S_ij be the likelihood a student would choose college j. 
We also let M_ij be the number of those students who participate in the online preference survey and
let s_ij be the proportion of the surveyed students who decide on college j.
 Clearly, M_ij=M_ji and the odds ratio is 1-s_ij/s_ij.
We define ω_ij = 2 s_ij M_ij + z^2/2 M_ij + 2 z^2 where z is the .975 percentile of the standard normal distribution.
Given s_ij and M_ij, the .95 confidence interval for S_ij, calculated by the Wilson score method, is

    ω_ij ± z/2 M_ij + 2 z^2√( 4 s_ij(1-s_ij) M_ij + z^2).

Replacing s_ij with E_i/A_i and M_ij with A_i in (<ref>), we obtain the Wilson confidence interval for P_ii.
Given s_ij and the confidence interval (<ref>), we can extract M_ij by Lemma <ref>.
Alternatively, we can also extract M_ij using s_ij and the confidence interval length 
z/M_ij + z^2√( 4 s_ij(1-s_ij) M_ij + z^2).

 
Let τ = 1 - s_ij and let η be the lower confidence bound in (<ref>). Then

    M_ij 
    = 
    τ s_ij-(s_ij-η)(1-2η) + √([(s_ij-η)(1-2η)-τ s_ij]^2 + 4η (1-η) (s_ij-η)^2)/2 (s_ij-η/z )^2.



Thus, N_ij s_ij M_ij is the number of students in the survey who are admitted by college i but decide to attend college j.
Consequently, ∑_k ≠ i N_ik is the number of students in the survey who are admitted by college i but decide not to attend college i.
This is about 1-P_ii of the students in the survey who are admitted by college i, if  each student has the same likelihood to take the survey.
Therefore, we estimate P_ij by

    P̂_ij  N_ij/∑_k≠ i N_ik (1 - E_i/A_i ),      j≠ i.

As stated in Theorem <ref>, P̂_ij is an unbiased estimate for P_ij given P_ii = E_i/A_i, the size of the survey data ∑_k ≠ i N_ik, and certain reasonable assumptions.
 
Assume all students accepted by college i have the same likelihood to independently participate in the preference survey. 
Then

    E [ P̂_ij |  . P_ii = P̂_ii, ∑_k ≠ i N_ik ] = P_ij,   ∀ j≠i.



 As the breadth and size of each college differ significantly across the education system, we mitigate the size effect in the enrollment without modifying the odds ratios.
The above estimation of P_ij for all j≠ i is based on unequal sample sizes.
The size of M_ij relies on the size of A_j, but A_j varies across all j≠ i.
To make P_ij and P_ik comparable roughly under a common sample size, we scale the length of the Wilson confidence interval (<ref>)
by √(E_j/max_k≠i E_k), i.e., the scaled Wilson confidence interval is

    ω_ij ± z/2 M_ij + 2 z^2√( 4 s_ij(1-s_ij) M_ij + z^2) √(E_j/max_k≠i E_k).

Before the scaling, the length is of order O ( 1/√(M_ij) ). 
After the scaling, it is of O  ( √(E_j/M_ij) ), 
ignoring the common denominator √(max_k≠i E_k).
If E_j/M_ij is close to a non-zero constant as E_j→∞, then the length of the scaled interval is 
of O ( 1/√(max_k≠i E_k) ) and 
the common sample size is of O  ( max_k≠i  E_k  ).
As P_ij only counts the students enrolling in college j, we  use E_j in (<ref>) to exclude the students who do not enroll in college j.
 Using max_k≠i E_k guarantees that the scaled intervals (<ref>) lie in (0,1).
The scaling by a constant is different from the actual increase of the sample size; the latter extracts more information from new data and thus reduces the uncertainty in estimation.
Scaling by the square root of sample size is a common practice to balance the breadth and depth, e.g., t-statistic with unequal sample size, the investor's breadth, and Grinold and Kahn (2011).
In summary, Algorithm <ref> estimates P_ij when the enrollment sizes E_j have a large variation:







 §.§ Confidence Interval of π

After estimating P, we solve the counterbalance equation π̂= π̂P̂ by Algorithm <ref> , where 1_n is the column vector with n ones.
According to the theory of Markov chains, π^(t) converges if P̂ satisfies certain properties specified in Theorem <ref>. 
After setting the final π^(t) to π̂,
we sort the vector π̂ from the largest value to the least. 
The college with the largest value ranks first, and the college with the least value ranks last. 


The uncertainty of π̂ comes from the estimation of P. 
We can capitalize on the Monte Carlo simulation to estimate the confidence interval of π̂,
by simulating 20,000 power transition matrices P.
For each simulated P, we calculate a ranking score π̂.
From these 20,000 sets of ranking scores π̂, we extract the .95 confidence intervals of π,
the mean and median ranking scores, and the .95 confidence intervals of ranks.
Algorithm <ref> can be used to simulate a P. 
Note that each row of the simulated P already sums to 1.






§ PROPERTIES OF THE SORTING

This section discusses a few properties of π. Some of them relate to the theory of homogeneous Markov chains. Hence, one could use the theory to find more properties.
 Besides, we highlight a few ways to improve a college's ranking score. 



 §.§ Endogenous Weighting

The explicit dimension of the matrix P is n.
Moreover, each P_ij is a result of choices made by many students who are admitted by both colleges i and j;
each student considers an indefinite number of reasons which are not listed in P_ij.
For any i ∈ℕ, we use college i as the benchmark or reference to rank all colleges. 
The ranking scores are a row vector of

    μ_i   (P_i1, P_i2, ⋯, P_in).
 
The more college j influences i, the higher its ranking score P_ij.
This way, we define n sets of ranking scores, μ_1, μ_2, ⋯, μ_n.
As each college is unique (i.e., not a linear combination of others), each reference college has its own dimensionality. 
Thus, we have n ranking scores on n-dimensional axes, in contrast to merely several axes as in the rankings by mainstream media.

In contrast to (<ref>),
we apply endogenous weighting to the references to combine these ranking scores μ_1, ⋯, μ_n. 
Essentially, we let colleges judge colleges themselves without any external interventions from business interests, mainstream media, advertising, or college administrators.
In doing so, we believe that more weight should be placed on a good reference (i.e., with large π_i), and less weight should be placed on a bad one (i.e., with small π_i).
Thus, we have weighted ranking scores ∑_i=1^n π_i μ_i = π P. 
As the weighted ranking scores also quantify the quality of the schools as references, π P should be a multiple of π.
By Theorem <ref>, there exists a unique equation that links the endogenously weighted scores π P to a constant multiple of π; and that equation is (<ref>).
When π is not assumed to have a unit sum, we can still have the equation π = π P as long as the sum of π P does not deviate from that of π.


If π P = βπ for some β>0, then β=1.





 §.§ Spillover Effects

The distribution π takes into consideration the spillover effects of direct influence, making non-competing colleges comparable and smoothing out controversial competitions so that they become consistent.
In a two-step spillover π = π P = π P^2, for example, 

    π_i = ∑_k=1^n ∑_z=1^n π_k P_kz P_zi,

i has a direct influence on z when P_zi>0 and z has a direct influence on k when P_kz>0. 
Then, i has an indirect influence on k even if i may have no direct influence on k;
π_i collects indirect influence from all players in ℕ.
For any integer m>1, we may also consider an m-step indirect influence using P^m. 

There are many similarities between π and the invariant measure in the theory of homogeneous Markov chains.
 We borrow the concepts of irreducibility and aperiodicity from the theory. 
In general, P satisfies both irreducibility and aperiodicity if we consider the top 300 U.S. colleges. 
In the preference survey data, for example, there is a chain of direct bilateral influence: Northern Virginia Community College (NVCC) ↔ George Mason University ↔ UVA ↔ Harvard.
Thus, seemingly unrelated Harvard and NVCC establish an indirect influence over each other through spillovers of up to three steps.
Besides, irreducibility and aperiodicity guarantee the convergence in Algorithm <ref>.
Finally, the solution to

    π = π P^m

is the same as the solution to (<ref>), as stated in Theorem <ref>.


If P is irreducible and aperiodic, then the solutions to (<ref>) and (<ref>) are equivalent.


 The ranking score π_i is college i's long-run influence in the higher education system and across all other colleges.
A straightforward implication of Theorem <ref> is that π is invariant to any steps of indirect influence. 
If we treat P^m as a new power transition matrix, then conflicts in P^m are less severe than those in P whenever m>1.
As m→∞, P^m gradually smooths out all conflicts in P. 
In the long-run in P^∞, all rows are π, and there are no more conflicts.
Moreover, the spillover effects take two further actions: amplification of real comparative advantages and off-setting of noised ones.
One consequence is that some elite schools have substantial authority compared to non-elite ones.




 §.§ Strategies to Improve π_i

For college i, a policy implication from the ranking is how to boost its relative strength in the higher education system.
Both the college and its students can improve π_i.
Collaboration with another college may also augment its ranking score.

We introduce a few more notations for the next two theorems. 
Let I_n be the n× n identity matrix and let π__-i be the transpose of π with π_i removed.
The column vector α_i takes the ith row of P and then drops its ith element, and the matrix Z_i is the transpose of P with the ith row and the ith column removed.
Also, the column vector γ_ij extracts the jth row from P with the jth element replaced with zero, and the ith element dropped.




  §.§.§ Unilateral Strategies

At the institutional level, college i could improve its π_i by increasing its enrollment rate P_ii. 
A university can attract its admitted students by such recruiting strategies as tuition discounts, research opportunities, and scholarships.
It could also market its reputation through college sports and alumni networks.
We should not, however, use the single criterion P_ii to rank the colleges
—a college could artificially inflate its enrollment rate by simply admitting non-competitive applicants.
Theorem <ref> specifies the exact effect of a small variation of P_ii on π.
Derivative (<ref>) measures the response multiplier of π_i for a given small shock of P_ii.
Not surprisingly, by (<ref>), rising P_ii has a non-positive effect on π_j for all j≠ i.



    dπ_i/d P_ii
    = 
    π_i/1-P_ii1_n-1' (I_n-1-Z_i)^-1α_i/1+1_n-1' (I_n-1-Z_i)^-1α_i≥ 0

and

    dπ__-i/d P_ii 
    =
    -  π_i/1-P_ii (I_n-1-Z_i  )^-1α_i/1+1_n-1' (I_n-1-Z_i)^-1α_i.

Therefore, dπ_j/d P_ii≤ 0 for all j≠ i.


At the student level, students could share their private information about college choices on the preference survey websites.
A higher π_i is in line with the interest of any student who is committed to enrolling in college i.
The student has private information about his or her personal choices, e.g., enrolling in college i and rejecting college j. 
Thus, he or she has a strategy to reveal or not the private information on the survey websites.
If the private information is released, then the jth row of P changes slightly as P_ji increases, but P_jj remains the same.
As each choice is counted in π, both the school and the student are better off if the private information is revealed, according to (<ref>) in Theorem <ref>.
Thus, there is no incentive for students to hide the private information, i.e., strategyproofness.
Derivative (<ref>) also implies that college i improves π_i more when recruiting students from a more competitive college j (i.e., with a more significant π_j), other things remaining constant.

 For any j≠ i,

    dπ_i/d P_ji
    = 
    π_j/1-P_ji-P_jj1_n-1' (I_n-1-Z_i)^-1γ_ij/1+1_n-1' (I_n-1-Z_i)^-1α_i≥ 0

and 

    dπ__-i/d P_ji
    = 
    π_j/1-P_ji-P_jj (I_n-1-Z_i)^-1 [ 1_n-1' (I_n-1-Z_i)^-1γ_ij/1+1_n-1' (I_n-1-Z_i)^-1α_iα_i - γ_ij ] .






  §.§.§ Bilateral Cooperation

The counterbalance equilibrium has a mixed cooperative and non-cooperative character.
From the cooperative side, player i would assist j in improving π_j whenever P_ji>0
because π_jP_ji is a component of π_i = ∑_k=1^n π_k P_ki.
From the non-cooperative side, as π1 = 1, an increase of π_j may mean a decrease of π_i.
 Thus, the trade-off is how much player i should assist player j without sacrificing himself or herself.

 A relevant policy question is how to identify the cooperators and the competitors for player i.
By (<ref>), dπ_j/d P_ji may also be positive. 
If this happens, then colleges i and j would form a cooperative partnership to improve their relative strength by slightly increasing P_ji.
Formation of this partnership does not involve a third party, so it is easily enforceable.
Once the partnership forms, (<ref>) calculates the effects on third parties.
Besides, a third party, say, college k, automatically acts as a battlefield for the competition between colleges i and j.
The reason is that both dπ_i/d P_ki and dπ_j/d P_kj are non-negative according to (<ref>), but P_ki directly conflicts with P_kj.






§ RESULTS FROM A MINIATURE RANKING
 
It is not our intention to generate a new college ranking to compete with the commercial ones. 
Also, data collection and computational costs are high if we sort all of approximately 3,000 U.S. colleges. 
Any truncation of the complete list, however, distorts the ranking results to some degree.
A moderate-sized ranking, based on the above methodology, should be in real-time and online, changing as often as the data of revealed preferences.
To illustrate the methodology in the last sections, however, we analyze the twelve colleges in Table <ref>, using the data from the websites mentioned above.
Table <ref> reports the ranks and their .95 confidence bands, ranking scores and their .95 confidence bands, and means and medians of 20,000 simulated sets of ranking scores.





 As a summary of Table <ref>, the elite private colleges are far ahead of their elite public peers.
Harvard and Stanford capture nearly half of the total authority, due to  the spillover's amplification effect.
 Because of this effect, for example, Harvard's ranking score is 40% higher than Stanford's while their odds ratios are much closer.
The amplification comes from Harvard's relative advantages over other colleges, compared to Stanford's.
Berkeley and Michigan have very close ranking scores; their confidence intervals overlap by 23%.
To separate them effectively, we could add more colleges to the ranking.
We could also perform secondary analyses based on the estimated ranking scores.
Regression of the scores on the tuitions, for example, finds which universities are undervalued and which are overvalued.

The estimated power transition matrix P̂ is
89
    [
    [   HRV   SFD  Yale   PRT   MIT   CCG   CIT   UCB  Mich  UCLA   UVA   UNC; .7942 .0464 .0411 .0257 .0242 .0167 .0128 .0100 .0132 .0069 .0062 .0026; .0723 .7581 .0402 .0260 .0266 .0158 .0338 .0079 .0090 .0046 .0024 .0032; .1070 .0634 .6753 .0299 .0468 .0196 .0249 .0046 .0148 .0067 .0035 .0036; .1059 .0640 .0514 .6403 .0426 .0215 .0401 .0086 .0089 .0073 .0054 .0040; .0616 .0721 .0281 .0340 .6956 .0177 .0543 .0099 .0099 .0089 .0046 .0033; .0765 .0560 .0905 .0528 .0689 .5221 .0434 .0160 .0446 .0160 .0056 .0076; .0849 .1093 .0288 .0498 .2050 .0329 .3972 .0345 .0210 .0149 .0114 .0104; .0407 .0786 .0522 .0432 .0582 .0377 .0993 .4174 .0229 .1428 .0040 .0030; .0823 .0582 .0544 .0456 .0787 .0867 .1033 .0428 .4061 .0232 .0093 .0094; .0349 .0577 .0344 .0307 .0795 .0344 .1508 .1564 .0229 .3907 .0042 .0033; .0563 .0617 .0443 .1563 .0628 .0714 .0682 .0219 .0232 .0152 .3856 .0330; .0482 .0423 .0464 .1147 .0218 .0369 .0571 .0247 .0405 .0206 .0702 .4767;       ] ].



Compared with s_ij in Table <ref>, P̂ also takes the sizes N_ij, E_i, and A_i into consideration. 
The matrix illustrates which college has the most direct influence on college i.
In the first column, for example, Harvard pulls in significant authority from other educational superpowers such as Yale, Princeton, CalTech, and Michigan, in the decreasing order of P_i1.
By the tenth row, Berkeley, CalTech, MIT, and Stanford are the top four influencers on UCLA.
The direct influence on UNC is the most evenly distributed, possibly due to its long distance from the other institutions.
Duke University may be added to the list to break the approximate evenness.

Using (<ref>), we calculate the response of π_i, in percentage, to the 1% shock of the enrollment rate P_ii, i.e., P_ii/π_idπ_i/d P_ii = dlogπ_i/dlog P_ii. 
The result is the following vector

89
    [
    [   HRV   SFD  Yale   PRT   MIT   CCG   CIT   UCB  Mich  UCLA   UVA   UNC;  2.79 2.496 1.841 1.629 1.975 1.045  .621  .699  .668  .629  .622  .904 ] ].



Among the private elites, CalTech has the lowest percentage increase of π_i with respect to a 1% hike in the enrollment rate.
Among the public elites, UNC has the highest sensitivity of π_i to the increase of the enrollment rate.
For Michigan to catch up with Berkeley, for example, its π_i needs a .0239/.0235-1=1.7% increase. 
This can be achieved by a 1.7/.668=2.55% increase in the enrollment rate, other things remaining unchanged.
Being situated at the top of the ranking, Harvard may have no incentive to raise its enrollment rate, though it also leads in the impulse response vector.
Also, the response function has a wide range, from .621 to 2.79.
If an econometric model is used to model the ranking scores by enrollment rates and other covariates, then the unknown coefficient for enrollment rates would presumably be a constant across all the colleges, which is highly artificial, as shown in the above vector.
Lastly, one could also use (<ref>) to find the response of π_j, in percentage, to the 1% shock of P_ii.

To locate the partners for college i, we look for all j≠ i for which dπ_j/d P_ji≥ 0, according to Theorem <ref>.
We list these partners, calculated from (<ref>) and (<ref>), in the following matrix:

89
    [
    [       HRV       SFD      Yale       PRT       MIT       CCG       CIT       UCB      Mich      UCLA       UVA       UNC;      2.79 .269/.013 .211/7e-3;                2.50 .157/3e-4 .114/4e-3 .195/.011 .120/1e-3 .129/3e-4 .086/5e-4 .088/7e-5; .159/1e-4      1.84 .103/2e-3 .105/6e-3 .039/2e-4 .120/1e-3 .073/1e-4 .099/2e-4;                1.63 .078/7e-4 .085/3e-3 .097/.004 .055/6e-4 .054/8e-4 .058/3e-4 .083/3e-3 .081/2e-3;                1.98 .112/2e-3 .208/.044 .100/2e-3 .095/2e-3 .111/3e-3 .113/4e-4;                1.05 .052/.001 .052/1e-3 .137/.010 .064/9e-4 .044/9e-4 .080/7e-4;            .256/.02 .088/1e-4      .621 .148/8e-3 .086/.004 .079/5e-3 .118/8e-4 .144/7e-4;                .699 .318/.061;           .094/.009 .074/9e-4      .688 .050/5e-4 .039/2e-4 .053/7e-4;           .213/.060      .629;           .017/4e-4 .015/7e-5      .622 .072/7e-3;           .013/5e-6 .012/3e-4 .023/3e-4 .100/6e-3      .904;           ] ].



At the ith row and the jth column, the numerator is the response of π_i, in percentage, to the 1% shock of P_ji;
the denominator is the response of π_j, in percentage, to the same shock.
Based on this matrix, the partnership at (j,i) does not automatically imply a partnership at (i,j). 
Many partnerships do exist at both (j,i) and (i,j), for example, (Harvard, Yale), (MIT, CalTech), and (Michigan, Chicago).
Three or more institutions could also form a partnership, such as (Michigan, UVA, UNC) and (CalTech, UVA, UNC).
Lastly, the numbers in the matrix show which partnership is the favorite one. 
In the tenth column, for example, UCLA has the largest response function .061 with Berkeley. 
Therefore, it would prefer the cooperation with Berkeley to that with Stanford, Princeton, MIT, or Chicago.

The above accounts of cooperation work only for a small change of P. 
For a large perturbation, we could conduct other analyses.
For example, in a scenario analysis in which CalTech hypothetically raises its enrollment rate from the current level 39.72% to the same level 69.56% as MIT, other things remaining the same,
the ranking scores for this scenario become


89
    [
    [   HRV   SFD  Yale   PRT   MIT   CCG   CIT   UCB  Mich  UCLA   UVA   UNC; .2761 .1986 .1159 .0842 .1268 .0432 .0737 .0235 .0232 .0185 .0088 .0076 ] ].



Compared with Table <ref>, CalTech increases its ranking score by 28% in this scenario. 
The increase, however, is still not enough for CalTech to surpass Princeton.




§ DISCUSSION

The authority-based ranking is subject to several vulnerabilities. 
The survey data could contain selection bias.
For example, STEM (science, technology, engineering, and mathematics) students may be more likely to create accounts on the websites of survey data than students majoring in the humanities. 
This bias would result in higher ranks for schools with strong STEM programs, according to Theorem <ref>.
Another way selection bias may creep in is through herding behaviors of the consumers of higher education, which compromise their uniqueness. 
In particular, many consumers have a mindset shaped by mainstream media.
As a consequence, the odds ratios in Table <ref> could have already been distorted by other rankings.
Secondly, one arguable assumption we make is that revealed preferences effectively capture the most important college comparisons.
 Of course, any additional data would help.
Last but not least, when π̂_i and π̂_j are too close to show a significant difference, it would be unfair to rank one higher and the other lower.
This is often the case when schools are ranked below the top 100.
Other information may be needed to distinguish them.
Otherwise, we could only list the ranking's .95 confidence range for each college, which ranks beyond the top 100.

We could apply the authority-distribution methodology in many similar situations. 
One example, as mentioned in Hu and Shapley (2003), is to sort academic journals.
The only context change is that P_ij is the proportion of journal j in all citations cited by journal i.
Journal sizes can be adjusted by their total citations using (<ref>).
Indeed, researchers (e.g., Baltagi, 1998; Kalaitzidakis, Mamuneas, and Stengos, 2003 and 2011) have used citation data to rank journals.
For another example, we could consider the relative competitiveness of each currency. 
In this case, we let P_ij be the proportion of the export to country j in the total production in country i, all in local currency. 
This example has a vivid power-in-and-out dynamic. 
Its results can help avoid unnecessary trade wars and identify the best trade partners. 
In sports, a team is ranked in the news media by how many times it wins throughout the season. 
This simple measurement does not account for how strong the opponents were in those games. 
A remedy could use the endogenous weighting system (<ref>) so that winning over a strong opponent counts more than winning over a weak one.
It could also account for by how much that team wins or loses in each game so that each earned or lost point is reflected in the ranking.
After being converted into percentages of the total points, the earned points versus lost points in a game make a bilateral odds ratio, like those in Table <ref>.

How to extend the authority-distribution framework remains a big challenge. 
First, one could analyze the properties of matrix P, such as its eigenvalues and eigenvectors.
If we do not normalize the rows of P, we may solve the equation π P = βπ for some unknown β>0. 
This solution π is the eigenvector centrality of the network, which could avoid scaling the enrollment sizes in Section <ref>.
In the sample ranking in Section <ref>, many other universities not listed in the ranking also have direct  impacts on UCLA, for example.
Thus, the tenth row of P̂ should have a sum of less than one while the diagonal of P̂ remains unchanged.
The shrinkage of the off-diagonal sums could solve the data truncation problem when we are only interested in the top 300 colleges, which have a national reputation.
The shrinkage size may negatively relate to the ranking score π_i.
 For this shrunk matrix, the Perron-Frobenius theorem (cf. Keener, 1993) asserts the existence of a positive eigenvector.
Secondly, any other data would help provide a more detailed profile of U.S. higher education. 
One could use data from the CDS, the NCES, and Avery et al. (2013) to conduct a multivariate logistic regression to estimate P.
Thirdly, one could also study a multiple-dimensional π. 
Another counterbalance equilibrium could address how colleges admit students, using acceptance and rejection by schools. 
This facet highly correlates with the one we study in Sections <ref> through <ref> and supplements the other side of the story in the many-to-many matching game.
Ignoring the correlation, the solution to this counterbalance equilibrium measures what types of applicants are competitive in applying for colleges.
However, dealing with qualitative and latent variables, such as recommendation letters, is a big hurdle to cross.
 Additionally, from a policy viewpoint, when n is large, we need a fast algorithm to identify the best cooperators and the worst competitors for each player.
Lastly, a consumer could select multiple choices of alternatives, revealing his or her preference over unselected ones.
In approval voting (e.g., Brams and Fishburn, 1978), for example, a voter can select a few from a shortlist of candidates. 
Then the ranking scores measure the confidence in the candidates by the voters.





§ CONCLUSION

College rankings have emerged because of public demand  and intense competition among institutions of higher learning.
They have become as necessary as high school education counselors and college campus visits. 
There are numerous college rankings in the United States, the most famous of which are generated by mainstream media outlets. 
These publications are supposed to provide useful guidelines for high school seniors and help them make the best college choices. 
However, researches have found that college rankings often mislead students and distort the landscape of higher education. 
We argue that the multiplicity of these rankings is due to the subjective selection of ranking criteria and the subjective weights on the criteria.
The consequence is to reduce higher education institutions to only one stereotype and to undermine selection diversity. 
In contrast, the starting point of our research is to recognize that each college is different, as is each student.
Each adds inclusiveness value to the educational system.
 Beyond that, choosing a college could embrace all aspects of the big data available for that college.
Of course, challenges exist regarding the volume, variety, and veracity of the data, as well as heterogeneity (cf. Fan, Han, and Liu, 2014).

 This paper deals with these challenges. 
For heterogeneity and variety, we let each college decide the selection criteria and their weights; we also let each student decide his or her considerations and their weights. 
These decisions altogether result in millions of matching games, in which we observe only a small fraction of the selection outcomes.
Also, revealed preference from each selection is a ranking between two colleges, derived from an individual's rational consideration of many factors.
 Hence, the preferences together could summarize the most relevant big data about colleges, eliminating voluminous efforts in data collection and storage for a college ranking agency.
The full preferences revealed in the outcomes, however, warrant no consistent utility function nor complete linear ordering of the colleges. 
To resolve this issue, we apply authority distribution to absorb the spillover effects in the direct bilateral influence. 
The result is unique and consistent;
 the solution mitigates the noise in data quality by offsetting the inconsistencies in direct bilateral comparisons.

Our ranking method is likely the most authoritative one compared with those used in popular rankings.
First, the results are comprehensive. 
They are based on foot voting by millions of students 
—each student uses his or her decision function. 
Secondly, the method is scientific and objective. 
We use weighting on colleges, but the weights are endogenously implied from the weighting system. 
They are not subjectively determined by a committee,
and they are the ranking scores.
Finally, it offers individual rationality and strategyproofness for students.
The ranking counts every rational choice made by students;
 there is no white noise, as in a regression model.





1cm


§ REFERENCES


=1.1em
=1
1. Academic Ranking of World Universities (ARWU),
<www.shanghairanking.com>. Accessed 1 January 2019.

=1.1em
=1
2. Avery CN, Glickman ME, Hoxby CM, Metrick A. 
A revealed preference ranking of U.S. colleges and universities. 
Quart J Econ. 2013;128:425-467.

=1.1em
=1   
3. Baltagi BH. 
Worldwide institutional rankings in econometrics: 1989-1995.
Econometric Theo. 1998;14:1-43.

=1.1em
=1   
4. Baucells M, Shapley LS.
Multiperson utility.
Games Econ Behav. 2008; 62:329-347.

=1.1em
=1   
5. Bastedo MN, Bowman NA. 
The U.S. News and World Report college rankings: modeling institutional effects on organizational reputation. 
Am J Educ. 2010;116:163-184.

=1.1em
=1   
6. Bastedo MN, Bowman NA.
College rankings as an interorganizational dependency: establishing the foundation for strategic and institutional accounts.
Research Higher Educ. 2011;52:3-23.

=1.1em
=1   
7. Brams SJ, Fishburn PC.
Approval voting.
Am Polit Sci Rev. 1978;72:831-847.

=1.1em
=1   
8. Bruni F.
How to make sense of college rankings.
New York Times. 29 October 2016.

=1.1em
=1
9. Chirinko RS, Schaller H.
A revealed preference approach to understanding corporate governance problems: Evidence from Canada.
J Financial Econ. 2004; 74:181-206.

=1.1em
=1   
10. Common Data Set Initiative (CDS), 
<https://www.commondataset.org>. Accessed 1 January 2019.

=1.6em
=1   
11. Corrente S, Grecoa S, Lowinski RS.
Robust ranking of universities evaluated by hierarchical and interacting criteria.
In: Huber S., Geiger M., de Almeida A., editors. Multiple Criteria Decision Making and Aiding.
Cham: Springer; 2018. p.145-192.

=1.6em
=1
12. Craig R.
College disrupted: the great unbundling of higher education.
New York: St. Martin's Press; 2015.

=1.6em
=1  
13. Crama I, Leruth L.
Control and voting power in corporate networks: Concepts and computational aspects.
Euro J Oper Res. 2007;178:879-893.

=1.6em
=1  
14. Dohmen T, Falk A.
Performance pay and multidimensional sorting: productivity, preferences, and gender.
Amer Econ Rev. 2011; 101:556-590.

=1.6em
=1  
15. Eeckhout J, Pinheiro R, Schmidheiny K. 
Spatial sorting.
J Polit Econ. 2014; 122:554-620.

=1.6em
=1  
16. Eeckhout J, Pinheiro R, Schmidheiny K. 
Spatial sorting: why New York, Los Angeles and Detroit attract the greatest minds as well as the unskilled.
CESifo Working Paper no. 3274, 2010.

=1.6em
=1   
17. Ehrenberg RG.
Method or madness? inside the U.S. News & World Report College rankings.
J College Admission. 2005;189:29-35.

=1.6em
=1   
18. Fan J, Han F, Liu H.
Challenges of big data analysis.
Nat Sci Rev. 2014; 1:293-314.

=1.6em
=1   
19. Forbes, <www.forbes.com>, retrieved January 1, 2019.

=1.6em
=1   
20. Gale D, Shapley LS.
College admissions and the stability of marriage.
Amer Math Mon. 1962;69:9-15.

=1.6em
=1   
21. Grabisch M, Rusinowska A.
A model of influence in a social network.
Theo Decision. 2010;69:69-96.

=1.6em
=1   
22. Grewal R, Dearden JA, Lilien GL.
The uiversity rankings game: modeling the competition among universities for ranking.
Am Stat. 2008;62:232-237.

=1.6em
=1   
23. Grinold RC, Kahn RN.
Breadth, skill, and time.
J Portfolio Manag. 2011;38:18-28.

=1.6em
=1   
24. Hu X, Shapley LS.
On authority distributions in organizations: equilibrium.
Games Econ Behav. 2003;45:132-152.

=1.6em
=1   
25. Kalaitzidakis P, Mamuneas TP, Stengos T.
Rankings of academic journals and institutions in economics.
J Euro Econ Assoc. 2003;1:1346-1366.

=1.6em
=1   
26. Kalaitzidakis P, Mamuneas TP, Stengos T.
An updated ranking of academic journals in economics.
Can J Econ. 2011;44:1525-1538.

=1.6em
=1   
27. Keener JP.
The Perron–Frobenius theorem and the ranking of football teams.
SIAM Rev. 1993; 35:80-93.

=1.6em
=1   
28. Langville AN, Meyer CD.
Google's PageRank and Beyond: The Science of Search Engine Rankings.
Princeton, NJ: Princeton University Press; 2012.


=1.6em
=1   
29. Luca M, Smith J.
Salience in quality disclosure: evidence from the U.S. News College rankings.
Harvard Business School Working Paper 2011;12-014.

=1.6em
=1   
30. Moed HF.
A critical comparative analysis of five world university rankings.
Scientometrics. 2017;110:967-990.

=1.6em
=1   
31. National Center for Education Statistics,
<https://nces.ed.gov/collegenavigator/>. Accessed 1 January 2019.

=1.6em
=1   
32. Niche,
<www.niche.com>. Accessed 1 January 2019.

=1.6em
=1   
33. Parchment,
<www.parchment.com>. Accessed 1 January 2019.

=1.6em
=1   
34. Perez-Pena R, Slotnik DE.
Gaming the college rankings. 
New York Times. 31 January 2012.

=1.6em
=1   
35. Princeton Review,
<www.princetonreview.com>. Accessed 1 January 2019.

=1.6em
=1   
36. Pritchard R.
Revealed Preference Methods for Studying Bicycle Route Choice - A Systematic Review.
Int J Environ Res Public Health. 2018; 15:470-507..

=1.6em
=1   
37. Quacquarelli Symonds (QS),
<www.topuniversities.com>. Accessed 1 January 2019.

=1.6em
=1   
38. Samuelson PA. 
Consumption theory in terms of revealed preference. 
Economica New Series. 1948;15:243-253. 

=1.6em
=1   
39. Tieskens KF, Van Zantena BT, Schulpa CJE, Verburga PH.
Aesthetic appreciation of the cultural landscape through social media: An analysis of revealed preference in the Dutch river landscape.
Landscape and Urban Planning 2018; 177:128-137.

=1.6em
=1   
40. Times Higher Education, <www.timeshighereducation.com>. Accessed 1 January 2019.

=1.6em
=1   
41. U.S. News & World Report, <www.usnews.com>. Accessed 1 January 2019.

=1.6em
=1   
42. Walls Street Journal, <www.wsj.com>. Accessed 1 January 2019.

=1.6em
=1   
43. Washington Monthly, <www.washingtonmonthly.com>. Accessed 1 January 2019.

=1.6em
=1   
44. Wilson EB. 
Probable inference, the law of succession, and statistical inference.
J Am Stat Assoc. 1927;22:209-212. 




































































Appendix



 §.§ A1. Proof of Lemma <ref>


As τ=1-s_ij and η is the lower confidence bound in (<ref>), 

    η 
    =
    2s_ij M_ij + z^2/2 M_ij + 2z^2 - z/2 M_ij + 2 z^2√(4τ s_ij M_ij + z^2).

Therefore

    √(4τ s_ij M_ij + z^2)
    =
    2s_ij M_ij + z^2 - 2η (M_ij + z^2)/z 
    =
    2(s_ij-η)/z M_ij + (1-2 η)z.

Next, we square both sides to get

    [                                             4τ s_ij M_ij + z^2                                                              = 4(s_ij-η)^2/z^2 M_ij^2 + 4(s_ij-η)(1-2 η) M_ij + (1-2 η)^2 z^2 ]

or simply,

    ( s_ij-η/z )^2 M_ij^2 + [(s_ij-η)(1-2 η)-τ s_ij] M_ij - η (1- η) z^2  = 0.

The above quadratic equation of M_ij has the solution expressed in Lemma <ref>.





 §.§ A2. Proof of Theorem <ref>

The probability density for Ñ_ij = ñ_ij for all j∈ℕ is
(A_i)!/∏_j (ñ_ij)!∏_j P_ij^ñ_ij
and the marginal probability density for Ñ_ii=E_i is
(A_i)!/ (E_i)! (A_i - E_i)! P_ii^E_i (1-P_ii)^A_i-E_i.
Thus, given Ñ_ii=E_i=ñ_ii, the conditional probability density for Ñ_ij = ñ_ij for all j≠i is

    (A_i)!/∏_j (ñ_ij)!∏_j P_ij^ñ_ij/(A_i)!/(E_i)! (A_i - E_i)! P_ii^E_i (1-P_ii)^A_i-E_i = (A_i-E_i)!/∏_j≠ i (ñ_ij)!∏_j≠ i (P_ij/1-P_ii )^ñ_ij.

Therefore, the conditional Ñ_ij given Ñ_ii=E_i also has a multinomial distribution, and the parameters are
A_i-E_i and P_ij/1-P_ii, ∀ j≠ i.

Let λ be the probability with which any student —admitted by college i— would participate in the survey. Then, the joint conditional probability of
Ñ_ij = ñ_ij and N_ij=n_ij for all j≠ i is

    [                                                                                                                                                                                                                                                                 (A_i-E_i)!/∏_j≠ i (ñ_ij)!∏_j≠ i (P_ij/1-P_ii )^ñ_ij∏_j≠ i [  ( [ ñ_ij; n_ij ] )λ^n_ij (1-λ)^ñ_ij - n_ij ];                                                                                                                                                                                    = (A_i-E_i )! λ^∑_j≠ i n_ij (1-λ)^A_i-E_i -∑_j≠ i n_ij/ (∑_j≠ i n_ij )!
    	 (A_i-E_i -∑_j≠ i n_ij )!× (∑_j≠ i (ñ_ij - n_ij)  )! ∏_j≠ i ( P_ij/1-P_ii )^ñ_ij - n_ij/∏_j≠ i (ñ_ij - n_ij)!;                                                                                                                                                                                                                                                                                                             × (∑_j≠ i n_ij )! ∏_j≠ i ( P_ij/1-P_ii )^n_ij/∏_j≠ i (n_ij)!. ]

In the above three fractions, the first one is the binomial distribution density for ∑_j≠ iN_ij, the total number of admitted students in the survey excluding those in E_i. 
The second one is the multinomial distribution density for Ñ_ij-N_ij for all j≠ i, the admitted students not in the survey also excluding those in E_i.
And the third one is the multinomial density for N_ij, conditional on ∑_j≠ iN_ij.
Finally, we factor out the first two fractions to get the conditional marginal density of N_ij = n_ij as 
(∑_j≠ i n_ij )!/∏_j≠ i (n_ij)!∏_j≠ i [P_ij/1-P_ii ]^n_ij.
This shows that N_ij, ∀ j≠ i, have a multinomial distribution, given the observation of ∑_k ≠ i N_ik. 
Therefore, N_ij/∑_k ≠ i N_ik has a conditional expectation P_ij/1-P_ii.





 §.§ A3. Proof of Theorem <ref>

We multiply 1_n on both right sides of the equation π P = βπ to get 
π P 1_n = βπ1_n.
As P 1_n = 1_n and π1_n = 1, we have

    β = βπ1_n = π P 1_n = π1_n = 1.





 §.§ A4. Proof of Theorem <ref>

When P is irreducible and aperiodic, P^t converges as t →∞; and
lim_t→∞ P^t =
1_n (ζ_1, ⋯, ζ_n)
for some row vector (ζ_1, ⋯, ζ_n).
Therefore, by (<ref>) and π1_n=1,

    π = π P 
    =π P^2
    = ⋯
    = lim_t→∞π P^t 
    = π1_n
    (ζ_1, ⋯, ζ_n)
    = (ζ_1, ⋯, ζ_n).

And by (<ref>),

    π = π P^m 
    =π P^2m
    = ⋯
    = lim_t→∞π P^tm 
    = π1_n
    (ζ_1, ⋯, ζ_n)
    = (ζ_1, ⋯, ζ_n).






 §.§ A5. Proof of Theorem <ref>

Let 0_n be the n× 1 zero vector.
When we make a small perturbation Δ P to P, the new authority distribution π + Δπ satisfies the counterbalance equation of

    A.1π + Δπ = (π + Δπ) [P + Δ P]

subject to Δ P 1_n = 0_n and Δπ1_n = 0.
After subtracting π = π P from (<ref>), we get
Δπ [I_n - P -Δ P] = πΔ P
and its first-order approximation

    A.2Δπ [I_n - P] ≈πΔ P.


Without loss of generality, let us increase P_11 by Δ P_11 and calculate the effect of the change on π. 
Then the elements of Δ P are all zeros except the first row.
By (<ref>), the other elements in the row decrease proportionally. 
Thus, the first row of Δ P is

    (Δ P_11, -Δ P_11/1-P_11 P_12, ⋯, -Δ P_11/1-P_11 P_1n )
    =
    Δ P_11 (1, -P_12/1-P_11, ⋯, -P_1n/1-P_11 ).

By (<ref>), the derivative of π with respect to P_11, i.e. dπ/d P_11, satisfies

    A.3dπ/d P_11 [ I_n -P  ] = πd P/d P_11 
    = π_1  (1, -P_12/1-P_11, ⋯, -P_1n/1-P_11 ).


Let the row vector β_i take the ith column of P and then drop the ith element.
We partition the transpose of P as 
P' =  (
[ P_11  β_1;  α_1  Z_1 ] ).
To solve dπ/d P_11 from (<ref>), we write the augmented matrix for the identity dπ/d P_111_n = 0 and the transpose of (<ref>) as
[ 
[              1         1_n-1'              0;         1-P_11           -β_1            π_1;           -α_1     I_n-1 -Z_1 -π_1/1-P_11α_1;                ]].
If we multiply 
[
[      1      0   0_n';      0      1 1_n-1';    0_n    0_n  I_n-1 ] ]
to the left side of the augmented matrix, then the second row becomes a zero vector. 
After dropping the second row, we get the new augmented matrix of

    A.4[ 
    [              1         1_n-1'              0;           -α_1     I_n-1 -Z_1 -π_1/1-P_11α_1;                ]].

We next multiple
[
[                      1 -1_n-1' (I_n-1-Z_1)^-1;                      0                  I_n-1 ] ]
to the left side of (<ref>) to get

    A.5[ 
    [         1+1_n-1' (I_n-1-Z_1)^-1α_1                             0_n-1' π_1/1-P_111_n-1' (I_n-1-Z_1)^-1α_1;                               -α_1                         I_n-1 -Z_1                     -π_1/1-P_11α_1;                                    ]].

Note that (I_n-1-Z_1)^-1 = I_n-1+Z_1 + Z_1^2 + Z_1^3 +⋯ has all non-negative elements.
Thus, 1_n-1' (I_n-1-Z_1)^-1α_1 ≥ 0.
By the first row of (<ref>), 

    dπ_1/d P_11 = π_1/1-P_111_n-1' (I_n-1-Z_1)^-1α_1/1+1_n-1' (I_n-1-Z_1)^-1α_1≥ 0.

By the second row of (<ref>), 
-dπ_1/d P_11α_1 +  (I_n-1-Z_1  ) dπ__-1/d P_11 = -π_1/1-P_11α_1
and thus

    [                                                       dπ__-1/d P_11                                                                   =                   (I_n-1-Z_1  )^-1 [ dπ_1/d P_11 - π_1/1-P_11 ] α_1;                                                                                                                                       = - π_1/(1-P_11)  [1+1_n-1' (I_n-1-Z_1)^-1α_1  ] (I_n-1-Z_1  )^-1α_1. ]

Therefore, dπ__-1/d P_11 is a non-positive vector.





 §.§ A6. Proof of Theorem <ref>

To show a student's strategyproofness, without loss of generality, we assume that the student enrolls in college 1 and rejects college 2.
If he or she reveals the private information in the preference survey, we want the effect of the revelation on π.

Whether revelation or not, the enrollment rates P_11 and P_22 do not change because they are official enrollment data
—the enrollment of the student in college 1 is already counted in P_11 and never counted in P_22.
Though M_12 and s_12 change with the revelation, their product N_12 remains unchanged because it is the number of students in the survey who are admitted by college 1 and enroll in college 2.
Thus, by (<ref>), the first row of P does not change with the revelation.

However, the estimated P_21 increases. 
Let P_21 have a small change Δ P_21.
By (<ref>), Δ P has non-zero values only in the second row which is


    (
    Δ P_21, 0, -Δ P_21P_23/1-P_21-P_22, ⋯, -Δ P_21P_2n/1-P_21-P_22 )
    =
    Δ P_21 (
    1, 0, -P_23/1-P_21-P_22, ⋯, -P_2n/1-P_21-P_22 ).


By (<ref>), the derivative of π with respect to P_21, i.e. dπ/d P_21, satisfies 

    A.6dπ/d P_21 [ I_n -P  ] = πd P/d P_21 
    = π_2  (1, 0, -P_23/1-P_21-P_22, ⋯, -P_2n/1-P_21-P_22 ).

In solving dπ/d P_21 from (<ref>), the augmented matrix for dπ/d P_211_n = 0 and the transpose of (<ref>) is
[ 
[                    1               1_n-1'                    0;               1-P_11                 -β_1                  π_2;                 -α_1           I_n-1 -Z_1 -π_2/1-P_21-P_22γ_12;                      ] ].
We apply the same operations as in the proof of Theorem <ref> to the matrix to get the new augmented matrix

    A.7[ 
    [               1+1_n-1' (I_n-1-Z_1)^-1α_1                                   0_n-1' π_2/1-P_21-P_221_n-1' (I_n-1-Z_1)^-1γ_12;                                     -α_1                               I_n-1 -Z_1                     -π_2/1-P_21-P_22γ_12;                                          ]].

Therefore, by the first row of (<ref>),

    dπ_1/d P_21
    = 
    π_2/1-P_21-P_221_n-1' (I_n-1-Z_1)^-1γ_12/1+1_n-1' (I_n-1-Z_1)^-1α_1≥ 0.

By the second row of (<ref>), 
-dπ_1/d P_21α_1 +  (I_n-1-Z_1  ) dπ__-1/d P_21 = -π_2/1-P_21-P_22γ_12
and thus

    [                                                                                                   dπ__-1/d P_21                                                                                                               =                                                       (I_n-1-Z_1  )^-1 [ dπ_1/d P_21α_1 - π_2/1-P_21-P_22γ_12 ];                                                                                                                                                                                                                               = (I_n-1-Z_1)^-1 [ π_2/1-P_21-P_221_n-1' (I_n-1-Z_1)^-1γ_12/1+1_n-1' (I_n-1-Z_1)^-1α_1α_1 - π_2/1-P_21-P_22γ_12 ];                                                                                                                                                                                                                               =              π_2/1-P_21-P_22 (I_n-1-Z_1)^-1 [ 1_n-1' (I_n-1-Z_1)^-1γ_12/1+1_n-1' (I_n-1-Z_1)^-1α_1α_1 - γ_12 ]. ]



