From pairwise comparisons to ranking - python
Pairwise comparison models such as that of Bradley and Terry can easily be extended to your case, when you have pairwise comparison probabilities instead of binary outcomes.

Let $N$ be the number of items, and let $p_{ij}$ be the probability that query $j$ is better than query $i$.
Then, the log-likelihood of the Bradley-Terry parameters $\lambda_1, \ldots, \lambda_N$ given probabilities $\{ p_{ij} \}$ is

$$
\sum_{i,j} p_{ij} [\log(\lambda_j) - \log(\lambda_i + \lambda_j)]
$$

This can be reparametrized into a convex function, and the maximum-likelihood parameters can be found by one of many convex opimization methods.

Here is a simple Python algorithm that will find the ML estimate, using a minorization-maximization approach.

import numpy as np

def mle(pmat, max_iter=100):
    n = pmat.shape[0]
    wins = np.sum(pmat, axis=0)
    params = np.ones(n, dtype=float)
    for _ in range(max_iter):
        tiled = np.tile(params, (n, 1))
        combined = 1.0 / (tiled + tiled.T)
        np.fill_diagonal(combined, 0)
        nxt = wins / np.sum(combined, axis=0)
        nxt = nxt / np.mean(nxt)
        if np.linalg.norm(nxt - params, ord=np.inf) &lt; 1e-6:
            return nxt
        params = nxt
    raise RuntimeError('did not converge')


Example usage:

import itertools

# Generating pairwise probability matrix.
pmat = np.zeros((10, 10))
for i, j in itertools.permutations(range(10), r=2):
    pmat[i][j] = (j + 1) / (i + j + 2)

# Estimating Bradley-Terry model parameters.
params = mle(pmat)

# Ranking (worst to best).
ranking = np.argsort(params)


Source: I am an author of a Python library for parameter inference in various statistical comparison models, choix.