How do IR researchers evaluate the ranks of documents?
If I understand what you're trying to achieve correctly, you can use this technique discounted cumulative gain.

$$DCG_p=\sum_{i=1}^{p}{\frac{2^{rel_i}-1}{\log_2{(i+1)}}}$$

$i$ is the rank, and $p$ is the number of results that you want to evaluate. For example, if you evaluate DCG for the first 10 results, then $p=10$.

NDCG is a form of DCG that involves normalizing the result. This is useful if p differs when you compare DCG scores.
$$NDCG=\frac{DCG}{IDCG}$$ where IDCG is the ideal DCG. This is found by calculating the DCG up to $p$ under ideal conditions (all documents have a relevance of 1).

Note that neither recall nor precision are used here. Relevance and rank are the factors to compare. You still need to give the documents a relevance factor, that's unavoidable, but you don't need to establish a threshold as you would with other evaluation metrics.