How to derive false positive and false negative from top-k accuracy? I am working on the following &quot;equality identification&quot; problem and become quite confused on how to reasonably define false positive and false negative in my case.
Problem:
Suppose I have a large set of graphs $G$, where for every graph $g \in G$, there is one and only one graph $g'$ that is identical to $g$. I am training a model $M$ which, given two graphs $g, g' \in G$, can decide how similar these two graphs are. The output of $M$, $o =M(g, g')$, denotes a similarity score of these two graphs. $o$ is a floating number, ranging from $[0, 1]$.
To evaluate the performance of $M$, I am measuring the top-$k$ accuracy of its predications in the following very standard way:
for g \in G:
    for t \in G:  // suppose here t != g and therefore comparison is meaningful
        accuracy[(g, t)] = M(g, t)

correct_prediction = 0
total_prediction = 0
for g \in G:
     total_prediction += 1
     sort_accuracy_array_on_graph (g); // we sort all comparisons on $g$ with other graphs \in G
     if g' in top-$k$ most similarity graphs with g: 
        // g' is the ground truth, the &quot;identical&quot; graph with g
        correct_prediction += 1

correct_prediction / total_prediction // the average top-$k$ accuracy

OK, so this seems very standard approach to computing the top-$k$ accuracy of $M$. However, currently I want to take one step further and somewhat measure the false positive and false negative of $M$. I understand that FP and FN are both very standard metrics in data mining, but just somewhat become very confused on how to define it in my problem. For instance, I can define False Negative in the following way:

False Negative of $M$: conceptually, FN denotes that $g'$, the identical graph of $g$, does not appear in top-$k$ most similarity graph of $g$. Then, perhaps FN is interchangeable with top-$k$ accuracy in my case?

False Positive of $M$: but if we accept that FN is interchangeable with top-$k$ accuracy in my case, then how to define FP of $M$? FP denotes that $M$ aggressively treats graph $t$, where $t != g'$, as the &quot;similar&quot; graph of $g$ and appear in its top-$k$ most similar set. Then isn't it indicating that FP is ALSO interchangeable with top-$k$ accuracy in my case? That seems suspicious because in my understanding, FP should not equal to FN? There must be something wrong.


Am I clear on this confusion? Any suggestion would be appreciated. Thank you!
I think your confusion comes from the fact that the false negative of $M$ as you thought it, is not top-$k$ accuracy but perhaps $1$ $-$ top-$k$. Moreover, how do you evaluate the condition if g' in top-$k$ most similarity graphs with g?
In any case, for your case I thought the following.
Let's say that if the model $M$ returns a value $o &gt; o^*$, where $o, o^* \in [0, 1]$ (and $o^*$ is a threshold value decided by a human), then we assume that $g \equiv g'$; $g \neq g'$ otherwise.
Thus, a false negative of $M$ is when $o &lt; o^*$ but in reality, $g \equiv g'$.
A false positive of $M$ instead is when $o &gt; o^*$ but in reality, $g \neq g'$.
Of course your result will depend on what value(s) of $o^*$ you choose, but now it should be more clear what are FN and FP.