Comparing AUCs: Discrimination of same Control from different Test Group - paired or unpaired? miRNA I want to compare two AUCs using a bootstraping method from pROC package (roc.test).
I have done a classification using Logistic Regression and Serum miRNAs as prediction factors. The Controls are in both cases the same, the test group is different.
1. Cancer (high risk) vs. Control = AUC 1
2. Cancer (low risk) vs. Control = AUC 2

If I want to compare this two AUCs - should I use a paired or unpaired test?
I would tend to using a paired test, because of exact the same Control group (Control groups are correlated).
Generally its paired. The bootstrapping personally I find concerning. I presume you are using it to augment your data set.
Bootstrapping at best tends to be a controversial statistical operation and could result in the non-significance you are observing. Anyway if its not significant between high and low - thats your answer.
You could of course perform a single ML classifying low, and high cancer risk against control. That would provide a method to understand the associations of miRNA.

Bootstrapping issue, I had a think about this. It depends on how the bootstrapping is done. In a paired sample this (I assume) samples with replacement at each paired position for both ROCs. If this is not the case - the test is junk. However, if this is the case, its actually a good test, because the paired structure of the data is always preserved no matter how many bootstrap samples are performed. So like if a 100 were done and the pairwise statistic (which we don't know what it is - but something like KS test) and its &gt;0.05 for all 100 samples - I don't think there's any doubt thats a robust result.
Again if the two ROCs are bootstrapped independently, in my opinion the test is junk.
Reason The concern of bootstrapping is it causes weird distributions which were not present in the original data. This is only okay if there's lots of experience about interpreting the result. For example, in phylogenetics there's decades of experience interpreting bootstrapping and a consensus about what it means. This often isn't the case for most other areas of statistics.
I note this particular issue is really ramping up the rep score ;0)