Dealing with bias in an FFT for frequency correction
Your approach is sensible: if $N$ is even, you can discard the DC and Nyquist bins, which results in $N/2-1$ values at positive and negative frequencies. This will result in a fair comparison.
If $N$ is odd, there is no Nyquist bin, and you just discard the DC bin, resulting in $(N-1)/2$ values at positive and negative frequencies.