Methods for calculating signal quality metric i'm sure this is a fairly large topic area, but I'm not quite sure where to start. I have a signal in which I can change several parameters, such as the frequency, gain, etc. The modulation is a differential BPSK, and my current metric for determining the signal quality is what percentage of my auto correlation (correlating the current bit signal with the previous bit) is above 0. If it's all above or below zero, a 100% signal quality. if it's half and half, then it's 0%, and then it's linear from there. Is there a better way of doing this? I'm wanting to be able to change my signal parameters in order to increase my signal quality, where a higher signal quality should be a higher reliability of receiving a message correctly.

I'm also looking at doing demodulation in the frequency domain, so then my calculations would probably need to change. Any input and references would be appreciated. Thanks!
From your question it sounds like you are looking for a SNDR metric.  In that case you are measuring a distortion level by comparing the distorted signal to a reference.
For a discrete-time signal $y$, comparable to reference $r$, the SNDR would be:
$$ SNDR = \frac{E[r^2]}{E[(y-r)^2]} $$