Two-GSMK channels demodulation: 3dB difference in BER
Yes that is correct, you are only getting half the signal: with cosine on both inputs half the signal moves to baseband and the other half moves to twice the frequency offset. I suspect the reason it is not working in the sine cosine case is the sign of that is such that you translated the entire signal upward to twice the frequency offset instead of to baseband. If that is the case you could invert the sine input and it should work without the 3 dB loss.