Distortion in phase difference measurement using FFT
The OP mentions &quot;demodulating&quot; but I don't actually see that in the process (meaning the phase modulation is not actually removed).  Without demodulation the  signal can possibly be spread across multiple DFT bins thus reducing the SNR in the bin that was used to measure the phase. If the signal bandwidth is less than one bin, and the noise is still white, then there would be further increase in SNR by the rate of $\sqrt{N}$ by increasing the number of samples by $N$.
The extent of this spreading due to modulation isn't clear since only the modulation index was provided; we would also need to know the modulation rate and type of waveform in order to predict the occupied bandwidth. With both the modulation index and rate with a continuous modulation waveform the occupied bandwidth can be approximated using Carson's Rule which is applicable to FM and PM given as:
$$BW \approx 2(\Delta F + F_m) \tag{1} \label{1}$$
With: 
$BW$: Approximate occupied BW 
$\Delta F$: frequency deviation $F_m$: the modulation rate.
And using the modulation index $\beta$ which is the peak phase angle (for sinusoidal PM) and related to $\Delta F$ and $F_m$ according to:
$$\beta = \frac{\Delta F}{F_m} \tag{2} \label{2}$$
From \ref{1} and \ref{2} we can establish the occupied bandwidth from the modulation rate and modulation index directly as follows:
$$BW \approx 2F_m(\beta + 1) \tag{3} \label{3}$$
From this we can confirm if the occupied bandwidth exceeds the resolution bandwidth of the measurement, at which point the SNR from a single bin will be maximized as limited by the DFT noise floor. If the occupied bandwidth is less, and the DFT noise floor still white in close proximity to the signal (meaning the signal is sufficiently stationary over that duration of time), then there will be a further improvement simply by increasing the number of samples.
The resolution bandwidth for each bin in the DFT given in Hz is given by the following formula as the &quot;Equivalent Noise BW&quot;:
$$\text{ENBW} = \frac{f_s}{N}w_b \tag{4} \label{4}$$
With: 
$\text{ENBW}$: Equivalent Noise Bandwidth (Hz) 
$f_s$: Sampling rate (Hz) 
$N$: Number of samples in DFT 
$w_b$: ENBW of the window function in units of DFT bins
Computing the ENBW of window functions is detailed here: Find the Equivalent Noise Bandwidth.  The equivalent noise BW of the Blackman Harris window is approximately 1.7 bins, so the equivalent noise bandwidth in Hz with the 10 MHz sampling rate and 32768 bins would be:
$$\text{ENBW} = \frac{10e6}{32768}1.7 = 518 \text{ Hz}$$
Using this result in \ref{3} results in the maximum modulation rate of
$$F_m \le \frac{\text{ENBW}}{2(\beta+1)} = 123.3 \text{ Hz}$$
Therefore, if the modulation rate is not significantly higher than 123 Hz, the signal will be entirely within the resolution bandwidth of the DFT and no further phase demodulation would be needed, within the SNR provided by the DFT as was done. If the modulation rate is significantly less, and the signal still stationary over longer duration (eventually phase noise of the signal source will limit this condition), then SNR can be further improved by $\sqrt{N}$ by increasing the number of samples by $N$, reducing the DFT noise floor. If the modulation rate is significantly more than 123 Hz, then there would be additional benefit in SNR to first demodulating the signal to remove the phase modulation. (Also to note that at higher modulation indexes, more signal energy is concentrated in the outer portions of the bandwidth such that there would be a decrease in SNR with further time duration or increasing $N$). When the modulation rate is less than the ENBW of the DFT, even though the signal is phase modulated, given the &quot;courseness&quot; of the DFT, the signal would appears just as a pure tone in this case since the DFT is providing us the average frequency, average phase over the duration of the capture.
So continuing the assumption of $F_m \le 123 \text{ Hz}$, with a phase variations of 0.15° and no motion (meaning this is phase noise); assuming the 0.15° was an rms measurement, that would be equivalent to an SNR as limited by noise due to phase of:
$$-20log_{10}((0.15) 2\pi/360) = 51.6 \text{ dB} $$
This is a 51.6 dB SNR in the resolution bandwidth of 518 Hz; if the signal for which the DFT was computed was real, then the SNR of the signal would be 3 dB higher given it would be at both bins 327 and 32441 and only one of those two bins was used for the measurement. The OP has not stated what bandwidth was used for the original 30 dB SNR metric of the signal, but this approach as outlined can be used to further quantify if the process is noisier than expected, or not. If the variation in phase was anything else such as peak to peak or peak measurements would translate to an even higher SNR, but that should be clarified as well for an accurate assessment. This should also help quantify the SNR requirements for a certain phase accuracy as needed for the final distance measurement.

When such phase noise is small, the small angle approximation $\sin(\theta) \approx \theta$ applies, thus the rms of any phase noise when normalized to the signal (typically given as &quot;dBc&quot;) is the SNR when both units are squared to be power terms (hence $20Log_{10}(\theta_{rms}$))