How to calculate SNR with sample noise and noisy signal only
The key issue here is whether the signal and noise are uncorrelated.  Assuming two real random variables x and y are both zero mean, the power of the combined signal is

$E\{(x+y)^2\} = E\{x^2+2xy+y^2\} = E\{x^2\}+E\{y^2\}+2E\{xy\}$

If the two variables are uncorrelated, then $E\{(x+y)^2\}=E\{x^2\}+E\{y^2\}$  In other words, the power of the combined signal and noise is the sum of the signal power and the the noise power if the signal and noise are uncorrelated and zero-mean.  If this assumption is valid - as if often the case in practice - then the approach you outlined would work.  It would also mean that $P_{signal+noise}=A_{signal}^2+A_{noise}^2$.  If the signal and noise are correlated, then the approach would not work.

One thing you might consider is whether you are trying to measure the signal-to-noise ratio or the signal-to-noise-and-distortion ratio.  When the audio is present, the recorded signal will suffer distortion that will degrade the overall signal quality beyond what would be predicted simply via the noise power.  This would cause the SNR estimate obtained via the above approach to be optimistic in predicting the overall signal quality.