“Subtracting” one audio stream from another audio stream
This setup shares some similarities with system identification problems, where $A$ would be the input of the LTI system you want to estimate the transfer function of, $B$ being the output; and the "ambient sound" being the additive noise. The LTI assumption is reasonable provided your converters/amplifiers/transducers are of decent quality.

So the steps would be:


Use a system identification technique to find the FIR filter $\hat{h}$ that minimizes the mean-square error between $\hat{h} \star A$ and $B$. A simple method, which might not be the most suitable here, is to divide the cross-correlation of $A$ and $B$ by the autocorrelation of $A$. Explanation here. The limitation is that it will not work well for long recordings (you might better compute your estimates on shorter segments and average them) - and that music is not the best "probe" signal to send into a system to estimate its response.
You can now use $\hat{h} \star A$ as an estimate of the original signal A as "heard" by the microphone and subtract it from $B$ to retrieve the ambient sound.


I gave a shot at this using a music clip (A), applying a reverb and slight amp model to simulate a speaker in a room, then mixing in a cat audio sample to get (B), then estimating an impulse response from the (A, B) pair, then subtracting the filtered A from B. This shows some results but a better FIR estimation technique might help here! (note that I truncated the estimated IR to its first 5000 samples to speed up computations).

Note that there are algorithms for doing this adaptively (such as LMS). This might be more suitable for your problem if $A$ and $B$ are processed in realtime rather than offline. Such algorithms form the basis of echo cancellation systems used in telecommunications.