Audio Signal Noise Filter Problem I'm currently working with audio signals and have a problem:

C = A*B + N, where

C = recorded signal from microphone consisting of:  

A = known music file data played on speakers next to microphone  

B = some convolution on the recorded A-sound due to speaker->mic roundtrip
(I mean the recorded signal won't be 100% the same as the audio data from the file before it is played to the speaker and recorded by mic. (Is this an impulse response?))  

N = some additional noise sounds recorded by microphone  

My goal: an approximate estimation as to whether there is a signal N and how loud it is.  

I don't have a need for accurate data!

Additional info:
I'm working with Apple's vDSP API. I have cross correlated the signals A and C, so I have the time window in which the signals overlap.

In the overlapping window, I have both signals in the time and frequency domain.

Currently I'm helpless if, for example, a Wiener filter is the right approach and if I'm capable to apply one with my known parameters(Is a known noise required? or the impulse response of the environment?).  I tried to apply a Wiener deconvolution by dividing C/A in the frequency domain with no success.  

Once more: I don't need accurate data, just a rough guess how much N is there in the signal C. Actually a SNR like measure would be sufficient.
Wiener filtering is one approach. It might even be the best approach. A Wiener filter is designed to minimize the noise (in the least squares sense) and invert the effect of the impulse response, given a known signal and a signal that is known to be tainted with noise and an impulse response. Once you have a Wiener filter, you can then compare the amplitude of the filtered signal with the amplitude of the direct signal to estimate the noise (I think).

Now I've never implemented a Wiener filter myself, so I will defer to a textbook with chapter dedicated to the subject: Advanced Digital Signal Processing (Electrical and Computer Engineering) by Glenn Zelniker and Fred J. Taylor. It's out of print, but I'm sure you can pick up a used copy for cheap somewhere. I suggest it because it's more mathematical than engineering in it's approach and that might appeal to you. Many other textbooks have info on the subject.

If a more ad-hoc approach suits you, here is a suggestion. I've never tried it, but it might work:

Create a filterbank, F, (a properly windowed FFT will work) to analyze both the known music file A and the input C. You will also need some measurement function, M. I would suggest $M(x) = | x |$, rather than $M(x) = x^2$, but both will work, and you may also want to do some smoothing over time. By comparing the results of $M(F(C))$ to $(M(F(A))$ when the noise is low, you should be able to determine the effect of B. Of course, you may not know that the noise is low, in which case, you'll have to get a bit more clever, creating some estimate based on some statistics. The goal, however, is to get to a point where you can predict the output of $(M(F(C))$ from $(M(F(A)))$, to within some bounds. The lower the bounds the better you will be at detecting the noise.

To estimate the noise, you will compare how far above those bounds the signal has actually gone. This will work well for band-limited noise, but for quieter, broadband noise, it may not work well. In that case, you may want to create an aggregate statistic based not just on how far above the bounds a signal has gone, but how many bounds have been crossed, or simply apply your measurement function directly to your unfiltered signal: $M(C)$ vs $(M(A))$.