Is it possible to remove the non-random noise in the signal by averaging?
Window-average filter is an example of lowpass filter. That means, if interesting frequencies in your signal are few times lower than noise, then yes, you can do that.
If they are close or higher, then no.
Use bandstop/highpass filter if your signal frequencies are much higher than noise, or adaptive filter like Recursive Least Squares instead, if your signal and noise frequencies overlap.
And don't forget, that your filter will affect your FRF measurements, so if you sample over the range that includes constant noise frequency, I'd suggest going with adaptive filter.
See also: What is the cut-off frequency of a moving average filter?