why is noise always a high frequency signal ? what then is frequency of a noise?
Noise is an unwanted signal, period. 

Typical sources of natural noise tend to be broadband, that is all frequencies from DC to very high indeed.

The electrical noise produced by resistors, Johnson noise, has a flat frequency spectrum from DC to very high. Flat spectrum means equal power per Hz of bandwidth. 

Consider listening to a flat noise signal. Looking at the noise power above and below (say) a mid A of 440Hz. There are only 440 1Hz bandwidths below it, but 10000 to 20000 (depending on how old you are and where your hearing cuts off) above it, so the noise signal sounds dominated by the high frequency hiss.

The noise produced by opamps however tends to be flat above some low frequency, usually in the audio band, most amps are flat above the 1kHz wher noise is typically specified, but then increases towards DC, due to what's called flicker noise. This is a big problem for DC instrumentation. High quality opamps intended for DC use will also have noise specs in the 0.1Hz to 10Hz range, and these numbers tend to dwarf the 1kHz numbers.

For DC signals and low frequency signals, eliminating the noise above their bandwidth can result in orders of magnitude reduction in noise voltage. Which is why it's done.

Other sources of unwanted signals can be narrow band. Your neighbour's garage door opener for instance can be removed by an RF filter. Mains hum can cause big problems in signal conditioning as the freqeuncy is so low, and can often be in the bandwidth of the signal. This then needs a reduction in the hum pickup, as filtering will do no good.