Input offset voltage, input bias current, input offset current of operational amplifier
They are all DC errors that may contribute to a significant DC error on the output signal. I say "may" because this depends on the actual circuit implementation and (some) component values.


  Can I get rid of these noises by using a high-pass filter?


Some of these DC errors can drift with time so, using a high pass filter with the wrong values may not effectively remove those errors.


  If not, how can I get rid of these noises?


It's important for any design that seeks to reduce errors to use the most appropriate component values and, if that is still insufficient, then use better op-amps.

One parameter that you have not mentioned is the change in CMRR with common-mode input level changes. This can create a substantial DC error if not taken care of. It can also produce a significant in-band AC error term if the input common mode signal level is changing. This latter error may not easily be got rid of by high pass filtering.