What could cause impulse noise on switch closure
A metal-dome switch generates a significant mechanical impulse when it is pressed. If your design includes ceramic capacitors (especially MLCCs), this impulse may be inducing microphonic effects in these capacitors. You can try to confirm this by tapping the board while it is in use.

Using a less mechanically "clicky" switch will solve the problem. So will using different types of capacitors.