How to dress for visit to math phd program for accepted students?
Math professors tend to be pretty casual dressers compared to those in other disciplines. Wearing khaki pants and a collared shirt (polo or button down) would put one towards the dressier end of the spectrum at most places, and jeans and a t-shirt / hoody are usually unlikely to cause one to stand out. 

I think that if you were to wear a suit and tie to these events you’d stand out. Not in a negative way mind you, it’s just that all of the faculty members you’ll be interacting with will be dressed much more casually. If you wear jeans or khakis with a collared shirt and sweater then you’ll look like you put in some effort and are taking this seriously, but aren’t dressed up so formally that you stand out.