What to do in the summer if math REUs reject me? Should I still apply to grad school?
REUs are a nice way to start to get a sense of what mathematics research is about, and can certainly be helpful for graduate admissions.  But they're not a prerequisite.  You don't need to give up on grad school just because you didn't get accepted to an REU.

As for what to do with the summer, if you are interested in research, consider trying to work on a research project with one of the professors at your university.  You say you want to do this next year, so it would be even better to start this summer, when you will have more time available and won't be distracted by coursework.  Start asking them about it now.

I think it is definitely a good idea to have some sort of research experience before applying to grad school.  Partly because it is helpful in admissions, but mostly because it will help you decide if it is actually something you want to do.  Mathematics research is a very different experience than coursework, and before committing to doing it for several years, it would be wise to see if you like it.