I want to quit my PhD and it's only been 6 weeks
Yes, it's OK to quit if what you signed up for is really not what you want (professionally). Plenty of people who start PhD's don't finish.
That said, in your particular case perhaps consider finishing the year. Use the time to learn the kind of applied mathematics you could use when (if) you return to physics. It's too late for your department to use your slot for someone else. They might save a little money if they are supporting you - but if you are earning that support by teaching or grading then your leaving has  no financial value to them.
I don't think you should decide this rashly. You should be able to talk about it with your advisor. Perhaps try to set up an appointment (virtual if necessary) rather than discussing in email.