After starting a PhD in Mathematics, I realized I am more interested in Statistics - can I transfer to a Statistics PhD?
Talk to your tutor or professor, they are there to help you and I always found mine very supportive. It sounds like you have thought about this. Find out what your options are they won't bite your head off for changing your mind. I changed direction twice before even getting to my Ph.D. then left that after 9 months when I realised it was not right for me. 

Don't spend years of your life doing something you don't care for, you will be far happier and productive doing something you truly love. There is a good chance your profs and tutors will have zig-zagged through their careers as well.