I still do not, exactly, know my field of interest. Is it relevant while applying for PhD?
I know this field.  Theoretical is enough of a preference for now.  Realistically there are many different areas in physics that rely on conceptual and mathematical ability.  At least you know you don't want to be an experimenter.  If you did, I would think there was a little more importance in having at least an initial hypothesis on what area to go into.  But I think theory is a more flexible, especially for someone new.

P.s.  Don't sleep on theoretical solid state.  E.g. density functional theory.