Changing interests after joining grad school
Things were said between the two professors that you couldn't control. It puts you in a difficult position, but might also be an opportunity. 

To switch fields you need to satisfy the university about some things, but also find an advisor in the new field. Either of these could be a stumbling block. The worst case, I think is that you need to find a new university, but that isn't entirely uncommon. It would cost you a bit of time, of course. 

But there is little worse than spending your life doing things that don't interest you. There is no replay button on life. 

Re-evaluate both your desires and your options. But also connect to those at the current university who see you in a positive light. They can also give you advice. 

FWIW, I think your advisor behaved badly in making accusations against you. I don't see your explorations as unethical in any way.