Moving to a different PhD program in the United States
Most programs will not fault you for wanting to change disciplines. If you can obtain the necessary application materials (letters of recommendation, specifically), changing programs will likely not be a massive issue. Be clear in your statements of purpose that you are interested in changing disciplines. If you are open as to why you are changing programs, I do not see application committees being overly turned off by such a decision. (I personally would not be at least. If a candidate is a good fit, I will pursue him or her). Explain the situation much like you have in your question. Emphasize that you feel that you could be a more successful researcher in a research program that is more inter-disciplinary. (Your background is likely to be more well-versed for an inter-disciplinary program anyway it sounds like). 

Do not obscure the fact that you are looking to start a new program. You have nothing to hide as far as I can tell. (This will obviously be contingent on potential complications you may or may not have fully disclosed). 

One comment I will briefly add deals with the "grass is always greener" aspect of graduate studies (and life). Make sure that changing programs will actually be a better situation than remaining with your current program. I would have a contingency plan in place for the event that you apply to new programs and are accepted into none of them. (Or maybe your plan can be to apply to one or two schools you know you will get into). There is the old adage "A bird in the hand is better than two in the bush." Make sure that if you put down the bird you have in your hand that the one in the bush is actually going to be a bigger, better bird. 

Because you provide no details about why choosing an advisor will be a problem, I am not going to be able to give any advice in that regard. 

As for career prospects, an inter-disciplinary PhD in math can be a blessing and a curse. My personal anecdotal experience has been that academia prefers more of a focused "single discipline" PhD (although this is changing to a certain extent) and industry prefers inter-disciplinary studies.  

As for what you need to tell your department right now, I would figure out first how you are going to get your letters of recommendation. If you feel that you have three people who will write you letters (and then maybe one backup), I would then speak to your graduate coordinator about terminating any support you have. Make sure that you close out any financial considerations that must take place. This will go a long way in ensuring you do not burn bridges when you leave. 

Some students invariably will leave a graduate program from time to time. Any admissions committee knows this. Declaring your intentions sooner rather than later will make your current program significantly more willing (and able) to help you find the best fit for your future studies. I would not be too concerned overall with trying to apply to other programs as long as you are open with your current program.