Is it possible to get a PhD in physics in 3 years? If not, why?
I have personally known only 2 people who got their PhD in 3 years in the US (though in Computer Science, not in Physics). I don't think it's impossible in Physics either, especially if you already have a masters going into the PhD, and if you have some research experience already so you can hit the ground running.

The problem generally with time to completion of a degree is simply that to do it in the near-minimum amount of time requires everything to go more or less according to plan, and you either have to get pretty lucky or just accept any outcome of your research efforts. I believe the root cause is simply that having a PhD where you have done the absolute minimum required, in the absolute minimum of time, has an uncertain (and perhaps limited) value. 

If you've learned all you really needed to learn, can demonstrate appropriate mastery, fulfill all program requirements, and have established the level of knowledge, skill, and experience required to move on to whatever you want to do after your PhD - fantastic! Based on the fact that average time to completion is higher than 3 years, this suggests that a large portion of people don't find that 3 years was enough for them.

There are myriad reasons for why time to completion of degrees is well beyond the minimum on average, but you could roughly condense them to: personal factors, adviser factors, and program factors. Some people just don't achieve the minimum to get approved for the PhD in 3 years, and some people just go way, way beyond the minimum before officially completing their program. What's most important is that you are clear - or get clear - on what you want to get out of your program, and what you want to do after you have completed it, and then plan appropriately to get the most out of the opportunity you have. That may mean a  completion time that is lower or higher than average, but what is most important is that you get what you came for in the first place.