Incompatibility between job and field of study
As a graduate PhD your options are staying in academia and moving to industry, as you correctly point out. If you stay in academia, be it as research staff or as postdoc, it is generally because there is a interest in your field of study. In industry however, especially with rather specialized studies, you are not hired for your research to this point. With a PhD you have proven you are capable of independently solving challenging problems. And math/physics PhDs are generally considered as being good at working with numbers and abstract concepts. This is why they end up in the banking sector or working for insurance companies. It doesn't matter if your PhD is from a high ranking university or not, once you move past a certain point of specialization there is simply no demand for the research you are working on, you are hired for the skills you developed during your PhD