How to edit how Commerce outputs the Billing and Shipping information in the checkout review table?
At time of writing, this isn't a viable solution, but I'm posting it because it might become one, one day.

Commerce Responsive UI module

There's a massive long 5-year-old issue matching my problems theming Commerce checkout in the Commerce issues queue. The "fix" was the creation of a module, Commerce Responsive UI, that aims to modernise the markup of the checkout (and other) pages and make them themable. It sounds like exactly what I'm looking for.

Problem is, there's no stable release, no release at all since November 2015, and it appears to have some serious unresolved issues, like breaking normal cart/checkout page paths. This might become a good answer to the question, but right now it's an unfinished work in progress.