How do I group taxonomy by hierarchy, not node?
I started a new view of taxonomy terms (sometimes the default taxonomy term view is overkill) which created a view of my taxonomy terms. I then created a relationship to Taxonomy term: Content with term and then added a field Content: Title using the relationship just created.

Finally I changed the Settings of view under Format to group by by term and hid the actual term field.

My output ended up like the following, Product One was tagged with Shirts and Ties, product Two just with ties as expected.


Shirts


Title: Product One
Title: Product Two

Ties


Title: Product One