How do I group taxonomy by hierarchy, not node? I am using Drupal 7 and Views.

I'm trying to use the generic Taxonomy/Term view and get a grouping I don't want.
I want this:


Taxonomy 1:


Node A
Node B

Taxonomy 2:


Node B
Node C



(Notice Node B is in both taxonomies.)

What I'm getting is the following one:


Taxonomy 1:


Node A

Taxonomy 2:


Node C

Taxonomy 1, Taxonomy 2


Node B



How do I get the first example to display,  where Node B just appears under the different Taxonomies and doesn't create a new heading with multiple taxonomies?
I started a new view of taxonomy terms (sometimes the default taxonomy term view is overkill) which created a view of my taxonomy terms. I then created a relationship to Taxonomy term: Content with term and then added a field Content: Title using the relationship just created.

Finally I changed the Settings of view under Format to group by by term and hid the actual term field.

My output ended up like the following, Product One was tagged with Shirts and Ties, product Two just with ties as expected.


Shirts


Title: Product One
Title: Product Two

Ties


Title: Product One