How to create a view based on taxonomy term of a node reference
OK. I've solved it myself!

I had to set up a relationship with "Team".

Then I added in a  "Content: Has taxonomy term ID" contextual filter. Within the settings there is a relationship option which I set to the one I set above.

Works as I expected!