Taxonomy Reference Based on Taxonomy Field
Two things come to my mind:


For the view make sure to do a taxonomy or user view and add a relationship to each other.
You can also use the private taxonomy module: http://drupal.org/project/private_taxonomy