Associate an user to a taxonomy term
User profiles are fieldable in Drupal 7, you can add a taxonomy field already, with nothing but core. You don't need Entity Reference or any other modules, you can simply  add field_taxonomy to your user. Sure, it is a bit opposite to "add users to terms" you described, but later, if you need it for Views etc, it doesn't matter which way it was created - there is a connection between user and term, and you can use it.