"Administer Taxonomy" is needed to make a user select a term
You don't need to give an user the permission to Administer Taxonomy in order to use a vocabulary. Assuming you are using Drupal 7, as the name implies Administer vocabularies and terms it's a permission that allows one to edit and/or *delete a term from any vocabularies. It can also be specified per vocabulary.