Can taxonomy terms contain special characters e.g. ampersands, apostrophes and dashes?
They are safe to use, however you may have issues with your url paths. so in your url if you have a term Theology &amp; Scripture", it becomes this: 'theology%20%2526%20scripture. This kind of urls are not good for search engine optimisation.

You can however use the pathauto module, to correct this issue.


  The Pathauto module automatically generates URL/path aliases for
  various kinds of content (nodes, taxonomy terms, users) without
  requiring the user to manually specify the path alias. This allows you
  to have URL aliases like /category/my-node-title instead of /node/123.
  The aliases are based upon a "pattern" system that uses tokens which
  the administrator can change.