title: How can I replace Unicode characters in Python?
body: <p>I'm pulling Twitter data via their API and one of the tweets has a special character (the right apostrophe) and I keep getting an error saying that Python can't map or character map the character. I've looked all over the Internet, but I have yet to find a solution for this issue. I just want to replace that character with either an apostrophe that Python will recognize, or an empty string (essentially removing it). I'm using Python 3.3. Any input on how to fix this problem? It may seem simple, but I'm a newbie at Python.</p>
  
  <p>Edit: Here is the function I'm using to try to filter out the unicode characters that throw errors.</p>
  
  <pre><code>@staticmethod
  def UnicodeFilter(var):
      temp = var
      temp = temp.replace(chr(2019), "'")
      temp = Functions.ToSQL(temp)
      return temp
  </code></pre>
  
  <p>Also, when running the program, my error is as follows.  </p>
  
  <blockquote>
    <p>'charmap' codec can't encode character '\u2019' in position 59: character maps to 'undefined'</p>
  </blockquote>
  
  <p>Edit: Here is a sample of my source code:</p>
  
  <pre><code>import json
  import mysql.connector
  import unicodedata
  from MySQLCL import MySQLCL
  
  class Functions(object):
  """This is a class for Python functions"""
  
  @staticmethod
  def Clean(string):
      temp = str(string)
      temp = temp.replace("'", "").replace("(", "").replace(")", "").replace(",", "").strip()
      return temp
  
  @staticmethod
  def ParseTweet(string):
      for x in range(0, len(string)):
          tweetid = string[x]["id_str"]
          tweetcreated = string[x]["created_at"]
          tweettext = string[x]["text"]
          tweetsource = string[x]["source"]
          truncated = string[x]["truncated"]
          inreplytostatusid = string[x]["in_reply_to_status_id"]
          inreplytouserid = string[x]["in_reply_to_user_id"]
          inreplytoscreenname = string[x]["in_reply_to_screen_name"]
          geo = string[x]["geo"]
          coordinates = string[x]["coordinates"]
          place = string[x]["place"]
          contributors = string[x]["contributors"]
          isquotestatus = string[x]["is_quote_status"]
          retweetcount = string[x]["retweet_count"]
          favoritecount = string[x]["favorite_count"]
          favorited = string[x]["favorited"]
          retweeted = string[x]["retweeted"]
          possiblysensitive = string[x]["possibly_sensitive"]
          language = string[x]["lang"]
  
          print(Functions.UnicodeFilter(tweettext))
          #print("INSERT INTO tweet(ExTweetID, TweetText, Truncated, InReplyToStatusID, InReplyToUserID, InReplyToScreenName, IsQuoteStatus, RetweetCount, FavoriteCount, Favorited, Retweeted, Language, TweetDate, TweetSource, PossiblySensitive) VALUES (" + str(tweetid) + ", '" + Functions.UnicodeFilter(tweettext) + "', " + str(truncated) + ", " + Functions.CheckNull(inreplytostatusid) + ", " + Functions.CheckNull(inreplytouserid) + ", '" + Functions.CheckNull(inreplytoscreenname) + "', " + str(isquotestatus) + ", " + str(retweetcount) + ", " + str(favoritecount) + ", " + str(favorited) + ", " + str(retweeted) + ", '" + str(language) + "', '" + Functions.ToSQL(tweetcreated) + "', '" + Functions.ToSQL(tweetsource) + "', " + str(possiblysensitive) + ")")
          #MySQLCL.Set("INSERT INTO tweet(ExTweetID, TweetText, Truncated, InReplyToStatusID, InReplyToUserID, InReplyToScreenName, IsQuoteStatus, RetweetCount, FavoriteCount, Favorited, Retweeted, Language, TweetDate, TweetSource, PossiblySensitive) VALUES (" + str(tweetid) + ", '" + tweettext + "', " + str(truncated) + ", " + Functions.CheckNull(inreplytostatusid) + ", " + Functions.CheckNull(inreplytouserid) + ", '" + Functions.CheckNull(inreplytoscreenname) + "', " + str(isquotestatus) + ", " + str(retweetcount) + ", " + str(favoritecount) + ", " + str(favorited) + ", " + str(retweeted) + ", '" + language + "', '" + tweetcreated + "', '" + str(tweetsource) + "', " + str(possiblysensitive) + ")")
  
  @staticmethod
  def ToBool(variable):
      if variable.lower() == 'true':
          return True
      elif variable.lower() == 'false':
          return False
  
  @staticmethod
  def CheckNull(var):
      if var == None:
          return ""
      else:
          return var
  
  @staticmethod
  def ToSQL(var):
      temp = var
      temp = temp.replace("'", "''")
      return str(temp)
  
  @staticmethod
  def UnicodeFilter(var):
      temp = var
      #temp = temp.replace(chr(2019), "'")
      unicodestr = unicode(temp, 'utf-8')
      if unicodestr != temp:
          temp = "'"
      temp = Functions.ToSQL(temp)
      return temp
  </code></pre>
  
  <p><a href="https://stackoverflow.com/users/984421/ekhumoro">ekhumoro's</a> response was correct.</p>