title: UnicodeError: URL contains non-ASCII characters (Python 2.7)
body: <p>So I've managed to make a crawler, and I'm searchng for all links and when I arrive at a product link I make some finds and I take all product information, but when it arrives to certain page it gives a unicode error :/</p>
  
  <pre><code>import urllib
  import urlparse
  from itertools import ifilterfalse
  from urllib2 import URLError, HTTPError
  
  from bs4 import BeautifulSoup
  
  urls = ["http://www.kiabi.es/"]
  visited = []
  
  
  def get_html_text(url):
      try:
          return urllib.urlopen(current_url).read()
      except (URLError, HTTPError, urllib.ContentTooShortError):
          print "Error getting " + current_url
  
  
  def find_internal_links_in_html_text(html_text, base_url):
      soup = BeautifulSoup(html_text, "html.parser")
      links = []
      for tag in soup.findAll('a', href=True):
          url = urlparse.urljoin(base_url, tag['href'])
          domain = urlparse.urlparse(base_url).hostname
          if domain in url:
              links.append(url)
      return links
  
  
  def is_url_already_visited(url):
      return url in visited
  
  
  while urls:
      current_url = urls.pop()
      word = '#C'
      if word in current_url:
          [do sth]
      #print "Parsing", current_url
      html_text = get_html_text(current_url)
      visited.append(current_url)
      found_urls = find_internal_links_in_html_text(html_text, current_url)
      new_urls = ifilterfalse(is_url_already_visited, found_urls)
      urls.extend(new_urls)
  </code></pre>
  
  <p>Error:</p>
  
  <pre><code>Traceback (most recent call last):
  
  File "&lt;ipython-input-1-67c2b4cf7175&gt;", line 1, in &lt;module&gt;
  runfile('S:/Consultas_python/Kiabi.py', wdir='S:/Consultas_python')
  
  File "C:\Anaconda2\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 685, in runfile
  execfile(filename, namespace)
  
  File "C:\Anaconda2\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 71, in execfile
  exec(compile(scripttext, filename, 'exec'), glob, loc)
  
  File "S:/Consultas_python/Kiabi.py", line 91, in &lt;module&gt;
  html_text = get_html_text(current_url)
  
  File "S:/Consultas_python/Kiabi.py", line 30, in get_html_text
  return urllib.urlopen(current_url).read()
  
  File "C:\Anaconda2\lib\urllib.py", line 87, in urlopen
  return opener.open(url)
  
  File "C:\Anaconda2\lib\urllib.py", line 185, in open
  fullurl = unwrap(toBytes(fullurl))
  
  File "C:\Anaconda2\lib\urllib.py", line 1070, in toBytes
  " contains non-ASCII characters")
  
  UnicodeError: URL u'http://www.kiabi.es/Barbapap\xe1_s1' contains non-ASCII characters
  </code></pre>
  
  <p>or</p>
  
  <pre><code>UnicodeError: URL u'http://www.kiabi.es/Petit-B\xe9guin_s2' contains non-ASCII characters
  </code></pre>
  
  <p>How can I fix it?</p>