title: Parsing spanish text and saving it in a db
body: <p>I'm parsing a web page written in spanish with <a href="http://scrapy.org" rel="nofollow">scrapy</a>. The problem is that I can't save the text because of the wrong encoding.</p>
  
  <p>This is the parse function:</p>
  
  <pre><code>def parse(self, response):
          hxs = HtmlXPathSelector(response)
          text = hxs.select('//text()').extract() # Ex: [u' Sustancia mineral, m\xe1s o menos dura y compacta, que no es terrosa ni de aspecto met\xe1lico.']
          s = "".join(text)
          db = dbf.Dbf("test.dbf", new=True)
          db.addField(
              ("WORD", "C", 25),
              ("DATA", "M", 15000), # Memo field
          )
          rec = db.newRecord()
          rec["WORD"] = "Stone"
          rec["DATA"] = s
          rec.store()
          db.close()
  </code></pre>
  
  <p>When I try to save it to a db(a dbf db) I get an ASCII(128) error. I tried decoding/encoding using 'utf-8' and 'latin1' but with no success. </p>
  
  <p>Edit:</p>
  
  <p>To save the db I'm using <a href="http://www.fiby.at/dbfpy/" rel="nofollow">dbfpy</a>. I added the dbf saving code in the parse function above.</p>
  
  <p>This is the error message:</p>
  
  <pre><code>Traceback (most recent call last):
    File "/usr/lib/python2.6/dist-packages/twisted/internet/base.py", line 1179, in mainLoop
      self.runUntilCurrent()
    File "/usr/lib/python2.6/dist-packages/twisted/internet/base.py", line 778, in runUntilCurrent
      call.func(*call.args, **call.kw)
    File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 280, in callback
      self._startRunCallbacks(result)
    File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 354, in _startRunCallbacks
      self._runCallbacks()
  --- &lt;exception caught here&gt; ---
    File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 371, in _runCallbacks
      self.result = callback(self.result, *args, **kw)
    File "/home/katy/Dropbox/proyectos/rae/rae/spiders/rae_spider.py", line 54, in parse
      rec.store()
    File "/home/katy/Dropbox/proyectos/rae/rae/spiders/record.py", line 211, in store
      self.dbf.append(self)
    File "/home/katy/Dropbox/proyectos/rae/rae/spiders/dbf.py", line 214, in append
      record._write()
    File "/home/katy/Dropbox/proyectos/rae/rae/spiders/record.py", line 173, in _write
      self.dbf.stream.write(self.toString())
    File "/home/katy/Dropbox/proyectos/rae/rae/spiders/record.py", line 223, in toString
      for (_def, _dat) in izip(self.dbf.header.fields, self.fieldData)
    File "/home/katy/Dropbox/proyectos/rae/rae/spiders/fields.py", line 215, in encodeValue
      return str(value)[:self.length].ljust(self.length)
  exceptions.UnicodeEncodeError: 'ascii' codec can't encode character u'\xf1' in position 18: ordinal not in range(128)
  </code></pre>