Reading BerkleyDB files from python: `\x01\x0b\x88\x0c\x01`?
Question: **This question in a nutshell: what do`\x04++HLMh7EjP3ILSfF\x00` and**

    
    
    '\x01\x0b\x88\x0c\x01-\x10\x02\x06!\x05"\x05#\n$\x0c\'\x0e%\x0b\x01&\x02\'\x06(\n\x00\x00'
    

**mean?**

Hi all,

I'm trying to read palm pre 2 database files. Some information is available
[in the documentation](https://developer.palm.com/content/api/dev-
guide/mojo/db8.html), but not enough for me to clearly understand the format.

Using the `file` command, I learnt that the format was `objects.db: Berkeley
DB (Btree, version 9, native byte-order)`.

Trying to open the database directly using `bsddb.open()` didn't work; I had
to use

    
    
    >>> env = bsddb.db.DBEnv()
    >>> env.open(None, bsddb.db.DB_CREATE | bsddb.db.DB_INIT_MPOOL)
    >>> internal_db = bsddb.db.DB(env)
    >>> internal_db.open('C:\objects.db', 'objects.db', bsddb.db.DB_BTREE, bsddb.db.DB_RDONLY)
    

Now I've open the database, but the keys and values are encoded in a format
which I don't understand: for example, here are some keys:
`'\x04++HMvu4v2GZbo1Ox\x00', '\x04++HMwBSPR8Zvwkt5\x00',
'\x04++HMwF4OJ0R+WeSS\x00'`, and a value:

    
    
    '\x01\x0b\xb7\r\x00\x05\xee\x89\x10\x029\x060\x04\x00/\x03\x04++HQqD0wWr_hZP75\x00\x00 \x02"\x06\x00$\x04inbox\x001\x02+\x04+33626320868\x00\x00%\x0e\x00\x00\x01.0\x19\xb3\x10&\x04Ok\x00-\rM\\/\x892\n:\r\x00\x05\xee\x89\'\x04sms\x003\n(\x04successful\x00.\rM\\/\x89\x00'
    

I tried to decode it from utf8, but I didn't get any convincing results. Do
you recognize which encoding is being used? I don't understand the `native
byte-order` part of the output of the `file` command, could it be related to
this?

Thanks!


Answer: ### Educated guess

From simple inspection, The message is probably "sms successful", with a phone
number:

> > > unicode(s, errors='ignore')
> u'\x01\x0b\r\x00\x05\x10\x029\x060\x04\x00/\x03\x04++HQqD0wWr_hZP75\x00\x00
> \x02"\x06\x00$\x04inbox\x001\x02+\x04+33626320868\x00\x00%\x0e\x00\x00\x01.0\x19\x10&\x04Ok\x00-\rM\/2\n:\r\x00\x05\'\x04sms\x003\n(\x04successful\x00.\rM\/\x00'

I think the other characters are binary data.

### Encodings

Decoding it did not help - Both chardet and BeautifulSoup detects this as
`windows-1252`:

    
    
    >>> s=u'\x01\x0b\xb7\r\x00\x05\xee\u2030\x10\x029\x060\x04\x00/\x03\x04++HQqD0wWr_hZP75\x00\x00 \x02"\x06\x00$\x04inbox\x001\x02+\x04+33626320868\x00\x00%\x0e\x00\x00\x01.0\x19\xb3\x10&\x04Ok\x00-\rM\\/\u20302\n:\r\x00\x05\xee\u2030\'\x04sms\x003\n(\x04successful\x00.\rM\\/\u2030\x00'
    >>> from BeautifulSoup import BeautifulSoup
    >>> soup = BeautifulSoup(s)
    >>> soup.originalEncoding
    'windows-1252'
    >>> chardet.detect(s)
    {'confidence': 0.5, 'encoding': 'windows-1252'}
    

However, the 1252 decoding gives nothing meaningful:

    
    
    >>> s.decode('windows-1252')
    u'\x01\x0b\xb7\r\x00\x05\xee\u2030\x10\x029\x060\x04\x00/\x03\x04++HQqD0wWr_hZP75\x00\x00 \x02"\x06\x00$\x04inbox\x001\x02+\x04+33626320868\x00\x00%\x0e\x00\x00\x01.0\x19\xb3\x10&\x04Ok\x00-\rM\\/\u20302\n:\r\x00\x05\xee\u2030\'\x04sms\x003\n(\x04successful\x00.\rM\\/\u2030\x00'
    

