Read python pickle data stream in Android
Question: I have this file which contains python pickle data stream. I've to read
contents of this file in Android.

For example, if I wanted to read this data stream in python, I'd just use the
following code

    
    
    queue = pickle.load(open('filename', 'rb'))
    

I want to achieve same thing in Android such that I can read this pickle
stream data and store it in some kind of collection.

How can I achieve this?


Answer: UPDATE: This only works for pickle protocols `2` and `3`.

I think the
[Unpickler](https://github.com/irmen/Pyrolite/blob/master/java/src/net/razorvine/pickle/Unpickler.java)
class from [Pyrolite](http://pythonhosted.org/Pyro4/pyrolite.html) (MIT
license) may be of particular interest to you. It is technically Java, but
Android is basically Java. To unpickle you would do something similar to the
following:

    
    
    InputStream stream = new FileInputStream("filename");
    Unpickler unpickler = new Unpickler();
    Object data = unpickler.load(stream);
    // And cast *data* to the appropriate type.
    

With the imports:

    
    
    import java.io.FileInputStream;
    import java.io.InputStream;
    import net.razorvine.pickle.Unpickler;
    

These are the objects supported by default:

    
    
    PYTHON    ---->     JAVA
    ------              ----
    None                null
    bool                boolean
    int                 int
    long                long or BigInteger  (depending on size)
    string              String
    unicode             String
    complex             net.razorvine.pickle.objects.ComplexNumber
    datetime.date       java.util.Calendar
    datetime.datetime   java.util.Calendar
    datetime.time       java.util.Calendar
    datetime.timedelta  net.razorvine.pickle.objects.TimeDelta
    float               double   (float isn't used) 
    array.array         array of appropriate primitive type (char, int, short, long, float, double)
    list                java.util.List<Object>
    tuple               Object[]
    set                 java.util.Set
    dict                java.util.Map
    bytes               byte[]
    bytearray           byte[]
    decimal             BigDecimal    
    custom class        Map<String, Object>  (dict with class attributes including its name in "__class__")
    

Please also note:

> The unpickler simply returns an Object. Because Java is a statically typed
> language you will have to cast that to the appropriate type. Refer to this
> table to see what you can expect to receive.

* * *

UPDATE: I ran tests using the various pickle protocols (`0-3`) and found that
it fails for `0` and `1`, but succeeds for `2` and `3`.

Here's the python code used to generate the pickled data:

    
    
    import pickle
    
    class Data(object):
        def __init__(self):
            self.x = 12
    
    data = Data()
    
    for p in [0, 1, 2]:
        with open('data.{}'.format(p), 'wb') as fh:
            pickle.dump(data, fh, protocol=p)
    
    # Python 3 only.
    with open('data.3', 'wb') as fh:
        pickle.dump(data, fh, protocol=3)
    

And the java code to unpickle it:

    
    
    import java.io.FileInputStream;
    import java.io.InputStream;
    import java.io.IOException;
    import java.util.Map;
    import net.razorvine.pickle.Unpickler;
    
    public class Test {
        public static void main(String[] args) throws IOException {
            String filename = args[0];
            InputStream inputStream = new FileInputStream(filename);
            Unpickler unpickler = new Unpickler();
            Map<String, Object> data = (Map<String, Object>)unpickler.load(inputStream);
        }
    }
    

When run with `data.0` and `data.1`, it fails with:

    
    
    Exception in thread "main" net.razorvine.pickle.PickleException: expected zero arguments for construction of ClassDict (for copy_reg._reconstructor)
      at net.razorvine.pickle.objects.ClassDictConstructor.construct(ClassDictConstructor.java:23)
      at net.razorvine.pickle.Unpickler.load_reduce(Unpickler.java:617)
      at net.razorvine.pickle.Unpickler.dispatch(Unpickler.java:170)
      at net.razorvine.pickle.Unpickler.load(Unpickler.java:84)
      at Test.main(Test.java:13)
    

When run with `data.2` and `data.3`, it succeeds.

