Parsing binary data into separate variables in Python?
Question: Say I open a file in Python that contains a series of binary data.

    
    
    with open(sys.argv[1]) as data_file:
        logData = data_file.read()
    

I basically want to create a loop saying:

    
    
    for each_word in logData:
           var1 = first 8 bytes
           var2 = next 16 bytes
           var3 = next 8 bytes
    

C code to generate my binary file:

    
    
    #include <stdio.h>
    #include <stdlib.h>
    
    int main(void)
    {
            FILE *file;
            int  buffer = 0x03000201;
            int  buffer2= 0x010203;
            file = fopen("test.bin", "wb");
    
            if (file != NULL)
            {
                    fwrite(&buffer, sizeof(buffer), 1, file);
                    fwrite(&buffer2, sizeof(buffer2), 1, file);
                    fclose(file);
            }
    
            return 0;
    }
    

and this basically continues until the loop is over, iterating through the
bytes of data. How is this possible?


Answer: Use the [`struct` module](http://docs.python.org/2/library/struct.html), it
will also allow you to interpret the binary data in many ways; you need to
define the types in a string format documented with that library:

    
    
    struct.unpack('=HHf255s', bytes)
    

The above example expects native byte-order, two unsigned shorts, a float and
a string of 255 characters.

Your code example becomes:

    
    
    for each_word in logData:
        var1, var2, var3 = struct.unpack('8s16s8s', each_word)
    

In case you get an error `TypeError: 'str' does not support the buffer
interface`, that is because `bytes` are expected, but you passed a `str`, so
convert it to bytes and specify the encoding (in this example, UTF-8):

    
    
    for each_word in logData:
        var1, var2, var3 = struct.unpack('8s16s8s', bytes(each_word, 'utf-8'))
    

But maybe the case that your 8/16 byte strings are long integers? In that case
use the appropriate format for struct.

_EDIT_ : it turns out you wanted to read 8 **bits** (not bytes), then the next
16 bits, then the next 8 bits, so you can read it as one (unsigned?) byte, one
short, and another byte. The format string you should use is `'=bhb'` (or
`'=BHB'` for unsigned). Example:

    
    
    import struct
    with open('test.bin','rb') as f:
        var1, var2, var3 = struct.unpack('=BHB', f.read(4))
        print(var1, var2, var3)
    

