Read data from binary file python
Question: I have a binary file with this format:

[![enter image description
here](http://i.stack.imgur.com/qHVBs.jpg)](http://i.stack.imgur.com/qHVBs.jpg)

and i use this code to open it:

    
    
    import numpy as np
    
    f = open("author_1", "r")
    
    dt = np.dtype({'names': ['au_id','len_au_name','au_name','nu_of_publ', 'pub_id', 'len_of_pub_id','pub_title','num_auth','len_au_name_1', 'au_name1','len_au_name_2', 'au_name2','len_au_name_3', 'au_name3','year_publ','num_of_cit','citid','len_cit_tit','cit_tit', 'num_of_au_cit','len_cit_au_name_1','au_cit_name_1', len_cit_au_name_2',
    'au_cit_name_2','len_cit_au_name_3','au_cit_name_3','len_cit_au_name_4',
    'au_cit_name_4', 'len_cit_au_name_5','au_cit_name_5','year_cit'],
             'formats': [int,int,'S13',int,int,int,'S61',          int,int,'S8',int,'S7',int,'S12',int,int,int,int,'S50',int,int,
                    'S7',int,'S7',int,'S9',int,'S8',int,'S1',int]})
    a = np.fromfile(f, dtype=dt, count=-1, sep="")
    

And I take this:

    
    
    array([ (1, 13, b'Scott Shenker', 200, 1, 61, b'Integrated services in the internet architecture: an overview', 3, 8, b'R Braden', 7, b'D Clark', 12, b'S Shenker\xe2\x80\xa6', 1994, 1000, 401, 50, b'[HTML] An architecture for differentiated services', 5, 7, b'D Black', 7, b'S Blake', 9, b'M Carlson', 8, b'E Davies', 1, b'Z', 1998),
    (402, 72, b'Resource rese', 1952544370, 544108393, 1953460848, b'ocol (RSVP)--Version 1 functional specification\x05\x00\x00\x00\x08\x00\x00\x00R Brad', 487013, 541851648, b'Zhang\x08', 1109414656, b'erson\x08', 542310400, b'Herzog\x07\x00\x00\x00S ', 1768776010, 511342, 103168, 22016, b'\x00A reliable multicast framework for light-weight s', 1769173861, 544435823, b'and app', 1633905004, b'tion le', 543974774, b'framing\x04', 458752, b'\x00\x00S Floy', 2660, b'', 1632247894),
    

Any idea how can open the whole file?


Answer: The data structure stored in this file is hierarchical, rather than "flat":
child arrays of different length are stored within each parent element. It is
not possible to represent such a data structure using numpy arrays (even
recarrays), and therefore it is not possible to read the file with
`np.fromfile()`.

What do you mean by "open the whole file"? What sort of python data structure
would you like to end up with?

It would be straightforward, but still not trivial, to write a function to
parse the file into a list of dictionaries.

