Why does sys.getsizeof() not return [size] in file.read([size]) in Python
Question: I have a large binary file that I would like to read in and unpack using
struct.unpack() The file consists of a number of lines each 2957 bytes long. I
read in the file using the following code:

    
    
    with open("bin_file", "rb") as f:
        line = f.read(2957)
    

My question is why, is the size returned by:

    
    
    import sys
    sys.getsizeof(line)
    

not equal to 2957 (in my case it is 2978)?


Answer: You misunderstand what `sys.getsizeof()` _does_. It returns the amount of
memory Python uses for a string object, not length of the line.

Python string objects track reference counts, the object type and other
metadata together with the actual characters, so 2978 bytes is _not_ the same
thing as the string length.

See the [`stringobject.h` definition of the
type](https://hg.python.org/cpython/file/3f13f1620d70/Include/stringobject.h#l35):

    
    
    typedef struct {
        PyObject_VAR_HEAD
        long ob_shash;
        int ob_sstate;
        char ob_sval[1];
    
        /* Invariants:
         *     ob_sval contains space for 'ob_size+1' elements.
         *     ob_sval[ob_size] == 0.
         *     ob_shash is the hash of the string or -1 if not computed yet.
         *     ob_sstate != 0 iff the string object is in stringobject.c's
         *       'interned' dictionary; in this case the two references
         *       from 'interned' to this object are *not counted* in ob_refcnt.
         */
    } PyStringObject;
    

where `PyObject_VAR_HEAD` is defined in
[`object.h`](https://hg.python.org/cpython/file/3f13f1620d70/Include/object.h#l77),
where the standard `ob_refcnt`, `ob_type` and `ob_size` fields are all
defined.

So a string of length 2957 takes 2958 bytes (string length + null) and the
remaining 20 bytes you see are to hold the reference count, the type pointer,
the object 'size' (string length here), the cached string hash and the
interned state flag.

Other object types will have different memory footprints, and the exact sizes
of the C types used differ from platform to platform as well.

