How to get the size of a Hugging Face pretrained model?
I keep getting a CUDA out of memory error when trying to fine-tune a Hugging Face pretrained XML Roberta model. So, the first thing I want to find out is the size of the pretrained model.
model = XLMRobertaForCausalLM.from_pretrained('xlm-roberta-base', config=config)
device = torch.device(&quot;cuda&quot;) if torch.cuda.is_available() else torch.device(&quot;cpu&quot;)

model.to(device)

I have tried to get the size of the model with
sys.getsizeof(model)

and, unsurprisingly, I get an incorrect result.  I get 56 as a result, which is the size of the python object.
But then, I tried model. element_size(), and I get the error
ModuleAttributeError: 'XLMRobertaForCausalLM' object has no attribute 'element_size'

I have searched in the Hugging Face documentation, but I have not found how to do it. Does anyone here know how to do it?

If you facing CUDA out of memory errors, the problem is mostly not the model, rather than the training data. You can reduce the batch_size (number of training examples used in parallel), so your gpu only need to handle a few examples each iteration and not a ton of.
However, to your question:
I would recommend you objsize. It is a library that calculates the &quot;real&quot; size (also known as &quot;deep&quot; size). So a straightforward solution would be:
import objsize
objsize.get_deep_size(model)

However, the documentation says:

Excluding non-exclusive objects. That is, objects that are also referenced from somewhere else in the program. This is true for calculating the object's deep size and for traversing its descendants.

This shouldn't be a problem, but if it still gets a too small size for your model you can use Pympler, another Library that calculates the &quot;deep&quot; size via recursion.
Another approach would be implementing a get_deep_size() function by yourself, e.g. from this article:
import sys

def get_size(obj, seen=None):
    &quot;&quot;&quot;Recursively finds size of objects&quot;&quot;&quot;
    size = sys.getsizeof(obj)
    if seen is None:
        seen = set()
    obj_id = id(obj)
    if obj_id in seen:
        return 0
    # Important mark as seen *before* entering recursion to gracefully handle
    # self-referential objects
    seen.add(obj_id)
    if isinstance(obj, dict):
        size += sum([get_size(v, seen) for v in obj.values()])
        size += sum([get_size(k, seen) for k in obj.keys()])
    elif hasattr(obj, '__dict__'):
        size += get_size(obj.__dict__, seen)
    elif hasattr(obj, '__iter__') and not isinstance(obj, (str, bytes, bytearray)):
        size += sum([get_size(i, seen) for i in obj])
    return size

