AttributeError: 'GPT2Model' object has no attribute 'gradient_checkpointing'
I am trying to load a GPT2 fine tuned model in flask initially. The model is being loaded during the init functions using:
app.modelgpt2 = torch.load('models/model_gpt2.pt', map_location=torch.device('cpu'))
app.modelgpt2tokenizer = GPT2Tokenizer.from_pretrained('gpt2')

But while performing the prediction task as followed in the snippet below:
from flask import current_app
input_ids = current_app.modelgpt2tokenizer.encode(&quot;sample sentence here&quot;, return_tensors='pt')
sample_outputs = current_app.modelgpt2.generate(input_ids,
                                                do_sample=True,
                                                top_k=50,
                                                min_length=30,
                                                max_length=300,
                                                top_p=0.95,
                                                temperature=0.7,
                                                num_return_sequences=1)

It throws the following error as mentioned in the question:
AttributeError: 'GPT2Model' object has no attribute 'gradient_checkpointing'

The error trace is listed starting from the model.generate function:
File &quot;/venv/lib/python3.8/site-packages/torch/autograd/grad_mode.py&quot;, line 28, in decorate_context
return func(*args, **kwargs)


File &quot;/venv/lib/python3.8/site-packages/transformers/generation_utils.py&quot;, line 1017, in generate
return self.sample(


File &quot;/venv/lib/python3.8/site-packages/transformers/generation_utils.py&quot;, line 1531, in sample
outputs = self(


File &quot;/venv/lib/python3.8/site-packages/torch/nn/modules/module.py&quot;, line 1102, in _call_impl
return forward_call(*input, **kwargs)


File &quot;/venv/lib/python3.8/site-packages/transformers/models/gpt2/modeling_gpt2.py&quot;, line 1044, in forward
transformer_outputs = self.transformer(


File &quot;/venv/lib/python3.8/site-packages/torch/nn/modules/module.py&quot;, line 1102, in _call_impl
return forward_call(*input, **kwargs)


File &quot;/venv/lib/python3.8/site-packages/transformers/models/gpt2/modeling_gpt2.py&quot;, line 861, in forward
print(self.gradient_checkpointing)


File &quot;/venv/lib/python3.8/site-packages/torch/nn/modules/module.py&quot;, line 1177, in getattr
raise AttributeError(&quot;'{}' object has no attribute '{}'&quot;.format(


AttributeError: 'GPT2Model' object has no attribute 'gradient_checkpointing'

Checked with modeling_gpt2.py, by default self.gradient_checkpointing is set False in the constructor of the class.

This issue is found to be occurring only if the framework is run using venv or deployment frameworks like uWSGI or gunicorn.
It is resolved when transformers version 4.10.0 is used instead of the latest package.
