Loading a Model with weights and optimizers without creating an instance in PyTorch
If you are open to using huggingface transformer for fine tuning which is really popular, here is a code sample:
import transformers
class Model(nn.Module):
    def __init__(self):
        super(Model,self).__init__()
        self.Bert = transformers.CamemBertModel.from_pretrained('camembert-base')
        self.fc0 = nn.Linear(768,1)

        nn.init.normal_(self.fc0.weight,std= 0.1)
        nn.init.normal_(self.fc0.bias ,0.)
        
    def forward(self,input_ids,attention_mask):
        hid= self.Bert(input_ids,attention_mask = attention_mask)
        hid= hid[0][:,0]
        x = self.fc0(hid)
        return x

You can change your last layer as you need.It is just a sample. It will load the pretrained weights from huggingface.You need not provide it.You can install transformers by the following line in terminal.
pip3 install transformers