Why yolo4 pytorch re-training loss seems high as like first time training?
Apparently, only the lower layers weights of the pretrained model are being updated. to change it open models.py and scroll down to the class definition of Yolov4.
As one can see, only down1-down5 and neck layers are being updated by the pretrained model weights while the head is starting fresh. you should change this code section to something similar to:
if yolov4conv137weight:
        self.neck = Neck()
        **self.head = Yolov4Head(output_ch)**
        _model = nn.Sequential(self.down1, self.down2, self.down3, self.down4, self.down5, self.neck, **self.head**)
        pretrained_dict = torch.load(yolov4conv137weight)

        model_dict = _model.state_dict()
        # 1. filter out unnecessary keys
        pretrained_dict = {k1: v for (k, v), k1 in zip(pretrained_dict.items(), model_dict)}
        # 2. overwrite entries in the existing state dict
        #print(&quot;update weights&quot;)
        model_dict.update(pretrained_dict)
        _model.load_state_dict(model_dict)
    # head
    **else:
        self.head = Yolov4Head(output_ch)**

Run again and observe the expected loss (hopefully).
Good luck!