From Coco annotation json to semantic segmentation image like VOC's .png in pytorch
I am trying to use COCO 2014 data for semantic segmentation training in PyTorch. 
I have a PSPNet model with a Cross Entropy loss function that worked perfectly on PASCAL VOC dataset from 2012. 
Now I am trying to use a portion of COCO pictures to do the same process. But Coco has json data instead of .png images for annotation and I somehow have to covert one to the other. I have noticed that there is annToMask in cocotools, but I cannot quiet figure out how to use that function in my case.
This is kind of what my dataloader's pull item looks like

def pull_item(self, index):

        I DONT KNOW WHAT TO DO HERE

        raw_img = self.transform(raw_img)
        anns_img = self.transform(anns_img)

        return raw_img, anns_img


Below is what my training function that uses data from dataloaders looks like.

 for images, labels in dataloaders_dict[phase]:

                images = images.to(device)

                labels = torch.squeeze(labels)
                labels = labels.to(device)

                with torch.set_grad_enabled(phase == 'train'):
                    outputs = net(images)

                    loss = criterion(outputs, labels.long())


I have worked on creating a Data Generator for the COCO dataset with PyCOCO and I think my experience can help you out. My post on medium documents the entire process from start to finish, including the creation of masks.

However, point to note, I was working with Tensorflow Keras and not pytorch. But the logic flow should largely be the same, so I am sure you can take back something useful from it.
