Instruction: GRU Loss decreased upto 0.9 but not further, PyTorch

Input:
the code that I am using for experimenting with GRU.
import torch
import torch.nn as nn
import torch.nn.functional as F

from collections import *

class N(nn.Module):
    def __init__(self):
        super().__init__()

        self.embed = nn.Embedding(5,2)

        self.layers = 4
        self.gru = nn.GRU(2, 512, self.layers, batch_first=True)
        self.bat = nn.BatchNorm1d(4)
        self.bat1 = nn.BatchNorm1d(4)
        self.bat2 = nn.BatchNorm1d(4)
        self.fc = nn.Linear(512,100)
        self.fc1 = nn.Linear(100,100)
        self.fc2 = nn.Linear(100,5)
        self.s = nn.Softmax(dim=-1)

    def forward(self,x):
        h0 = torch.zeros(self.layers, x.size(0), 512).requires_grad_()

        x = self.embed(x)

        x,hn = self.gru(x,h0)
        x  = self.bat(x)
        x = self.fc(x)
        x = nn.functional.relu(x)

        x = self.bat1(x)
        x = self.fc1(x)
        x = nn.functional.relu(x)

        x = self.bat2(x)
        x  = self.fc2(x)
        softmaxed = self.s(x)
        return  softmaxed

inp = torch.tensor([[4,3,2,1],[2,3,4,1],[4,1,2,3],[1,2,3,4]])
out = torch.tensor([[3,2,1,4],[3,2,4,1],[1,2,3,4],[2,3,4,1]])
k = 0
n = N()
opt = torch.optim.Adam(n.parameters(),lr=0.0001)

while k<10000:
    print(inp.shape)
    o = n(inp)

    o = o.view(-1, o.size(-1))
    out = out.view(-1)
    loss = nn.functional.cross_entropy(o.view(-1,o.size(-1)),out.view(-1)-1)

    acc = ((torch.argmax(o, dim=1) == (out -1)).sum().item() / out.size(0))
    if k==10000:
        print(torch.argmax(o, dim=1))
        print(out-1)
        exit()

    print(loss,acc)
    loss.backward()
    opt.step()
    opt.zero_grad()
    k+=1
print(o[0])

Shrinked Output:
torch.Size([4, 4])
tensor(0.9593, grad_fn=<NllLossBackward>) 0.9375
torch.Size([4, 4])
tensor(0.9593, grad_fn=<NllLossBackward>) 0.9375
tensor([4.8500e-01, 9.7813e-06, 5.1498e-01, 6.2428e-06, 7.5929e-06],
       grad_fn=<SelectBackward>)

The Loss is 0.9593 and accuracy reached up to 0.9375. For this simple input data, the GRU loss is this big. What is the reason? Is there anything wrong in this code? I used cross_entropy as loss function and Adam as the optimizer. Learning rate is 0.001. I tried multiple learning rates but all gave the same final result. I added batch normalization, it speed up the training, but the same loss and accuracy. Why loss does not decrease up to 0.2 or something.

Output:
I think it's because you are using cross entropy loss function which in PyTorch combines log-softmax and negative log likelihood. Since your model already performs softmax before returning the output, you actually end up calculating the negative log likelihood for softmax of softmax. Try removing the final softmax from your model.
PyTorch documentation for cross entropy loss: https://pytorch.org/docs/stable/nn.functional.html#cross-entropy


Instruction: calculating the mean and std on an array of torch tensors

Input:
I am trying to calculate to mean and std for an array of torch tensors. My dataset has 720 training images and each of these images has 4 landmarks with X and Y representing a 2D point on the image.
to_tensor = transforms.ToTensor()

landmarks_arr = []

for i in range(len(train_dataset)):
    landmarks_arr.append(to_tensor(train_dataset[i]['landmarks']))

mean = torch.mean(torch.stack(landmarks_arr, dim=0))#, dim=(0, 2, 3))
std = torch.std(torch.stack(landmarks_arr, dim=0)) #, dim=(0, 2, 3))

print(mean.shape)
print("mean is {} and std is {}".format(mean, std))

Result:
torch.Size([])
mean is nan and std is nan

There is a couple of problems above:

Why to_tensor is not converting the values between 0 and 1?
how to calculate mean correctly?
Should I divide by 255 and where?

I have:
len(landmarks_arr)

720

and
landmarks_arr[0].shape

torch.Size([1, 4, 2])

and
landmarks_arr[0]

tensor([[[502.2869, 240.4949],
         [688.0000, 293.0000],
         [346.0000, 317.0000],
         [560.8283, 322.6830]]], dtype=torch.float64)

Output:
From the pytorch docs of ToTensor():

Converts a PIL Image or numpy.ndarray (H x W x C) in the range [0, 255] to a
torch.FloatTensor of shape (C x H x W) in the range [0.0, 1.0] if the PIL Image
belongs to one of the modes (L, LA, P, I, F, RGB, YCbCr, RGBA, CMYK, 1) or if the
numpy.ndarray has dtype = np.uint8
In the other cases, tensors are returned without scaling.

Since your Landmark values are not a PIL image, and not within [0, 255], no scaling is applied.

Your calculation appears correct. It seems, that you might have some NaN value within your data.

You can try something like
for i in range(len(train_dataset)):
    landmarks = to_tensor(train_dataset[i]['landmarks'])
    landmarks[landmarks != landmarks] = 0  # this will set all nan to zero
    landmarks_arr.append(landmarks)

within your loop. Or assert for nan within the loop to find the culprit(s):
for i in range(len(train_dataset)):
    landmarks = to_tensor(train_dataset[i]['landmarks'])
    assert(not torch.isnan(landmarks).any()), f'nan encountered in sample {i}'  # will trigger if a landmark contains nan
    landmarks_arr.append(landmarks)

No, see 1). You could divide by the max coordinates of the landmarks though to constrain them to [0, 1] if you so desire.


Instruction: Increase of GPU memory usage during training

Input:
I was training the network on usual MNIST dataset, and encountered the next problem:
when i start to add valid_metrics to a loss_list and accuracy_list, amount of GPU memory that is being used starts increasing with every 1 or 2 epochs.
This is the code of train_loop:
def train_model(model: torch.nn.Module,
                train_dataset: torch.utils.data.Dataset,
                valid_dataset: torch.utils.data.Dataset,
                loss_function: torch.nn.Module = torch.nn.CrossEntropyLoss(),
                optimizer_class: Type[torch.optim.Optimizer] = torch.optim,
                optimizer_params: Dict = {},
                initial_lr = 0.01,
                lr_scheduler_class: Any = torch.optim.lr_scheduler.ReduceLROnPlateau,
                lr_scheduler_params: Dict = {},
                batch_size = 64,
                max_epochs = 1000,
                early_stopping_patience = 20):

    optimizer = torch.optim.Adam(model.parameters(), lr=initial_lr, **optimizer_params)
    lr_scheduler = lr_scheduler_class(optimizer, **lr_scheduler_params)

    train_loader = torch.utils.data.DataLoader(train_dataset, shuffle=True, batch_size=batch_size)
    valid_loader = torch.utils.data.DataLoader(valid_dataset, batch_size=batch_size)

    best_valid_loss = None
    best_epoch = None
    loss_list = list()
    accuracy_list = list()

    for epoch in range(max_epochs):

        print(f'Epoch {epoch}')

        start = timer()

        train_single_epoch(model, optimizer, loss_function, train_loader)

        valid_metrics = validate_single_epoch(model, loss_function, valid_loader)
        loss_list.append(valid_metrics['loss'])
        accuracy_list.append(valid_metrics['accuracy'])

        print('time:', timer() - start)
        print(f'Validation metrics: \n{valid_metrics}')

        lr_scheduler.step(valid_metrics['loss'])

        if best_valid_loss is None or best_valid_loss > valid_metrics['loss']:
            print(f'Best model yet, saving')
            best_valid_loss = valid_metrics['loss']
            best_epoch = epoch
            torch.save(model, './best_model.pth')

        if epoch - best_epoch > early_stopping_patience:
            print('Early stopping triggered')
            return loss_list, accuracy_list

and the code of validate_single_epoch:
def validate_single_epoch(model: torch.nn.Module,
                          loss_function: torch.nn.Module,
                          data_loader: torch.utils.data.DataLoader):

    loss_total = 0
    accuracy_total = 0

    for data in data_loader:

        X, y = data
        X, y = X.view(-1, 784), y.to(device)
        X = X.to(device)

        output = model(X)

        loss = loss_function(output, y)
        loss_total += loss

        y_pred = output.argmax(dim = 1, keepdim=True).to(device)
        accuracy_total += y_pred.eq(y.view_as(y_pred)).sum().item()

    loss_avg = loss_total / len(data_loader.dataset)
    accuracy_avg = 100.0 * accuracy_total / len(data_loader.dataset)

    return {'loss' : loss_avg, 'accuracy' : accuracy_avg}

I use GeForce MX250 as GPU

Output:
The problem is likely because the gradients are being computed and stored in the validation loop. To solve that, perhaps the easiest way is to wrap the validation call in a no_grad context:
with torch.no_grad():
    valid_metrics = validate_single_epoch(model, loss_function, valid_loader)

If you prefer, you can also decorate the validate_single_epoch(...) with @torch.no_grad():
@torch.no_grad()
def validate_single_epoch(...):
    # ...

Not related to your problem, but pay attention that you're using a model in training mode during validation, which may not be what you want. Perhaps there is a missing call to model.eval() in the validation function.


Instruction: Pytorch couldn't build multi scaled kernel nested model

Input:
I'm trying to create a modified MNIST model which takes input 1x28x28 MNIST tensor images, and it kind of branches into different models with different sized kernels, and accumulates at the end, so as to give a multi-scale-kerneled response in the spatial domain of the images. I'm worried about the model, since, I'm unable to construct it.

import numpy as np
import torch
import torch.nn as nn
import torch.optim as optim
import torch.utils.data as Data
from torchvision import datasets, transforms
import torch.nn.functional as F
import timeit
import unittest

torch.manual_seed(0)
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False
np.random.seed(0)

# check availability of GPU and set the device accordingly

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

# define a transforms for preparing the dataset
transform = transforms.Compose([
        transforms.ToTensor(), # convert the image to a pytorch tensor
        transforms.Normalize((0.1307,), (0.3081,)) # normalise the images with mean and std of the dataset
        ])

# Load the MNIST training, test datasets using `torchvision.datasets.MNIST` using the transform defined above

train_dataset = datasets.MNIST('./data',train=True,transform=transform,download=True)
test_dataset =  datasets.MNIST('./data',train=False,transform=transform,download=True)

# create dataloaders for training and test datasets
# use a batch size of 32 and set shuffle=True for the training set

train_dataloader = Data.DataLoader(dataset=train_dataset, batch_size=32, shuffle=True)
test_dataloader = Data.DataLoader(dataset=test_dataset, batch_size=32, shuffle=True)

# My Net

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()

        # define a conv layer with output channels as 16, kernel size of 3 and stride of 1
        self.conv11 = nn.Conv2d(1, 16, 3, 1) # Input = 1x28x28  Output = 16x26x26
        self.conv12 = nn.Conv2d(1, 16, 5, 1) # Input = 1x28x28  Output = 16x24x24
        self.conv13 = nn.Conv2d(1, 16, 7, 1) # Input = 1x28x28  Output = 16x22x22

        # define a conv layer with output channels as 32, kernel size of 3 and stride of 1
        self.conv21 = nn.Conv2d(16, 32, 3, 1) # Input = 16x26x26 Output = 32x24x24
        self.conv22 = nn.Conv2d(16, 32, 5, 1) # Input = 16x24x24 Output = 32x20x20
        self.conv23 = nn.Conv2d(16, 32, 7, 1) # Input = 16x22x22 Output = 32x16x16

        # define a conv layer with output channels as 64, kernel size of 3 and stride of 1
        self.conv31 = nn.Conv2d(32, 64, 3, 1) # Input = 32x24x24 Output = 64x22x22
        self.conv32 = nn.Conv2d(32, 64, 5, 1) # Input = 32x20x20 Output = 64x16x16
        self.conv33 = nn.Conv2d(32, 64, 7, 1) # Input = 32x16x16 Output = 64x10x10

        # define a max pooling layer with kernel size 2
        self.maxpool = nn.MaxPool2d(2), # Output = 64x11x11
        # define dropout layer with a probability of 0.25
        self.dropout1 = nn.Dropout(0.25)
        # define dropout layer with a probability of 0.5
        self.dropout2 = nn.Dropout(0.5)
        # define a linear(dense) layer with 128 output features
        self.fc11 = nn.Linear(64*11*11, 128)
        self.fc12 = nn.Linear(64*8*8, 128)      # after maxpooling 2x2
        self.fc13 = nn.Linear(64*5*5, 128)

        # define a linear(dense) layer with output features corresponding to the number of classes in the dataset
        self.fc21 = nn.Linear(128, 10)
        self.fc22 = nn.Linear(128, 10)
        self.fc23 = nn.Linear(128, 10)

        self.fc33 = nn.Linear(30,10)

    def forward(self, x1):
        # Use the layers defined above in a sequential way (folow the same as the layer definitions above) and
        # write the forward pass, after each of conv1, conv2, conv3 and fc1 use a relu activation.

        x = F.relu(self.conv11(x1))
        x = F.relu(self.conv21(x))
        x = F.relu(self.maxpool(self.conv31(x)))
        #x = torch.flatten(x, 1)
        x = x.view(-1,64*11*11)
        x = self.dropout1(x)
        x = F.relu(self.fc11(x))
        x = self.dropout2(x)
        x = self.fc21(x)

        y = F.relu(self.conv12(x1))
        y = F.relu(self.conv22(y))
        y = F.relu(self.maxpool(self.conv32(y)))
        #x = torch.flatten(x, 1)
        y = y.view(-1,64*8*8)
        y = self.dropout1(y)
        y = F.relu(self.fc12(y))
        y = self.dropout2(y)
        y = self.fc22(y)

        z = F.relu(self.conv13(x1))
        z = F.relu(self.conv23(z))
        z = F.relu(self.maxpool(self.conv33(z)))
        #x = torch.flatten(x, 1)
        z = z.view(-1,64*5*5)
        z = self.dropout1(z)
        z = F.relu(self.fc13(z))
        z = self.dropout2(z)
        z = self.fc23(z)

        out = self.fc33(torch.cat((x, y, z), 0))

        output = F.log_softmax(out, dim=1)
        return output

import unittest

class TestImplementations(unittest.TestCase):

    # Dataloading tests
    def test_dataset(self):
        self.dataset_classes = ['0 - zero',
                                '1 - one',
                                '2 - two',
                                '3 - three',
                                '4 - four',
                                '5 - five',
                                '6 - six',
                                '7 - seven',
                                '8 - eight',
                                '9 - nine']
        self.assertTrue(train_dataset.classes == self.dataset_classes)
        self.assertTrue(train_dataset.train == True)

    def test_dataloader(self):
        self.assertTrue(train_dataloader.batch_size == 32)
        self.assertTrue(test_dataloader.batch_size == 32)

    def test_total_parameters(self):
        model = Net().to(device)
        #self.assertTrue(sum(p.numel() for p in model.parameters()) == 1015946)

suite = unittest.TestLoader().loadTestsFromModule(TestImplementations())
unittest.TextTestRunner().run(suite)

def train(model, device, train_loader, optimizer, epoch):
    model.train()
    for batch_idx, (data, target) in enumerate(train_loader):
        # send the image, target to the device
        data, target = data.to(device), target.to(device)
        # flush out the gradients stored in optimizer
        optimizer.zero_grad()
        # pass the image to the model and assign the output to variable named output
        output = model(data)
        # calculate the loss (use nll_loss in pytorch)
        loss = F.nll_loss(output, target)
        # do a backward pass
        loss.backward()
        # update the weights
        optimizer.step()

        if batch_idx % 100 == 0:
            print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(
                epoch, batch_idx * len(data), len(train_loader.dataset),
                100. * batch_idx / len(train_loader), loss.item()))

def test(model, device, test_loader):
    model.eval()
    test_loss = 0
    correct = 0
    with torch.no_grad():
        for data, target in test_loader:

            # send the image, target to the device
            data, target = data.to(device), target.to(device)
            # pass the image to the model and assign the output to variable named output
            output = model(data)
            test_loss += F.nll_loss(output, target, reduction='sum').item() # sum up batch loss

            pred = output.argmax(dim=1, keepdim=True)  # get the index of the max log-probability
            correct += pred.eq(target.view_as(pred)).sum().item()

    test_loss /= len(test_loader.dataset)

    print('\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format(
        test_loss, correct, len(test_loader.dataset),
        100. * correct / len(test_loader.dataset)))

model = Net().to(device)

## Define Adam Optimiser with a learning rate of 0.01
optimizer =  torch.optim.Adam(model.parameters(),lr=0.01)

start = timeit.default_timer()
for epoch in range(1, 11):
  train(model, device, train_dataloader, optimizer, epoch)
  test(model, device, test_dataloader)
stop = timeit.default_timer()
print('Total time taken: {} seconds'.format(int(stop - start)) )

Here is my full code. I couldn't understand what could possibly go wrong...
It is giving
<ipython-input-72-194680537dcc> in forward(self, x1)
     46         x = F.relu(self.conv11(x1))
     47         x = F.relu(self.conv21(x))
---> 48         x = F.relu(self.maxpool(self.conv31(x)))
     49         #x = torch.flatten(x, 1)
     50         x = x.view(-1,64*11*11)

TypeError: 'tuple' object is not callable

Error.
P.S.: Pytorch Noob here.

Output:
You have mistakenly placed a comma at the end of the line where you define self.maxpool : self.maxpool = nn.MaxPool2d(2), # Output = 64x11x11 see?
This comma makes self.maxpool a tuple instead of a torch.nn.modules.pooling.MaxPool2d. Drop the comma at the end and this error is fixed.


Instruction: CIFAR10 dataloader sampler split

Input:
i am trying to split the training data of CIFAR10 so the last 5000 of the training set is used for validation. my code
size = len(CIFAR10_training)
dataset_indices = list(range(size))
val_index = int(np.floor(0.9 * size))
train_idx, val_idx = dataset_indices[:val_index], dataset_indices[val_index:]
train_sampler = SubsetRandomSampler(train_idx)
val_sampler = SubsetRandomSampler(val_idx)

train_dataloader = torch.utils.data.DataLoader(CIFAR10_training,
                                          batch_size=config['batch_size'],
                                          shuffle=False,  sampler = train_sampler)
valid_dataloader = torch.utils.data.DataLoader(CIFAR10_training,
                                           batch_size=config['batch_size'],
                                           shuffle=False,  sampler = val_sampler)
print(len(train_dataloader.dataset),len(valid_dataloader.dataset),

but the last print statement prints 50000 and 10000. should it not be 45000 and 5000
when i print the train_idx and val_idx it prints the right values([0:44999],[45000:49999]
is there anything wrong with my code

Output:
I cannot replicate your results, when I execute your code, the print statements outputs twice the same number : the number of elements in train_CIFAR10. So I guess you made a mistake when copying your code, and valid_dataloader is actually given CIFAR10_test (or something like that) as parameter. In the following, I'm gonna assume that it's the case, and that your print outputs (50000, 50000), which is the size of the training part of Pytorch's CIFAR10 dataset.
Then it is completely expected, and no it should not output (45000, 5000). You are asking for the length of train_dataloader.dataset and valid_dataloader.dataset, i.e the length of the underlying datasets. For both your loaders, this dataset is CIFAR10_training. Therefore you will get twice the size of this dataset (i.e 50000).
You cannot ask for len(train_dataloader) either, because you that would yield the number of batches in your dataset (approximately 45000/batch_size).
If you need to know the size of your splits, then you have to compute the length of your samplers:
print(len(train_dataloader.sampler), len(valid_dataloader.sampler))

Besides this, your code is fine, you are correctly splitting your data.


Instruction: ValueError: Expected tensor to be a tensor image of size (C, H, W). Got tensor.size() = torch.Size([8, 8])

Input:
I am trying to normalize my targets (landmarks) here in which each image has 4 landmarks with x and y values for each landmark (keypoint). Here the batch size is 8.
network = Network()
network.cuda()

criterion = nn.MSELoss()
optimizer = optim.Adam(network.parameters(), lr=0.0001)

loss_min = np.inf
num_epochs = 1

start_time = time.time()
for epoch in range(1,num_epochs+1):

    loss_train = 0
    loss_test = 0
    running_loss = 0

    network.train()
    print('size of train loader is: ', len(train_loader))

    for step in range(1,len(train_loader)+1):

        batch = next(iter(train_loader))
        images, landmarks = batch['image'], batch['landmarks']
        #RuntimeError: Given groups=1, weight of size [64, 3, 7, 7], expected input[64, 600, 800, 3] to have 3 channels, but got 600 channels instead
        #using permute below to fix the above error
        images = images.permute(0,3,1,2)

        images = images.cuda()

        landmarks = landmarks.view(landmarks.size(0),-1).cuda()

        norm_image = transforms.Normalize([0.3809, 0.3810, 0.3810], [0.1127, 0.1129, 0.1130])
        for image in images:
            image = image.float()
            ##image = to_tensor(image) #TypeError: pic should be PIL Image or ndarray. Got <class 'torch.Tensor'>
            image = norm_image(image)

        norm_landmarks = transforms.Normalize(0.4949, 0.2165)
        landmarks = norm_landmarks(landmarks)
        ##landmarks = torchvision.transforms.Normalize(landmarks) #Do I need to normalize the target?

        predictions = network(images)

        # clear all the gradients before calculating them
        optimizer.zero_grad()

        print('predictions are: ', predictions.float())
        print('landmarks are: ', landmarks.float())
        # find the loss for the current step
        loss_train_step = criterion(predictions.float(), landmarks.float())

        loss_train_step = loss_train_step.to(torch.float32)
        print("loss_train_step before backward: ", loss_train_step)

        # calculate the gradients
        loss_train_step.backward()

        # update the parameters
        optimizer.step()

        print("loss_train_step after backward: ", loss_train_step)

        loss_train += loss_train_step.item()

        print("loss_train: ", loss_train)
        running_loss = loss_train/step
        print('step: ', step)
        print('running loss: ', running_loss)

        print_overwrite(step, len(train_loader), running_loss, 'train')

    network.eval()
    with torch.no_grad():

        for step in range(1,len(test_loader)+1):

            batch = next(iter(train_loader))
            images, landmarks = batch['image'], batch['landmarks']
            images = images.permute(0,3,1,2)
            images = images.cuda()
            landmarks = landmarks.view(landmarks.size(0),-1).cuda()

            predictions = network(images)

            # find the loss for the current step
            loss_test_step = criterion(predictions, landmarks)

            loss_test += loss_test_step.item()
            running_loss = loss_test/step

            print_overwrite(step, len(test_loader), running_loss, 'Validation')

    loss_train /= len(train_loader)
    loss_test /= len(test_loader)

    print('\n--------------------------------------------------')
    print('Epoch: {}  Train Loss: {:.4f} Valid Loss: {:.4f}'.format(epoch, loss_train, loss_test))
    print('--------------------------------------------------')

    if loss_test < loss_min:
        loss_min = loss_test
        torch.save(network.state_dict(), '../moth_landmarks.pth')
        print("\nMinimum Valid Loss of {:.4f} at epoch {}/{}".format(loss_min, epoch, num_epochs))
        print('Model Saved\n')

print('Training Complete')
print("Total Elapsed Time : {} s".format(time.time()-start_time))

But I get this error:
size of train loader is:  90

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-13-3e4770ad7109> in <module>
     40
     41         norm_landmarks = transforms.Normalize(0.4949, 0.2165)
---> 42         landmarks = norm_landmarks(landmarks)
     43         ##landmarks = torchvision.transforms.Normalize(landmarks) #Do I need to normalize the target?
     44

~/anaconda3/lib/python3.7/site-packages/torchvision/transforms/transforms.py in __call__(self, tensor)
    210             Tensor: Normalized Tensor image.
    211         """
--> 212         return F.normalize(tensor, self.mean, self.std, self.inplace)
    213
    214     def __repr__(self):

~/anaconda3/lib/python3.7/site-packages/torchvision/transforms/functional.py in normalize(tensor, mean, std, inplace)
    282     if tensor.ndimension() != 3:
    283         raise ValueError('Expected tensor to be a tensor image of size (C, H, W). Got tensor.size() = '
--> 284                          '{}.'.format(tensor.size()))
    285
    286     if not inplace:

ValueError: Expected tensor to be a tensor image of size (C, H, W). Got tensor.size() = torch.Size([8, 8]).

How should I normalize my landmarks?

Output:
    norm_landmarks = transforms.Normalize(0.4949, 0.2165)
    landmarks = landmarks.unsqueeze_(0)
    landmarks = norm_landmarks(landmarks)

Adding
landmarks = landmarks.unsqueeze_(0)

Fixed the problem.


Instruction: How to upgrade to pytorch-nightly in google colab?

Input:
I got into problem ,
how can i solve this?
I want to run pytorch-nightly on colab, I have all codes in pytorch-nightly version because of new packages in it,
I tried to search about this and tried this code but it is not working even after restarting runtime
from os.path import exists
cuda_output = !ldconfig -p|grep cudart.so|sed -e 's/.*\.\([0-9]*\)\.\([0-9]*\)$/cu\1\2/'
accelerator = cuda_output[0] if exists('/dev/nvidia0') else 'cpu'
!pip install torchvision_nightly
!pip install torch_nightly -f https://download.pytorch.org/whl/nightly/{accelerator}/torch_nightly.html

Help!!

Output:
You're using the wrong package name, as mentioned on the pytorch website use this:
!pip install --pre torch torchvision -f https://download.pytorch.org/whl/nightly/cu102/torch_nightly.html -U

Here, -U option is for upgrade (as there is already pytorch stable version installed on colab).


Instruction: How can I extract the weight and bias of Linear layers in PyTorch?

Input:
In model.state_dict(), model.parameters() and model.named_parameters() weights and biases of nn.Linear() modules are contained separately, e.q. fc1.weight and fc1.bias. Is there a simple pythonic way to get both of them?
Expected example looks similar to this:
layer = model['fc1']
print(layer.weight)
print(layer.bias)

Output:
You can recover the named parameters for each linear layer in your model like so:
from torch import nn

for layer in model.children():
    if isinstance(layer, nn.Linear):
        print(layer.state_dict()['weight'])
        print(layer.state_dict()['bias'])


Instruction: How do I create a shear matrix for PyTorch's F.affine_grid & F.grid_sample?

Input:
I need to create a shear matrix that is autograd compatible, works on B,C,H,W tensors, and takes input values (possibly generated randomly) for the shear values. How can I generate the shear matrix for this?
import torch
import torch.nn.functional as F
import torchvision.transforms as transforms
from PIL import Image

# Load image
def preprocess_simple(image_name, image_size):
    Loader = transforms.Compose([transforms.Resize(image_size), transforms.ToTensor()])
    image = Image.open(image_name).convert('RGB')
    return Loader(image).unsqueeze(0)

# Save image
def deprocess_simple(output_tensor, output_name):
    output_tensor.clamp_(0, 1)
    Image2PIL = transforms.ToPILImage()
    image = Image2PIL(output_tensor.squeeze(0))
    image.save(output_name)

def get_shear_mat(theta):
    ...
    return shear_mat

def shear_img(x, theta, dtype):
    shear_mat = get_shear_mat(theta)
    grid = F.affine_grid(shear_mat , x.size()).type(dtype)
    x = F.grid_sample(x, grid)
    return x

# Shear tensor
test_input = # Test image
shear_values = (3,4) # Example values
sheared_tensor = shear_img(test_input, shear_values)

Output:
Say m is the shear factor, then theta = atan(1/m) is the shear angle.
You can now pick either horizontal shear or vertical shear. Here's how you implement get_shear_mat such that you can pick horizontal shear by setting ax=0 and vertical shear by setting ax=1:
def get_shear_mat(theta, ax=0):
    assert ax in [0, 1]
    m = 1 / torch.tan(torch.tensor(theta))
    if ax == 0: # Horizontal shear
        shear_mat = torch.tensor([[1, m, 0],
                         [0, 1, 0]])
    else: # Vertical shear
        shear_mat = torch.tensor([[1, 0, 0],
                         [m, 1, 0]])
    return shear_mat

Notice that a shear mapping is just a mapping of point (x,y) in the original image to the point (x+my,y) for horizontal shear, and (x,y+mx) for vertical shear. This is exactly what we do here by defining the shear_mat as above.
An optional modification to shear_img to support the operation for a batched input in the first row. Also adding an argument - ax to shear_img to define whether we want a horizontal (ax=0) or vertical(ax=1) shear:
def shear_img(x, ax, theta, dtype):
    shear_mat = get_shear_mat(theta, ax)[None, ...].type(dtype).repeat(x.shape[0], 1, 1)
    grid = F.affine_grid(shear_mat , x.size()).type(dtype)
    x = F.grid_sample(x.type(dtype), grid)
    return x

Let's test this implementation on an image:
# Let im be a 4D tensor of shape BxCxHxW (an image or a batch of images):
dtype = torch.cuda.FloatTensor if torch.cuda.is_available() else torch.FloatTensor # Set type of data
sheared_im = shear_img(im, 0, np.pi/4, dtype) #Horizontal shear by shear angle of pi/4
plt.imshow(sheared_im.squeeze(0).permute(1,2,0)/255)
plt.show()

If im is our dancing cat with a skirt:

Then our plot will be:

If we want a vertical shear:
sheared_im = shear_img(im, 1, np.pi/4, dtype) # Vertical shear by shear angle of pi/4
plt.imshow(sheared_im.squeeze(0).permute(1, 2, 0)/255)
plt.show()

We obtain:

Hooray!


Instruction: How can I make my deeplearning chatbot use the correct path to my dataset?

Input:
I am trying to get the code from a deeplearning chatbot to work. This chatbot uses pytorch and the dataset from Cornell movie corpus. But the code can't seem to find the path to the dataset, and I don't know how to code it in. This is the source for the deeplearning chatbot code  [https://colab.research.google.com/github/pytorch/tutorials/blob/gh-pages/_downloads/chatbot_tutorial.ipynb][1]
This is as far as I've gotten with it.
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals

import torch
from torch.jit import script, trace
import torch.nn as nn
from torch import optim
import torch.nn.functional as F
import csv
import random
import re
import os
import unicodedata
import codecs
from io import open
import itertools
import math

USE_CUDA = torch.cuda.is_available()
device = torch.device("cuda" if USE_CUDA else "cpu")

corpus_name = "cornell movie-dialogs corpus"
corpus = os.path.join("data", corpus_name)

def printLines(file, n=10):
    with open(file, 'rb') as datafile:
        lines = datafile.readlines()
    for line in lines[:n]:
        print(line)

printLines(os.path.join(corpus, "movie_lines.txt"))

And this is my error log
D:\Documents\Python\python pycharm files\pythonProject4\3.9 Chatbot.py:26: SyntaxWarning: 'str' object is not callable; perhaps you missed a comma?
  corpus = "D:\Documents\Python\intents\cornell_movie_dialogs_corpus.zip\cornell movie-dialogs corpus\\"("data", corpus_name)
Traceback (most recent call last):
  File "D:\Documents\Python\python pycharm files\pythonProject4\3.9 Chatbot.py", line 26, in <module>
    corpus = "D:\Documents\Python\intents\cornell_movie_dialogs_corpus.zip\cornell movie-dialogs corpus\\"("data", corpus_name)
TypeError: 'str' object is not callable

I hope there is a solution that doesn't alter the source code too much, but any tips or help are welcome.

Output: I solved my problem by removing anything but the cornell movie-dialogs corpus map. And then replacing the word "data" in line 26 with the directory of that map. This fixed it for me


Instruction: Low accuracy binary classification with Pytorch

Input:
In practicing deep learning for binary classification with Pytorch on Breast-Cancer-Wisconsin-Diagnostic-DataSet.
I've tried different approaches, and the best I can get as below, the accuracy is still low at 61%.
What's the way to improve the accuracy?
Thank you.
import pandas as pd
import io

dataset = pd.read_excel(base_dir + "Breast-Cancer-Wisconsin-Diagnostic.xlsx")

number_of_columns = dataset.shape[1]

# training and testing split of 70:30
dataset['diagnosis'] = pd.Categorical(dataset['diagnosis']).codes
dataset = dataset.sample(frac=1, random_state=1234)
train_input = dataset.values[:398, :number_of_columns-1]
train_target = dataset.values[:398, number_of_columns-1]
test_input = dataset.values[398:, :number_of_columns-1]
test_target = dataset.values[398:, number_of_columns-1]

import torch
torch.manual_seed(1234)
hidden_units = 5
net = torch.nn.Sequential(
torch.nn.Linear(number_of_columns-1, hidden_units),
torch.nn.ReLU(),
torch.nn.Linear(hidden_units, 2))

# choose optimizer and loss function
criterion = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(net.parameters(), lr=0.1,momentum=0.9)

# train
epochs = 50
for epoch in range(epochs):
    inputs = torch.autograd.Variable(torch.Tensor(train_input).float())
    targets = torch.autograd.Variable(torch.Tensor(train_target).long())
    optimizer.zero_grad()
    out = net(inputs)
    loss = criterion(out, targets)
    loss.backward()
    optimizer.step()
    if epoch == 0 or (epoch + 1) % 10 == 0:
        print('Epoch %d Loss: %.4f' % (epoch + 1, loss.item()))

# Epoch 1 Loss: 412063.1250
# Epoch 10 Loss: 0.6628
# Epoch 20 Loss: 0.6639
# Epoch 30 Loss: 0.6592
# Epoch 40 Loss: 0.6587
# Epoch 50 Loss: 0.6588

import numpy as np
inputs = torch.autograd.Variable(torch.Tensor(test_input).float())
targets = torch.autograd.Variable(torch.Tensor(test_target).long())
optimizer.zero_grad()
out = net(inputs)
_, predicted = torch.max(out.data, 1)
error_count = test_target.size - np.count_nonzero((targets == predicted).numpy())

print('Errors: %d; Accuracy: %d%%' % (error_count, 100 * torch.sum(targets == predicted) // test_target.size))

# Errors: 65; Accuracy: 61%

Output:
Features Representing samples are in different range. So, First thing you should do is to normalize the data.
You should plot the loss and acc over the training epochs for training and validation/test dataset to understand whether the model overfits on training data or underfit.
Furthermore, you can try with more complex (deeper) model. And since your training dataset has few number of samples, you can consider augmentation and transfer learning as well if possible.


Instruction: PyTorch: defined layer was not involved in the forward propagation but influenced the loss value

Input:
Recently, I met a confusing phenomenon when I used Pytorch to do simple experiments on logistic regression.
The question is when I fixed the random seed like this:
def set_seed(seed, cuda=True):
    np.random.seed(seed)
    torch.manual_seed(seed)
    if cuda:
        torch.cuda.manual_seed(seed)

and defined the following model with 2 layers:
class net(nn.Module):
    def __init__(self):
        super(net, self).__init__()
        self.hidden = nn.Linear(784, 100)
        self.output = nn.Linear(100, 10)

    def forward(self, x):
        x = self.hidden(x)
        x = self.output(x)
        return x

trained the network with:
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(net.parameters(), lr=0.1)

The orignal loss value was 0.6422, which is reproducible.
However, when I added an additional layer that was not involoved in the forward process like this:
class net(nn.Module):
    def __init__(self):
        super(net, self).__init__()
        self.hidden = nn.Linear(784, 100)
        self.output = nn.Linear(100, 10)
        self.add = nn.Linear(10,10)

    def forward(self, x):
        x = self.hidden(x)
        x = self.output(x)
        return x

The original loss value changed to 0.7431, which was not equal to the previous one and the model performance dropped simultaneously.
I really wonder the reason to this. Thank you!

Output:
This is completely expected if there are other sources of randomness (something that consumes the RNG) before computing the loss. As you didn't provide a Minimal, Reproducible Example, I'd guess that you're using a DataLoader with shuffle=True. In this case, even though you do not use the self.add layer, when you initialize it, it consumes the RNG; therefore leading to a different order to the samples. If the randomness is coming from a DataLoader with shuffle=True, you can control that by providing a different RNG to the DataLoader. Something like this:
import numpy as np

import torch
from torch import nn

import torchvision
from torchvision.transforms import ToTensor

def set_seed(seed, cuda=True):
    np.random.seed(seed)
    torch.manual_seed(seed)
    if cuda:
        torch.cuda.manual_seed(seed)

class net(nn.Module):
    def __init__(self):
        super(net, self).__init__()
        self.hidden = nn.Linear(784, 100)
        self.output = nn.Linear(100, 10)
        # self.add = nn.Linear(10, 10)  # try with and without
    def forward(self, x):
        x = self.hidden(x)
        x = self.output(x)
        return x

set_seed(0)

m = net()

bs = 4
ds = torchvision.datasets.MNIST(root=".", train=True, transform=ToTensor(), download=True)
rng_dl = torch.Generator()
dl = torch.utils.data.DataLoader(ds, batch_size=bs, shuffle=True, num_workers=0, generator=rng_dl)

criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(m.parameters(), lr=0.1)

for x, y in dl:
    y_hat = m(x.view(bs, -1))
    l = criterion(y_hat, y)
    print(l)
    exit()

Keep in mind that it could be several other things, such as data augmentation and other calls to functions that rely on random ops. If you can provide an MRE, I could try and give a more specific answer.


Instruction: torchvision.transforms.Normalize() slows down learning when adding to torchvision,transforms.Compose()

Input:
when i use
train_transforms = torchvision.transforms.Compose([
  torchvision.transforms.ToTensor(),
  torchvision.transforms.Normalize((0.1307,), (0.3081,))
])

for loading MNIST dataset, it slows down learning even with mean = 0 and std = 1.

Output:
The transformations are performed on CPU, and it doesn't matter if the mean/std are all zeros (BTW, don't set std to 0). To speed up the transform you have two options:

If you don't have any data augmentations in your flow, just transform the data and save it as normalized tensors (pickled or something).
You can also use torch.utils.data.DataLoader with some arguments: for example num_workers specifies how many CPU processes to use to transform the data. THere is also pin_memory which will speed up the whole thing if you are using CUDA.


Instruction: Space efficient way to store and read massive 3d dataset?

Input:
I am trying to train a neural network on sequential data. My dataset will consist of 3.6 million training examples. Each example will be a 30 x 32 ndarray (32 features observed over 30 days).
My question is what is the most space-efficient way to write and read this data?
Essentially it will have shape (3.6m, 30, 32) and np.save() seems convenient but I can't hold this whole thing in memory so I can't really save it using np.save() (or load it back using np.load()). CSV also won't work because my data has 3 dimensions.
My plan to create the thing is to process entries in batches and append them to some file so that I can keep memory free as I go.
Eventually, I am going to use the data file as an input for a PyTorch IterableDataset so it must be something that can be loaded one line at a time (like a .txt file, but I'm hoping there is some better way to save this data that is more true to its tabular, 3-dimensional nature). Any ideas are appreciated!

Output:
Since you are planning on using an iterable dataset you shouldn't need random access (IterableDataset doesn't support shuffle samplers). In that case, why not just write everything to a binary file and iterate over that? I find in practice this often is much faster than alternative solutions. This should be much faster than saving as a text file since you avoid the overhead of converting text to numbers.
An example implementation may look something like the following. First we could build a binary file as follows (containing random data as a placeholder)
import numpy as np
from tqdm import tqdm

filename = 'data.bin'
num_samples = 3600000
rows, cols = 30, 32
dtype = np.float32

# format: <num_samples> <rows> <cols> <sample0> <sample1>...
with open(filename, 'wb') as fout:
    # write a header that contains the total number of samples and the rows and columns per sample
    fout.write(np.array((num_samples, rows, cols), dtype=np.int32).tobytes())
    for i in tqdm(range(num_samples)):
        # random placeholder
        sample = np.random.randn(rows, cols).astype(dtype)
        # write data to file
        fout.write(sample.tobytes())

Then we could define an IterableDataset as follows
import numpy as np
from torch.utils.data import IterableDataset, DataLoader
from tqdm import tqdm

def binary_reader(filename, start=None, end=None, dtype=np.float32):
    itemsize = np.dtype(dtype).itemsize
    with open(filename, 'rb') as fin:
        num_samples, rows, cols = np.frombuffer(fin.read(3 * np.dtype(np.int32).itemsize), dtype=np.int32)
        start = start if start is not None else 0
        end = end if end is not None else num_samples
        blocksize = itemsize * rows * cols
        start_offset = start * blocksize
        fin.seek(start_offset, 1)
        for _ in range(start, end):
            yield np.frombuffer(fin.read(blocksize), dtype=dtype).reshape(rows, cols).copy()

class BinaryIterableDataset(IterableDataset):
    def __init__(self, filename, start=None, end=None, dtype=np.float32):
        super().__init__()
        self.filename = filename
        self.start = start
        self.end = end
        self.dtype = dtype

    def __iter__(self):
        return binary_reader(self.filename, self.start, self.end, self.dtype)

From a quick test of this dataset on my system (which uses SSD storage) I find I am able to iterate over all 3.6 million samples in about 10 seconds
dataset = BinaryIterableDataset('data.bin')
for sample in tqdm(dataset):
    pass

3600000it [00:09, 374026.17it/s]

Using a DataLoader with batch_size=256 it takes me about 20 seconds to iterate over the whole dataset (converting to tensors and creating batches has some overhead). For this dataset I found that the overhead of transferring data to and from shared memory when using parallel loading is actually quite a bit slower than just using 0 workers. Therefore I recommend using num_workers=0. As with any iterable dataset you would need to add extra logic to support num_workers > 1, though I'm not sure it would be worth it in this case.
loader = DataLoader(dataset, batch_size=256, num_workers=0)
for batch in tqdm(loader):
    # batch is a tensor of shape (256, 30, 32)
    pass

14063it [00:19, 710.49it/s]

Note that the data.bin file would not be portable across systems that use different byte order. Though modifications could be made to support that.


Instruction: How do I create a scale matrix for rescaling a PyTorch tensor, and then how do I use it?

Input:
I need to create a scale matrix that is autograd compatible, works on B,C,H,W tensors, and takes input values (possibly generated randomly) for controlling the scaling. How can I generate and use a scale matrix for this?
import torch
import torch.nn.functional as F
import torchvision.transforms as transforms
from PIL import Image

# Load image
def preprocess_simple(image_name, image_size):
    Loader = transforms.Compose([transforms.Resize(image_size), transforms.ToTensor()])
    image = Image.open(image_name).convert('RGB')
    return Loader(image).unsqueeze(0)

# Save image
def deprocess_simple(output_tensor, output_name):
    output_tensor.clamp_(0, 1)
    Image2PIL = transforms.ToPILImage()
    image = Image2PIL(output_tensor.squeeze(0))
    image.save(output_name)

def get_scale_mat(theta):
    ...
    return scale_mat

def scale_img(x, theta, dtype):
    scale_mat = get_scale_mat(theta)

    # Can F.affine_grid & F.grid_sample be used with a scale matrix?
    grid = F.affine_grid(scale_mat , x.size()).type(dtype)
    x = F.grid_sample(x, grid)
    return x

# Shear tensor
test_input = # Test image
scale = 5 # Example value
scaled_tensor = scale_img(test_input, scale)

Output:
This is how you create and use a 3x2 scale matrix with F.affine_grid and F.grid_sample:
def get_scale_mat(m, device, dtype):
    scale_mat = torch.tensor([[m, 0., 0.],
                              [0., m, 0.]])
    return scale_mat

def scale_tensor(x, scale):
    assert scale > 0
    scale_matrix = get_scale_mat(scale, x.device, x.dtype)[None, ...].repeat(x.shape[0],1,1)
    grid = F.affine_grid(scale_matrix, x.size())
    x = F.grid_sample(x, grid)
    return x


Instruction: How can I unroll a PyTorch Tensor?

Input:
I have a tensor:
t1 = torch.randn(564, 400)

I want to unroll it to a 1-d tensor that's 225600 long.
How can I do this?

Output:
Note the difference between view and reshape as suggested by Kris -
From reshape's docstring:

When possible, the returned tensor will be a view
of input. Otherwise, it will be a copy. Contiguous inputs and inputs with compatible strides can be reshaped without copying...

So in case your tensor is not contiguous calling reshape should handle what one would have had to handle had one used view instead; That is, call t1.contiguous().view(...) to handle non-contiguous tensors.
Also, one could use faltten: t1 = t1.flatten() as an equivalent of view(-1), which is more readable.


Instruction: How could I get predictions for my PyTorch Image Classifier?

Input:
I have implemented this code for My Keras model that worked accurately but now I wanna implement this for my PyTorch Model, but I am unable to configure it for making predictions, Below is my full code, I need help with the classify function:
# importing libraries

import tkinter as tk
from tkinter import *
# import PIL
from tkinter import filedialog

import numpy
from PIL import Image, ImageTk
import torch

# importing model

model=torch.load('corelK_model_0.pt')

classes = {
    0:'africa',
    1: 'beach',
    2: 'tallbuilding',
    3: 'buses',
    4: 'dinosaurs',
    5: 'elephants',
    6: 'Roses',
    7: 'horses',
    8: 'mountains',
    9: 'food'
}

def upload_image():
    file_path = filedialog.askopenfilename()
    uploaded = Image.open(file_path)
    uploaded.thumbnail(((top.winfo_width() / 2.25, (top.winfo_height() / 2.25))))
    im = ImageTk.PhotoImage(uploaded)
    sign_image.configure(image=im)
    sign_image.image = im
    label.configure(text=' ')
    show_classify_button(file_path)

def show_classify_button(file_path):
    classify_btn = Button(top, text="Classify Image", command=lambda: classify(file_path), padx=10, pady=5)
    classify_btn.configure(background="#364156", foreground="white", font=('arial', 10, 'bold'))
    classify_btn.place(relx=0.79, rely=0.46)

def classify(file_path):
    image = Image.open(file_path)
    image = image.resize((32, 32))
    image = numpy.expand_dims(image, axis=0)
    image = numpy.array(image)
    pred = model.predict_classes([image])[0]
    sign = classes[pred]

    print(sign)
    label.configure(foreground='#011638', text=sign)

# initialize GUI
top = tk.Tk()  # calling the constructor or creating the object of tk class
top.geometry('800x600')  # set height and width
top.title("Image Classification CIFAR10")
top.configure(background="#CDCDCD")

# set Heading

heading = Label(top, text="Image Classifier", pady=20, font=('arial', 20, 'bold'))
heading.configure(background="#CDCDCD", foreground='#364156')
heading.pack()

upload = Button(top, text="Upload an image", command=upload_image, padx=10, pady=5)
heading.configure(background="#364156", foreground='white', font=('arial', 10, 'bold'))
upload.pack(side=BOTTOM, pady=50)

# upload image
sign_image = Label(top)
sign_image.pack(side=BOTTOM, expand=True)

# predicted class

label = Label(top, background="#CDCDCD", font=('arial', 15, 'bold'))
label.pack(side=BOTTOM, expand=True)

top.mainloop()

The model I have used is VGG16 after transfer learning, I have saved it using the torch.save .

Output:
Can you try the following?

After loading the model, set it to evaluation mode using this statement:

model=torch.load('corelK_model_0.pt')
model.eval()

You're not applying the same image transformations (on the test image) that were used while model training. Your classify function should look like this:

def classify(file_path):
    image = Image.open(file_path)
    image = image.resize((32, 32))
    image = numpy.expand_dims(image, axis=0)
    image = numpy.array(image)

    # Start of transformations
    # .......................
    # End of transformations

    pred = model.predict_classes([image])[0]
    sign = classes[pred]

    print(sign)
    label.configure(foreground='#011638', text=sign)


Instruction: When I run train.py with YOLACT, I get the error KeyError: 0

Input:
I'm new to machine learning and program.
Now I'm trying to develop YOLACT AI using my own data.
However, when I run train.py, I get the following error and cannot learn.
What can I do to overcome this error?`
(yolact) tmori@tmori-Lenovo-Legion-Y740-15IRHg:~/yolact$ python train.py --config=can_config  --save_interval=2000

    loading annotations into memory...
    Done (t=0.00s)
    creating index...
    index created!
    loading annotations into memory...
    Done (t=0.00s)
    creating index...
    index created!
    Initializing weights...
    Begin training!

    /home/tmori/yolact/utils/augmentations.py:309: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
      mode = random.choice(self.sample_options)
    /home/tmori/yolact/utils/augmentations.py:309: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
      mode = random.choice(self.sample_options)
    /home/tmori/yolact/utils/augmentations.py:309: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
      mode = random.choice(self.sample_options)
    /home/tmori/yolact/utils/augmentations.py:309: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
      mode = random.choice(self.sample_options)
    [  0]       0 || B: 4.840 | C: 16.249 | M: 4.682 | S: 2.749 | T: 28.521 || ETA: 9:18:44 || timer: 3.352
    /home/tmori/yolact/utils/augmentations.py:309: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
      mode = random.choice(self.sample_options)
    /home/tmori/yolact/utils/augmentations.py:309: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
      mode = random.choice(self.sample_options)
    /home/tmori/yolact/utils/augmentations.py:309: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
      mode = random.choice(self.sample_options)
    /home/tmori/yolact/utils/augmentations.py:309: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
      mode = random.choice(self.sample_options)
    [  1]      10 || B: 4.535 | C: 9.228 | M: 4.379 | S: 1.867 | T: 20.008 || ETA: 3:25:24 || timer: 0.864
    /home/tmori/yolact/utils/augmentations.py:309: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
      mode = random.choice(self.sample_options)
    /home/tmori/yolact/utils/augmentations.py:309: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
      mode = random.choice(self.sample_options)
    /home/tmori/yolact/utils/augmentations.py:309: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
      mode = random.choice(self.sample_options)
    /home/tmori/yolact/utils/augmentations.py:309: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
      mode = random.choice(self.sample_options)

    Computing validation mAP (this may take a while)...

    Traceback (most recent call last):
      File "train.py", line 504, in <module>
        train()
      File "train.py", line 371, in train
        compute_validation_map(epoch, iteration, yolact_net, val_dataset, log if args.log else None)
      File "train.py", line 492, in compute_validation_map
        val_info = eval_script.evaluate(yolact_net, dataset, train_mode=True)
      File "/home/tmori/yolact/eval.py", line 956, in evaluate
        prep_metrics(ap_data, preds, img, gt, gt_masks, h, w, num_crowd, dataset.ids[image_idx], detections)
      File "/home/tmori/yolact/eval.py", line 427, in prep_metrics
        detections.add_bbox(image_id, classes[i], boxes[i,:],   box_scores[i])
      File "/home/tmori/yolact/eval.py", line 315, in add_bbox
        'category_id': get_coco_cat(int(category_id)),
      File "/home/tmori/yolact/eval.py", line 293, in get_coco_cat
        return coco_cats[transformed_cat_id]
    KeyError: 0

I'm trying to develop an AI that finds cans and segments them.
First I annotated only one item of "can" with labelme, and then created a COCO format json file with labelme2coco.py.
After that, I modified config.py according to "Custom Datasets" on YOLACT's GitHub and ran train.py.
My development environment is as follows.
OS:Ubuntu20.04LTS
Anaconda:4.8.3
Python: 3.6.12
Pytorch: 1.4.0
CUDA Toolkit: 10.1
cuDNN: 7.6.5

Output:
Your class id in annotations.json should start from 1 not 0. If they are starting from 0, try this in config.py in your "my_custom_dataset" in label map add this
'label_map': { 0:  1, 1:  2, 2:  3... and so on}

In this case there 3 classes!
Also in yolact_base_config in same script num_classes should be 1 greater than your number of classes for example in this case it would be 4.


Instruction: How do I use two loss for two dataset in one pytorch nn?

Input:
I am quite new to pytorch and deep learning. Here is my question. I have two different datasets with the same feature domain sharing one neural network for a regression problem. The input is the features and the output is the target value. The first dataset uses a normal loss while the second dataset, I am trying to create a new loss for it.
I have searched multi-loss problems, people usually have two loss summed up for the backward process. But I want to use the loss in turn. (When I train the first dataset, the nn uses the first loss and when I train the second dataset, the nn uses the other loss)
Is this possible to do? Appreciate if anyone has some idea.

Output:
The loss function does not necessarily have to do with network topology. You can use the corresponding loss with each dataset you use, e.g.
if first_task:
    dataloader = torch.utils.data.DataLoader(first_dataset)
    loss_fn = first_loss_fn
else:
    dataloader = torch.utils.data.Dataloader(second_dataset)
    loss_fn = second_loss_fn

# The pytorch training loop, very roughly
for batch in dataloader:
   x, y = batch
   optimizer.zero_grad()
   loss = loss_fn(network.forward(x), y) # calls the corresponding loss function
   loss.backward()
   optimizer.step()

You can do this for the two datasets sequentially (meaning you interleave by epochs):
for batch in dataloader_1:
    ...
    loss = first_loss_fn(...)

for batch in dataloader_2:
    ...
    loss = second_loss_fn(...)

or better
dataset = torch.utils.data.ChainDataset([first_dataset, second_dataset])
dataloader = torch.utils.data.DataLoader(dataset)

You can also do simultaneously (interleave by examples). The standard way I think would be to use torch.utils.data.ConcatDataset
dataset = torch.utils.data.ConcatDataset([first_dataset, second_dataset])
dataloader = torch.utils.data.DataLoader(dataset)

Note that here you need each sample to store information about the dataset it comes from so you can determine which cost to apply.
A simpler way would be to interleave by batches (then you apply the same cost to the entire batch). For this case one way proposed here is to use separate dataloaders (this way you get flexibility on how often to sample each of them).


Instruction: Deep learning chatbot specific Index error list index out of range

Input:
I am trying to follow a tutorial on how to make a deeplearning chatbot with pytorch. However, this code is quite complex for me and it has stopped with a "IndexError: list index out of range". I looked the error up and get the gist of what it usually means, but seeing as this code is very complex for me I can't figure out how to solve the error.
this is the source tutorial: [https://colab.research.google.com/github/pytorch/tutorials/blob/gh-pages/_downloads/chatbot_tutorial.ipynb#scrollTo=LTzdbPF-OBL9][1]
Line 198 seems to be causing the error
return len(p[0].split(' ')) < MAX_LENGTH and len(p[1].split(' ')) < MAX_LENGTH

This is the error log
Start preparing training data ...
Reading lines...
Traceback (most recent call last):
  File "D:\Documents\Python\python pycharm files\pythonProject4\3.9 Chatbot.py", line 221, in <module>
    voc, pairs = loadPrepareData(corpus, corpus_name, datafile, save_dir)
  File "D:\Documents\Python\python pycharm files\pythonProject4\3.9 Chatbot.py", line 209, in loadPrepareData
    pairs = filterPairs(pairs)
  File "D:\Documents\Python\python pycharm files\pythonProject4\3.9 Chatbot.py", line 202, in filterPairs
    return [pair for pair in pairs if filterPair(pair)]
  File "D:\Documents\Python\python pycharm files\pythonProject4\3.9 Chatbot.py", line 202, in <listcomp>
    return [pair for pair in pairs if filterPair(pair)]
  File "D:\Documents\Python\python pycharm files\pythonProject4\3.9 Chatbot.py", line 198, in filterPair
    return len(p[0].split(' ')) < MAX_LENGTH and len(p[1].split(' ')) < MAX_LENGTH
IndexError: list index out of range
Read 442563 sentence pairs

Process finished with exit code 1

And this is my code copied from my pycharm up to the block with the error. Seeing as its a huge code I could not copy the entire code. The rest of the code can be found in the github source link above.
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals

import torch
from torch.jit import script, trace
import torch.nn as nn
from torch import optim
import torch.nn.functional as F
import csv
import random
import re
import os
import unicodedata
import codecs
from io import open
import itertools
import math

USE_CUDA = torch.cuda.is_available()
device = torch.device("cuda" if USE_CUDA else "cpu")

corpus_name = "cornell movie-dialogs corpus"
corpus = os.path.join("D:\Documents\Python\intents", corpus_name)

def printLines(file, n=10):
    with open(file, 'rb') as datafile:
        lines = datafile.readlines()
    for line in lines[:n]:
        print(line)

printLines(os.path.join(corpus, "movie_lines.txt"))

# Splits each line of the file into a dictionary of fields
def loadLines(fileName, fields):
    lines = {}
    with open(fileName, 'r', encoding='iso-8859-1') as f:
        for line in f:
            values = line.split(" +++$+++ ")
            # Extract fields
            lineObj = {}
            for i, field in enumerate(fields):
                lineObj[field] = values[i]
            lines[lineObj['lineID']] = lineObj
    return lines

# Groups fields of lines from `loadLines` into conversations based on *movie_conversations.txt*
def loadConversations(fileName, lines, fields):
    conversations = []
    with open(fileName, 'r', encoding='iso-8859-1') as f:
        for line in f:
            values = line.split(" +++$+++ ")
            # Extract fields
            convObj = {}
            for i, field in enumerate(fields):
                convObj[field] = values[i]
            # Convert string to list (convObj["utteranceIDs"] == "['L598485', 'L598486', ...]")
            lineIds = eval(convObj["utteranceIDs"])
            # Reassemble lines
            convObj["lines"] = []
            for lineId in lineIds:
                convObj["lines"].append(lines[lineId])
            conversations.append(convObj)
    return conversations

# Extracts pairs of sentences from conversations
def extractSentencePairs(conversations):
    qa_pairs = []
    for conversation in conversations:
        # Iterate over all the lines of the conversation
        for i in range(len(conversation["lines"]) - 1):  # We ignore the last line (no answer for it)
            inputLine = conversation["lines"][i]["text"].strip()
            targetLine = conversation["lines"][i+1]["text"].strip()
            # Filter wrong samples (if one of the lists is empty)
            if inputLine and targetLine:
                qa_pairs.append([inputLine, targetLine])
    return qa_pairs

# Define path to new file
datafile = os.path.join(corpus, "formatted_movie_lines.txt")

delimiter = '\t'
# Unescape the delimiter
delimiter = str(codecs.decode(delimiter, "unicode_escape"))

# Initialize lines dict, conversations list, and field ids
lines = {}
conversations = []
MOVIE_LINES_FIELDS = ["lineID", "characterID", "movieID", "character", "text"]
MOVIE_CONVERSATIONS_FIELDS = ["character1ID", "character2ID", "movieID", "utteranceIDs"]

# Load lines and process conversations
print("\nProcessing corpus...")
lines = loadLines(os.path.join(corpus, "movie_lines.txt"), MOVIE_LINES_FIELDS)
print("\nLoading conversations...")
conversations = loadConversations(os.path.join(corpus, "movie_conversations.txt"),
                                  lines, MOVIE_CONVERSATIONS_FIELDS)

# Write new csv file
print("\nWriting newly formatted file...")
with open(datafile, 'w', encoding='utf-8') as outputfile:
    writer = csv.writer(outputfile, delimiter=delimiter)
    for pair in extractSentencePairs(conversations):
        writer.writerow(pair)

# Print a sample of lines
print("\nSample lines from file:")
printLines(datafile)

# Default word tokens
PAD_token = 0  # Used for padding short sentences
SOS_token = 1  # Start-of-sentence token
EOS_token = 2  # End-of-sentence token

class Voc:
    def __init__(self, name):
        self.name = name
        self.trimmed = False
        self.word2index = {}
        self.word2count = {}
        self.index2word = {PAD_token: "PAD", SOS_token: "SOS", EOS_token: "EOS"}
        self.num_words = 3  # Count SOS, EOS, PAD

    def addSentence(self, sentence):
        for word in sentence.split(' '):
            self.addWord(word)

    def addWord(self, word):
        if word not in self.word2index:
            self.word2index[word] = self.num_words
            self.word2count[word] = 1
            self.index2word[self.num_words] = word
            self.num_words += 1
        else:
            self.word2count[word] += 1

    # Remove words below a certain count threshold
    def trim(self, min_count):
        if self.trimmed:
            return
        self.trimmed = True

        keep_words = []

        for k, v in self.word2count.items():
            if v >= min_count:
                keep_words.append(k)

        print('keep_words {} / {} = {:.4f}'.format(
            len(keep_words), len(self.word2index), len(keep_words) / len(self.word2index)
        ))

        # Reinitialize dictionaries
        self.word2index = {}
        self.word2count = {}
        self.index2word = {PAD_token: "PAD", SOS_token: "SOS", EOS_token: "EOS"}
        self.num_words = 3 # Count default tokens

        for word in keep_words:
            self.addWord(word)

MAX_LENGTH = 10  # Maximum sentence length to consider

# Turn a Unicode string to plain ASCII, thanks to
# http://stackoverflow.com/a/518232/2809427
def unicodeToAscii(s):
    return ''.join(
        c for c in unicodedata.normalize('NFD', s)
        if unicodedata.category(c) != 'Mn'
    )

# Lowercase, trim, and remove non-letter characters
def normalizeString(s):
    s = unicodeToAscii(s.lower().strip())
    s = re.sub(r"([.!?])", r" \1", s)
    s = re.sub(r"[^a-zA-Z.!?]+", r" ", s)
    s = re.sub(r"\s+", r" ", s).strip()
    return s

# Read query/response pairs and return a voc object
def readVocs(datafile, corpus_name):
    print("Reading lines...")
    # Read the file and split into lines
    lines = open(datafile, encoding='utf-8').\
        read().strip().split('\n')
    # Split every line into pairs and normalize
    pairs = [[normalizeString(s) for s in l.split('\t')] for l in lines]
    voc = Voc(corpus_name)
    return voc, pairs

# Returns True iff both sentences in a pair 'p' are under the MAX_LENGTH threshold
def filterPair(p):
    # Input sequences need to preserve the last word for EOS token
    return len(p[0].split(' ')) < MAX_LENGTH and len(p[1].split(' ')) < MAX_LENGTH

# Filter pairs using filterPair condition
def filterPairs(pairs):
    return [pair for pair in pairs if filterPair(pair)]

# Using the functions defined above, return a populated voc object and pairs list
def loadPrepareData(corpus, corpus_name, datafile, save_dir):
    print("Start preparing training data ...")
    voc, pairs = readVocs(datafile, corpus_name)
    print("Read {!s} sentence pairs".format(len(pairs)))
    pairs = filterPairs(pairs)
    print("Trimmed to {!s} sentence pairs".format(len(pairs)))
    print("Counting words...")
    for pair in pairs:
        voc.addSentence(pair[0])
        voc.addSentence(pair[1])
    print("Counted words:", voc.num_words)
    return voc, pairs

# Load/Assemble voc and pairs
save_dir = os.path.join("data", "save")
voc, pairs = loadPrepareData(corpus, corpus_name, datafile, save_dir)
# Print some pairs to validate
print("\npairs:")
for pair in pairs[:10]:
    print(pair)

MIN_COUNT = 3    # Minimum word count threshold for trimming

I really hope someone can help me fix this problem and help me understand why it happens.

Output:
In the end I changed
return len(p[0].split(' ')) < MAX_LENGTH and len(p[1].split(' ')) < MAX_LENGTH

to
try:
    return len(p[0].split(' ')) < MAX_LENGTH and len(p[1].split(' ')) < MAX_LENGTH
except:
    return False

And now the code seems to be working.


Instruction: torch.cat along negative dimension

Input:
In the following,
 x_6 = torch.cat((x_1, x_2_1, x_3_1, x_5_1), dim=-3)
 Sizes of tensors x_1, x_2_1, x_3_1, x_5_1 are
 torch.Size([1, 256, 7, 7])
 torch.Size([1, 256, 7, 7])
 torch.Size([1, 256, 7, 7])
 torch.Size([1, 256, 7, 7]) respectively.

 The size of x_6 turns out to be torch.Size([1, 1024, 7, 7])

I couldn't understand & visualise this concatenation along a negative dimension(-3 in this case).
What exactly is happening here?
How does the same go if dim = 3?
Is there any constraint on dim for a given set of tensors?

Output:
The answer by danin is not completely correct, actually wrong when looked from the perspective of tensor algebra, since the answer indicates that the problem has to do with accessing or indexing a Python list. It isn't.
The -3 means that we concatenate the tensors along the 2nd dimension. (you could've very well used 1 instead of the confusing -3).

From taking a closer look at the tensor shapes, it seems that they represent (b, c, h, w) where b stands for batch_size, c stands for number of channels, h stands for height and w stands for width.
This is usually the case, somewhere at the final stages of encoding (possibly) images in a deep neural network and we arrive at these feature maps.
The torch.cat() operation with dim=-3 is meant to say that we concatenate these 4 tensors along the dimension of channels c (see above).
4 * 256 => 1024
Hence, the resultant tensor ends up with a shape torch.Size([1, 1024, 7, 7]).

Notes: It is hard to visualize a 4 dimensional space since we humans live in an inherently 3D world. Nevertheless, here are some answers that I wrote a while ago which will help to get some mental picture.

How to understand the term tensor in TensorFlow?

Very Basic Numpy array dimension visualization


Instruction: How to view all possible distributions that can be used in PyTorch kl_divergence function?

Input: I would like to know what are all of the possible combinations of distributions I can plug into PyTorch's kl_divergence function out of the box. How can I view all of the possibilities?

Output:
Here is a function to display all of the registered combinations:
from torch.distributions.kl import _KL_REGISTRY

def view_kl_options():
    """
    Displays all combinations of distributions that can be used in
    torch's kl_divergence function. Iterates through the registry
    and prints out the registered name combos.
    """
    names = [(k[0].__name__, k[1].__name__) for k in _KL_REGISTRY.keys()]
    max_name_len = max([len(t[0]) for t in names])
    for arg1, arg2 in sorted(names):
        print(f"  {arg1:>{max_name_len}} || {arg2}")

Calling view_kl_options() will give you an output like
                  Bernoulli || Bernoulli
                  Bernoulli || Poisson
                       Beta || Beta
                       Beta || ContinuousBernoulli
                       Beta || Exponential
                       Beta || Gamma
                       Beta || Normal
                       Beta || Pareto
                       Beta || Uniform
                   Binomial || Binomial
                Categorical || Categorical
                     Cauchy || Cauchy
        ContinuousBernoulli || ContinuousBernoulli
        ContinuousBernoulli || Exponential
        ContinuousBernoulli || Normal
        ContinuousBernoulli || Pareto
        ContinuousBernoulli || Uniform
                  Dirichlet || Dirichlet
                Exponential || Beta
                Exponential || ContinuousBernoulli
                Exponential || Exponential
                Exponential || Gamma
                Exponential || Gumbel
                Exponential || Normal
                Exponential || Pareto
                Exponential || Uniform
          ExponentialFamily || ExponentialFamily
                      Gamma || Beta
                      Gamma || ContinuousBernoulli
                      Gamma || Exponential
                      Gamma || Gamma
                      Gamma || Gumbel
                      Gamma || Normal
                      Gamma || Pareto
                      Gamma || Uniform
                  Geometric || Geometric
                     Gumbel || Beta
                     Gumbel || ContinuousBernoulli
                     Gumbel || Exponential
                     Gumbel || Gamma
                     Gumbel || Gumbel
                     Gumbel || Normal
                     Gumbel || Pareto
                     Gumbel || Uniform
                 HalfNormal || HalfNormal
                Independent || Independent
                    Laplace || Beta
                    Laplace || ContinuousBernoulli
                    Laplace || Exponential
                    Laplace || Gamma
                    Laplace || Laplace
                    Laplace || Normal
                    Laplace || Pareto
                    Laplace || Uniform
  LowRankMultivariateNormal || LowRankMultivariateNormal
  LowRankMultivariateNormal || MultivariateNormal
         MultivariateNormal || LowRankMultivariateNormal
         MultivariateNormal || MultivariateNormal
                     Normal || Beta
                     Normal || ContinuousBernoulli
                     Normal || Exponential
                     Normal || Gamma
                     Normal || Gumbel
                     Normal || Normal
                     Normal || Pareto
                     Normal || Uniform
          OneHotCategorical || OneHotCategorical
                     Pareto || Beta
                     Pareto || ContinuousBernoulli
                     Pareto || Exponential
                     Pareto || Gamma
                     Pareto || Normal
                     Pareto || Pareto
                     Pareto || Uniform
                    Poisson || Bernoulli
                    Poisson || Binomial
                    Poisson || Poisson
    TransformedDistribution || TransformedDistribution
                    Uniform || Beta
                    Uniform || ContinuousBernoulli
                    Uniform || Exponential
                    Uniform || Gamma
                    Uniform || Gumbel
                    Uniform || Normal
                    Uniform || Pareto
                    Uniform || Uniform


Instruction: PyTorch equivalent of numpy reshape function

Input:
Hi I have these to functions to flatten my complex type data to feed it to NN and reconstruct NN prediction to the original form.
def flatten_input64(Input): #convert (:,4,4,2) complex matrix to (:,64) real vector
  Input1 = Input.reshape(-1, 32, order='F')
  Input_vector=np.zeros([19957,64],dtype = np.float64)
  Input_vector[:,0:32] = Input1.real
  Input_vector[:,32:64] = Input1.imag
  return Input_vector

def convert_output64(Output): #convert (:,64) real vector to (:,4,4,2) complex matrix
  Output1 = Output[:,0:32] + 1j * Output[:,32:64]
  output_matrix = Output1.reshape(-1, 4 ,4 ,2 , order = 'F')
  return output_matrix

I am writing a customized loss that required all operation to be in torch and I should rewrite my conversion functions in PyTorch. The problem is that PyTorch doesn't have 'F' order reshape. I tried to write my own version of F reorder but, it doesn't work.
Do you have any idea what is my mistake?
def convert_output64_torch(input):
   # number_of_samples = defined
   for i in range(0, number_of_samples):
     Output1 = input[i,0:32] + 1j * input[i,32:64]
     Output2 = Output1.view(-1,4,4,2).permute(3,2,1,0)
   if i == 0:
     Output3 = Output2
   else:
     Output3 = torch.cat((Output3, Output2),0)
return Output3

Update: following @a_guest comment I tried to recreate my matrix with transpose and reshape and I got this code working  same as F order reshape in numy:
def convert_output64_torch(input):
   Output1 = input[:,0:32] + 1j * input[:,32:64]
   shape = (-1 , 4 , 4 , 2)
   Output3 = torch.transpose(torch.transpose(torch.reshape(torch.transpose(Output1,0,1),shape[::-1]),1,2),0,3)
return Output3

Output:
In both, Numpy and PyTorch, you can get the equivalent with the following operation: a.T.reshape(shape[::-1]).T (where a is either an array or a tensor):
>>> a = np.arange(16).reshape(4, 4)
>>> a
array([[ 0,  1,  2,  3],
       [ 4,  5,  6,  7],
       [ 8,  9, 10, 11],
       [12, 13, 14, 15]])
>>> shape = (2, 8)
>>> a.reshape(shape, order='F')
array([[ 0,  8,  1,  9,  2, 10,  3, 11],
       [ 4, 12,  5, 13,  6, 14,  7, 15]])
>>> a.T.reshape(shape[::-1]).T
array([[ 0,  8,  1,  9,  2, 10,  3, 11],
       [ 4, 12,  5, 13,  6, 14,  7, 15]])


Instruction: Convert a list of numpy array to torch tensor list

Input:
The numpy arrays in the list are 2D array that have different sizes, let's say:
1x1, 4x4, 8x8, etc
about 7 arrays in total.
I know how to convert each on of them, by:
torch.from_numpy(a1by1).type(torch.FloatTensor)
torch.from_numpy(a4by4).type(torch.FloatTensor)
etc..

Is there a way to convert the entire list in one command?
I found these 2 question:
How to convert a list or numpy array to a 1d torch tensor?

How to convert a list of tensors into a torch::Tensor?

but it's not what Im' looking for

Output:
If by one command you mean one-liner then
Here, We can use list-comprehension
lst = [a1by1, a4by4, a8by8]
lst = [torch.from_numpy(item).float() for item in lst]


Instruction: Get the polygon coordinates of predicted output mask in YOLACT/YOLACT++

Input:
I am using Yolact https://github.com/dbolya/yolact ,an instance segmentation algorithm which outputs the test image with a mask on the detected object. As the input images are given with the coordinates of polygons around the input classes in the annotations.json, I want to get an output like this. But I can't figure out how to extract the coordinates of those contours/polygons.
As far as I understood from this script https://github.com/dbolya/yolact/blob/master/eval.py the output is list of tensors for detected objects. It contains classes, scores, boxes and mask for evaluated image. The eval.py script returns recognized image with all this information. Recognition is saved in 'preds' in evalimg function (line 595), and post-processing of predict result is in the "def prep_display" (line 135)
Now how do I extract those polygon coordinates and save it in .JSON file or whatever else?
I also tried to look at these but couldn't figure out sadly!
https://github.com/dbolya/yolact/issues/286
and
https://github.com/dbolya/yolact/issues/256

Output:
You need to create a complete post-processing pipeline that is specific to your task. Here's small pseudocode that could be added to the prep_disply() in eval.py
with timer.env('Copy'):
    if cfg.eval_mask_branch:

        # Add the below line to get all the predicted objects as a list
        all_objects_mask = t[3][:args.top_k]

        # Convert each object mask to binary and then
        # Use OpenCV's findContours() method to extract the contour points for each object


Instruction: Pytorch - RuntimeError: invalid multinomial distribution (encountering probability entry < 0)

Input:
I am using Stable Baselines 3 to train an agent to play Connect 4 game. I am trying to take the case into account when an agent starts a game as a second player.
self.env = self.ks_env.train([opponent, None])

When I am trying to run the code, I am getting the following error:
invalid multinomial distribution (encountering probability entry < 0)
/opt/conda/lib/python3.7/site-packages/torch/distributions/categorical.py in sample(self, sample_shape)
samples_2d = torch.multinomial(probs_2d, sample_shape.numel(), True).T

However, there is no problem when an agent is first player:
self.env = self.ks_env.train([None, opponent])

I think problem is related to the Pytorch library. My question is how can I fix this issue?

Output:
After checking your provided code, the problem doesn't seem to come from what agent starts the game but from not restarting the environment after a game is done.
I just changed your step function as shown:
def step(self, action):
    # Check if agent's move is valid
    is_valid = (self.obs['board'][int(action)] == 0)
    if is_valid:  # Play the move
        self.obs, old_reward, done, _ = self.env.step(int(action))
        reward = self.change_reward(old_reward, done)
    else:  # End the game and penalize agent
        reward, done, _ = -10, True, {}
    if done:
        self.reset()
    return board_flip(self.obs.mark,
                      np.array(self.obs['board']).reshape(1, self.rows, self.columns) / 2),
                      reward, done, _

With this, the model was able to train and you can check that it works as expected with the following snippet:
done = True
for step in range(500):
    if done:
        state = env.reset()
    state, reward, done, info = env.step(env.action_space.sample())
    print(reward)

Link to my version of your notebook


Instruction: Problems loading model; Pytorch

Input:
I am trying to load a model I saved with the following and it gives me this error:
import torch

model = torch.load('./grunet.pkl')
model.eval()

This is the error:
  Traceback (most recent call last):
  File ".\eval.py", line 4, in <module>
    model.eval()
  AttributeError: 'NoneType' object has no attribute 'eval'

Please help!

Output:
You are missing one step:
model = YourModelClass()
model.load_state_dict(torch.load("./grunet.pkl"))
model.eval()

# do something with the model