Instruction: pytorch training loop ends with ''int' object has no attribute 'size' exception

Input:
The code I am posting below is just a small part of the application:
def train(self, training_reviews, training_labels):

        # make sure out we have a matching number of reviews and labels
        assert(len(training_reviews) == len(training_labels))

        # Keep track of correct predictions to display accuracy during training
        correct_so_far = 0

        # Remember when we started for printing time statistics
        start = time.time()

        criterion = nn.CrossEntropyLoss()
        optimizer =  torch.optim.SGD(self.parameters(), lr=self.learning_rate)

        # loop through all the given reviews and run a forward and backward pass,
        # updating weights for every item
        for i in range(len(training_reviews)):

            # TODO: Get the next review and its correct label
            review = training_reviews[i]
            label = training_labels[i]
            print('processing item ',i)
            self.update_input_layer(review)
            output = self.forward(torch.from_numpy(self.layer_0).float())
            target = self.get_target_for_label(label)
            print('output ',output)
            print('target ',target)
            loss = criterion(output, target)

...
mlp = SentimentNetwork(reviews[:-1000],labels[:-1000], learning_rate=0.1)
mlp.train(reviews[:-1000],labels[:-1000])

and it ends with the exception in the title line when evaluating:
loss = criterion(output, target)

prior to that, the variables are as follows:
output  tensor([[0.5803]], grad_fn=<SigmoidBackward>)
target  1

Output:
Target should be a torch.Tensor variable. Use torch.tensor([target]).
Additionally, you may want to use batches (so there are N samples and shape of torch.tensor is (N,), same for target).
Also see introductory tutorial about PyTorch, as you're not using batches, not running optimizer or not using torch.utils.data.Dataset and torch.utils.data.DataLoader as you probably should.


Instruction: Pytorch: multiply two high dimensions tensor, (2, 5, 3) * (2, 5) into (2, 5, 3)

Input:
I want to multiply two high dimensions tensor, (2, 5, 3) * (2, 5) into (2, 5, 3), which multiply each row vector by a scalar.
E.g.
emb = nn.Embedding(6, 3)

input = torch.tensor([[1, 2, 3, 4, 5,],
                      [2, 3, 1, 4, 5,]])
input_emb = emb(input)

print(input.shape)
> torch.Size([2, 5])

print(input_emb.shape)
> torch.Size([2, 5, 3])

print(input_emb)
> tensor([[[-1.9114, -0.1580,  1.2186],
         [ 0.4627,  0.9119, -1.1691],
         [ 0.6452, -0.6944,  1.9659],
         [-0.5048,  0.6411, -1.3568],
         [-0.2328, -0.9498,  0.7216]],

        [[ 0.4627,  0.9119, -1.1691],
         [ 0.6452, -0.6944,  1.9659],
         [-1.9114, -0.1580,  1.2186],
         [-0.5048,  0.6411, -1.3568],
         [-0.2328, -0.9498,  0.7216]]], grad_fn=<EmbeddingBackward>)

I want to multiply may as follows:
// It is written in this way for convenience, not mathematical true.

// multiply each row vector by a scalar
[[
         [-1.9114, -0.1580,  1.2186] * 1
         [ 0.4627,  0.9119, -1.1691] * 2
         [ 0.6452, -0.6944,  1.9659] * 3
         [-0.5048,  0.6411, -1.3568] * 4
         [-0.2328, -0.9498,  0.7216] * 5
]
[
         [ 0.4627,  0.9119, -1.1691] * 2
         [ 0.6452, -0.6944,  1.9659] * 3
         [-1.9114, -0.1580,  1.2186] * 1
         [-0.5048,  0.6411, -1.3568] * 4
         [-0.2328, -0.9498,  0.7216] * 5
]]

Except for the multi-for-loops ways, how to implement it in a concise way by PyTorch APIs?
Thanks in advances.

Output:
You can by correctly aligning the dimensions of both tensors:
import torch
from torch.nn import Embedding

emb = Embedding(6, 3)
inp = torch.tensor([[1, 2, 3, 4, 5,],
                      [2, 3, 1, 4, 5,]])
input_emb = emb(inp)

inp[...,None] * input_emb

tensor([[[-0.3069, -0.7727, -0.3772],
         [-2.8308,  1.3438, -1.1167],
         [ 0.6366,  0.6509, -3.2282],
         [-4.3004,  3.2342, -0.6556],
         [-3.0045, -0.0191, -7.4436]],

        [[-2.8308,  1.3438, -1.1167],
         [ 0.6366,  0.6509, -3.2282],
         [-0.3069, -0.7727, -0.3772],
         [-4.3004,  3.2342, -0.6556],
         [-3.0045, -0.0191, -7.4436]]], grad_fn=<MulBackward0>)


Instruction: Running GPU on multiple PyTorch tensor operators

Input:
I have the following PyTorch tensor:
X = np.array([[1, 3, 2, 3], [2, 3, 5, 6]])
X = torch.FloatTensor(X).cuda()

I was wondering if there is any difference (especially in the speed) between Scenario A or B below if I run the following multiple PyTorch operators in one line?
Scenario A:
X_sq_sum = (X**2).cuda().sum(dim = 1).cuda()

Scenario B:
X_sq_sum = (X**2).sum(dim = 1).cuda()

ie. Scenario A has two .cuda() whereas Scenario B only has one .cuda().
Many thanks in advance.

Output:
They will perform equally as the CUDA conversion is only done once.
As described in the docs, repeated .cuda() calls will be no-ops if the object is already in CUDA memory.


Instruction: How to create a balancing cycling iterator in PyTourch?

Input:
Say I have 2 classes. And for one I have only 17 samples and for another 83. I want to always have equal amount of data from each class per epoch (meaning 17 by 17 in this case). Also, I want to slide sampling a window across the class where I have more data each epoch (first 17, next 17, ...).
Currently I have a looping sampling iterator like this:
class CyclicIterator:
    def __init__(self, loader, sampler):
        self.loader = loader
        self.sampler = sampler
        self.epoch = 0
        self._next_epoch()

    def _next_epoch(self):
        self.iterator = iter(self.loader)
        self.epoch += 1

    def __len__(self):
        return len(self.loader)

    def __iter__(self):
        return self

    def __next__(self):
        try:
            return next(self.iterator)
        except StopIteration:
            self._next_epoch()
            return next(self.iterator)

I wonder how to force all samples from each class to be of equal count per epoch?

Output:
For a balanced batch, which means equal (or close to equal) number of samples per category in each batch, there are some approaches:
-Oversampling (make minor sized classes oversample until reach highest number of samples). In this approach you can use following code:
https://github.com/galatolofederico/pytorch-balanced-batch
-Undersampling (delivers number of samples for all categories based in smallest category number). In my experience, below function does like that using PyTorch library:
torch.utils.data.sampler.WeightedRandomSampler(weights, len(weights))

Where weights is the probability of each sample, it depends in how many samples per category you have, for instance, if you data is simple as that data = [0, 1, 0, 0, 1], class '0' count is 3, and class '1' count is 2 So weights vector is [1/3, 1/2, 1/3, 1/3, 1/2]. With that you can call WeightedRamdomSampler and it will make it for you.
You need to call it in Dataloader. The code to setup it is:
sampler = torch.utils.data.sampler.WeightedRandomSampler(weights, len(weights))
train_dataloader = DataLoader(dataset_train, batch_size=mini_batch,
                              sampler=sampler, shuffle=False,
                              num_workers=1)


Instruction: Tensorboard only showing dots, line not showing

Input:
I am using tensorboard through pytorch and have created a scaler which shows accuracy, loss, and validation accuracy.  When I open the graph it only shows dot and dosen't show a line connecting these dots.
Here is a snippet of my code
tb = SummaryWriter(comment=f'-{epoch}')

tb.add_scalar("Accuracy", float(correct/len(train_tensor)*100), epoch)
tb.add_scalar("Loss", loss, epoch)
tb.add_scalar("ValiAccuracy", float(correct/len(train_tensor)*100), epoch)
tb.close()

What is leading tensorboard to not show these lines?

Output:
You are always creating a new directory with your "comment" (depends on your epoch) in the definition of the SummaryWriter. Each subdirectory will be treated as a different experiment in the tensorboard. Thats why they have different colors and only show dots instead of a connected line.
You can try to define your SummaryWriter without a comment:
tb = SummaryWriter()


Instruction: Element wise batch matrix multiplication of a row with every other row in matrix, in PyTorch

Input:
I have two matrices of sizes (30, 24, 512) respectively where 30 is the batch size. Let us call them A and B.
Now what I need to do is this:
For every batch in A, I want to compute element-wise batch matrix multiplication of each row in a single batch of A with each row in a single batch of B and sum them.
In other words, for every batch, I have a (24, 512) matrix on left-hand side (A) and on right-hand side (B). Now for each row (1 * 512) in A, I want to compute element-wise matrix multiplication of that row with each of the (24 * 512) rows in B, one by one, and sum them.
This operation would thus result in a (1 * 512) sized vector.
Done for each row in A, it would result in a (24 * 512) sized matrix and then, done for each batch it would result in (30, 24, 512) sized matrix, which is the preferred dimension of my output.
How do I go about doing this?
I do not want to use for loops, since that would be inefficient. I also do not want to use repeat method for the same reason.

Output:
I figured out the answer to my own question. Instead of computing the element-wise product of each row and then wanting to compute the sum, I summed the rows and then took the element-wise product, which made the problem (and the solution) much simpler.
import torch

a = torch.randn(30, 24, 512)
b = torch.randn(30, 24, 512)

# Step 1: Summing b along dimension 1 and unsqueezing
# Dimension of b_sum is 30*1*512
b_sum = torch.sum(b, dim=1).unsqueeze(1)

# Step 2: Element-wise product
result = a * b_sum


Instruction: Copy large github file into existing repo without downloading/cloning

Input:
How to i add bin file associated to this repo https://github.com/graykode/gpt-2-Pytorch to my existing private repo without downloading/cloning?  There is a 500mb file that needs downloading to get the codebase to work that I need in my repo and don't want to download into my small hard disk during the cloning process.
It's a noob question but i'm at a complete loss as to how to proceed. Please be nice.

Output:
You can't, sorry. The maximum file size on Github is 100 MB. And for good reason, Git does not work well with large, binary files.

Simplest thing is to find a USB key or other external storage and use it as temporary space to store the file while you add it with git-lfs.
Or don't add the file to your repository. Download and cache it as part of your build process.


Instruction: Loading json file using torchtext

Input:
I'm working on the dailydialog dataset, which I've converted into a
JSON file which looks something like this:
[{"response": "You know that is tempting but is really not good for our fitness.", "message": "Say, Jim, how about going for a few beers after dinner?"}, {"response": "Do you really think so? I don't. It will just make us fat and act silly. Remember last time?", "message": "What do you mean? It will help us to relax."}, {"response": "I suggest a walk over to the gym where we can play singsong and meet some of our friends.", "message": "I guess you are right. But what shall we do? I don't feel like sitting at home."}, {"response": "Sounds great to me! If they are willing, we could ask them to go dancing with us.That is excellent exercise and fun, too.", "message": "That's a good idea. I hear Mary and Sally often go there to play pingpong.Perhaps we can make a foursome with them."}, {"response": "All right.", "message": "Please lie down over there."}]
So, each item has two keys - response and message.
This is my first time using PyTorch, so I was following a few online available resources. These are the relevant snippets of my code:
def tokenize_en(text):
    return [tok.text for tok in spacy_en.tokenizer(text)]

src = Field(tokenize = tokenize_en,
            init_token = '<sos>',
            eos_token = '<eos>',
            lower = True)

fields = {'response': ('r', src)}

train_data, test_data, validation_data = TabularDataset.splits(
                                        path = 'FilePath',
                                        train = 'trainset.json',
                                        test = 'testset.json',
                                        validation = 'validationset.json',
                                        format = 'json',
                                        fields = fields
)

Although no errors are raised, despite having many items in my JSON file, the train, test and validation datasets strangely have only 1 example each, as seen in this image:
Image Showing the length of train_data, test_data and validation_data
I'd be really grateful if someone could point out the error to me.
Edit: I found out that the whole file is being treated as a single text string due to lack of indents in the file. But if I indent the JSON file, the TabularDataset function throws a JSONDecodeError to me, suggesting it can no more decode the file. How can I get rid of this problem?

Output:
I think the code is alright, but the issue is with your JSON file. Can you try removing the square brackets("[]") at the beginning and the end of the file?
Probably that is the reason that Your Python file is reading it as one single object.


Instruction: Can numpy arrays run in GPUs?

Input:
I am using PyTorch. I have the following code:
import numpy as np
import torch

X = np.array([[1, 3, 2, 3], [2, 3, 5, 6], [1, 2, 3, 4]])
X = torch.DoubleTensor(X).cuda()

X_split = np.array_split(X.numpy(),
                         indices_or_sections = 2,
                         axis = 0)
X_split

but I am getting this error:
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-121-870b5d3f67b6> in <module>()
----> 1 X_prime_class_split = np.array_split(X_prime_class.numpy(),
      2                                      indices_or_sections = 2,
      3                                      axis = 0)
      4 X_prime_class_split

TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

The error message is clear and I know how to fix this error by just including .cpu(), ie. X_prime_class.cpu().numpy(). I am just curious to know if this confirms that numpy arrays cannot run in GPUs/Cuda?

Output:
No you cannot generally run numpy functions on GPU arrays. PyTorch reimplements much of the functionality in numpy for PyTorch tensors. For example torch.chunk works similarly to np.array_split so you could do the following:
X = np.array([[1, 3, 2, 3], [2, 3, 5, 6], [1, 2, 3, 4]])
X = torch.DoubleTensor(X).cuda()
X_split = torch.chunk(X, chunks=2, dim=0)

which splits X into multiple tensors without ever moving the X off the GPU.


Instruction: Im not running anything on GPU. But GPU stat is not clearing my usage

Input:
GPUstat image
Im not running anything on gpu.
My ID is suil5044. GPUstat is not clearing my usage.
I think I finished it in the IDE but actually the code is still running on the server.
How do I kill my code still running on the server?
And not affect other users
Thanks!

Output:
It would be better if you could show us which process are using your GPU. Because sometimes many utils process use the GPU too under same user name.
However, please try,

Find out all running process pid which are using the GPU. This tool may help.
Kill process by PID by this command: kill -9 <pid> (be sure this pid is under your username)


Instruction: How can I incorporate PReLU in a quantized model?

Input:
I'm trying to quantize a model which uses PReLU. Replacing PReLU with ReLU is not possible as it drastically affects the network performance to the point its useless.
As far as I know, PReLU is not supported in Pytorch when it comes to quantization. So I tried to rewrite this module manually and implement the multiplication and additions using torch.FloatFunctional() to get around this limitation.
This is what I have come up so far:
class PReLU_Quantized(nn.Module):
    def __init__(self, prelu_object):
        super().__init__()
        self.weight = prelu_object.weight
        self.quantized_op = nn.quantized.FloatFunctional()
        self.quant = torch.quantization.QuantStub()
        self.dequant = torch.quantization.DeQuantStub()

    def forward(self, inputs):
        # inputs = torch.max(0, inputs) + self.weight * torch.min(0, inputs)
        self.weight = self.quant(self.weight)
        weight_min_res = self.quantized_op.mul(self.weight, torch.min(inputs)[0])
        inputs = self.quantized_op.add(torch.max(inputs)[0], weight_min_res).unsqueeze(0)
        self.weight = self.dequant(self.weight)
        return inputs

and for the replacement :
class model(nn.Module):
     def __init__(self)
         super().__init__()
         ....
        self.prelu = PReLU()
        self.prelu_q = PReLU_Quantized(self.prelu)
         ....

Basically, I read the learned parameter of the existing prelu module, and run the calculation myself in a new module. The module seems to be working in the sense, its not failing the whole application.
However, in order to assess whether my implementation is actually correct and yields the same result as the original module, I tried to test it.
Here is a counterpart for normal models (i.e. not quantized model):
For some reason, the error between the actual PReLU and my implementation is very large!
Here are sample diffs in different layers:
diff : 1.1562038660049438
diff : 0.02868632599711418
diff : 0.3653906583786011
diff : 1.6100226640701294
diff : 0.8999372720718384
diff : 0.03773299604654312
diff : -0.5090572834014893
diff : 0.1654307246208191
diff : 1.161868691444397
diff : 0.026089997962117195
diff : 0.4205571115016937
diff : 1.5337920188903809
diff : 0.8799554705619812
diff : 0.03827812895178795
diff : -0.40296515822410583
diff : 0.15618863701820374

and the diff is calculated like this in the forward pass:
def forward(self, x):
    residual = x
    out = self.bn0(x)
    out = self.conv1(out)
    out = self.bn1(out)

    out = self.prelu(out)
    out2 = self.prelu2(out)
    print(f'diff : {( out - out2).mean().item()}')

    out = self.conv2(out)
...

This is the normal implementation which I used on ordinary model (i.e. not quantized!) to asses whether it produces correct result and then move on to quantized version:
class PReLU_2(nn.Module):
    def __init__(self, prelu_object):
        super().__init__()
        self.prelu_weight = prelu_object.weight
        self.weight = self.prelu_weight

    def forward(self, inputs):
        x = self.weight
        tmin, _ = torch.min(inputs,dim=0)
        tmax, _ = torch.max(inputs,dim=0)
        weight_min_res = torch.mul(x, tmin)
        inputs = torch.add(tmax, weight_min_res)
        inputs = inputs.unsqueeze(0)
        return inputs

What am I missing here?

Output:
I figured it out! I made a huge mistake in the very begining. I needed to calculate
PReLU(x)=max(0,x)+a∗min(0,x)

or

and not the actual torch.min! or torch.max! which doesn't make any sense!
Here is the final solution for normal models (i.e not quantized)!:
class PReLU_2(nn.Module):
    def __init__(self, prelu_object):
        super().__init__()
        self.prelu_weight = prelu_object.weight
        self.weight = self.prelu_weight

    def forward(self, inputs):
        pos = torch.relu(inputs)
        neg = -self.weight * torch.relu(-inputs)
        inputs = pos + neg
        return inputs

and this is the quantized version :
class PReLU_Quantized(nn.Module):
    def __init__(self, prelu_object):
        super().__init__()
        self.prelu_weight = prelu_object.weight
        self.weight = self.prelu_weight
        self.quantized_op = nn.quantized.FloatFunctional()
        self.quant = torch.quantization.QuantStub()
        self.dequant = torch.quantization.DeQuantStub()

    def forward(self, inputs):
        # inputs = max(0, inputs) + alpha * min(0, inputs)
        self.weight = self.quant(self.weight)
        weight_min_res = self.quantized_op.mul(-self.weight, torch.relu(-inputs))
        inputs = self.quantized_op.add(torch.relu(inputs), weight_min_res)
        inputs = self.dequant(inputs)
        self.weight = self.dequant(self.weight)
        return inputs

Side note:
I also had a typo where I was calculating the diff :
    out = self.prelu(out)
    out2 = self.prelu2(out)
    print(f'diff : {( out - out2).mean().item()}')

    out = self.conv2(out)

needs to be
    out1 = self.prelu(out)
    out2 = self.prelu2(out)
    print(f'diff : {( out1 - out2).mean().item()}')
    out = self.conv2(out1)

Update:
In case you face issues in quantization, you may try this version :
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.nn.quantized as nnq
from torch.quantization import fuse_modules

class QPReLU(nn.Module):
    def __init__(self, num_parameters=1, init: float = 0.25):
        super(QPReLU, self).__init__()
        self.num_parameters = num_parameters
        self.weight = nn.Parameter(torch.Tensor(num_parameters).fill_(init))
        self.relu1 = nn.ReLU()
        self.relu2 = nn.ReLU()
        self.f_mul_neg_one1 = nnq.FloatFunctional()
        self.f_mul_neg_one2 = nnq.FloatFunctional()
        self.f_mul_alpha = nnq.FloatFunctional()
        self.f_add = nnq.FloatFunctional()
        self.quant = torch.quantization.QuantStub()
        self.dequant = torch.quantization.DeQuantStub()
        self.quant2 = torch.quantization.QuantStub()
        self.quant3 = torch.quantization.QuantStub()
        # self.dequant2 = torch.quantization.QuantStub()
        self.neg_one = torch.Tensor([-1.0])

    def forward(self, x):
        x = self.quant(x)

        # PReLU, with modules only
        x1 = self.relu1(x)

        neg_one_q = self.quant2(self.neg_one)
        weight_q = self.quant3(self.weight)
        x2 = self.f_mul_alpha.mul(
            weight_q, self.f_mul_neg_one2.mul(
                self.relu2(
                    self.f_mul_neg_one1.mul(x, neg_one_q),
                ),
            neg_one_q)
        )

        x = self.f_add.add(x1, x2)
        x = self.dequant(x)
        return x

m1 = nn.PReLU()
m2 = QPReLU()

# check correctness in fp
for i in range(10):
    data = torch.randn(2, 2) * 1000
    assert torch.allclose(m1(data), m2(data))

# toy model
class M(nn.Module):
    def __init__(self):
        super(M, self).__init__()
        self.prelu = QPReLU()

    def forward(self, x):
        x = self.prelu(x)
        return x

# quantize it
m = M()
m.qconfig = torch.quantization.default_qconfig
torch.quantization.prepare(m, inplace=True)
# calibrate
m(torch.randn(4, 4))
# convert
torch.quantization.convert(m, inplace=True)
# run some data through
res = m(torch.randn(4, 4))
print(res)

and make sure to read the ralted notes here


Instruction: Reshape rows into groups of columns

Input:
I have a number of row vectors which I would like to batch as column vectors and use as input for Conv1d. As an example I'd like to reshape the tensor x into y i.e. making two groups of two column vectors.
# size = [4, 3]
x = torch.tensor([
    [0,  1,  2],
    [3,  4,  5],
    [6,  7,  8],
    [9, 10, 11]
])

# size = [2, 3, 2]
y = torch.tensor([
    [[0,  3],
     [1,  4],
     [2,  5]],
    [[6,  9],
     [7, 10],
     [8, 11]]
])

Is there a way to do this with just reshape and similar functions? The only way I can think of is using loops and copying into a new tensor.

Output:
You need to use permute as well as reshape:
x.reshape(2, 2, 3).permute(0, 2, 1)
Out[*]:
tensor([[[ 0,  3],
         [ 1,  4],
         [ 2,  5]],

        [[ 6,  9],
         [ 7, 10],
         [ 8, 11]]])

First, you split the vectors into 2 x.reshape(2,2,3) placing the extra dimension in the middle. Then using permute you change the order of dimensions to be as you expected.


Instruction: Deployment of a PyTorch computer vision deep learning model into a windows desktop app

Input:
I have trained a yolov3 model in PyTorch with my dataset and I also have written some utility codes for it that run alongside the model all in python language. Now I want to deploy this model and my utilities into a windows desktop app that takes a video and runs the model on its frames. How can I do this task with minimum change in my code or rewrite it in another language? What framework is the best option for designing the UI of the app?
Thanks.

Output:
I would first use your model with PyTorch to detect each frame and use numpy ImageDraw to draw around your object(to be detected). Here is an article on this: Drawing a rectangle inside a 2D numpy array

Then I would use OpenCV(cv2) to append all the frames together to make a video you could also use ffmpeg. Here is a article on this(OpenCV): How to make a movie out of images in python

Then for your UI framework you could use PyQt5 to display your video: Load an opencv video frame by frame using PyQT. But you could also use Kivy with Gstreamer: Kivy VideoPlayer fullscreen, loop, and hide controls

Finally to turn your .py file into a .exe(executable for windows) I would use PyInstaller for that: http://www.pyinstaller.org/


Instruction: Truncated backpropagation in PyTorch (code check)

Input:
I am trying to implement truncated backpropagation through time in PyTorch, for the simple case where K1=K2. I have an implementation below that produces reasonable output, but I just want to make sure it is correct. When I look online for PyTorch examples of TBTT, they do inconsistent things around detaching the hidden state and zeroing out the gradient, and the ordering of these operations. Please let me know if I have made a mistake.
In the code below, H maintains the current hidden state, and model(weights, H, x) outputs the prediction and the new hidden state.
while i < NUM_STEPS:
    # Grab x, y for ith datapoint
    x = data[i]
    target = true_output[i]

    # Run model
    output, new_hidden = model(weights, H, x)
    H = new_hidden

    # Update running error
    error += (output - target)**2

    if (i+1) % K == 0:
        # Backpropagate
        error.backward()
        opt.step()
        opt.zero_grad()
        error = 0
        H = H.detach()

    i += 1

Output:
So the idea of your code is to isolate the last variables after each Kth step. Yes, your implementation is absolutely correct and this answer confirms that.
# truncated to the last K timesteps
while i < NUM_STEPS:
    out = model(out)
    if (i+1) % K == 0:
        out.backward()
        out.detach()
out.backward()

You can also follow this example for your reference.
import torch

from ignite.engine import Engine, EventEnum, _prepare_batch
from ignite.utils import apply_to_tensor

class Tbptt_Events(EventEnum):
    """Aditional tbptt events.

    Additional events for truncated backpropagation throught time dedicated
    trainer.
    """

    TIME_ITERATION_STARTED = "time_iteration_started"
    TIME_ITERATION_COMPLETED = "time_iteration_completed"

def _detach_hidden(hidden):
    """Cut backpropagation graph.

    Auxillary function to cut the backpropagation graph by detaching the hidden
    vector.
    """
    return apply_to_tensor(hidden, torch.Tensor.detach)

def create_supervised_tbptt_trainer(
    model, optimizer, loss_fn, tbtt_step, dim=0, device=None, non_blocking=False, prepare_batch=_prepare_batch
):
    """Create a trainer for truncated backprop through time supervised models.

    Training recurrent model on long sequences is computationally intensive as
    it requires to process the whole sequence before getting a gradient.
    However, when the training loss is computed over many outputs
    (`X to many <https://karpathy.github.io/2015/05/21/rnn-effectiveness/>`_),
    there is an opportunity to compute a gradient over a subsequence. This is
    known as
    `truncated backpropagation through time <https://machinelearningmastery.com/
    gentle-introduction-backpropagation-time/>`_.
    This supervised trainer apply gradient optimization step every `tbtt_step`
    time steps of the sequence, while backpropagating through the same
    `tbtt_step` time steps.

    Args:
        model (`torch.nn.Module`): the model to train.
        optimizer (`torch.optim.Optimizer`): the optimizer to use.
        loss_fn (torch.nn loss function): the loss function to use.
        tbtt_step (int): the length of time chunks (last one may be smaller).
        dim (int): axis representing the time dimension.
        device (str, optional): device type specification (default: None).
            Applies to batches.
        non_blocking (bool, optional): if True and this copy is between CPU and GPU,
            the copy may occur asynchronously with respect to the host. For other cases,
            this argument has no effect.
        prepare_batch (callable, optional): function that receives `batch`, `device`,
            `non_blocking` and outputs tuple of tensors `(batch_x, batch_y)`.

    .. warning::

        The internal use of `device` has changed.
        `device` will now *only* be used to move the input data to the correct device.
        The `model` should be moved by the user before creating an optimizer.

        For more information see:

        * `PyTorch Documentation <https://pytorch.org/docs/stable/optim.html#constructing-it>`_
        * `PyTorch's Explanation <https://github.com/pytorch/pytorch/issues/7844#issuecomment-503713840>`_

    Returns:
        Engine: a trainer engine with supervised update function.

    """

    def _update(engine, batch):
        loss_list = []
        hidden = None

        x, y = batch
        for batch_t in zip(x.split(tbtt_step, dim=dim), y.split(tbtt_step, dim=dim)):
            x_t, y_t = prepare_batch(batch_t, device=device, non_blocking=non_blocking)
            # Fire event for start of iteration
            engine.fire_event(Tbptt_Events.TIME_ITERATION_STARTED)
            # Forward, backward and
            model.train()
            optimizer.zero_grad()
            if hidden is None:
                y_pred_t, hidden = model(x_t)
            else:
                hidden = _detach_hidden(hidden)
                y_pred_t, hidden = model(x_t, hidden)
            loss_t = loss_fn(y_pred_t, y_t)
            loss_t.backward()
            optimizer.step()

            # Setting state of engine for consistent behaviour
            engine.state.output = loss_t.item()
            loss_list.append(loss_t.item())

            # Fire event for end of iteration
            engine.fire_event(Tbptt_Events.TIME_ITERATION_COMPLETED)

        # return average loss over the time splits
        return sum(loss_list) / len(loss_list)

    engine = Engine(_update)
    engine.register_events(*Tbptt_Events)
    return engine


Instruction: Pytorch with Modified Derivatives

Input:
I'm in the process of rewriting the TRACX2 model, a variation of a recurrent neural network used for training encodings in the context of word segmentation from continuous speech or text. The writer of the original code manually wrote the network in Numpy, while I want to optimize it with Pytorch. However, they implement something they call "temperature" and a "fahlman offset":

This clearly isn't the actual derivative of tanh(x), one of their activation functions, but they used this derivative instead. How would I go about implementing this modification in Pytorch?

Output:
Basically, you add a backward hook like so:
a = Variable(torch.randn(2,2), requires_grad=True)
m = nn.Linear(2,1)
m(a).mean().backward()
print(a.grad)
# shows a 2x2 tensor of non-zero values
temperature = 0.3
fahlmanOffset = .1

def hook(module, grad_input, grad_output):
    # Use custom gradient output
    return grad_output * temperature + fahlmanOffset

m.register_backward_hook(hook)

a.grad.zero_()
m(a).mean().backward()
print(a.grad)
# shows a 2x2 tensor with modified gradient

(courtesy of this answer)


Instruction: Can't parse arguments (deep learning tutorial using pytorch)

Input:
I'm following this tutorial.
The first scripts run fine and I have a "data" folder in the folder of my scripts containing the MRI data downloaded from MRnet.
However when it comes to the "train" script I get an error. Here's the full script and the error (using jupyter notebook):
import shutil
import os
import time
from datetime import datetime
import argparse
import numpy as np
from tqdm import tqdm

import torch
import torch.nn as nn
import torch.optim as optim
from torch.autograd import Variable
from torchsample.transforms import RandomRotate, RandomTranslate, RandomFlip, ToTensor, Compose, RandomAffine
from torchvision import transforms
import torch.nn.functional as F
from tensorboardX import SummaryWriter
import nbimporter

from dataloader import MRDataset
import model

from sklearn import metrics

def train_model(model, train_loader, epoch, num_epochs, optimizer, writer, current_lr, log_every=100):
    _ = model.train()

    if torch.cuda.is_available():
        model.cuda()

    y_preds = []
    y_trues = []
    losses = []

    for i, (image, label, weight) in enumerate(train_loader):
        optimizer.zero_grad()

        if torch.cuda.is_available():
            image = image.cuda()
            label = label.cuda()
            weight = weight.cuda()

        label = label[0]
        weight = weight[0]

        prediction = model.forward(image.float())

        loss = torch.nn.BCEWithLogitsLoss(weight=weight)(prediction, label)
        loss.backward()
        optimizer.step()

        loss_value = loss.item()
        losses.append(loss_value)

        probas = torch.sigmoid(prediction)

        y_trues.append(int(label[0][1]))
        y_preds.append(probas[0][1].item())

        try:
            auc = metrics.roc_auc_score(y_trues, y_preds)
        except:
            auc = 0.5

        writer.add_scalar('Train/Loss', loss_value,
                          epoch * len(train_loader) + i)
        writer.add_scalar('Train/AUC', auc, epoch * len(train_loader) + i)

        if (i % log_every == 0) & (i > 0):
            print('''[Epoch: {0} / {1} |Single batch number : {2} / {3} ]| avg train loss {4} | train auc : {5} | lr : {6}'''.
                  format(
                      epoch + 1,
                      num_epochs,
                      i,
                      len(train_loader),
                      np.round(np.mean(losses), 4),
                      np.round(auc, 4),
                      current_lr
                  )
                  )

    writer.add_scalar('Train/AUC_epoch', auc, epoch + i)

    train_loss_epoch = np.round(np.mean(losses), 4)
    train_auc_epoch = np.round(auc, 4)
    return train_loss_epoch, train_auc_epoch

def evaluate_model(model, val_loader, epoch, num_epochs, writer, current_lr, log_every=20):
    _ = model.eval()

    if torch.cuda.is_available():
        model.cuda()

    y_trues = []
    y_preds = []
    losses = []

    for i, (image, label, weight) in enumerate(val_loader):

        if torch.cuda.is_available():
            image = image.cuda()
            label = label.cuda()
            weight = weight.cuda()

        label = label[0]
        weight = weight[0]

        prediction = model.forward(image.float())

        loss = torch.nn.BCEWithLogitsLoss(weight=weight)(prediction, label)

        loss_value = loss.item()
        losses.append(loss_value)

        probas = torch.sigmoid(prediction)

        y_trues.append(int(label[0][1]))
        y_preds.append(probas[0][1].item())

        try:
            auc = metrics.roc_auc_score(y_trues, y_preds)
        except:
            auc = 0.5

        writer.add_scalar('Val/Loss', loss_value, epoch * len(val_loader) + i)
        writer.add_scalar('Val/AUC', auc, epoch * len(val_loader) + i)

        if (i % log_every == 0) & (i > 0):
            print('''[Epoch: {0} / {1} |Single batch number : {2} / {3} ] | avg val loss {4} | val auc : {5} | lr : {6}'''.
                  format(
                      epoch + 1,
                      num_epochs,
                      i,
                      len(val_loader),
                      np.round(np.mean(losses), 4),
                      np.round(auc, 4),
                      current_lr
                  )
                  )

    writer.add_scalar('Val/AUC_epoch', auc, epoch + i)

    val_loss_epoch = np.round(np.mean(losses), 4)
    val_auc_epoch = np.round(auc, 4)
    return val_loss_epoch, val_auc_epoch

def get_lr(optimizer):
    for param_group in optimizer.param_groups:
        return param_group['lr']

def run(args):
    log_root_folder = "./logs/{0}/{1}/".format(args.task, args.plane)
    if args.flush_history == 1:
        objects = os.listdir(log_root_folder)
        for f in objects:
            if os.path.isdir(log_root_folder + f):
                shutil.rmtree(log_root_folder + f)

    now = datetime.now()
    logdir = log_root_folder + now.strftime("%Y%m%d-%H%M%S") + "/"
    os.makedirs(logdir)

    writer = SummaryWriter(logdir)

    augmentor = Compose([
        transforms.Lambda(lambda x: torch.Tensor(x)),
        RandomRotate(25),
        RandomTranslate([0.11, 0.11]),
        RandomFlip(),
        transforms.Lambda(lambda x: x.repeat(3, 1, 1, 1).permute(1, 0, 2, 3)),
    ])

    train_dataset = MRDataset('./data/', args.task,
                              args.plane, transform=augmentor, train=True)
    train_loader = torch.utils.data.DataLoader(
        train_dataset, batch_size=1, shuffle=True, num_workers=11, drop_last=False)

    validation_dataset = MRDataset(
        './data/', args.task, args.plane, train=False)
    validation_loader = torch.utils.data.DataLoader(
        validation_dataset, batch_size=1, shuffle=-True, num_workers=11, drop_last=False)

    mrnet = model.MRNet()

    if torch.cuda.is_available():
        mrnet = mrnet.cuda()

    optimizer = optim.Adam(mrnet.parameters(), lr=args.lr, weight_decay=0.1)

    if args.lr_scheduler == "plateau":
        scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(
            optimizer, patience=3, factor=.3, threshold=1e-4, verbose=True)
    elif args.lr_scheduler == "step":
        scheduler = torch.optim.lr_scheduler.StepLR(
            optimizer, step_size=3, gamma=args.gamma)

    best_val_loss = float('inf')
    best_val_auc = float(0)

    num_epochs = args.epochs
    iteration_change_loss = 0
    patience = args.patience
    log_every = args.log_every

    t_start_training = time.time()

    for epoch in range(num_epochs):
        current_lr = get_lr(optimizer)

        t_start = time.time()

        train_loss, train_auc = train_model(
            mrnet, train_loader, epoch, num_epochs, optimizer, writer, current_lr, log_every)
        val_loss, val_auc = evaluate_model(
            mrnet, validation_loader, epoch, num_epochs, writer, current_lr)

        if args.lr_scheduler == 'plateau':
            scheduler.step(val_loss)
        elif args.lr_scheduler == 'step':
            scheduler.step()

        t_end = time.time()
        delta = t_end - t_start

        print("train loss : {0} | train auc {1} | val loss {2} | val auc {3} | elapsed time {4} s".format(
            train_loss, train_auc, val_loss, val_auc, delta))

        iteration_change_loss += 1
        print('-' * 30)

        if val_auc > best_val_auc:
            best_val_auc = val_auc
            if bool(args.save_model):
                file_name = f'model_{args.prefix_name}_{args.task}_{args.plane}_val_auc_{val_auc:0.4f}_train_auc_{train_auc:0.4f}_epoch_{epoch+1}.pth'
                for f in os.listdir('./models/'):
                    if (args.task in f) and (args.plane in f) and (args.prefix_name in f):
                        os.remove(f'./models/{f}')
                torch.save(mrnet, f'./models/{file_name}')

        if val_loss < best_val_loss:
            best_val_loss = val_loss
            iteration_change_loss = 0

        if iteration_change_loss == patience:
            print('Early stopping after {0} iterations without the decrease of the val loss'.
                  format(iteration_change_loss))
            break

    t_end_training = time.time()
    print(f'training took {t_end_training - t_start_training} s')

def parse_arguments():
    parser = argparse.ArgumentParser()
    parser.add_argument('-t', '--task', type=str, required=True,
                        choices=['abnormal', 'acl', 'meniscus'])
    parser.add_argument('-p', '--plane', type=str, required=True,
                        choices=['sagittal', 'coronal', 'axial'])
    parser.add_argument('--prefix_name', type=str, required=True)
    parser.add_argument('--augment', type=int, choices=[0, 1], default=1)
    parser.add_argument('--lr_scheduler', type=str,
                        default='plateau', choices=['plateau', 'step'])
    parser.add_argument('--gamma', type=float, default=0.5)
    parser.add_argument('--epochs', type=int, default=50)
    parser.add_argument('--lr', type=float, default=1e-5)
    parser.add_argument('--flush_history', type=int, choices=[0, 1], default=0)
    parser.add_argument('--save_model', type=int, choices=[0, 1], default=1)
    parser.add_argument('--patience', type=int, default=5)
    parser.add_argument('--log_every', type=int, default=100)
    args = parser.parse_args()
    return args

if __name__ == "__main__":
    args = parse_arguments()
    run(args)

Error:
usage: ipykernel_launcher.py [-h] -t {abnormal,acl,meniscus} -p
                             {sagittal,coronal,axial} --prefix_name
                             PREFIX_NAME [--augment {0,1}]
                             [--lr_scheduler {plateau,step}] [--gamma GAMMA]
                             [--epochs EPOCHS] [--lr LR]
                             [--flush_history {0,1}] [--save_model {0,1}]
                             [--patience PATIENCE] [--log_every LOG_EVERY]
ipykernel_launcher.py: error: the following arguments are required: -t/--task, -p/--plane, --prefix_name

%tb:
    ---------------------------------------------------------------------------
SystemExit                                Traceback (most recent call last)
<ipython-input-3-e6a34ab63dc0> in <module>
    275
    276 if __name__ == "__main__":
--> 277     args = parse_arguments()
    278     run(args)

<ipython-input-3-e6a34ab63dc0> in parse_arguments()
    270     parser.add_argument('--patience', type=int, default=5)
    271     parser.add_argument('--log_every', type=int, default=100)
--> 272     args = parser.parse_args()
    273     return args
    274

~\anaconda3\envs\Pytorch\lib\argparse.py in parse_args(self, args, namespace)
   1753     # =====================================
   1754     def parse_args(self, args=None, namespace=None):
-> 1755         args, argv = self.parse_known_args(args, namespace)
   1756         if argv:
   1757             msg = _('unrecognized arguments: %s')

~\anaconda3\envs\Pytorch\lib\argparse.py in parse_known_args(self, args, namespace)
   1785         # parse the arguments and exit if there are any errors
   1786         try:
-> 1787             namespace, args = self._parse_known_args(args, namespace)
   1788             if hasattr(namespace, _UNRECOGNIZED_ARGS_ATTR):
   1789                 args.extend(getattr(namespace, _UNRECOGNIZED_ARGS_ATTR))

~\anaconda3\envs\Pytorch\lib\argparse.py in _parse_known_args(self, arg_strings, namespace)
   2020         if required_actions:
   2021             self.error(_('the following arguments are required: %s') %
-> 2022                        ', '.join(required_actions))
   2023
   2024         # make sure all required groups had one option present

~\anaconda3\envs\Pytorch\lib\argparse.py in error(self, message)
   2506         self.print_usage(_sys.stderr)
   2507         args = {'prog': self.prog, 'message': message}
-> 2508         self.exit(2, _('%(prog)s: error: %(message)s\n') % args)

~\anaconda3\envs\Pytorch\lib\argparse.py in exit(self, status, message)
   2493         if message:
   2494             self._print_message(message, _sys.stderr)
-> 2495         _sys.exit(status)
   2496
   2497     def error(self, message):

SystemExit: 2

I have no clue on how to move forwards. I'm stranded here. Anyone know where to go from here?

Output:
I will guess.
ArgumentParser was created to get arguments when you run it in console/terminal not Juputer
python script.py -t abnormal -p axial --prefix_name abc

and Python puts these arguments as list in sys.argv and ArgumentParser uses automatically values from sys.argv in parser.parse_args()
If you want to run it in juputer then you have to send arguments manually as list
args = parser.parse_args( ["-t", "abnormal", "-p", "axial", "--prefix_name", "abc"] )

or you have to append argument to sys.argv
sys.argv.append("-t")
sys.argv.append("abnormal")
sys.argv.append("-p")
sys.argv.append("axial")
sys.argv.append("--prefix_name")
sys.argv.append("abc")

or using .extend( list )
sys.argv.extend( ["-t", "abnormal", "-p", "axial", "--prefix_name", "abc"] )

or using string with split(" ")
sys.argv.extend( "-t abnormal -p axial --prefix_name abc".split(' ') )

but if you run it in Jupyter many times with different arguments then it will remeber all arguments and you will need to remove previous arguments
sys.argv.clear()
sys.argv.extend( ["-t", "abnormal", "-p", "axial", "--prefix_name", "abc"] )

or replace all elements (except first which sometimes can be useful)
sys.argv[1:] = ["-t", "abnormal", "-p", "axial", "--prefix_name", "abc"]

if __name__ == "__main__":

    sys.argv[1:] = ["-t", "abnormal", "-p", "axial", "--prefix_name", "abc"]

    args = parse_arguments()
    run(args)

Eventually you can change function parse_arguments() to make it more universal.
You can set
def parse_arguments(arguments=None):

    # ... code ...

    args = parser.parse_args(arguments)

and then you can use it in script which you run in console/terminal
args = parse_arguments()

or in Jupyter
args = parse_arguments( ["-t", "abnormal", "-p", "axial", "--prefix_name", "abc"] )

BTW:
sys.argv.append() and sys.argv.extend() can be also useful to add some options to all executions when you run in console/terminal
There is module shlex to split arguments with spaces inside -msg "Hello World"
Normal text.split(" ") will create incorrect list ['-msg', '"Hello', 'World"']
shlex.split(text) will create correct list ['-msg', 'Hello World']


Instruction: PyTorch - Element-wise signed min/max?

Input:
I may be missing something obvious, but I can't find a way to compute this.
Given two tensors, I want to keep the minimum elements in each one of them as well as the sign.
I thought about
sign_x = torch.sign(x)
sign_y = torch.sign(y)
min = torch.min(torch.abs(x), torch.abs(y))

in order to eventually multiply the signs with the obtained minimums, but then I have no method to multiply the correct sign to each element that was kept and must choose one of the two tensors.

Output:
Here is one way to do it. Multiply torch.sign(x) and torch.sign(y) by a tensor of booleans representing whether x or y is the result of the min calculation. Then take the logical or (|) of the two resulting tensors to combine them, and multiply that by the min calculation.
mins = torch.min(torch.abs(x), torch.abs(y))

xSigns = (mins == torch.abs(x)) * torch.sign(x)
ySigns = (mins == torch.abs(y)) * torch.sign(y)
finalSigns = xSigns.int() | ySigns.int()

result = mins * finalSigns

If x and y have the same absolute value for a certain element, in the code above the sign of x takes precedence. For y to take precedence, swap the order and use finalSigns = ySigns.int() | xSigns.int() instead.


Instruction: PyTorch: What is the difference between tensor.cuda() and tensor.to(torch.device("cuda:0"))?

Input:
In PyTorch, what is the difference between the following two methods in sending a tensor (or model) to GPU:
Setup:
X = np.array([[1, 3, 2, 3], [2, 3, 5, 6], [1, 2, 3, 4]]) # X = model()
X = torch.DoubleTensor(X)

Method 1
Method 2

X.cuda()
device = torch.device("cuda:0")X = X.to(device)

(I don't really need a detailed explanation of what is happening in the backend, just want to know if they are both essentially doing the same thing)

Output:
There is no difference between the two.
Early versions of pytorch had .cuda() and .cpu() methods to move tensors and models from cpu to gpu and back. However, this made code writing a bit cumbersome:
if cuda_available:
  x = x.cuda()
  model.cuda()
else:
  x = x.cpu()
  model.cpu()

Later versions introduced .to() that basically takes care of everything in an elegant way:
device = torch.device('cuda') if cuda_available else torch.device('cpu')
x = x.to(device)
model = model.to(device)


Instruction: tensorflow's Timedistributed equivalent in pyTorch

Input:
Is there any equivalent implementation of tensorflow.keras.layers.Timedistributed for pytorch?
I am trying to build something like
Timedistributed(Resnet50()).

Output:
Credit to miguelvr on this topic.
You can use this code which is a PyTorch module developed to mimic the Timeditributed wrapper.
import torch.nn as nn

class TimeDistributed(nn.Module):
    def __init__(self, module, batch_first=False):
        super(TimeDistributed, self).__init__()
        self.module = module
        self.batch_first = batch_first

    def forward(self, x):

        if len(x.size()) <= 2:
            return self.module(x)

        # Squash samples and timesteps into a single axis
        x_reshape = x.contiguous().view(-1, x.size(-1))  # (samples * timesteps, input_size)

        y = self.module(x_reshape)

        # We have to reshape Y
        if self.batch_first:
            y = y.contiguous().view(x.size(0), -1, y.size(-1))  # (samples, timesteps, output_size)
        else:
            y = y.view(-1, x.size(1), y.size(-1))  # (timesteps, samples, output_size)

        return y


Instruction: PyTorch value thresholding and zeroing all other values

Input:
I have 2D tensor in PyTorch, representing model confidences. I want:

if 2nd value in row is greater or equal to threshold, all other values should be changed to 0
else values should not change

The simple approach would be:

iterate through rows
check 2nd value
if value is greater or equal, create row of zeroes, change 2nd value to the 2nd value from row and replace row
else don't do anything

It is inefficient, however. Is there a vectorized / tensorized way to do this?

Output:
I would do this by first constructing a new zero matrix, and then moving items from your matrix to the zero matrix as needed. You copy all rows that are in a row whose second element is below the threshold. For all other rows, you only copy the second element.
import torch
threshold = .2

X = torch.rand((100, 10))
new = torch.zeros_like(X)
mask = X[:, 2] <= threshold
new[mask] = X[mask]
new[~mask, 2] = X[~mask, 2]


Instruction: How does one use torch.optim.lr_scheduler.OneCycleLR()?

Input:
I wanted to use torch.optim.lr_scheduler.OneCycleLR() while training. Can some kindly explain to me how to use it?
What i got from the documentation was that it should be called after each train_batch.
My confusions are as follows:

Does the max_lr parameter has to be same with the optimizer lr parameter?

Can this scheduler be used with Adam optimizer. How is the momentum calculated then?

Let’s say i trained my model for some number of epochs at a stretch now, i wanted to train for some more epochs. Would i have to reset the the scheduler?

Can anybody provide me a sort of a toy example/training loop that implements this scheduler?
I am kind of new to deep learning & PyTorch so my question might be somewhat silly.

Output:
You might get some use out of this thread: How to use Pytorch OneCycleLR in a training loop (and optimizer/scheduler interactions)?
But to address your points:

Does the max_lr parameter has to be same with the optimizer lr parameter? No, this is the max or highest value -- a hyperparameter that you will experiment with. Notice in the paper the use of max_lr: https://arxiv.org/pdf/1708.07120.pdf

Can this scheduler be used with Adam optimizer. How is the momentum calculated then? Yes.

Let’s say i trained my model for some number of epochs at a stretch now, i wanted to train for some more epochs. Would i have to reset the the scheduler?  Depends, are you loading the model from a saved checkpoint or not? Check PyTorch's tutorials: https://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html#sphx-glr-beginner-blitz-neural-networks-tutorial-py


Instruction: Create a tensor view (no data copy) where one row/col is removed

Input:
TL;DR: Is there a way to delete one row or column of a torch tensor's view without creating a copy of the underlying data?
Background:
Starting from a base tensor (samples, features) of shape (2000,10000) (about 90MB), I want to create a 2-tuple of new views for each feature that consist of the one-feature-training-target with shape (2000,1) and the training inputs with shape (2000,9999) containing the remaining features.
As I want to train one rather simple model for each feature (10k) and as many as possible in parallel, I would like to have this data available in parallel, but my memory does not allow for this unless the input tensors are views.
While slicing creates views and can easily be used for the targets, so far I haven't found a slicing or function that can be used for the inputs. Creating two slices and concatenating them again creates a copy. The same happens when I use a mask to remove the unwanted feature.

Output:
Currently, this is not possible.

PyTorch's dense tensors are always represented by a dense data array and the view only changes where the indices break into the next dimension.

PyTorch's sparse tensors only do the opposite of what you intend, they allow to spread the densely stored data to be interpreted as being spread out.

That being said, you could design your models to take the (2000, 10000) sized tensor as input and effectively remove the target input-feature from the output of your first layer.
Let's say your first layer is a torch.nn.Linear which means that there is an underlying matrix of shape (10000, d_out) (where d_out is the intended output size of the layer - actually, the current implementation uses the transposed weight matrix but I'll ignore that in the examples unless it's explicitly layer.weight that's being used). So your first internal feature tensor is mathematically equivalent to:
layer_output = x.matmul(weights) + biases

which you can rid of input feature i_target by:
layer_output = x.matmul(weights) + biases - x[:,i_target].unsqueeze(dim=-1) * weights[i_target]

or for layer = torch.nn.Linear(10000, d_out):
layer_output = layer(x) - x[:,i_target].unsqueeze(dim=-1) * layer.weight[:,i_target]

So the rest of the network needs no adjustments. A similar approach is also possible for convolution layers.
Also note that this is only necessary for training. If you want your final model to take samples with 9999 features you can just erase the i_target-th column from the linear layer's weights matrix and remove the
- x[:,i_target].unsqueeze(dim=-1) * weights[i_target]

part.
Final note:
Personally I wouldn't train 10k individual models but rather have a single model with 10k outputs where each of the outputs only depends on 9999 of the input features. Of course this makes the model design much more difficult as it requires a lot of care to make sure that each layer's output contains features that don't depend on one of the input features, but it removes a lot of communication overhead that comes with processing lots of small operations on the GPU (which can be devastating in my experience).


Instruction: Is it possible to use non-pytoch augmentation in transform.compose

Input:
I am working on a data classification problem that takes images as an input in Pytorch. I would like the use the imgaug library, but unfortunatly I keep on getting errors. Here is my code.
#import necessary libraries
from torch import nn
from torchvision import models
import imgaug as ia
import imgaug.augmenters as iaa
from torchvision import datasets
from torch.utils.data.dataloader import DataLoader
from torchvision import transforms
from torch import optim
import numpy as np
from PIL import Image
import glob
from matplotlib import image

#preprocess images
#create data transformers
seq = iaa.Sequential([iaa.Sometimes(0.5,iaa.GaussianBlur(sigma=(0,3.0))),
                      iaa.Sometimes(0.5,iaa.LinearContrast((0.75,1.5))),
                      iaa.AdditiveGaussianNoise(loc=0,scale=(0.0,0.05*255),per_channel=0.5),
                      iaa.Sometimes(0.5,iaa.Affine(
        scale={"x": (0.8, 1.2), "y": (0.8, 1.2)},
        translate_percent={"x": (-0.2, 0.2), "y": (-0.2, 0.2)},
        rotate=(-25, 25),
        shear=(-8, 8)))],random_order=True)

train_transformation = transforms.Compose([transforms.RandomResizedCrop(300),
                                           seq,
                                           transforms.ToTensor()])

train_data = datasets.ImageFolder(root = 'train')
train_loader = DataLoader(train_data,shuffle = True,batch_size = 32,num_workers = 0)
train_iter = iter(train_loader)
train_iter.next()

Jupyter Server: local
Python 3.8.4 64-bit: Idle
CNN Cancer Detector
Melanoma
Intro
Skin cancer is the most common form of cancer, with 1 in 5 Americans developping it by the time they reach 70 years old. Over 2 people die of skin cancer in the US every hour.[1] Early detection is key in saving peoples lives with skin cancer, with the early detection 5 year survival rate being 99%[1]. Dermatologist have to look at patients one by one, and must assess by eye whether or not a blemish is malignant or benign. Dermatologist's have around a 66% accuracy rate in assessing 752 different skin diseases, while CNN's, such as the one detailed in *Dermatologist-level classification of skin cancer with deep neural networks* published in Nature have achieved greater accuracy levels then dermatologist's, around 72.1%[2].
By converting cancer detection to easily deployable software, you could allow people to get accurate cancer testing at home, saving resources and time. By making cancer detection more accesible, people would be more likely to get tested, saving lives in the process. Below I will detail my process and results from a melanoma (the most deadly form of skin cancer) detector model using CNN's.

[2]

from PIL import Image
import glob
from matplotlib import image

[3]

#preprocess images
#create data transformers
seq = iaa.Sequential([iaa.Sometimes(0.5,iaa.GaussianBlur(sigma=(0,3.0))),
                      iaa.Sometimes(0.5,iaa.LinearContrast((0.75,1.5))),
                      iaa.AdditiveGaussianNoise(loc=0,scale=(0.0,0.05*255),per_channel=0.5),
                      iaa.Sometimes(0.5,iaa.Affine(
        scale={"x": (0.8, 1.2), "y": (0.8, 1.2)},
        translate_percent={"x": (-0.2, 0.2), "y": (-0.2, 0.2)},
        rotate=(-25, 25),
        shear=(-8, 8)))],random_order=True)
…train_iter = iter(train_loader)
train_iter.next()
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
 in
     20 train_loader = DataLoader(train_data,shuffle = True,batch_size = 32,num_workers = 0)
     21 train_iter = iter(train_loader)
---> 22 train_iter.next()

D:\Python\lib\site-packages\torch\utils\data\dataloader.py in __next__(self)
    343
    344     def __next__(self):
--> 345         data = self._next_data()
    346         self._num_yielded += 1
    347         if self._dataset_kind == _DatasetKind.Iterable and \

D:\Python\lib\site-packages\torch\utils\data\dataloader.py in _next_data(self)
    383     def _next_data(self):
    384         index = self._next_index()  # may raise StopIteration
--> 385         data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
    386         if self._pin_memory:
    387             data = _utils.pin_memory.pin_memory(data)

D:\Python\lib\site-packages\torch\utils\data\_utils\fetch.py in fetch(self, possibly_batched_index)
     45         else:
     46             data = self.dataset[possibly_batched_index]
---> 47         return self.collate_fn(data)

D:\Python\lib\site-packages\torch\utils\data\_utils\collate.py in default_collate(batch)
     77     elif isinstance(elem, container_abcs.Sequence):
     78         transposed = zip(*batch)
---> 79         return [default_collate(samples) for samples in transposed]
     80
     81     raise TypeError(default_collate_err_msg_format.format(elem_type))

D:\Python\lib\site-packages\torch\utils\data\_utils\collate.py in (.0)
     77     elif isinstance(elem, container_abcs.Sequence):
     78         transposed = zip(*batch)
---> 79         return [default_collate(samples) for samples in transposed]
     80
     81     raise TypeError(default_collate_err_msg_format.format(elem_type))

D:\Python\lib\site-packages\torch\utils\data\_utils\collate.py in default_collate(batch)
     79         return [default_collate(samples) for samples in transposed]
     80
---> 81     raise TypeError(default_collate_err_msg_format.format(elem_type))

TypeError: default_collate: batch must contain tensors, numpy arrays, numbers, dicts or lists; found

I am aware that the input to the imgaug transformer must be a numpy array, but I am not sure how to incorporate that into my transform.compose (if I can at all that is.). When the imgaug seq is not in the transform.compose it works properly.
Thank you for the help!

Output:
Looking at the documentation of transforms in pytorch gives us a hint of how to do it: https://pytorch.org/docs/stable/torchvision/transforms.html#generic-transforms
I would try something like:
train_transformation = transforms.Compose([transforms.RandomResizedCrop(300),
                                           transforms.Lambda(lambda x: seq(x)),
                                           transforms.ToTensor()])


Instruction: Why does an indexed tensor retain the original tensor's stride?

Input:
My understanding of stride is the number of steps you'd need to take to jump to the next part in an axis. So if you have a stride of (3,1) you'd need to take 3 steps to get to the next row and 1 step to get to the next column (assuming the first axis is row and the second axis is column).
Yet when I index into a pytorch tensor, b[1:, 1:] with shape (3,3) lopping off the first row
and the first column, then query its stride, I get (3,1) instead of (2,1).
Why is this the case?
import unittest
import torch
class TestChapter3(unittest.TestCase):
    def setUp(self):
        self.a = torch.tensor(list(range(9)))
        self.b = self.a.view(3,3)
        self.c = self.b[1:, 1:]

    def test_index_of_view(self):
        print(self.c)
        self.assertEqual(self.c.size(), torch.Size([2, 2]))
        self.assertEqual(self.c.storage_offset(), 4)
        self.assertEqual(self.c.stride(), (2, 1)) # self.c.stride() is actually (3,1)

if __name__ == "__main__":
    unittest.main()

Output:
The self.c tensor still uses the same underlaying data storage as self.a and self.b. They are all just different views on the same storage.
self.c.storage_offset() tells you where the view of self.c begins (passing the first entire row and the first element of the second row = 4 elements in total) but the data is still stored in memory as one long array.
Since the underlying storage is still the same, the memory address to go one row down is still 3*sizeof(float) bytes or three elements further, which is what self.c.stride()[0] says.
In your example we have:
self.a = (visible=[0, 1, 2, 3, 4, 5, 6, 7, 8], data=[0, 1, 2, 3, 4, 5, 6, 7, 8])
self.b = (visible=[
    [0, 1, 2],
    [3, 4, 5],
    [6, 7, 8]
], data=[0, 1, 2, 3, 4, 5, 6, 7, 8])
self.c = (visible=[

       [4, 5],
       [7, 8]
], data=[4, 5, 6, 7, 8])

And yes, the 6 in the last data array is not a mistake. The last data array is just an offset view of the same memory that self.a and self.b are referencing.


Instruction: TypeError: pic should be PIL Image or ndarray. Got

Input:
I would like to split images using Open CV then feed to the by PyTorch model to count object in each image. I am getting the following error message when running the code: TypeError: pic should be PIL Image or ndarray. Got <class ‘bool’>
import os

import numpy as np
import torch
from PIL import Image
from torch.utils.data import Dataset
from torchvision import transforms, utils
#from torchvision.transforms import Grayscalei
import pandas as pd
import pdb
import cv2

class CellsDataset(Dataset):
    # a very simple dataset

    def __init__(self, root_dir, transform=None, return_filenames=False):
        self.root = root_dir
        self.transform = transform
        self.return_filenames = return_filenames
        self.files = [os.path.join(self.root,filename) for filename in os.listdir(self.root)]
        self.files = [path for path in self.files
                      if os.path.isfile(path) and os.path.splitext(path)[1]=='.png']

    def __len__(self):
        return len(self.files)

    def __getitem__(self, idx):
        path = self.files[idx]
        sample = Image.open(path)

        sample = cv2.imread(path)
        b,g,r=cv2.split(sample)
        sample=cv2.imwrite('sample.png', g)

        #transform3 = Grayscale(num_output_channels=3)
        #sample = transform3(sample) # convert to a 3 channel grayscale, as it needs to be 3 channel.
        if self.transform:
            sample = self.transform(sample)

        if self.return_filenames:
            return sample, path
        else:
            return sample

Output:
The variable sample is reassigned to what cv2.imwrite returns in this line:
sample=cv2.imwrite('sample.png', g)

That replaces your ndarray with something else.
Try replacing that line with simply cv2.imwrite('sample.png', g) without assigning the value of the evaluated expression to sample.


Instruction: How to adjust the batch data by the amount of labels in PyTorch

Input:
I have made n-grams / doc-ids for document classification,
def create_dataset(tok_docs, vocab, n):
  n_grams = []
  document_ids = []
  for i, doc in enumerate(tok_docs):
    for n_gram in [doc[0][i:i+n] for i in range(len(doc[0]) - 1)]:
       n_grams.append(n_gram)
       document_ids.append(i)
  return n_grams, document_ids

def create_pytorch_datasets(n_grams, doc_ids):
  n_grams_tensor = torch.tensor(n_grams)
  doc_ids_tensor = troch.tensor(doc_ids)
  full_dataset = TensorDataset(n_grams_tensor, doc_ids_tensor)
  return full_dataset

create_dataset returns pair of (n-grams, document_ids) like below:
n_grams, doc_ids = create_dataset( ... )
train_data = create_pytorch_datasets(n_grams, doc_ids)
>>> train_data[0:100]
(tensor([[2076, 517, 54, 3647, 1182, 7086],
         [517, 54, 3647, 1182, 7086, 1149],
         ...
         ]),
 tensor(([0, 0, 0, 0, 0, ..., 3, 3, 3]))

train_loader = DataLoader(train_data, batch_size = batch_size, shuffle = True)

The first of tensor content means n-grams and the second one does doc_id.
But as you know, by the length of documents, the amount of training data according to the label would changes.
If one document has very long length, there would be so many pairs that have its label in training data.
I think it can cause overfitting in model, because the classification model tends to classify inputs to long length documents.
So, I want to extract input batches from a uniform distribution for label (doc_ids). How can I fix it in code above?
p.s)
If there is train_data like below, I want to extract batch by the probability like that:
  n-grams        doc_ids
([1, 2, 3, 4],      1)       ====> 0.33
([1, 3, 5, 7],      2)       ====> 0.33
([2, 3, 4, 5],      3)       ====> 0.33 * 0.25
([3, 5, 2, 5],      3)       ====> 0.33 * 0.25
([6, 3, 4, 5],      3)       ====> 0.33 * 0.25
([2, 3, 1, 5],      3)       ====> 0.33 * 0.25

Output:
In pytorch you can specify a sampler or a batch_sampler to the dataloader to change how the sampling of datapoints is done.
docs on the dataloader:
https://pytorch.org/docs/stable/data.html#data-loading-order-and-sampler
documentation on the sampler: https://pytorch.org/docs/stable/data.html#torch.utils.data.Sampler
For instance, you can use the WeightedRandomSampler to specify a weight to every datapoint. The weighting can be the inverse length of the document for instance.
I would make the following modifications in the code:
def create_dataset(tok_docs, vocab, n):
  n_grams = []
  document_ids = []
  weights = []  # << list of weights for sampling
  for i, doc in enumerate(tok_docs):
    for n_gram in [doc[0][i:i+n] for i in range(len(doc[0]) - 1)]:
       n_grams.append(n_gram)
       document_ids.append(i)
       weights.append(1/len(doc[0]))  # << ngrams of long documents are sampled less often
  return n_grams, document_ids, weights

sampler = WeightedRandomSampler(weights, 1, replacement=True) # << create the sampler

train_loader = DataLoader(train_data, batch_size=batch_size, shuffle=False, sampler=sampler)  # << includes the sampler in the dataloader


Instruction: GPU memory leakage when creating objects from sentence-transformers

Input:
Description
I am creating a function in R that embeds sentences using the sentence_transformers library from Python.
For some unknown reason, creating the object multiple times under the same variable name ends up in insufficient memory space to allocate the transformer. To reproduce:
sentence_transformers <- reticulate::import("sentence_transformers")
for (i in 1:10) {
  print(i)
  bert_encoder <- sentence_transformers$SentenceTransformer("bert-large-nli-stsb-mean-tokens")
}

However, doing the same operation directly on Python does not produce an error
from sentence_transformers import SentenceTransformer
for i in range(10):
    print(i)
    bert_encoder = SentenceTransformer("bert-large-nli-stsb-mean-tokens")
}

This happens with any model that is allocated in GPU. On my NVIDIA GTX 1060 it reaches the 4th cycle, but on smaller GPUs it crashes earlier. One temporal solution is to create the model outside only once, and then pass the model as a parameter to the function as many times as wanted, but I would rather avoid that because it adds an extra step and in any case calling multiple models might just make it crash as well.
Expected behaviour
The for loop finishes without an error
Observed behaviour
Error in py_call_impl(callable, dots$args, dots$keywords) :
RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 6.00 GiB total capacity; 2.95 GiB already allocated; 16.11 MiB free; 238.68 MiB cached)
Unsuccesful attemps at solving it

The solutions proposed here
Using numba as suggested here
Declaring the variable explicitely on Python via reticulate::py_run_string() and then doing del bert_encoder and calling the garbage collector

Details
Windows 10 Home
Python 3.7.4
R 4.0.1
Reticulate 1.16
Torch 1.3.1
Tensorflow 2.2.0
Transformers 2.11.0
sentence_transformers 0.2.6

Output:
Ok so I am posting my solution for anyone else having this issue.
After each call to the model as
sentence_transformers <- import("sentence_transformers")
encoder <- sentence_transformers$SentenceTransformer("bert-large-nli-stsb-mean-tokens")

I release GPU memory using
  # Has this been done on a GPU?
  py <- reticulate::py_run_string("import torch
is_cuda_available = torch.cuda.is_available()")

  # Release GPU
  if (isTRUE(reticulate::py$is_cuda_available)) {

    tryCatch(reticulate::py_run_string("del encoder"),
             warning = function(e) {},
             error = function(e) {})

    tryCatch(rm(encoder),
             warning = function(e) {},
             error = function(e) {})

    gc(full = TRUE, verbose = FALSE)

    py <- reticulate::py_run_string("import torch
torch.cuda.empty_cache()")

  }

and it works perfectly.


Instruction: Pytorch geometric Data object edge_attr for undirected graphs

Input:
How would one construct a edge_attr list for undirected graphs in pytorch geometric Data object. Say we had an undirected graph such as this one. The graph COO matrix required by the pytorch geometric object is:

[['a', 'b', 'a', 'c', 'b', 'c']  ['b', 'a', 'c', 'a', 'c', 'b']]

How would one construct the edge_attr list then (which is an array of one-hot encoded vectors for the features of each edge). Since the graph is undirected, would one simply append two of the same one-hot encoded feature vectors at a time. For example, say these are the edge features :

(a, b) = [0,0,0,1]  (a, c) = [1,0,0,0]  (b, c) = [0,1,0,0]

Would the edge_attr list look like:

[[0,0,0,1], [0,0,0,1], [1,0,0,0], [1,0,0,0], [0,1,0,0], [0,1,0,0]

Note how each one-hot encoded feature vector is repeated twice, and the feature vector index in edge_attr corresponds its respective edge in the graph COO matrix. Because the graph is undirected, we just use the same feature matrix. Is the right way to do it, or is there some other way?

Output: The answer I figured out is yes. In an undirected graph for pytorch_geometric, the edge features would need to be repeated twice, once for each respective connection present in the COO matrix.


Instruction: Pytorch dynamic amount of Layers?

Input:
I am trying to specify a dynamic amount of layers, which I seem to be doing wrong.
My issue is that when I define the 100 layers here, I will get an error in the forward step.
But when I define the layer properly it works?
Below simplified example
class PredictFromEmbeddParaSmall(LightningModule):
    def __init__(self, hyperparams={'lr': 0.0001}):
        super(PredictFromEmbeddParaSmall, self).__init__()
        #Input is something like tensor.size=[768*100]
        self.TO_ILLUSTRATE = nn.Linear(768, 5)
        self.enc_ref=[]
        for i in range(100):
            self.enc_red.append(nn.Linear(768, 5))
        # gather the layers output sth
        self.dense_simple1 = nn.Linear(5*100, 2)
        self.output = nn.Sigmoid()
    def forward(self, x):
        # first input to enc_red
        x_vecs = []
        for i in range(self.para_count):
            layer = self.enc_red[i]
            # The first dim is the batch size here, output is correct
            processed_slice = x[:, i * 768:(i + 1) * 768]
            # This works and give the out of size 5
            rand = self.TO_ILLUSTRATE(processed_slice)
            #This will fail? Error below
            ret = layer(processed_slice)
            #more things happening we can ignore right now since we fail earlier

I get this error when executing "ret = layer.forward(processed_slice)"

RuntimeError: Expected object of device type cuda but got device type
cpu for argument #1 'self' in call to _th_addmm

Is there a smarter way to program this? OR solve the error?

Output:
You should use a ModuleList from pytorch instead of a list: https://pytorch.org/docs/master/generated/torch.nn.ModuleList.html . That is because Pytorch has to keep a graph with all modules of your model, if you just add them in a list they are not properly indexed in the graph, resulting in the error you faced.
Your coude should be something alike:
class PredictFromEmbeddParaSmall(LightningModule):
    def __init__(self, hyperparams={'lr': 0.0001}):
        super(PredictFromEmbeddParaSmall, self).__init__()
        #Input is something like tensor.size=[768*100]
        self.TO_ILLUSTRATE = nn.Linear(768, 5)
        self.enc_ref=nn.ModuleList()                     # << MODIFIED LINE <<
        for i in range(100):
            self.enc_red.append(nn.Linear(768, 5))
        # gather the layers output sth
        self.dense_simple1 = nn.Linear(5*100, 2)
        self.output = nn.Sigmoid()
    def forward(self, x):
        # first input to enc_red
        x_vecs = []
        for i in range(self.para_count):
            layer = self.enc_red[i]
            # The first dim is the batch size here, output is correct
            processed_slice = x[:, i * 768:(i + 1) * 768]
            # This works and give the out of size 5
            rand = self.TO_ILLUSTRATE(processed_slice)
            #This will fail? Error below
            ret = layer(processed_slice)
            #more things happening we can ignore right now since we fail earlier

Then it should work all right!
Edit: alternative way.
Instead of using ModuleList you can also just use nn.Sequential, this allows you to avoid using the for loop in the forward pass. That also means that you will not have access to intermediary activations, so that is not the solution for you if you need them.
class PredictFromEmbeddParaSmall(LightningModule):
    def __init__(self, hyperparams={'lr': 0.0001}):
        super(PredictFromEmbeddParaSmall, self).__init__()
        #Input is something like tensor.size=[768*100]
        self.TO_ILLUSTRATE = nn.Linear(768, 5)
        self.enc_ref=[]
        for i in range(100):
            self.enc_red.append(nn.Linear(768, 5))

        self.enc_red = nn.Seqential(*self.enc_ref)       # << MODIFIED LINE <<
        # gather the layers output sth
        self.dense_simple1 = nn.Linear(5*100, 2)
        self.output = nn.Sigmoid()
    def forward(self, x):
        # first input to enc_red
        x_vecs = []
        out = self.enc_red(x)                            # << MODIFIED LINE <<


Instruction: Pytorch Import Error when deploying django on AWS

Input:
My django app works locally. When i try to deploy it and then open it on AWS, i get the following error:

I included torch==1.5.1 in my requirements.txt
Requirements.txt:

When I comment out the parts of the application that require torch and remove it from requirements.txt, it works fine. What can I do to ensure that torch is installed like the other modules in requirements.txt? Here is the link to requirements.txt

Output:
By default torch will require CUDA and GPU. Depending on your ec2 instance this may not be viable option, thus leading to the errors you are observing.
You can install non-cuda version of torch as follows for tests and see if this will have any benefits:
pip3 install torch==1.5.1+cpu torchvision==0.6.1+cpu -f https://download.pytorch.org/whl/torch_stable.html


Instruction: Problem with output of neural network in a cross-entropy method attempt at solving CartPole-v0

Input:
I am trying to implement the cross-entropy policy-based method to the classic CartPole-v0 environment. I am actually reformatting a working implementation of this algorithm on the MountainCarContinuous-v0, but when I try to get the agent learning, I get this error message:
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
 in
      4
      5 agent = Agent(env)
----> 6 scores = agent.learn()
      7
      8 # plot the scores

~/cross_entropy.py in learn(self, n_iterations, max_t, gamma, print_every, pop_size, elite_frac, sigma)
     83         for i_iteration in range(1, n_iterations+1): # loop over all the training iterations
     84             weights_pop = [best_weight + (sigma*np.random.randn(self.get_weights_dim())) for i in range(pop_size)] # population of the weights/policies
---> 85             rewards = np.array([self.evaluate(weights, gamma, max_t) for weights in weights_pop]) # rewards from the policies resulting from all individual weights
     86
     87             # get the best policies

~/cross_entropy.py in (.0)
     83         for i_iteration in range(1, n_iterations+1): # loop over all the training iterations
     84             weights_pop = [best_weight + (sigma*np.random.randn(self.get_weights_dim())) for i in range(pop_size)] # population of the weights/policies
---> 85             rewards = np.array([self.evaluate(weights, gamma, max_t) for weights in weights_pop]) # rewards from the policies resulting from all individual weights
     86
     87             # get the best policies

~/cross_entropy.py in evaluate(self, weights, gamma, max_t)
     56             action = self.forward(state)
     57             #action = torch.argmax(action_vals).item()
---> 58             state, reward, done, _ = self.env.step(action)
     59             episode_return += reward * math.pow(gamma, t)
     60             if done:

/gym/wrappers/time_limit.py in step(self, action)
     14     def step(self, action):
     15         assert self._elapsed_steps is not None, "Cannot call env.step() before calling reset()"
---> 16         observation, reward, done, info = self.env.step(action)
     17         self._elapsed_steps += 1
     18         if self._elapsed_steps >= self._max_episode_steps:

/gym/envs/classic_control/cartpole.py in step(self, action)
    102     def step(self, action):
    103         err_msg = "%r (%s) invalid" % (action, type(action))
--> 104         assert self.action_space.contains(action), err_msg
    105
    106         x, x_dot, theta, theta_dot = self.state

AssertionError: tensor([ 0.3987,  0.6013]) () invalid

I found this is because the MountainCarContinuous-v0 environment has an action_space of type Box(2) whereas CartPole-v0 is Discrete(2), meaning that I only want an integer as action selection.
I have tried working around this notion by applying a softmax activation function and then took the index of the higher value as the action.
action_vals = self.forward(state)
action = torch.argmax(action_vals).item()

This gets rid of the error but when I train the agent, it seems to learn incredibly fast which is kind of an indicator that something is wrong. This is my full agent class:
class Agent(nn.Module):
    def __init__(self, env, h_size=16):
        super().__init__()
        self.env = env
        # state, hidden layer, action sizes
        self.s_size = env.observation_space.shape[0]
        self.h_size = h_size
        self.a_size = env.action_space.n
        # define layers
        self.fc1 = nn.Linear(self.s_size, self.h_size)
        self.fc2 = nn.Linear(self.h_size, self.a_size)

        self.device = torch.device('cpu')

    def set_weights(self, weights):
        s_size = self.s_size
        h_size = self.h_size
        a_size = self.a_size
        # separate the weights for each layer
        fc1_end = (s_size*h_size)+h_size
        fc1_W = torch.from_numpy(weights[:s_size*h_size].reshape(s_size, h_size))
        fc1_b = torch.from_numpy(weights[s_size*h_size:fc1_end])
        fc2_W = torch.from_numpy(weights[fc1_end:fc1_end+(h_size*a_size)].reshape(h_size, a_size))
        fc2_b = torch.from_numpy(weights[fc1_end+(h_size*a_size):])
        # set the weights for each layer
        self.fc1.weight.data.copy_(fc1_W.view_as(self.fc1.weight.data))
        self.fc1.bias.data.copy_(fc1_b.view_as(self.fc1.bias.data))
        self.fc2.weight.data.copy_(fc2_W.view_as(self.fc2.weight.data))
        self.fc2.bias.data.copy_(fc2_b.view_as(self.fc2.bias.data))

    def get_weights_dim(self):
        return (self.s_size+1)*self.h_size + (self.h_size+1)*self.a_size

    def forward(self, x):
        x = F.relu(self.fc1(x))
        x = F.softmax(self.fc2(x))
        return x

    def evaluate(self, weights, gamma=1.0, max_t=5000):
        self.set_weights(weights)
        episode_return = 0.0
        state = self.env.reset()
        for t in range(max_t):
            state = torch.from_numpy(state).float().to(self.device)
            action_vals = self.forward(state)
            action = torch.argmax(action_vals).item()
            state, reward, done, _ = self.env.step(action)
            episode_return += reward * math.pow(gamma, t)
            if done:
                break
        return episode_return

    def learn(self, n_iterations=500, max_t=1000, gamma=1.0, print_every=10, pop_size=50, elite_frac=0.2, sigma=0.5):
        """PyTorch implementation of the cross-entropy method.

        Params
        ======
            n_iterations (int): maximum number of training iterations
            max_t (int): maximum number of timesteps per episode
            gamma (float): discount rate
            print_every (int): how often to print average score (over last 100 episodes)
            pop_size (int): size of population at each iteration
            elite_frac (float): percentage of top performers to use in update
            sigma (float): standard deviation of additive noise
        """
        n_elite=int(pop_size*elite_frac) # number of elite policies from the population

        scores_deque = deque(maxlen=100) # list of the past 100 scores
        scores = [] # list of all the scores
        best_weight = sigma*np.random.randn(self.get_weights_dim()) # initialize the first best weight randomly

        for i_iteration in range(1, n_iterations+1): # loop over all the training iterations
            weights_pop = [best_weight + (sigma*np.random.randn(self.get_weights_dim())) for i in range(pop_size)] # population of the weights/policies
            rewards = np.array([self.evaluate(weights, gamma, max_t) for weights in weights_pop]) # rewards from the policies resulting from all individual weights

            # get the best policies
            ##
            elite_idxs = rewards.argsort()[-n_elite:]
            elite_weights = [weights_pop[i] for i in elite_idxs]
            ##

            best_weight = np.array(elite_weights).mean(axis=0) # take the average of the best weights

            reward = self.evaluate(best_weight, gamma=1.0) # evaluate this new policy
            scores_deque.append(reward) # append the reward
            scores.append(reward) # also append the reward

            torch.save(self.state_dict(), 'checkpoint.pth') # save the agent

            if i_iteration % print_every == 0: # print every 100 steps
                print('Episode {}\tAverage Score: {:.2f}'.format(i_iteration, np.mean(scores_deque)))

            if np.mean(scores_deque)>=195.0: # print if environment is solved
                print('\nEnvironment solved in {:d} iterations!\tAverage Score: {:.2f}'.format(i_iteration-100, np.mean(scores_deque)))
                break
        return scores

If anyone has an idea on how to get the agent training properly, please give me any suggestions.

Output:
Turns out all I needed was to add an act() method to the Agent class.
def act(self, state):
        state = state.unsqueeze(0)
        probs = self.forward(state).cpu()
        m = Categorical(probs)
        action = m.sample()
        return action.item()


Instruction: Pytorch convert a pd.DataFrame which is variable length sequence to tensor

Input:
I get a pandas DataFrame as follows and want to convert it to torch.tensor for embedding.
# output first 5 rows examples
print(df['col'].head(5))

                      col
0             [a, bc, cd]
1      [d, ed, fsd, g, h]
2  [i, hh, ihj, gfw, hah]
3                 [a, cb]
4                   [sad]

train_tensor = torch.from_numpy(train)

But it gets an error:

TypeError: can't convert np.ndarray of type numpy.str_. The only supported types are: float64, float32, float16, int64, int32, int16, int8, uint8, and bool.

It seems that from_numpy() doesn't support the variable lenght sequences.
So if want to initialize tensor form it what is the proper way?
And after getting the corresponding tensor I will try to add padding to variable length sequences and do embedding layer for it.
Could anyone help me?
Thanks in advances.

Output:
There are multiple steps involved here
words to IDs

Pretrained: If you are using a pretrained embeddings like Glove/word2vec you will have to map each word to its ID in the vocabulary so that the embedding layer can load the pretrained embeddings.
In case you want to train your own embeddings you will have to map each word to an ID and save the map for later use (during predictions). This is normally called vocabulary

# Vocabulary to our own ID
def to_vocabulary_id(df):
  word2id = {}
  sentences = []
  for v in df['col'].values:
    row = []
    for w in v:
      if w not in word2id:
        word2id[w] = len(word2id)+1
      row.append(word2id[w])

    sentences.append(row)
  return sentences, word2id

df = pd.DataFrame({'col': [
                           ['a', 'bc', 'cd'],
                           ['d', 'ed', 'fsd', 'g', 'h'],
                           ['i', 'hh', 'ihj', 'gfw', 'hah'],
                           ['a', 'cb'],
                           ['sad']]})
sentences, word2id = to_vocabulary_id(df)

Embedding layer
If our vocabulary size is say 100 and embedding size is 8, then we will create an embedding layer as below
embedding = nn.Embedding(100, 8)

Pad variable length sentences to 0 and create Tensor
data = pad_sequence([torch.LongTensor(s) for s in sentences], batch_first=True, padding_value=0)

Run through the embedding layer
Finally
import torch
from torch.nn.utils.rnn import pad_sequence

data = pad_sequence([torch.LongTensor(s) for s in sentences], batch_first=True, padding_value=0)

embedding = nn.Embedding(100, 8)
embedding(data).shape

Output:
torch.Size([5, 5, 8])

As you can see we have passed 5 sentences and the max length is 5. So we get embeddings of size 5 X 5 X 8 ie. 5 sentences, 5 words each one having embedding of size 8.


Instruction: Accuracy per epoch in PyTorch

Input:
I have made a chatbot using pytorch and would like to display accuracy on every epoch. I am not quite understanding how to do that. I can display loss but cant figure out how to display my accuracy
Here is my code :-
from nltk_utils import tokenize, stem, bag_of_words
import json
import numpy as np
import torch
import torch.nn as nn
from torch.utils.data import Dataset, DataLoader
from model import NeuralNet
from torch.autograd import Variable

all_words=[]
tags=[]
xy=[]
questionsP1=[]
questionsP2=[]
questionsP3=[]
questionsP4=[]
questionTag={}

with open('new.json', encoding="utf8") as file:
        data = json.load(file)

for intent in data["intents"]:
    for proficiency in intent["proficiency"]:
        for questions in proficiency["questions"]:
            for responses in questions["responses"]:
                wrds = tokenize(responses)
                all_words.extend(wrds)
                xy.append((wrds, questions["tag"]))

            if questions["tag"] in tags:
                print(questions["tag"])

            if questions["tag"] not in tags:
                tags.append(questions["tag"])

            if proficiency["level"] == "P1":
                questionsP1.append(questions["question"])
                questionTag[questions["question"]]=questions["tag"]

            if proficiency["level"] == "P2":
                questionsP2.append(questions["question"])
                questionTag[questions["question"]]=questions["tag"]

            if proficiency["level"] == "P3":
                questionsP3.append(questions["question"])
                questionTag[questions["question"]]=questions["tag"]

            if proficiency["level"] == "P4":
                questionsP4.append(questions["question"])
                questionTag[questions["question"]]=questions["tag"]

ignore_words = ['?', '!', '.', ',']
all_words = [stem(x) for x in all_words if x not in ignore_words]
all_words = sorted(set(all_words))
tags = sorted(set(tags))

X_train = []
y_train = []

for tokenized_response, tag in xy:
    bag = bag_of_words(tokenized_response, all_words)
    print(bag)
    X_train.append( bag )

    label = tags.index( tag )
    y_train.append( label )
print(y_train)

X_train = np.array( X_train )
y_train = np.array( y_train )

class ChatDataset(Dataset):
    def __init__(self):
        self.n_samples = len(X_train)
        self.x_data = X_train
        self.y_data = y_train

    def __getitem__(self, index):
        return self.x_data[index], self.y_data[index]

    def __len__(self):
        return self.n_samples

#HyperParameters
batch_size = 8
hidden_size = 8
output_size = len(tags)
input_size = len(X_train[0])
learning_rate = 0.001
num_epochs = 994

dataset = ChatDataset()
train_loader = DataLoader(dataset = dataset, batch_size=batch_size, shuffle = True, num_workers = 2)

device = 'cpu'
model = NeuralNet(input_size, hidden_size, output_size).to(device)

#loss and optimizer
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr = learning_rate)

for epoch in range( num_epochs ):
    for (words, labels) in train_loader:
        words = words.to(device)
        labels = labels.to(device)

        #Forward
        outputs = model(words)
        loss = criterion(outputs, labels)

        #backward and optimizer step
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

        print(f'epoch {epoch + 1}/ {num_epochs}, loss={loss.item(): .4f}')

print(f'final loss, loss={loss.item(): .4f}')

data = {
    "model_state": model.state_dict(),
    "input_size": input_size,
    "output_size": output_size,
    "hidden_size": hidden_size,
    "all_words": all_words,
    "tags": tags,
}

FILE = "data.pth"
torch.save(data, FILE)

with open('new.json', 'r') as f:
    intents = json.load(f)

bot_name = "Sam"
while True:
    sentence = input("You: ")
    if sentence == 'quit':
        break
    sentence = tokenize(sentence)
    X = bag_of_words(sentence, all_words)
    X = X.reshape( 1, X.shape[0])
    X = torch.from_numpy( X )

    output = model( X )
    _, predicted = torch.max(output, dim=1)
    tag = tags[predicted.item()]
    print(tag)

    probs = torch.softmax(output, dim=1)
    probs = probs[0][predicted.item()]

    print( probs.item() )

    if probs.item() > 0.75:
        for intent in intents["intents"]:
            for proficiency in intent["proficiency"]:
                for questions in proficiency["questions"]:
                    if questions["tag"] == tag:
                        print(f'{bot_name}: {questions["question"]}')
    else:
        print(f'{bot_name}: Probability Too Low')

print(f'Training Complete. File saved to {FILE}')

My chatbot is working inverselt... i am trying to map the answer to the right question.
Any help would be appreciated.

Output:
According to your code labels contains the indices that should have the highest values in outputs in order for the samples to be counted as correct predictions.
So to calculate the validation accuracy:
correct = 0
total = 0
model.eval()
with torch.no_grad():
    for (words, labels) in validation_loader:
        words = words.to(device)
        labels = labels.to(device)
        total += labels.shape[0]

        outputs = model(words)

        correct += torch.sum(labels == outputs.argmax(dim=-1))

accuracy = correct / total


Instruction: How to get count number of equal elements in two torch tensors that also equal a specific value

Input:
I'm working in pytorch and trying to count the number of equal elements in 2 torch tensors, that also equal a specific value.
That is, if tensor a=[0,1,2,0,1,2] and tensor b = [0,2,1,0,2,1]
I want it to return:

3 when I check how many element are equal in the a,b and also equals 0. sum(a == b and a == 0 and b == 0) = 2,
0 when I check how many element are equal in the a,b and also equals 1 or 2. sum(a == b and a == 1,2 and b == 1,2) = 0.

Thanks!

Output:
A straight forward approach would be:
torch.sum((a==b) * (a==val))

if val is the value you look for.