Instruction: where I can deploy ml algorithm without time out?

Input:
I used Heroku to deploy ml algorithm with Pytorch but when I sent request to app to train model with 100 epochs then I got this
error

at=error code=H12 desc="Request Timeout" method=POST path="/train"

I searched about request time out and I find that the limit to request time
from Heroku is 30 sec
What's the solution of time out problem and is there an platform for training model without time out?

Output:
Model training can be an expensive and lengthy operation that can take from minutes to hours (depending on context and amount of data), so it is likely that you will have a request timeout on other providers too.
The solution is to make the model train a background task:

create a worker Dyno which schedule the model training (ie everyday at h11:00)

OR

keep the web Dyno but make sure the /train request spawns a background thread (hence the web request completes quickly and the training happens in background)


Instruction: Torch: how to concatenate tensors of different sizes?

Input:
I have two tensors:
rc of size: torch.Size([128, 16, 1])
xt of size: torch.Size([128, 40, 1])

I would like to concatenate xt to rc along dimension 2 so that the final size of rc_xt is:
rc_xt = torch.Size([128, 40, 2])

In short, I want to 'increase' the size of rc's dimension 1 (16) to 40 -- through any means, even just repeating elements -- and then concatenate them both along dimension 2.
I have tried to google how to do this but I cannot get it working no matter what I do, I'm a little confused about how to go about doing this.
Thank you in advance.

Output:
"Increasing" the size of rc can be done simply by padding.
For instance, you can pad it by zeros:
p_rc = nnf.pad(rc, (0, 0, 0, xt.shape[1]-rc.shape[1], 0, 0), 'constant', 0)

Once you have a padded version of rc you can concat:
rc_xt = torch.concat((p_rc, xt), dim=-1)


Instruction: Variational Autoencoder (VAE) returns consistent output

Input:
I'm working on the signal compression and reconstruction with VAE. I've trained 1600 fragments but the values of 1600 reconstructed signals are very similar. Moreover, results from same batch are almost consistent. As using the VAE, loss function of the model contains binary cross entropy (BCE) and the output of the train model should be located between 0 to 1 (The input data also normalized to 0~1).
VAE model(LSTM) :
class LSTM_VAE(nn.Module):
def __init__(self,
             input_size=3000,
             hidden=[1024, 512, 256, 128, 64],
             latent_size=64,
             num_layers=8,
             bidirectional=True):
    super().__init__()

    self.input_size = input_size
    self.hidden = hidden
    self.latent_size = latent_size
    self.num_layers = num_layers
    self.bidirectional = bidirectional

    self.actv = nn.LeakyReLU()

    self.encode = nn.LSTM(input_size=self.input_size,
                          hidden_size=self.hidden[0],
                          num_layers=self.num_layers,
                          batch_first=True,
                          bidirectional=True)
    self.bn_encode = nn.BatchNorm1d(1)

    self.decode = nn.LSTM(input_size=self.latent_size,
                          hidden_size=self.hidden[2],
                          num_layers=self.num_layers,
                          batch_first=True,
                          bidirectional=True)
    self.bn_decode = nn.BatchNorm1d(1)

    self.fc1 = nn.Linear(self.hidden[0]*2, self.hidden[1])
    self.fc2 = nn.Linear(self.hidden[1], self.hidden[2])
    self.fc31 = nn.Linear(self.hidden[2], self.latent_size)
    self.fc32 = nn.Linear(self.hidden[2], self.latent_size)
    self.bn1 = nn.BatchNorm1d(1)
    self.bn2 = nn.BatchNorm1d(1)
    self.bn3 = nn.BatchNorm1d(1)

    self.fc4 = nn.Linear(self.hidden[2]*2, self.hidden[1])
    self.fc5 = nn.Linear(self.hidden[1], self.hidden[0])
    self.fc6 = nn.Linear(self.hidden[0], self.input_size)
    self.bn4 = nn.BatchNorm1d(1)
    self.bn5 = nn.BatchNorm1d(1)
    self.bn6 = nn.BatchNorm1d(1)

def encoder(self, x):
    x = torch.unsqueeze(x, 1)
    x, _ = self.encode(x)
    x = self.actv(x)
    x = self.fc1(x)
    x = self.actv(x)
    x = self.fc2(x)
    x = self.actv(x)

    mu = self.fc31(x)
    log_var = self.fc32(x)

    return mu, log_var

def decoder(self, z):
    z, _ = self.decode(z)
    z = self.bn_decode(z)
    z = self.actv(z)
    z = self.fc4(z)
    z = self.bn4(z)
    z = self.fc5(z)
    z = self.bn5(z)
    z = self.fc6(z)
    z = self.bn6(z)
    z = torch.sigmoid(z)

    return torch.squeeze(z)

def sampling(self, mu, log_var):
    std = torch.exp(0.5 * log_var)
    eps = torch.randn_like(std)

    return mu + eps * std

def forward(self, x):
    mu, log_var = self.encoder(x.view(-1, self.input_size))
    z = self.sampling(mu, log_var)
    z = self.decoder(z)

    return z, mu, log_var

Loss function and Train code :
def lossF(recon_x, x, mu, logvar, input_size):
BCE = F.binary_cross_entropy(recon_x, x.view(-1, input_size), reduction='sum')
KLD = -0.5 * torch.sum(1 + logvar - mu.pow(2) - logvar.exp())

return BCE + KLD

optim = torch.optim.Adam(model.parameters(), lr=opt.lr)

for epoch in range(opt.epoch):
    for batch_idx, data in enumerate(train_set):
        data = data.to(device)
        optim.zero_grad()
        recon_x, mu, logvar = model(data)
        loss = lossF(recon_x, data, mu, logvar, opt.input_size)
        loss.backward()
        train_loss += loss.item()
        optim.step()

I built the code by refer the example codes of others and only changed very few parameters. I rebuilt the code, change the dataset, update parameters but nothing worked. If you have any suggestion to solve this problem, PLEASE let me know.

Output:
I've find out the reason of the issue. It turns out that the decoder model derives output value in the range of 0.4 to 0.6 to stabilize the BCE loss. BCE loss can't be 0 even if the prediction is correct to answer. Also the loss value is non-linear to the range of the output. The easiest way to lower the loss is give 0.5 for the output, and my model did.
To avoid this error, I standardize my data and added some outlier data to avoid BCE issue. VAE is such complicated network for sure.


Instruction: Having problems with linear pytorch model initialization

Input:
I am trying to figure out what is wrong with my initialization of the neural network model. I have already set a pdb trace to see that the defining neural network part is the source of error. Also, I get yellow marks on the defining neural network code because the module is expected to be returned but if I return the module, it causes a recursion error. It is a linear model that has to have an input dimension of the batch size * 81 and an output dimension of the batch size * 1. I am relatively new at pytorch and defining deep neural networks so this may not be a good question. My syntax may also be very bad. Any help is appreciated. The code below is the defining of the neural network and training of the pytorch model.
def get_nnet_model(module_list=nn.ModuleList(), input_dim: int = 8100, layer_dim: int = 100) -> nn.Module:
    """ Get the neural network model
    @return: neural network model
    """
    device = torch.device('cpu')

    module_list.append(nn.Linear(input_dim, layer_dim))
    module_list[-1].weight.data.normal_(0, 0.1)
    module_list[-1].bias.data.zero_()

def train_nnet(nnet: nn.Module, states_nnet: np.ndarray, outputs: np.ndarray, batch_size: int = 100, num_itrs: int = 10000, train_itr: int = 10000, device: torch.device, lr=0.01, lr_d=1):
    nnet.train()

    criterion = nn.MSELoss()
    optimizer = optim.Adam(nnet.parameters(), lr=lr)

    while train_itr < num_itrs:
        optimizer.zero_grad()

        lr_itr = lr + (lr_d ** train_itr)
        for param_group in optimizer.param_groups:
            param_group['lr'] = lr_itr

        data = pickle.load(open("data/data.pkl", "rb"))
        nnet_inputs_np, nnet_targets_np = data
        nnet_inputs_np = nnet_inputs_np.astype(np.float32)

        nnet_inputs = torch.tensor(nnet_inputs_np, device=device)
        nnet_targets = torch.tensor(nnet_targets_np, device=device)

        nnet_inputs = nnet_inputs.float()
        nnet_outputs = nnet(nnet_inputs)

        loss = criterion(nnet_outputs, nnet_targets)
        loss.backward()
        optimizer.step()

Output:
Based on your comment, somewhere else in your code you have something like:
nnet = get_nnet_model(...)

However, get_nnet_model(...) isn't returning anything. Change the def get_nnet_model to:
def get_nnet_model(module_list=nn.ModuleList(), input_dim: int = 8100, layer_dim: int = 100) -> nn.Module:
    """ Get the neural network model
    @return: neural network model
    """
    device = torch.device('cpu')

    module_list.append(nn.Linear(input_dim, layer_dim))
    module_list[-1].weight.data.normal_(0, 0.1)
    module_list[-1].bias.data.zero_()

    return module_list  # add this one


Instruction: what is wrong with the following implementation of Conv1d?

Input:
I am trying to implement a Conv1d layer with Batch Normalization but I keep getting the following error:
RuntimeError                              Traceback (most recent call last)
<ipython-input-32-ef6e122ea50c> in <module>()
----> 1 test()
      2 for epoch in range(1, n_epochs + 1):
      3   train(epoch)
      4   test()

7 frames
/usr/local/lib/python3.7/dist-packages/torch/nn/modules/conv.py in _conv_forward(self, input, weight, bias)
    258                             _single(0), self.dilation, self.groups)
    259         return F.conv1d(input, weight, bias, self.stride,
--> 260                         self.padding, self.dilation, self.groups)
    261
    262     def forward(self, input: Tensor) -> Tensor:

RuntimeError: Expected 3-dimensional input for 3-dimensional weight [25, 40, 5], but got 2-dimensional input of size [32, 40] instead

The data is passed on in batches of 32 using DataLoader class and it has 40 features and 10 labels. Here is my model:
class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        #self.flatten=nn.Flatten()
        self.net_stack=nn.Sequential(
            nn.Conv1d(in_channels=40, out_channels=25, kernel_size=5, stride=2), #applying batch norm
            nn.ReLU(),
            nn.BatchNorm1d(25, affine=True),
            nn.Conv1d(in_channels=25, out_channels=20, kernel_size=5, stride=2), #applying batch norm
            nn.ReLU(),
            nn.BatchNorm1d(20, affine=True),
            nn.Linear(20, 10),
            nn.Softmax(dim=1))

    def forward(self,x):
        #x=torch.reshape(x, (1,-1))
        result=self.net_stack(x)
        return result

I have tried given in other answers like unsqueezing the input tensor, but none of the models in such questions is using Conv1d with batchnorm1d so I am not able to narrow down the problem to which layer must be causing the error. I have just started with using Pytorch and was able to implement a simple linear NN model, but I am facing this error while using a convolutional NN for the same data.

Output:
You need to add a batch dimension to your input (and also change the number of input channels).
A conv1d layer accepts inputs of shape [B, C, L], where B is the batch size, C is the number of channels and L is the width/length of your input. Also, your conv1d layer expects 40 input channels:
nn.Conv1d(in_channels=40, out_channels=25, kernel_size=5, stride=2)

hence, your input tensor x must have shape [B, 40, L] while now it has shape [32, 40].
Try:
def forward(self,x):
    result=self.net_stack(x[None])
    return result

you will get another error complaining about dimensions mismatch, suggesting you need to change the number of input channels to 40.


Instruction: How to resolve the error "mat1 and mat2 shapes cannot be multiplied" for the following CNN architecture?

Input:
I am trying to implement a Conv1d model with Batch Normalization but I am getting the error :
RuntimeError                              Traceback (most recent call last)
<ipython-input-117-ef6e122ea50c> in <module>()
----> 1 test()
      2 for epoch in range(1, n_epochs + 1):
      3   train(epoch)
      4   test()

7 frames
/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py in linear(input, weight, bias)
   1751     if has_torch_function_variadic(input, weight):
   1752         return handle_torch_function(linear, (input, weight), input, weight, bias=bias)
-> 1753     return torch._C._nn.linear(input, weight, bias)
   1754
   1755

RuntimeError: mat1 and mat2 shapes cannot be multiplied (32x140 and 100x10)

I use a batch size of 32, with the number of features of the data being 40. I've been trying to calculate where the 32 x 140 is coming from but I wasn't able to do that. Here is the architecture for the CNN I am trying to use:
class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        #self.flatten=nn.Flatten()
        self.net_stack=nn.Sequential(
            nn.Conv1d(in_channels=1, out_channels=25, kernel_size=5, stride=2), #applying batch norm
            nn.ReLU(),
            nn.BatchNorm1d(25, affine=True),
            nn.Conv1d(in_channels=25, out_channels=20, kernel_size=5, stride=2), #applying batch norm
            nn.ReLU(),
            nn.BatchNorm1d(20, affine=True),
            nn.Flatten()
            nn.Linear(20*5, 10),
            nn.Softmax(dim=1))

    def forward(self,x):
        # result=self.net_stack(x[None])
        result=self.net_stack(x[:, None, :])
        return result

Output:
This fully-connnected should change from:
nn.Linear(20*5, 10)

to:
nn.Linear(20*7, 10)

Why?
If your input data length is 40, then (B is the batch size):

Output after first conv (K=25): B x 25 x 18
Output after second conv (K=20): B x 20 x 7
Output after nn.Flatten(): B x 140, i.e., if B=32, then 32x140


Instruction: [python]Colab crashes for unknown reason while training gan(pytorch)

Input:
I tried to train gan on some monkey pics but it crashes colab for unknown reason if try to train it.
I am using 1370 128*128 monkey images.
I have no idea where the issue might be, please respond
btw the runtime is gpu, so the problem doesn't linked to that
from torch import optim
import torchvision
from torchvision import transforms
import torch, torch.nn as nn

batch_size = 4

generic_transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.ToPILImage(),
    transforms.Resize((128,128)),
    transforms.ToTensor(),
    transforms.Normalize((0., 0., 0.), (6, 6, 6)),
    transforms.Grayscale(),
])

trainset=torchvision.datasets.ImageFolder(root='drive/My Drive/monkeys', transform=generic_transform)

trainloader = torch.utils.data.DataLoader(trainset,batch_size=batch_size, shuffle=True)

def _init_weights(m):
    if isinstance(m, nn.Conv2d):
        nn.init.normal_(m.weight, 0.0, 0.02)

def gen_noise(noise_shape, n_samples, device='cuda:0'):
    return torch.randn(noise_shape, n_samples).to(device)

class Discriminator(nn.Module):
 #convolutional discriminator
  def __init__(self) -> None:
    super(Discriminator, self).__init__()

    self.hidden_dim = 64

    self.relu = nn.ReLU(inplace=False)
    self.sigmoid = nn.Sigmoid()

    self.conv_1 = nn.Conv2d(in_channels=1, out_channels=32, kernel_size=32, stride = 2)
    self.maxPooling_1 = nn.MaxPool2d(kernel_size=3)
    self.conv_2 = nn.Conv2d(in_channels=32, out_channels=16, kernel_size=8, stride = 2)
    self.maxPooling_2 = nn.MaxPool2d(kernel_size=2)
    self.linear_layer = nn.Linear(in_features=self.hidden_dim, out_features=1)

  def forward(self, x) -> float:
    self.x = x
    self.x = self.relu(self.conv_1(self.x))
    self.x = self.maxPooling_1(self.x)
    self.x = self.relu(self.conv_2(self.x))
    self.x = self.maxPooling_2(self.x)
    print(self.x.shape)
    self.x = self.x.view(self.x.shape[0],
                         self.x.shape[1]*self.x.shape[2]*self.x.shape[3])
    self.x = self.sigmoid(self.linear_layer(self.x))

    return self.x

 class Generator(nn.Module):
    #fully connected generator
    def __init__(self, hidden_dim, output_dim, z_dim) -> None:
    super(Generator, self).__init__()

    self.relu = nn.ReLU(inplace=False)
    self.hidden_dim = hidden_dim
    self.output_dim = output_dim
    self.z_dim = z_dim

    self.linear_layer_1 = nn.Linear(in_features=self.z_dim, out_features=self.hidden_dim)
    self.linear_layer_2 = nn.Linear(in_features=self.hidden_dim, out_features=self.hidden_dim*2)
    self.linear_layer_3 = nn.Linear(in_features=self.hidden_dim*2, out_features=self.output_dim)

    def forward(self, x) -> torch.tensor:
      self.x = x
      self.x = self.relu(self.linear_layer_1(self.x))
      self.x = self.relu(self.linear_layer_2(self.x))
      self.x = self.relu(self.linear_layer_3(self.x))
      return self.x

class GAN():
  def __init__(self, hidden_dim, output_dim, z_dim, criterion, device="cuda:0") -> None:
    if device == "cuda:0":
      assert torch.cuda.is_available(), "apply gpu"

    self.hidden_dim = hidden_dim
    self.output_dim = output_dim
    self.device = device
    self.criterion = criterion
    self.z_dim = torch.tensor(z_dim).long()

    self.discriminator = Discriminator().to(self.device)
    self.d_opt = optim.Adam(self.discriminator.parameters(), lr=0.0001)

    self.generator = Generator(hidden_dim=self.hidden_dim, output_dim=self.output_dim, z_dim=self.z_dim).to(self.device)
    self.g_opt = optim.Adam(self.generator.parameters(), lr=0.0001)

    self.generator = self.generator.apply(_init_weights)
    self.discriminator = self.discriminator.apply(_init_weights)

class GAN_Trainer():
  def __init__(self, z_dim, model, device="cuda:0") -> None:
    self.device = device
    self.gan = model
    self.z_dim = z_dim

    self._d_mean_train_loss = None
    self._g_mean_train_loss = None

  def train(self, batch) -> None:
    print(1)
    self.batch = batch.to(self.device)

    self.noise = gen_noise(self.batch.shape[0],self.z_dim).to(self.device)
    self.gan.g_opt.zero_grad()
    self._g_output = self.gan.generator.forward(self.noise.to(self.device))

    self._g_output = self._g_output.view(self.batch.shape[0],
                                        1,
                                        torch.sqrt(torch.tensor(self.gan.output_dim)).int(),
                                        torch.sqrt(torch.tensor(self.gan.output_dim)).int())

    print(self._g_output.shape)

    self._d_for_g_pred = self.gan.discriminator.forward(self._g_output)
    self._g_loss = self.gan.criterion(self._d_for_g_pred, torch.zeros_like(self._d_for_g_pred))
    self._g_loss.backward()
    self.gan.g_opt.step()

    self.gan.d_opt.zero_grad()
    self._d_fake_pred = self.gan.discriminator.forward(self._g_output)
    self._d_fake_loss = self.gan.criterion(self._g_output, torch.zeros_like(self._g_output))

    self._d_real_pred = self.gan.discriminator.forward(self.batch)
    self._d_real_loss = self.gan.criterion(self.batch, torch.ones_like(self.batch))

    self._d_mean_loss = torch.mean(torch.cat((self._d_fake_loss, self._d_real_loss),0))
    self._d_mean_loss.backward(retain_graph=True)
    self.gan.d_opt.step()

    self._d_mean_train_loss = self._d_mean_train_loss + self._d_mean_loss.detach()
    self._g_mean_train_loss = self._g_mean_train_loss + self._g_loss.detach()

torch.cuda.empty_cache()

gan = GAN(hidden_dim=1200,
          output_dim=16384,
          z_dim = 1000,
          criterion=nn.BCEWithLogitsLoss())

trainer = GAN_Trainer(model=gan, z_dim=1000)

#here is where it crashes
from tqdm import trange
torch.cuda.empty_cache()

image = trainset[0][0].to("cuda:0").view(1,
                                         trainset[0][0].shape[0],
                                         trainset[0][0].shape[1],
                                         trainset[0][0].shape[2])
trainer.train(batch=image)

please help! I am starting to lose my sanity already, thanks! <3

Output:
I've debugged your code a bit, and found that the crash is happening at line:
self._d_mean_loss = torch.mean(torch.cat((self._d_fake_loss, self._d_real_loss),0))

I have tried to find out the reason it is crashing, it looks like your operations are not properly done and some inplace operations are changing the graph and causing pytorch to malfunction.
There are some major issues in the code that you need to change in terms of GAN logic. Here:
self._d_real_loss = self.gan.criterion(self.batch, torch.ones_like(self.batch))

Real loss should be based on the output of discriminator self._d_real_pred. In general you should have three backward calls:

You feed the discriminator with real batch and expect it to output 'real' class.
Generate fake images with generator. Feed it to discriminator and expect it to output 'fake' class.
Lastly for generator, you feed the discriminator fake images again and expect it to fail, optimize for 'fake' class so that the generator can learn outputting better looking fakes.

I would highly recommend this tutorial: https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html . It is a great tutorial for understanding how GANs are trained.
Lastly, I would remove all self.... storing in forward and train functions. Any operations are done tensors update the graph by storing tensors as members and modifying them may cause problems and cause gradients to wipe etc.


Instruction: What output_padding does in nn.ConvTranspose2d?

Input:
What is the working of Output_padding in Conv2dTranspose? Please Help me to understand this?
Conv2dTranspose(1024, 512, kernel_size=3, stride=2, padding=1, output_padding=1)

Output:
According to documentation here: https://pytorch.org/docs/stable/generated/torch.nn.ConvTranspose2d.html  when applying Conv2D operation with Stride > 1 you can get same output dimensions with different inputs. For example, 7x7 and 8x8 inputs would both return 3x3 output with Stride=2:
import torch

conv_inp1 = torch.rand(1,1,7,7)
conv_inp2 = torch.rand(1,1,8,8)

conv1 = torch.nn.Conv2d(1, 1, kernel_size = 3, stride = 2)

out1 = conv1(conv_inp1)
out2 = conv1(conv_inp2)
print(out1.shape)         # torch.Size([1, 1, 3, 3])
print(out2.shape)         # torch.Size([1, 1, 3, 3])

And when applying the transpose convolution, it is ambiguous that which output shape to return, 7x7 or 8x8 for stride=2 transpose convolution. Output padding helps pytorch to determine 7x7 or 8x8 output with output_padding parameter. Note that, it doesn't pad zeros or anything to output, it is just a way to determine the output shape and apply transpose convolution accordingly.
conv_t1 = torch.nn.ConvTranspose2d(1, 1, kernel_size=3, stride=2)
conv_t2 = torch.nn.ConvTranspose2d(1, 1, kernel_size=3, stride=2, output_padding=1)
transposed1 = conv_t1(out1)
transposed2 = conv_t2(out2)

print(transposed1.shape)      # torch.Size([1, 1, 7, 7])
print(transposed2.shape)      # torch.Size([1, 1, 8, 8])


Instruction: Updating packages in conda

Input:
I have a problem with updating packages in conda. The list of my installed packages is:
#
# Name                    Version                   Build  Channel
_anaconda_depends         2020.07                  py38_0
_ipyw_jlab_nb_ext_conf    0.1.0                    py38_0
alabaster                 0.7.12             pyhd3eb1b0_0
anaconda                  custom                   py38_1
anaconda-client           1.7.2                    py38_0
anaconda-navigator        1.10.0                   py38_0
anaconda-project          0.9.1              pyhd3eb1b0_1
argh                      0.26.2                   py38_0
argon2-cffi               20.1.0           py38h2bbff1b_1
ase                       3.21.0                   pypi_0    pypi
asn1crypto                1.4.0                      py_0
astroid                   2.5              py38haa95532_1
astropy                   4.2.1            py38h2bbff1b_1
async_generator           1.10               pyhd3eb1b0_0
atomicwrites              1.4.0                      py_0
attrs                     20.3.0             pyhd3eb1b0_0
autopep8                  1.5.6              pyhd3eb1b0_0
babel                     2.9.0              pyhd3eb1b0_0
backcall                  0.2.0              pyhd3eb1b0_0
backports                 1.0                pyhd3eb1b0_2
backports.functools_lru_cache 1.6.3              pyhd3eb1b0_0
backports.shutil_get_terminal_size 1.0.0              pyhd3eb1b0_3
backports.tempfile        1.0                pyhd3eb1b0_1
backports.weakref         1.0.post1                  py_1
bcrypt                    3.2.0            py38he774522_0
beautifulsoup4            4.9.3              pyha847dfd_0
bitarray                  1.9.1            py38h2bbff1b_1
bkcharts                  0.2                      py38_0
blas                      1.0                         mkl
bleach                    3.3.0              pyhd3eb1b0_0
blosc                     1.21.0               h19a0ad4_0
bokeh                     2.3.1            py38haa95532_0
boto                      2.49.0                   py38_0
bottleneck                1.3.2            py38h2a96729_1
brotli                    1.0.9                ha925a31_2
brotlipy                  0.7.0           py38h2bbff1b_1003
bzip2                     1.0.8                he774522_0
ca-certificates           2021.1.19            haa95532_1
certifi                   2020.12.5        py38haa95532_0
cffi                      1.14.5           py38hcd4344a_0
chardet                   4.0.0           py38haa95532_1003
charls                    2.2.0                h6c2663c_0
click                     7.1.2              pyhd3eb1b0_0
cloudpickle               1.6.0                      py_0
clyent                    1.2.2                    py38_1
colorama                  0.4.4              pyhd3eb1b0_0
comtypes                  1.1.9           py38haa95532_1002
conda                     4.10.0           py38haa95532_0
conda-build               3.20.5                   py38_1
conda-env                 2.6.0                         1
conda-package-handling    1.7.3            py38h8cc25b3_1
conda-verify              3.4.2                      py_1
console_shortcut          0.1.1                         4
contextlib2               0.6.0.post1                py_0
cpuonly                   1.0                           0    pytorch
cryptography              3.4.7            py38h71e12ea_0
curl                      7.71.1               h2a8f88b_1
cycler                    0.10.0                   py38_0
cython                    0.29.22          py38hd77b12b_0
cytoolz                   0.11.0           py38he774522_0
dask                      2021.4.0           pyhd3eb1b0_0
dask-core                 2021.4.0           pyhd3eb1b0_0
decorator                 5.0.6              pyhd3eb1b0_0
defusedxml                0.7.1              pyhd3eb1b0_0
diff-match-patch          20200713                   py_0
distributed               2021.4.0         py38haa95532_0
docutils                  0.17             py38haa95532_1
entrypoints               0.3                      py38_0
et_xmlfile                1.0.1                   py_1001
fastcache                 1.1.0            py38he774522_0
filelock                  3.0.12             pyhd3eb1b0_1
flake8                    3.9.0              pyhd3eb1b0_0
flask                     1.1.2              pyhd3eb1b0_0
freetype                  2.10.4               hd328e21_0
fsspec                    0.9.0              pyhd3eb1b0_0
future                    0.18.2                   py38_1
get_terminal_size         1.0.0                h38e98db_0
gevent                    21.1.2           py38h2bbff1b_1
giflib                    5.2.1                h62dcd97_0
glob2                     0.7                pyhd3eb1b0_0
gmpy2                     2.0.8            py38h7edee0f_3
googledrivedownloader     0.4                      pypi_0    pypi
greenlet                  1.0.0            py38hd77b12b_2
h5py                      2.10.0           py38h5e291fa_0
hdf5                      1.10.4               h7ebc959_0
heapdict                  1.0.1                      py_0
html5lib                  1.1                        py_0
icc_rt                    2019.0.0             h0cc432a_1
icu                       58.2                 ha925a31_3
idna                      2.10               pyhd3eb1b0_0
imagecodecs               2021.3.31        py38h5da4933_0
imageio                   2.9.0              pyhd3eb1b0_0
imagesize                 1.2.0              pyhd3eb1b0_0
importlib-metadata        3.10.0           py38haa95532_0
importlib_metadata        3.10.0               hd3eb1b0_0
iniconfig                 1.1.1              pyhd3eb1b0_0
intel-openmp              2020.2                      254
intervaltree              3.1.0                      py_0
ipykernel                 5.3.4            py38h5ca1d4c_0
ipynb                     0.5.1                    pypi_0    pypi
ipython                   7.22.0           py38hd4e2768_0
ipython_genutils          0.2.0              pyhd3eb1b0_1
ipywidgets                7.6.3              pyhd3eb1b0_1
isodate                   0.6.0                    pypi_0    pypi
isort                     5.8.0              pyhd3eb1b0_0
itsdangerous              1.1.0              pyhd3eb1b0_0
jdcal                     1.4.1                      py_0
jedi                      0.17.1                   py38_0
jinja2                    2.11.3             pyhd3eb1b0_0
joblib                    1.0.1              pyhd3eb1b0_0
jovian                    0.2.28                   pypi_0    pypi
jpeg                      9b                   hb83a4c4_2
json5                     0.9.5                      py_0
jsonschema                3.2.0                      py_2
jupyter                   1.0.0                    py38_7
jupyter_client            6.1.12             pyhd3eb1b0_0
jupyter_console           6.4.0              pyhd3eb1b0_0
jupyter_core              4.7.1            py38haa95532_0
jupyterlab                2.2.6                      py_0
jupyterlab_pygments       0.1.2                      py_0
jupyterlab_server         1.2.0                      py_0
jupyterlab_widgets        1.0.0              pyhd3eb1b0_1
keyring                   22.3.0           py38haa95532_0
kiwisolver                1.3.1            py38hd77b12b_0
krb5                      1.18.2               hc04afaa_0
lazy-object-proxy         1.6.0            py38h2bbff1b_0
lcms2                     2.12                 h83e58a3_0
lerc                      2.2.1                hd77b12b_0
libaec                    1.0.4                h33f27b4_1
libarchive                3.4.2                h5e25573_0
libcurl                   7.71.1               h2a8f88b_1
libdeflate                1.7                  h2bbff1b_5
libiconv                  1.15                 h1df5818_7
liblief                   0.10.1               ha925a31_0
libllvm9                  9.0.1                h21ff451_0
libpng                    1.6.37               h2a8f88b_0
libsodium                 1.0.18               h62dcd97_0
libspatialindex           1.9.3                h6c2663c_0
libssh2                   1.9.0                h7a1dbc1_1
libtiff                   4.2.0                hd0e1b90_0
libuv                     1.40.0               he774522_0
libxml2                   2.9.10               hb89e7f3_3
libxslt                   1.1.34               he774522_0
libzopfli                 1.0.3                ha925a31_0
llvmlite                  0.36.0           py38h34b8924_4
locket                    0.2.1            py38haa95532_1
lxml                      4.6.3            py38h9b66d53_0
lz4-c                     1.9.3                h2bbff1b_0
lzo                       2.10                 he774522_2
m2w64-gcc-libgfortran     5.3.0                         6
m2w64-gcc-libs            5.3.0                         7
m2w64-gcc-libs-core       5.3.0                         7
m2w64-gmp                 6.1.0                         2
m2w64-libwinpthread-git   5.0.0.4634.697f757               2
markupsafe                1.1.1            py38he774522_0
matplotlib                3.3.4            py38haa95532_0
matplotlib-base           3.3.4            py38h49ac443_0
mccabe                    0.6.1                    py38_1
menuinst                  1.4.16           py38he774522_1
mistune                   0.8.4           py38he774522_1000
mkl                       2020.2                      256
mkl-service               2.3.0            py38h196d8e1_0
mkl_fft                   1.3.0            py38h46781fe_0
mkl_random                1.1.1            py38h47e9c7a_0
mock                      4.0.3              pyhd3eb1b0_0
more-itertools            8.7.0              pyhd3eb1b0_0
mpc                       1.1.0                h7edee0f_1
mpfr                      4.0.2                h62dcd97_1
mpir                      3.0.0                hec2e145_1
mpmath                    1.2.1            py38haa95532_0
msgpack-python            1.0.2            py38h59b6b97_1
msys2-conda-epoch         20160418                      1
multipledispatch          0.6.0                    py38_0
navigator-updater         0.2.1                    py38_0
nbclient                  0.5.3              pyhd3eb1b0_0
nbconvert                 6.0.7                    py38_0
nbformat                  5.1.3              pyhd3eb1b0_0
nest-asyncio              1.5.1              pyhd3eb1b0_0
networkx                  2.5                        py_0
ninja                     1.10.2               h6d14046_1
nltk                      3.6.1              pyhd3eb1b0_0
nose                      1.3.7           pyhd3eb1b0_1006
notebook                  6.3.0            py38haa95532_0
numba                     0.53.1           py38hf11a4ad_0
numexpr                   2.7.3            py38hcbcaa1e_0
numpy                     1.19.2           py38hadc3359_0
numpy-base                1.19.2           py38ha3acd2a_0
numpydoc                  1.1.0              pyhd3eb1b0_1
olefile                   0.46                       py_0
opencv-python             4.5.1.48                 pypi_0    pypi
openjpeg                  2.3.0                h5ec785f_1
openpyxl                  3.0.7              pyhd3eb1b0_0
openssl                   1.1.1k               h2bbff1b_0
packaging                 20.9               pyhd3eb1b0_0
pandas                    1.2.3            py38hf11a4ad_0
pandoc                    2.12                 haa95532_0
pandocfilters             1.4.3            py38haa95532_1
paramiko                  2.7.2                      py_0
parso                     0.7.0                      py_0
partd                     1.2.0              pyhd3eb1b0_0
path                      15.1.2           py38haa95532_0
path.py                   12.5.0                        0
pathlib2                  2.3.5            py38haa95532_2
pathtools                 0.1.2                      py_1
patsy                     0.5.1                    py38_0
pep8                      1.7.1                    py38_0
pexpect                   4.8.0              pyhd3eb1b0_3
pickleshare               0.7.5           pyhd3eb1b0_1003
pillow                    8.2.0            py38h4fa10fc_0
pip                       20.2.4           py38haa95532_0
pkginfo                   1.7.0            py38haa95532_0
pluggy                    0.13.1           py38haa95532_0
ply                       3.11                     py38_0
powershell_shortcut       0.0.1                         3
prometheus_client         0.10.1             pyhd3eb1b0_0
prompt-toolkit            3.0.17             pyh06a4308_0
prompt_toolkit            3.0.17               hd3eb1b0_0
protobuf                  3.14.0                   pypi_0    pypi
psutil                    5.8.0            py38h2bbff1b_1
ptyprocess                0.7.0              pyhd3eb1b0_2
py                        1.10.0             pyhd3eb1b0_0
py-lief                   0.10.1           py38ha925a31_0
pycodestyle               2.6.0              pyhd3eb1b0_0
pycosat                   0.6.3            py38h2bbff1b_0
pycparser                 2.20                       py_2
pycurl                    7.43.0.6         py38h7a1dbc1_0
pydocstyle                6.0.0              pyhd3eb1b0_0
pyerfa                    1.7.2            py38h2bbff1b_0
pyflakes                  2.2.0              pyhd3eb1b0_0
pygments                  2.8.1              pyhd3eb1b0_0
pylint                    2.7.4            py38haa95532_1
pynacl                    1.4.0            py38h62dcd97_1
pyodbc                    4.0.30           py38ha925a31_0
pyopenssl                 20.0.1             pyhd3eb1b0_1
pyparsing                 2.4.7              pyhd3eb1b0_0
pyqt                      5.9.2            py38ha925a31_4
pyreadline                2.1                      py38_1
pyrsistent                0.17.3           py38he774522_0
pysocks                   1.7.1            py38haa95532_0
pytables                  3.6.1            py38ha5be198_0
pytest                    6.2.3            py38haa95532_2
python                    3.8.5                h5fd99cc_1
python-dateutil           2.8.1              pyhd3eb1b0_0
python-jsonrpc-server     0.4.0                      py_0
python-language-server    0.35.1                     py_0
python-libarchive-c       2.9                pyhd3eb1b0_1
python-louvain            0.15                     pypi_0    pypi
pytorch                   1.7.1               py3.8_cpu_0  [cpuonly]  pytorch
pytz                      2021.1             pyhd3eb1b0_0
pywavelets                1.1.1            py38he774522_2
pywin32                   227              py38he774522_1
pywin32-ctypes            0.2.0                 py38_1000
pywinpty                  0.5.7                    py38_0
pyyaml                    5.4.1            py38h2bbff1b_1
pyzmq                     20.0.0           py38hd77b12b_1
qdarkstyle                3.0.2              pyhd3eb1b0_0
qt                        5.9.7            vc14h73c81de_0
qtawesome                 1.0.2              pyhd3eb1b0_0
qtconsole                 5.0.3              pyhd3eb1b0_0
qtpy                      1.9.0                      py_0
rdflib                    5.0.0                    pypi_0    pypi
regex                     2021.4.4         py38h2bbff1b_0
requests                  2.25.1             pyhd3eb1b0_0
rope                      0.18.0                     py_0
rtree                     0.9.4            py38h21ff451_1
ruamel_yaml               0.15.100         py38h2bbff1b_0
scikit-image              0.18.1           py38hf11a4ad_0
scikit-learn              0.24.1           py38hf11a4ad_0
scipy                     1.6.2            py38h14eb087_0
seaborn                   0.11.1             pyhd3eb1b0_0
send2trash                1.5.0              pyhd3eb1b0_1
setuptools                52.0.0           py38haa95532_0
simplegeneric             0.8.1                    py38_2
singledispatch            3.6.1           pyhd3eb1b0_1001
sip                       4.19.13          py38ha925a31_0
six                       1.15.0           py38haa95532_0
snappy                    1.1.8                h33f27b4_0
snowballstemmer           2.1.0              pyhd3eb1b0_0
sortedcollections         2.1.0              pyhd3eb1b0_0
sortedcontainers          2.3.0              pyhd3eb1b0_0
soupsieve                 2.2.1              pyhd3eb1b0_0
sphinx                    3.2.1                      py_0
sphinxcontrib             1.0                      py38_1
sphinxcontrib-applehelp   1.0.2              pyhd3eb1b0_0
sphinxcontrib-devhelp     1.0.2              pyhd3eb1b0_0
sphinxcontrib-htmlhelp    1.0.3              pyhd3eb1b0_0
sphinxcontrib-jsmath      1.0.1              pyhd3eb1b0_0
sphinxcontrib-qthelp      1.0.3              pyhd3eb1b0_0
sphinxcontrib-serializinghtml 1.1.4              pyhd3eb1b0_0
sphinxcontrib-websupport  1.2.4                      py_0
spyder                    4.1.5                    py38_0
spyder-kernels            1.9.4                    py38_0
sqlalchemy                1.4.7            py38h2bbff1b_0
sqlite                    3.35.4               h2bbff1b_0
statsmodels               0.12.2           py38h2bbff1b_0
sympy                     1.7.1            py38haa95532_0
tbb                       2020.3               h74a9793_0
tblib                     1.7.0                      py_0
tensorboardx              2.1                      pypi_0    pypi
terminado                 0.9.4            py38haa95532_0
testpath                  0.4.4              pyhd3eb1b0_0
threadpoolctl             2.1.0              pyh5ca1d4c_0
tifffile                  2021.3.31          pyhd3eb1b0_1
tk                        8.6.10               he774522_0
toml                      0.10.2             pyhd3eb1b0_0
toolz                     0.11.1             pyhd3eb1b0_0
torch-cluster             1.5.8                    pypi_0    pypi
torch-geometric           1.6.3                    pypi_0    pypi
torch-scatter             2.0.5                    pypi_0    pypi
torch-sparse              0.6.8                    pypi_0    pypi
torch-spline-conv         1.2.0                    pypi_0    pypi
torchaudio                0.7.2                      py38    pytorch
torchvision               0.8.2                  py38_cpu  [cpuonly]  pytorch
tornado                   6.1              py38h2bbff1b_0
tqdm                      4.59.0             pyhd3eb1b0_1
traitlets                 5.0.5              pyhd3eb1b0_0
typing_extensions         3.7.4.3            pyha847dfd_0
ujson                     4.0.2            py38hd77b12b_0
unicodecsv                0.14.1                   py38_0
urllib3                   1.26.4             pyhd3eb1b0_0
uuid                      1.30                     pypi_0    pypi
vc                        14.2                 h21ff451_1
vs2015_runtime            14.27.29016          h5e58377_2
watchdog                  1.0.2            py38haa95532_1
wcwidth                   0.2.5                      py_0
webencodings              0.5.1                    py38_1
werkzeug                  1.0.1              pyhd3eb1b0_0
wheel                     0.36.2             pyhd3eb1b0_0
widgetsnbextension        3.5.1                    py38_0
win_inet_pton             1.1.0            py38haa95532_0
win_unicode_console       0.5                      py38_0
wincertstore              0.2                      py38_0
winpty                    0.4.3                         4
wrapt                     1.12.1           py38he774522_1
xlrd                      2.0.1              pyhd3eb1b0_0
xlsxwriter                1.3.8              pyhd3eb1b0_0
xlwings                   0.23.0           py38haa95532_0
xlwt                      1.3.0                    py38_0
xmltodict                 0.12.0                     py_0
xz                        5.2.5                h62dcd97_0
yaml                      0.2.5                he774522_0
yapf                      0.31.0             pyhd3eb1b0_0
zeromq                    4.3.3                ha925a31_3
zfp                       0.5.5                hd77b12b_6
zict                      2.0.0              pyhd3eb1b0_0
zipp                      3.4.1              pyhd3eb1b0_0
zlib                      1.2.11               h62dcd97_4
zope                      1.0                      py38_1
zope.event                4.5.0                    py38_0
zope.interface            5.3.0            py38h2bbff1b_0
zstd                      1.4.5                h04227a9_0

and I want to update
torch-cluster             1.5.8                    pypi_0    pypi
torch-geometric           1.6.3                    pypi_0    pypi
torch-scatter             2.0.5                    pypi_0    pypi
torch-sparse              0.6.8                    pypi_0    pypi
torch-spline-conv         1.2.0                    pypi_0    pypi

However, when I enter the command
conda update torch-cluster

I get
PackageNotInstalledError: Package is not installed in prefix.
  prefix: C:\Users\Rostyslav\anaconda3
  package name: torch-cluster

I may be wrong about it, but I think the reason of the error is that the packages that I want to update are located in different channels, namely pypi. But even if it true, I do not understand how to reach this channel.
Can anybody explain?
Thanks a lot!

Output:
Channel pypi means that the package was installed with pip.
You may need to upgrade it with pip as well
pip install torch-cluster --upgrade


Instruction: sampler argument in DataLoader of Pytorch

Input:
While using Pytorch's DataLoader utility, in sampler what is the purpose of RandomIdentitySampler? And in RandomIdentitySampler there is an argument instances. Does instances depends upon number of workers? If there is are 4 workers then should there be 4 instances as well?
Following is the chunk of code:
c_dataloaders = DataLoader(Preprocessor(cluster_dataset.train_set,
                                        root=cluster_dataset.images_dir,
                                        transform=train_transformer),
                                        batch_size=args.batch_size_stage2,
                                        num_workers=args.workers,
                                        sampler=RandomIdentitySampler(cluster_dataset.train_set,
                                        args.batch_size_stage2,
                                        args.instances)

Output:
This sampler is not part of the PyTorch or any other official lib (torchvision, torchtext, etc.). Anyway, there is a RandomIdentitySampler in the torchreid from KaiyangZhou. Assuming this is the case:

While using Pytorch's DataLoader utility, in sampler what is the purpose of RandomIdentitySampler?

As you can see in the DataLoader documentation: the sampler "defines the strategy to draw samples from the dataset". More specifically, based on RandomIdentitySampler documentation, it "randomly samples N identities each with K instances".

And in RandomIdentitySampler there is an argument instances. Does instances depends upon number of workers?

Based on the previous answer, you can note that instances does not depend on the number of workers. It simply sets the number of instances of each identity that will be drawn from the dataset for each batch.

If there are 4 workers then should there be 4 instances as well?

Not necessarily. The only constraint is that the number of instances should not be smaller than the batch size.


Instruction: Difference between two implemantations of Seq layers - pytorch

Input:
I have implemented a model using the traditional way of implementation. and the code is like this.
    def __init__(self):
    super(enhance_net_nopool, self).__init__()
    number_f = 32

    self.relu = nn.ReLU(inplace=True)
    self.e_conv1 = nn.Conv2d(number_f, number_f, 3, 1, 1, bias=True)
    self.e_conv2 = nn.Conv2d(number_f, number_f, 3, 1, 1, bias=True)
    self.e_conv3 = nn.Conv2d(number_f, number_f, 3, 1, 1, bias=True)

def forward(self, x):
    x1 = self.relu(self.e_conv1(x))
    x2 = self.relu(self.e_conv2(x1))
    x3 = self.relu(self.e_conv3(x2))

I need to know is it possible to rewrite that code using
seq_layers=nn.Sequential(*layers)

this *layers.
If it's implemented like this.
self.conv_block = [
        relu = nn.ReLU(inplace=True)
        conv = nn.Conv2d(number_f, number_f, 3, 1, 1, bias=True)
    ]

are they the same?

Output:
Your code can be rewritten as:
layers = [nn.Conv2d(number_f, number_f, 3, 1, 1, bias=True), nn.ReLU(inplace=True),
    nn.Conv2d(number_f, number_f, 3, 1, 1, bias=True), nn.ReLU(inplace=True),
    nn.Conv2d(number_f, number_f, 3, 1, 1, bias=True), nn.ReLU(inplace=True)
]

seq_layers = nn.Sequential(*layers)

or
def get_conv(number_f):
    return nn.Sequential(
                nn.Conv2d(number_f, number_f, 3, 1, 1, bias=True),
                nn.ReLU(inplace=True)
                )

layers = [get_conv(number_f) for _ in range(3)]
seq_layers = nn.Sequential(*layers)
print(seq_layers)


Instruction: RuntimeError: expected scalar type Double but found Float

Input:
I'm a newbie in PyTorch and I got the following error from my cnn layer: "RuntimeError: expected scalar type Double but found Float". I converted each element into .astype(np.double) but the error message remains. Then after converting Tensor tried to use .double() and again the error message remains.
Here is my code for a better understanding:
import torch.nn as nn
class CNN(nn.Module):

    # Contructor
    def __init__(self, shape):
        super(CNN, self).__init__()
        self.cnn1 = nn.Conv1d(in_channels=shape, out_channels=32, kernel_size=3)
        self.act1 = torch.nn.ReLU()
    # Prediction
    def forward(self, x):
        x = self.cnn1(x)
        x = self.act1(x)
    return x

    X_train_reshaped = np.zeros([X_train.shape[0],int(X_train.shape[1]/depth),depth])

    for i in range(X_train.shape[0]):
        for j in range(X_train.shape[1]):
            X_train_reshaped[i][int(j/3)][j%3] = X_train[i][j].astype(np.double)

    X_train = torch.tensor(X_train_reshaped)
    y_train = torch.tensor(y_train)

    # Dataset w/o any tranformations
    train_dataset_normal = CustomTensorDataset(tensors=(X_train, y_train), transform=None)
    train_loader = torch.utils.data.DataLoader(train_dataset_normal, shuffle=True, batch_size=16)

    model = CNN(X_train.shape[1]).to(device)

    # Loss and optimizer
    criterion = nn.CrossEntropyLoss()
    optimizer = torch.optim.Adam(model.parameters())

    # Train the model
    #how to implement batch_size??
    for epoch in range(epochno):
        #for i, (dataX, labels) in enumerate(X_train_reshaped,y_train):
        for i, (dataX, labels) in enumerate(train_loader):
            dataX = dataX.to(device)
            labels = labels.to(device)

            # Forward pass
            outputs = model(dataX)
            loss = criterion(outputs, labels)

            # Backward and optimize
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()

            if (i+1) % 100 == 0:
                print ('Epoch [{}/{}], Step [{}/{}], Loss: {:.4f}'
                       .format(epoch+1, num_epochs, i+1, total_step, loss.item()))

And following is the error I received:
RuntimeError                              Traceback (most recent call last)
<ipython-input-39-d99b62b3a231> in <module>
     14
     15         # Forward pass
---> 16         outputs = model(dataX.double())
     17         loss = criterion(outputs, labels)
     18

~\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)
    887             result = self._slow_forward(*input, **kwargs)
    888         else:
--> 889             result = self.forward(*input, **kwargs)
    890         for hook in itertools.chain(
    891                 _global_forward_hooks.values(),

<ipython-input-27-7510ac2f1f42> in forward(self, x)
     22     # Prediction
     23     def forward(self, x):
---> 24         x = self.cnn1(x)
     25         x = self.act1(x)

~\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)
    887             result = self._slow_forward(*input, **kwargs)
    888         else:
--> 889             result = self.forward(*input, **kwargs)
    890         for hook in itertools.chain(
    891                 _global_forward_hooks.values(),

~\torch\nn\modules\conv.py in forward(self, input)
    261
    262     def forward(self, input: Tensor) -> Tensor:
--> 263         return self._conv_forward(input, self.weight, self.bias)
    264
    265

~\torch\nn\modules\conv.py in _conv_forward(self, input, weight, bias)
    257                             weight, bias, self.stride,
    258                             _single(0), self.dilation, self.groups)
--> 259         return F.conv1d(input, weight, bias, self.stride,
    260                         self.padding, self.dilation, self.groups)
    261

RuntimeError: expected scalar type Double but found Float

Output: I don't know It's me or Pytorch but the error message is trying to say convert into float somehow. Therefore inside forward pass I resolved the problem by converting dataX to float as following: outputs = model(dataX.float())


Instruction: Where I can find an intuitive explanation of PyTorch's Tensor.unfold() being used to get image patches?

Input:
Recently I came across some code that extracted (sliding-window style) a number of square patches from an RGB image (or set of them) of shape N x B x H x W. They did this as follows:
patch_width = 3
patches = image.permute(0,2,3,1).unfold(dim = 1, size = patch_width, stride = patch_width) \
        .unfold(dim = 2, size = patch_width, stride = patch_width)

I understand that the unfold() method "returns all all slices of size size from self tensor in the dimension dim," from reading the documentation, but try as I might, I just can't get a good intuition for why stacking two .unfold() calls produces square patches. I get what happens when you use unfold() once on a tensor. I don't get what happens when you call it twice successively along two different dimensions.
I've seen this approach used multiple times, always without a good explanation as to why it works (1, 2), and it's driving me bonkers. Why are the spatial dimensions H and W permuted to be dims 1 and 2, while the channel dim is set to 3? Why does unfolding the same way on dim 1, then on dim 2 result in square patch_width by patch_width patches?
Any insight would be hugely appreciated, even if it's just a link to an article I missed. I've been Googling for well over an hour now and have met very little success. Thank you!
[1]PyTorch forum post
[2]Another forum post doing the same thing

Output:
I suppose, there are two distinct parts in your question, first one is why you need to permute, and second how two unfolds combined produce square image slices.
First moment is rather technical - unfold places produced slices in the new dimension of tensor, being 'inserted at the end of the shape'. permute here is needed to place it near channel or depth dimension, for merging them natural way using view later.
Now second part. Consider a deck of imaginary cards, each card is a picture channel. Take a card and cut it on vertical slices, then place slices on top of each other. Take a second card and do the same, placing result on the first one, do it with all cards. Now repeat the procedure, with cutting slices horisontaly. At the end you have much thinner but taller deck, where former cards become subdecks of patches.


Instruction: Still error trying to run libtorch helloworld pod install on apple silicon m1

Input:
When trying to create the cocoapod for ios pytorch helloworld on my mac mini m1 (big sur 11.2.1)
with installed miniconda3, homebrew 3.1.1, ruby 3.0.1.p64, gem 3.2.15, pod 1.10.1
and PATH set to
/opt/homebrew/lib/ruby/gems/3.0.0/bin:/opt/homebrew/opt/ruby/bin:$PATH
according to pod install giving error related to ruby gems and libffi
running pod install dies with a segmentation fault, see the following ruby crash file
So maybe someone can give me an idea how to get ios 14.4 and m1 big sur 11  working together?
Process:               ruby [80736]
Path:                  /opt/homebrew/*/ruby
Identifier:            ruby
Version:               0
Code Type:             ARM-64 (Native)
Parent Process:        bash [70856]
Responsible:           Terminal [70854]
User ID:               474179448

Date/Time:             2021-04-15 16:58:57.213 +0200
OS Version:            macOS 11.2.1 (20D74)
Report Version:        12
Anonymous UUID:        5893C804-2FF8-FBB1-5889-A883C6E0135A

Sleep/Wake UUID:       0044F30E-9B3E-4B4D-B6D2-B508B271D6B1

Time Awake Since Boot: 85000 seconds
Time Since Wake:       2300 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGABRT)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000000000001
Exception Note:        EXC_CORPSE_NOTIFY

VM Regions Near 0x1:
-->
    __TEXT                      10234c000-102350000    [   16K] r-x/r-x SM=COW  /opt/homebrew/*

Application Specific Information:
abort() called

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib          0x0000000185430cec __pthread_kill + 8
1   libsystem_pthread.dylib         0x0000000185461c24 pthread_kill + 292
2   libsystem_c.dylib               0x00000001853a9864 abort + 104
3   libruby.3.0.dylib               0x00000001026923a0 die + 12
4   libruby.3.0.dylib               0x00000001026923f8 rb_bug_for_fatal_signal + 88
5   libruby.3.0.dylib               0x0000000102774490 sigsegv + 96
6   libsystem_platform.dylib        0x00000001854a9c44 _sigtramp + 56
7   ???                             0xffff80019719d8cc 0 + 18446603343051217100
8   libcurl.4.dylib                 0x000000019719d8cc curl_easy_getinfo + 40
9   libffi.dylib                    0x0000000191ef4050 ffi_call_SYSV + 80
10  libffi.dylib                    0x0000000191efc9d8 ffi_call_int + 944
11  ffi_c.bundle                    0x00000001042394cc rbffi_CallFunction + 260 (Call.c:400)
12  ffi_c.bundle                    0x000000010423d250 attached_method_invoke + 44 (MethodHandle.c:174)
13  libffi.dylib                    0x0000000191efce10 ffi_closure_SYSV_inner + 800
14  libffi.dylib                    0x0000000191ef41e8 ffi_closure_SYSV + 56
15  libruby.3.0.dylib               0x00000001027e0b84 vm_call_cfunc_with_frame + 228
16  libruby.3.0.dylib               0x00000001027db078 vm_sendish + 1116
17  libruby.3.0.dylib               0x00000001027c7f70 vm_exec_core + 6948
18  libruby.3.0.dylib               0x00000001027d72c8 rb_vm_exec + 1652
19  libruby.3.0.dylib               0x00000001027e3df8 invoke_block_from_c_bh + 616
20  libruby.3.0.dylib               0x00000001027d1bf4 rb_yield + 180
21  libruby.3.0.dylib               0x00000001026359a0 rb_ary_each + 84
22  libruby.3.0.dylib               0x00000001027e0b84 vm_call_cfunc_with_frame + 228
23  libruby.3.0.dylib               0x00000001027db078 vm_sendish + 1116
24  libruby.3.0.dylib               0x00000001027c7f10 vm_exec_core + 6852
25  libruby.3.0.dylib               0x00000001027d72c8 rb_vm_exec + 1652
26  libruby.3.0.dylib               0x00000001027e3df8 invoke_block_from_c_bh + 616
27  libruby.3.0.dylib               0x00000001027d1bf4 rb_yield + 180
28  libruby.3.0.dylib               0x00000001026359a0 rb_ary_each + 84
29  libruby.3.0.dylib               0x00000001027e0b84 vm_call_cfunc_with_frame + 228
30  libruby.3.0.dylib               0x00000001027db078 vm_sendish + 1116
31  libruby.3.0.dylib               0x00000001027c7f10 vm_exec_core + 6852
32  libruby.3.0.dylib               0x00000001027d72c8 rb_vm_exec + 1652
33  libruby.3.0.dylib               0x00000001027e3df8 invoke_block_from_c_bh + 616
34  libruby.3.0.dylib               0x00000001027d1bf4 rb_yield + 180
35  libruby.3.0.dylib               0x0000000102639ca8 rb_ary_collect + 168
36  libruby.3.0.dylib               0x00000001027e0b84 vm_call_cfunc_with_frame + 228
37  libruby.3.0.dylib               0x00000001027db078 vm_sendish + 1116
38  libruby.3.0.dylib               0x00000001027c7f10 vm_exec_core + 6852
39  libruby.3.0.dylib               0x00000001027d72c8 rb_vm_exec + 1652
40  libruby.3.0.dylib               0x00000001026dd2cc load_iseq_eval + 200
41  libruby.3.0.dylib               0x00000001026db528 rb_load_internal + 56
42  libruby.3.0.dylib               0x00000001026dc850 rb_f_load + 180
43  libruby.3.0.dylib               0x00000001027e0b84 vm_call_cfunc_with_frame + 228
44  libruby.3.0.dylib               0x00000001027db078 vm_sendish + 1116
45  libruby.3.0.dylib               0x00000001027c7f70 vm_exec_core + 6948
46  libruby.3.0.dylib               0x00000001027d72c8 rb_vm_exec + 1652
47  libruby.3.0.dylib               0x0000000102698f8c rb_ec_exec_node + 268
48  libruby.3.0.dylib               0x0000000102698e20 ruby_run_node + 96
49  ruby                            0x000000010234fec0 main + 92
50  libdyld.dylib                   0x000000018547df34 start + 4

Thread 1:
0   libsystem_kernel.dylib          0x0000000185430e04 poll + 8
1   libruby.3.0.dylib               0x00000001027a9530 timer_pthread_fn + 124
2   libsystem_pthread.dylib         0x000000018546206c _pthread_start + 320
3   libsystem_pthread.dylib         0x000000018545cda0 thread_start + 8

Thread 2:: FFI Callback Dispatcher
0   libsystem_kernel.dylib          0x000000018542c488 __psynch_cvwait + 8
1   libsystem_pthread.dylib         0x0000000185462568 _pthread_cond_wait + 1192
2   ffi_c.bundle                    0x000000010423b80c async_cb_wait + 88 (Function.c:605)
3   libruby.3.0.dylib               0x00000001027a1c48 rb_nogvl + 280
4   ffi_c.bundle                    0x000000010423afa0 async_cb_event + 140 (Function.c:545)
5   libruby.3.0.dylib               0x00000001027a86b8 thread_start_func_2 + 1104
6   libruby.3.0.dylib               0x00000001027a8174 thread_start_func_1 + 152
7   libsystem_pthread.dylib         0x000000018546206c _pthread_start + 320
8   libsystem_pthread.dylib         0x000000018545cda0 thread_start + 8

Thread 3:: open3.rb:403
0   libsystem_kernel.dylib          0x000000018542c488 __psynch_cvwait + 8
1   libsystem_pthread.dylib         0x0000000185462568 _pthread_cond_wait + 1192
2   libruby.3.0.dylib               0x000000010279f7b0 native_cond_timedwait + 120
3   libruby.3.0.dylib               0x00000001027a81e0 thread_start_func_1 + 260
4   libsystem_pthread.dylib         0x000000018546206c _pthread_start + 320
5   libsystem_pthread.dylib         0x000000018545cda0 thread_start + 8

Thread 0 crashed with ARM Thread State (64-bit):
...

Binary Images:
       0x10234c000 -        0x10234ffff +ruby (0) <F0E1ACB6-DADC-3709-A7C9-B20CA91730D9> /opt/homebrew/*/ruby
       0x1023ec000 -        0x1023effff +encdb.bundle (0) <0512F22A-C22F-3133-B7DF-C0630FBB7291> /opt/homebrew/*/encdb.bundle
       0x102400000 -        0x102403fff +transdb.bundle (0) <0749C9D7-559E-3089-8034-3F04F71589F5> /opt/homebrew/*/transdb.bundle
       0x102414000 -        0x102417fff +monitor.bundle (0) <46C2FFE4-EE5B-3C46-98D9-E6FE36564805> /opt/homebrew/*/monitor.bundle
       0x102428000 -        0x10242ffff +pathname.bundle (0) <52304C1D-6B1E-3CF4-927D-65D050FB8E7F> /opt/homebrew/*/pathname.bundle
       0x102440000 -        0x102443fff +escape.bundle (0) <80C6BFF1-ECCE-3004-96A4-3BC8A836F380> /opt/homebrew/*/escape.bundle
       0x102454000 -        0x102483fff +date_core.bundle (0) <5BEB2E2A-A969-3022-91F0-91F2A901D94B> /opt/homebrew/*/date_core.bundle
       0x1024a0000 -        0x1024affff +bigdecimal.bundle (0) <1292C18E-90FF-3113-A794-70FC5E7A61E6> /opt/homebrew/*/bigdecimal.bundle
       0x1024c0000 -        0x1024c7fff +stringio.bundle (0) <1A3F1181-6690-3E8C-BD5F-27F8A4A2F38D> /opt/homebrew/*/stringio.bundle
       0x1024d8000 -        0x1024dbfff +etc.bundle (0) <1E536D8B-A4A9-30F3-84E5-ADF950666B25> /opt/homebrew/*/etc.bundle
       0x1024ec000 -        0x1024effff +digest.bundle (0) <F596BA9F-A8BF-3D95-B5EB-6C617E4F0927> /opt/homebrew/*/digest.bundle
       0x102500000 -        0x102503fff +strscan.bundle (0) <9548F035-0164-3C07-A923-BE0DC07C4229> /opt/homebrew/*/strscan.bundle
       0x102514000 -        0x102517fff +wait.bundle (0) <AFC11C5F-E127-30EF-86C1-9DAD1B8027DA> /opt/homebrew/*/wait.bundle
       0x10252c000 -        0x1025a7fff  dyld (832.7.3) <4AB185B3-DC20-3C03-A193-67C0E6C589D7> /usr/lib/dyld
       0x102630000 -        0x10289bfff +libruby.3.0.dylib (0) <F87CC4ED-8475-3AEE-AD7B-2AF63B4EA539> /opt/homebrew/*/libruby.3.0.dylib
       0x103e1c000 -        0x103e3bfff +socket.bundle (0) <278655A9-7512-3AFC-AA68-6A824BA8C2FA> /opt/homebrew/*/socket.bundle
       0x103e54000 -        0x103e5ffff +zlib.bundle (0) <34A11F55-C4B0-3781-84CD-7847CC56B29E> /opt/homebrew/*/zlib.bundle
       0x103e74000 -        0x103e77fff +windows_31j.bundle (0) <610116E4-A042-3F9E-8CEF-0A0B4A608E4F> /opt/homebrew/*/windows_31j.bundle
       0x103e88000 -        0x103e8ffff +parser.bundle (0) <E08CB57E-FA0E-368E-8C2A-988494BD82A6> /opt/homebrew/*/parser.bundle
       0x103ea0000 -        0x103ea7fff +generator.bundle (0) <EA3FFFF6-797E-3172-9750-EFA602F9DA15> /opt/homebrew/*/generator.bundle
       0x103eb8000 -        0x103ef7fff +openssl.bundle (0) <D2ACCAC6-35B7-365D-A590-DC0A5683A383> /opt/homebrew/*/openssl.bundle
       0x103f24000 -        0x103f6ffff +libssl.1.1.dylib (0) <387D1B0F-EE36-3D80-96B0-21BCDA8B4E62> /opt/homebrew/*/libssl.1.1.dylib
       0x103fa4000 -        0x104123fff +libcrypto.1.1.dylib (0) <6703626D-9366-31E0-8697-C74FFA806779> /opt/homebrew/*/libcrypto.1.1.dylib
       0x1041c4000 -        0x1041c7fff +nonblock.bundle (0) <4285445A-338C-3B5F-9501-8132C66F1367> /opt/homebrew/*/nonblock.bundle
       0x1041d8000 -        0x1041dffff +psych.bundle (0) <6E68A386-AEB1-38A8-9AA3-4CB61FFA0439> /opt/homebrew/*/psych.bundle
       0x1041f0000 -        0x104207fff +libyaml-0.2.dylib (0) <86720355-4A4B-32E0-8BD0-18AB73A96463> /opt/homebrew/*/libyaml-0.2.dylib
       0x104218000 -        0x10421bfff +sha2.bundle (0) <F641AD10-D5E2-3D7D-BBFA-56F8294B442B> /opt/homebrew/*/sha2.bundle
       0x10422c000 -        0x104247fff +ffi_c.bundle (0) <6ADCB3BE-9056-33F0-AD30-10DAB38241E2> /opt/homebrew/*/ffi_c.bundle
       0x104270000 -        0x104277fff  libffi-trampolines.dylib (27) <ADFD2779-8444-3C1E-8AF1-F5BDCFDDA05B> /usr/lib/libffi-trampolines.dylib
       0x104280000 -        0x104283fff +sha1.bundle (0) <B63454E5-2319-352D-864B-8CEFF2C94880> /opt/homebrew/*/sha1.bundle
       0x1851be000 -        0x1851bffff  libsystem_blocks.dylib (78) <9B6D4883-03E9-3785-851E-EA79FA64ADC1> /usr/lib/system/libsystem_blocks.dylib
       0x1851c0000 -        0x1851f7fff  libxpc.dylib (2038.80.3) <BD0DFD42-0DC3-3F3D-9C04-5A2B3D93794D> /usr/lib/system/libxpc.dylib
       0x1851f8000 -        0x18520ffff  libsystem_trace.dylib (1277.80.2) <4A466196-D2DD-367B-80AB-988F281EC3B8> /usr/lib/system/libsystem_trace.dylib
       0x185210000 -        0x185284fff  libcorecrypto.dylib (1000.80.5) <9BD8FED7-2A36-3602-A5A7-0CA87C03FB84> /usr/lib/system/libcorecrypto.dylib
       0x185285000 -        0x1852b0fff  libsystem_malloc.dylib (317.40.8) <21120432-52C1-34E4-BF01-623722FA3A41> /usr/lib/system/libsystem_malloc.dylib
       0x1852b1000 -        0x1852f4fff  libdispatch.dylib (1271.40.12) <F5BFBD55-EF70-3659-854D-9061325EB26D> /usr/lib/system/libdispatch.dylib
       0x1852f5000 -        0x18532dfff  libobjc.A.dylib (818.2) <B03625B0-501E-3AC1-8E16-08B621120EAD> /usr/lib/libobjc.A.dylib
       0x18532e000 -        0x185330fff  libsystem_featureflags.dylib (28.60.1) <297CC4DD-AFA4-3BA3-B4E1-0DF47E49C21E> /usr/lib/system/libsystem_featureflags.dylib
       0x185331000 -        0x1853b1fff  libsystem_c.dylib (1439.40.11) <A7147E08-E7C0-3842-916E-F2270A689F47> /usr/lib/system/libsystem_c.dylib
       0x1853b2000 -        0x18540dfff  libc++.1.dylib (904.4) <B139607F-1E80-3A8E-870D-0AC022069EA1> /usr/lib/libc++.1.dylib
       0x18540e000 -        0x185427fff  libc++abi.dylib (904.4) <1DD3A1C9-D765-34FB-B8C1-87BF52CE49C0> /usr/lib/libc++abi.dylib
       0x185428000 -        0x18545afff  libsystem_kernel.dylib (7195.81.3) <55FCA547-4877-3075-8A08-FE1620BFC682> /usr/lib/system/libsystem_kernel.dylib
       0x18545b000 -        0x185466fff  libsystem_pthread.dylib (454.80.2) <8E907E6C-C227-312E-944C-767093692AFF> /usr/lib/system/libsystem_pthread.dylib
       0x185467000 -        0x1854a5fff  libdyld.dylib (832.7.3) <EF759BF3-97FA-30EA-A1CA-EDECFEA726FE> /usr/lib/system/libdyld.dylib
       0x1854a6000 -        0x1854acfff  libsystem_platform.dylib (254.80.2) <8633A39C-10A2-3B44-93F7-617AB09FF640> /usr/lib/system/libsystem_platform.dylib
       0x1854ad000 -        0x1854d8fff  libsystem_info.dylib (542.40.3) <4CC96CFC-7198-3F26-8C8C-20FB010CDF98> /usr/lib/system/libsystem_info.dylib
       0x1854d9000 -        0x185982fff  com.apple.CoreFoundation (6.9 - 1774.101) <EA76C90A-23ED-3791-8FBC-8292916F0B16> /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
       0x185983000 -        0x185bb5fff  com.apple.LaunchServices (1122.11 - 1122.11) <B79A592B-8036-3E24-AD9D-3FB4E7BE2983> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/LaunchServices
       0x185bb6000 -        0x185c8efff  com.apple.gpusw.MetalTools (1.0 - 1) <ED9E3F77-4900-3B5B-978A-70AA6762DFBA> /System/Library/PrivateFrameworks/MetalTools.framework/Versions/A/MetalTools
       0x185c8f000 -        0x185ee9fff  libBLAS.dylib (1336.40.1) <96EAD889-D898-3884-A36C-F433DC2C64DD> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
       0x185eea000 -        0x185f34fff  com.apple.Lexicon-framework (1.0 - 86.1) <81EA9F2C-6059-322A-B336-56CD7F3AB6C2> /System/Library/PrivateFrameworks/Lexicon.framework/Versions/A/Lexicon
       0x185f35000 -        0x185f97fff  libSparse.dylib (106) <1A70E696-43E3-3D8B-A3E1-ADB624729BF4> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libSparse.dylib
       0x185f98000 -        0x18601ffff  com.apple.SystemConfiguration (1.20 - 1.20) <DD6AB615-BB7B-3634-9D33-3923E5038BAA> /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration
       0x186020000 -        0x186053fff  libCRFSuite.dylib (50) <79C1501B-B0F6-341A-96CC-F4FE066E3D59> /usr/lib/libCRFSuite.dylib
       0x186054000 -        0x18627efff  libmecabra.dylib (929.1.1) <29B77781-FA91-3180-AFE8-608A355AE97E> /usr/lib/libmecabra.dylib
       0x18627f000 -        0x186610fff  com.apple.Foundation (6.9 - 1774.101) <8F7A0D5B-0E89-36F6-AC84-D3BEC2C44792> /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
       0x186611000 -        0x186702fff  com.apple.LanguageModeling (1.0 - 247.1) <66B05273-1979-3DB2-9F68-C0E3A6CD88B1> /System/Library/PrivateFrameworks/LanguageModeling.framework/Versions/A/LanguageModeling
       0x18728c000 -        0x1875eafff  com.apple.security (7.0 - 59754.80.3) <C76855AD-6EE4-3413-9E6E-CC450BDB20E2> /System/Library/Frameworks/Security.framework/Versions/A/Security
       0x1875eb000 -        0x18785cfff  libicucore.A.dylib (66109) <3CE58F97-7FC3-37D8-BB99-D6BECFC86DD2> /usr/lib/libicucore.A.dylib
       0x18785d000 -        0x187867fff  libsystem_darwin.dylib (1439.40.11) <B790A863-2D74-300E-9698-A25B5602B32F> /usr/lib/system/libsystem_darwin.dylib
       0x187868000 -        0x187b57fff  com.apple.CoreServices.CarbonCore (1307 - 1307) <3EC22291-65E5-3EB6-9498-9A1244C90147> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/CarbonCore.framework/Versions/A/CarbonCore
       0x187b93000 -        0x187bcefff  com.apple.CSStore (1122.11 - 1122.11) <52D8D7A1-4879-3488-9D39-5F2C7696EFA8> /System/Library/PrivateFrameworks/CoreServicesStore.framework/Versions/A/CoreServicesStore
       0x187bcf000 -        0x187c99fff  com.apple.framework.IOKit (2.0.2 - 1845.81.1) <516911DA-18D7-3D17-8646-BBF7C75CD070> /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
       0x187c9a000 -        0x187ca5fff  libsystem_notify.dylib (279.40.4) <A7B6BDA8-5371-352E-8A36-95D46C4B07F1> /usr/lib/system/libsystem_notify.dylib
       0x1890bc000 -        0x1897c6fff  libnetwork.dylib (2288.80.2) <07EC53A0-293C-3403-8394-755AE0BDDFA4> /usr/lib/libnetwork.dylib
       0x1897c7000 -        0x189c54fff  com.apple.CFNetwork (1220.1 - 1220.1) <3C5F5D1E-DB7C-3027-BBB0-91E6DEA3E264> /System/Library/Frameworks/CFNetwork.framework/Versions/A/CFNetwork
       0x189c55000 -        0x189c64fff  libsystem_networkextension.dylib (1295.80.3) <B6BD1267-BE59-3E42-B2B5-2BF13F17D02D> /usr/lib/system/libsystem_networkextension.dylib
       0x189c65000 -        0x189c66fff  libenergytrace.dylib (22) <C5CFEF87-BB69-3351-A0C8-9B601383A45C> /usr/lib/libenergytrace.dylib
       0x189c67000 -        0x189cbbfff  libMobileGestalt.dylib (978.80.1) <93C6E288-C098-357F-B8A5-3E133DF39ECE> /usr/lib/libMobileGestalt.dylib
       0x189cbc000 -        0x189cd3fff  libsystem_asl.dylib (385) <31E28E59-1CDD-3B83-8BF0-56C675227FA2> /usr/lib/system/libsystem_asl.dylib
       0x189cd4000 -        0x189cedfff  com.apple.TCC (1.0 - 1) <C55FE947-0C86-3AAC-9306-9EFA7C033D07> /System/Library/PrivateFrameworks/TCC.framework/Versions/A/TCC
       0x18aea2000 -        0x18aeb9fff  com.apple.ProtocolBuffer (1 - 285.23.11.29.1) <38163CA8-14FF-34A9-8AE4-D7D69B8C8854> /System/Library/PrivateFrameworks/ProtocolBuffer.framework/Versions/A/ProtocolBuffer
       0x18aeba000 -        0x18b061fff  libsqlite3.dylib (321.1) <8592B35B-9EA3-3C84-8453-9C86FB5C039C> /usr/lib/libsqlite3.dylib
       0x18b1c8000 -        0x18b23bfff  com.apple.AE (918.0.1 - 918.0.1) <7D13C9B5-D195-3E9E-B6C7-254F95A925C6> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/AE.framework/Versions/A/AE
       0x18b23c000 -        0x18b243fff  libdns_services.dylib (1310.80.1) <64D4BA25-C388-3AB8-BDA8-2E81459DA46A> /usr/lib/libdns_services.dylib
       0x18b244000 -        0x18b24bfff  libsystem_symptoms.dylib (1431.40.36) <0657E539-C0CE-30F8-B630-FBAE36109542> /usr/lib/system/libsystem_symptoms.dylib
       0x18b24c000 -        0x18b3a8fff  com.apple.Network (1.0 - 1) <486C55B3-900C-3D09-AB0D-F99A152CFB84> /System/Library/Frameworks/Network.framework/Versions/A/Network
       0x18b3a9000 -        0x18b3cefff  com.apple.analyticsd (1.0 - 1) <E47FE17B-2ED2-3BE2-A5AB-046DB3C02EA0> /System/Library/PrivateFrameworks/CoreAnalytics.framework/Versions/A/CoreAnalytics
       0x18b3cf000 -        0x18b3d1fff  libDiagnosticMessagesClient.dylib (112) <20AD555E-DF00-3C91-A95B-AB2AD23780AA> /usr/lib/libDiagnosticMessagesClient.dylib
       0x18b3d2000 -        0x18b41dfff  com.apple.spotlight.metadata.utilities (1.0 - 2150.7.5) <5400DF7A-9249-30E9-B692-AC431C7F74D9> /System/Library/PrivateFrameworks/MetadataUtilities.framework/Versions/A/MetadataUtilities
       0x18b41e000 -        0x18b4b7fff  com.apple.Metadata (10.7.0 - 2150.7.5) <4B03E6F8-1568-338B-AA75-480F9D824516> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Metadata
       0x18b4b8000 -        0x18b4befff  com.apple.DiskArbitration (2.7 - 2.7) <7ED2211D-BA3C-37EC-BBA4-4320FBBC8A6A> /System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration
       0x18b4bf000 -        0x18b7ddfff  com.apple.vImage (8.1 - 544.2) <B1B84588-8B57-3F98-9D50-AAC142DFF36E> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vImage.framework/Versions/A/vImage
       0x18bcda000 -        0x18bce7fff  com.apple.OpenDirectory (11.2 - 230.40.1) <C509DC91-F994-34B7-A5C5-A108E7DA5E4E> /System/Library/Frameworks/OpenDirectory.framework/Versions/A/OpenDirectory
       0x18bce8000 -        0x18bd07fff  com.apple.CFOpenDirectory (11.2 - 230.40.1) <A5449895-6129-3BDF-864B-49ACA82E3052> /System/Library/Frameworks/OpenDirectory.framework/Versions/A/Frameworks/CFOpenDirectory.framework/Versions/A/CFOpenDirectory
       0x18bd08000 -        0x18bd10fff  com.apple.CoreServices.FSEvents (1290.40.2 - 1290.40.2) <72CF142E-3792-318C-B2D6-B60C5E219312> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/FSEvents.framework/Versions/A/FSEvents
       0x18bd11000 -        0x18bd35fff  com.apple.coreservices.SharedFileList (144 - 144) <E4152CCC-4A03-3959-B788-D7DD1ADFF8A6> /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/SharedFileList.framework/Versions/A/SharedFileList
       0x18bd36000 -        0x18bd38fff  libapp_launch_measurement.dylib (14.1) <A5637442-ADCB-30F0-AAB4-72FA1C5E3811> /usr/lib/libapp_launch_measurement.dylib
       0x18bd39000 -        0x18bd7cfff  com.apple.CoreAutoLayout (1.0 - 21.10.1) <6026D662-F75F-3C82-8C80-F6CEBF6369BF> /System/Library/PrivateFrameworks/CoreAutoLayout.framework/Versions/A/CoreAutoLayout
       0x18bd7d000 -        0x18be68fff  libxml2.2.dylib (34.9) <E170FFEE-EB9F-3252-9B16-4A47FBAC425A> /usr/lib/libxml2.2.dylib
       0x18ccb9000 -        0x18cccafff  libsystem_containermanager.dylib (318.80.2) <0B742EA4-AFA2-36B6-AB4B-2F8ACA7211AA> /usr/lib/system/libsystem_containermanager.dylib
       0x18cccb000 -        0x18ccdcfff  com.apple.IOSurface (289.3 - 289.3) <64E3394D-C908-378C-B5CE-B89C6BE61E9F> /System/Library/Frameworks/IOSurface.framework/Versions/A/IOSurface
       0x18ccdd000 -        0x18cce6fff  com.apple.IOAccelerator (439.52 - 439.52) <2995471C-4A7A-342A-B702-496519D138E2> /System/Library/PrivateFrameworks/IOAccelerator.framework/Versions/A/IOAccelerator
       0x18cce7000 -        0x18cdddfff  com.apple.Metal (244.32.7 - 244.32.7) <E8BC8E6D-4359-3A04-87CB-6F0D4F8F5C4B> /System/Library/Frameworks/Metal.framework/Versions/A/Metal
       0x18d8e4000 -        0x18d93bfff  com.apple.MetalPerformanceShaders.MPSCore (1.0 - 1) <F07355C4-C893-3534-B74E-DD5FBCBEC76C> /System/Library/Frameworks/MetalPerformanceShaders.framework/Versions/A/Frameworks/MPSCore.framework/Versions/A/MPSCore
       0x18d93c000 -        0x18d940fff  libsystem_configuration.dylib (1109.60.2) <AEC5E654-A5B8-343E-80B7-27D5D0D856D9> /usr/lib/system/libsystem_configuration.dylib
       0x18d941000 -        0x18d945fff  libsystem_sandbox.dylib (1441.60.4) <D7CDDE27-978E-3511-AE6F-296D901290B2> /usr/lib/system/libsystem_sandbox.dylib
       0x18d946000 -        0x18d947fff  com.apple.AggregateDictionary (1.0 - 1) <516D38F6-E0E1-36B4-AC96-E5079ECC6ED4> /System/Library/PrivateFrameworks/AggregateDictionary.framework/Versions/A/AggregateDictionary
       0x18d948000 -        0x18d94bfff  com.apple.AppleSystemInfo (3.1.5 - 3.1.5) <E6509790-A434-3A6A-AF9E-EA1FDBF15F6A> /System/Library/PrivateFrameworks/AppleSystemInfo.framework/Versions/A/AppleSystemInfo
       0x18d94c000 -        0x18d94dfff  liblangid.dylib (136) <12979BA7-28E3-3E74-AC24-65166A921235> /usr/lib/liblangid.dylib
       0x18d94e000 -        0x18d9dcfff  com.apple.CoreNLP (1.0 - 245.1) <92E28F08-9AB8-3B02-A889-677A716E393C> /System/Library/PrivateFrameworks/CoreNLP.framework/Versions/A/CoreNLP
       0x18d9dd000 -        0x18d9e4fff  com.apple.LinguisticData (1.0 - 399) <2B3E7B26-D669-38C0-9B7F-FFB2E94BC23B> /System/Library/PrivateFrameworks/LinguisticData.framework/Versions/A/LinguisticData
       0x18d9e5000 -        0x18de9afff  libBNNS.dylib (288.80.1) <695BEB14-BA36-3386-8C72-A219A73C9601> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBNNS.dylib
       0x18de9b000 -        0x18df4dfff  libvDSP.dylib (760.40.6) <9FAC0A5B-0CD5-3999-815A-3C663EC71F65> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libvDSP.dylib
       0x18df4e000 -        0x18df60fff  com.apple.CoreEmoji (1.0 - 128) <0FCD33E9-8DC4-3FB1-86DC-8ECA2FA10E37> /System/Library/PrivateFrameworks/CoreEmoji.framework/Versions/A/CoreEmoji
       0x18df61000 -        0x18df6bfff  com.apple.IOMobileFramebuffer (343.0.0 - 343.0.0) <57F4592F-D6FA-3ED9-ACEF-B1D7A7359DC1> /System/Library/PrivateFrameworks/IOMobileFramebuffer.framework/Versions/A/IOMobileFramebuffer
       0x18e26b000 -        0x18e2f0fff  com.apple.securityfoundation (6.0 - 55240.40.4) <6D4E864F-4AAC-31AF-85AF-C308528C0F0B> /System/Library/Frameworks/SecurityFoundation.framework/Versions/A/SecurityFoundation
       0x18e2f1000 -        0x18e2fafff  com.apple.coreservices.BackgroundTaskManagement (1.0 - 104) <8981AD5E-DC12-3B54-BAC0-E9E113C4B1A0> /System/Library/PrivateFrameworks/BackgroundTaskManagement.framework/Versions/A/BackgroundTaskManagement
       0x18e2fb000 -        0x18e300fff  com.apple.xpc.ServiceManagement (1.0 - 1) <7B4325ED-9CF7-3D9E-A39C-F0AE7819F0C0> /System/Library/Frameworks/ServiceManagement.framework/Versions/A/ServiceManagement
       0x18e301000 -        0x18e303fff  libquarantine.dylib (119.40.2) <66942A5C-57B3-3524-BA49-0F2DA4A584D1> /usr/lib/system/libquarantine.dylib
       0x18e304000 -        0x18e313fff  libCheckFix.dylib (31) <05E93C9D-45F9-3758-95FD-481E5EA2D5EF> /usr/lib/libCheckFix.dylib
       0x1913f9000 -        0x19140cfff  libsasl2.2.dylib (214) <D2C32F92-0728-3C67-B774-5CFC95F83497> /usr/lib/libsasl2.2.dylib
       0x191cb4000 -        0x191cc4fff  com.apple.Kerberos (3.0 - 1) <862CCDF8-B5F2-3EDE-B728-B62CE0F158F7> /System/Library/Frameworks/Kerberos.framework/Versions/A/Kerberos
       0x191cc5000 -        0x191d0dfff  com.apple.GSS (4.0 - 2.0) <F4CC9D06-2046-3621-B654-96E7575115ED> /System/Library/Frameworks/GSS.framework/Versions/A/GSS
       0x191d0e000 -        0x191d1dfff  com.apple.CommonAuth (4.0 - 2.0) <DEE913BB-9741-3BB8-933A-64623CBCCC70> /System/Library/PrivateFrameworks/CommonAuth.framework/Versions/A/CommonAuth
       0x191eea000 -        0x191eeafff  liblaunch.dylib (2038.80.3) <327A64BF-8C2C-303D-91E4-70A168C5D87F> /usr/lib/system/liblaunch.dylib
       0x191eec000 -        0x191efdfff  libffi.dylib (27) <92403FAB-6424-3AA3-ADB4-98FCD46D892A> /usr/lib/libffi.dylib

Output:
After ffi has been fixed, I was able to successfully run pod install
without Rosetta
by running sudo gem install ethon with my homebrewn gem that installed ethon-0.13.0
and sudo gem install ffi that installed ffi-1.15.0
Thanks for trying to help


Instruction: How to calculate mean on second axis until row-specific column in Pytorch?

Input:
I am looking for a fast way to calculate the mean for each row of a 2d matrix but only until a specific column. The remaining values of each row can be ignored. The column is different for each row.
In Numpy, it could be coded like this. However, I am hoping to find a solution without a for loop which also does not break the gradients.
import numpy as np

arr = np.linspace(0, 10, 15).reshape(3,5)
cols = [2,0,4]

for row, col in enumerate(cols):
    arr[row, col+1:] = np.nan

result = np.nanmean(arr, axis=1)

Any suggestions?
Edit: Best solution I have found so far:
result = torch.stack([arr[i, 0:cols[i]+1].mean() for i in range(len(arr))])

But I would still like to avoid the for loop.

Output:
Try creating a mask:
t = torch.tensor(arr)
mask = torch.arange(t.shape[1]) <= torch.tensor(cols).unsqueeze(-1)

result = (t*mask).sum(1)/mask.sum(1)

Output:
tensor([0.7143, 3.5714, 8.5714], dtype=torch.float64)


Instruction: How to manually obtain the minus log-likelihood in Pytorch?

Input:
I'm implementing a VAE and I want to obtain the negativev log-likelihood manually (not using an existing function). The given equation is equation1, and I have also found it can as well be expressed as equation2. I have been stuck on this for a couple of days now, and don't know where my code is wrong.
def loss_loglik(y_mean, y_logvar, x):
    out_1 = (x.size()[2]*x.size()[3] / 2) * np.log(2 * np.pi)
    out_2 = (x.size()[2]*x.size()[3] / 2) * torch.log(y_logvar.exp())
    x_diff = x - y_mean
    out_3 = torch.sum(x_diff.pow(2)) / (2 * y_logvar.exp())
    loss = out_1 + out_2 + out_3

The shape of the three arguments is (batch_size, 1, 28, 28).

Output:
It seems like eq. 2 is wrong.
It should have been something like. I did not derive it, just trying to match with the input... so, please verify.

I modified your function below.
def loss_loglik(y_mean, y_logvar, x):
    m, n  = x.size()[2], x.size()[3]
    b = x.size()[0]

    y_mean = y_mean.reshape(B, -1)
    y_logvar = y_loagvar.reshape(B, -1)

    out_1 = (m*n / 2) * np.log(2 * np.pi)

    out_2 = (1 / 2) * y_logvar.sum(dim=1)

    x_diff = x - y_mean
    # note sigma is inside the sum
    out_3 = torch.sum(x_diff.pow(2) / (2 * y_logvar.exp()), dim=1)

    loss = out_1 + out_2 + out_3

    return -loss
    # shape of loss will be (batchsize,)


Instruction: reshape tensor in a determined way

Input:
I have some tensor x3. I got it in the following way:
x = torch.tensor([0, 0, 0, 0, 1, 0, 0, 0, 0])
x2 = torch.stack(5 * [x], 0)
x2 = x2.reshape(-1)
x3 = torch.stack(4 * [x2], 0)
x3 = torch.stack(6 * [x3], -1)
x3 = torch.stack(7 * [x3], -1)

In short it means that
x[0, :9, 0, 0] = [0, 0, 0, 0, 1, 0, 0, 0, 0]
x[0, 9:18, 0, 0] = [0, 0, 0, 0, 1, 0, 0, 0, 0]

and so on.
Then I want to reshape it that every nine values of the 1st dimension go to the new dimension. In other words, I want x3[0, 0, 0, 0, :] to give me tensor([0, 0, 0, 0, 1, 0, 0, 0, 0])
I tried to do:
x3.reshape(4, 5, 6, 7, 9)[0, 0, 0, 0, :]
tensor([0, 0, 0, 0, 0, 0, 0, 0, 0])
x3.reshape(4, 9, 6, 7, 5).transpose(1, -1)[0, 0, 0, 0, :]
tensor([0, 0, 0, 0, 0, 0, 0, 0, 1])

As you see, none of it gives me the right answer
UPD: added x3 = torch.stack(7 * [x3], -1)

Output:
If you want to modify the 1st dimension and create a new dimension at the end, you need to first move that dimension to last by using permute. Something like this should do the trick:
xpermuted = x3.permute(0, 2, 3, 1)
xreshaped = xpermuted.reshape(xpermuted.shape[0], xpermuted.shape[1], xpermuted.shape[2], int(xpermuted.shape[3] / 9), 9)

print(xreshaped[0, 0, 0, 0, :])     # tensor([0, 0, 0, 0, 1, 0, 0, 0, 0])
print(xreshaped[0, 0, 0, 1, :])     # tensor([0, 0, 0, 0, 1, 0, 0, 0, 0])
print(xreshaped[0, 0, 0, 2, :])     # tensor([0, 0, 0, 0, 1, 0, 0, 0, 0])

After that, you can restore the initial dimension order by using permute again if you need the original order of dimensions:
xrestored = xreshaped.permute(0, 3, 1, 2, 4)
print(xrestored.shape)              # torch.Size([4, 5, 6, 7, 9])

Technically you don't have to move the first dimension to last initially, you can also do the reverse first reshape, then permute. Actually, now that I think about it, this is better since it has one less permute:
xreshaped = x3.reshape(x3.shape[0], int(x3.shape[1]/9), 9, x3.shape[2], x3.shape[3])
xrestored = xreshaped.permute(0, 1, 3, 4, 2)

print(xrestored.shape)              # torch.Size([4, 5, 6, 7, 9])
print(xrestored[0, 0, 0, 0, :])     # tensor([0, 0, 0, 0, 1, 0, 0, 0, 0])
print(xrestored[0, 1, 0, 0, :])     # tensor([0, 0, 0, 0, 1, 0, 0, 0, 0])


Instruction: Flutter- Connecting to Python/Pytorch Backend

Input:
I'm building a mobile app using Flutter that translates American Sign Language (ASL) to written English. As a prototype, I've limited my scope to translating photos of single alphanumeric characters. I have two parts to the project right now:

A python script that takes the path to an image, processes it with Torch/Torchvision and a custom model, and returns the alphanumeric character that is most likely to have been in the image. This part works great.

A Flutter front end that connects to the user's camera and saves the path to a taken image.

Q: What is the best way to connect these two parts of my project?
So far, I've tried two methods:

Using Starflut, which allows me to call Python code from Flutter and receive return values. I've tested it with some basic Python code (no dependencies) and it works great. However, for my more complex use case, where my Python code imports "torch", the code crashes.

Error Message:
:/data/data/com.example.asl/files/flutter_assets/starfiles/image_prediction.py, run failed

ModuleNotFoundError: No module named 'torch'

This happens when calling the following line of code:
    var result = await widget.srvGroup.loadRawModule("python", "",
    resPath + "/flutter_assets/starfiles/" + "image_prediction.py", false);

I've also tried Pytorch_mobile Flutter package, but beyond not being working exactly in the way I want and implemented in my own Python script, also crashes.

Error Message:
java.lang.NullPointerException: Attempt to invoke virtual method 'org.pytorch.IValue org.pytorch.Module.forward(org.pytorch.IValue[])' on a null object reference

This happens when calling the following line of code:
String prediction =
    await TorchMobile.getPrediction(_image, maxWidth: 256, maxHeight: 256);

Any suggestions as to how to get Starflut functioning would be ideal, as I'd like to use my Python interface, but if that's not possible, I'd love suggestions as to how to approach this problem.

Output:
starflut package is used to compile python programs or any other scripting language in Flutter. This package will not help you to integrate your backend model with the application
To connect backend with your mobile application you'll have to write your own API or you can use Pytorch_mobile Flutter package.
Your app is crashing at -
String prediction =
    await TorchMobile.getPrediction(_image, maxWidth: 256, maxHeight: 256);

because it seems you forgot to initialize _image. Unfortunately, NULL is being stored in _image. So recheck your code where you've initialized _image.
To get a clear understanding you can definitely check out the following videos

Building An App From Scratch: Connecting Python Backend with Flutter Frontend | #5
How To Use Pytorch Mobile on Flutter


Instruction: How to load a learning rate scheduler state dict?

Input:
I have a model and a learning rate scheduler. I'm saving the model and optimizer using the state dict method that is shown here.
import torch
import torch.nn as nn
import torch.optim as optim

class net_x(nn.Module):
        def __init__(self):
            super(net_x, self).__init__()
            self.fc1=nn.Linear(2, 20)
            self.fc2=nn.Linear(20, 20)
            self.out=nn.Linear(20, 4)

        def forward(self, x):
            x=self.fc1(x)
            x=self.fc2(x)
            x=self.out(x)
            return x

nx = net_x()

r = torch.tensor([1.0,2.0])
optimizer = optim.Adam(nx.parameters(), lr = 0.1)
scheduler = torch.optim.lr_scheduler.CyclicLR(optimizer, base_lr=1e-2, max_lr=0.1, step_size_up=1, mode="triangular2", cycle_momentum=False)

path = 'opt.pt'
for epoch in range(10):
    optimizer.zero_grad()
    net_predictions = nx(r)
    loss = torch.sum(torch.randint(0,10,(4,)) - net_predictions)
    loss.backward()
    optimizer.step()
    scheduler.step()
    print('loss:' , loss)
    torch.save({    'epoch': epoch,
                    'net_x_state_dict': nx.state_dict(),
                    'optimizer_state_dict': optimizer.state_dict(),
                    'scheduler': scheduler,
                    }, path)

PATH = control_path
checkpoint = torch.load(path)
nx.load_state_dict(checkpoint['net_x_state_dict'])
optimizer.load_state_dict(checkpoint['optimizer_state_dict'])
scheduler.load_state_dict(checkpoint['scheduler'])

The code runs just fine without the part that I'm loading the scheduler state dict, so I'm not sure what I'm doing wrong. I'm trying to load the state dict as mentioned here, but I'm getting this error:
TypeError                                 Traceback (most recent call last)
<ipython-input-7-e3217d6dd870> in <module>
     42 nx.load_state_dict(checkpoint['net_x_state_dict'])
     43 optimizer.load_state_dict(checkpoint['optimizer_state_dict'])
---> 44 scheduler.load_state_dict(checkpoint['scheduler'])

~/anaconda3/lib/python3.7/site-packages/torch/optim/lr_scheduler.py in load_state_dict(self, state_dict)
     92                 from a call to :meth:`state_dict`.
     93         """
---> 94         self.__dict__.update(state_dict)
     95
     96     def get_last_lr(self):

TypeError: 'CyclicLR' object is not iterable

Output:
Since we have to extract the state_dict() values from scheduler before saving i.e in torch.save() method
Below code will work
import torch
import torch.nn as nn
import torch.optim as optim

class net_x(nn.Module):
        def __init__(self):
            super(net_x, self).__init__()
            self.fc1=nn.Linear(2, 20)
            self.fc2=nn.Linear(20, 20)
            self.out=nn.Linear(20, 4)

        def forward(self, x):
            x=self.fc1(x)
            x=self.fc2(x)
            x=self.out(x)
            return x

nx = net_x()

r = torch.tensor([1.0,2.0])
optimizer = optim.Adam(nx.parameters(), lr = 0.1)
scheduler = torch.optim.lr_scheduler.CyclicLR(optimizer, base_lr=1e-2, max_lr=0.1, step_size_up=1, mode="triangular2", cycle_momentum=False)

path = 'opt.pt'
for epoch in range(10):
    optimizer.zero_grad()
    net_predictions = nx(r)
    loss = torch.sum(torch.randint(0,10,(4,)) - net_predictions)
    loss.backward()
    optimizer.step()
    scheduler.step()
    print('loss:' , loss)
    torch.save({    'epoch': epoch,
                    'net_x_state_dict': nx.state_dict(),
                    'optimizer_state_dict': optimizer.state_dict(),
                    'scheduler': scheduler.state_dict(),    # HERE IS THE CHANGE
                    }, path)

PATH = control_path
checkpoint = torch.load(path)
nx.load_state_dict(checkpoint['net_x_state_dict'])
optimizer.load_state_dict(checkpoint['optimizer_state_dict'])
scheduler.load_state_dict(checkpoint['scheduler'])


Instruction: TypeError: conv2d(): argument 'input' (position 1) must be Tensor, not str

Input:
When I try to train the model I get the following error:

TypeError: conv2d(): argument 'input' (position 1) must be Tensor, not
str

The code I'm using is:
class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(1, 6, kernel_size=5)
        self.conv2 = nn.Conv2d(6, 16, kernel_size=5)
        self.dropout = nn.Dropout2d()
        self.fc1 = nn.Linear(256, 64)
        self.fc2 = nn.Linear(64, 1)
        self.hybrid = Hybrid(qiskit.Aer.get_backend('qasm_simulator'), 100, np.pi / 2)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        x = F.max_pool2d(x, 2)
        x = F.relu(self.conv2(x))
        x = F.max_pool2d(x, 2)
        x = self.dropout(x)
        x = x.view(1, -1)
        x = F.relu(self.fc1(x))
        x = self.fc2(x)
        x = self.hybrid(x)
        return torch.cat((x, 1 - x), -1)

model = Net()
optimizer = optim.Adam(model.parameters(), lr=0.001)
loss_func = nn.NLLLoss()

epochs = 20
loss_list = []

model.train()

for epoch in range(epochs):
    total_loss = []

    for i, data in enumerate(train_ldr, 0):
        # get the inputs; data is a list of [inputs, labels]
        X_train, Y_train = data
        print(data)
        optimizer.zero_grad()

        # Forward pass
        output = model(X_train)
        # Calculating loss
        loss = loss_func(output, Y_train)
        # Backward pass
        loss.backward()
        # Optimize the weights
        optimizer.step()

        total_loss.append(loss.item())
    loss_list.append(sum(total_loss)/len(total_loss))
    print('Training [{:.0f}%]\tLoss: {:.4f}'.format(
        100. * (epoch + 1) / epochs, loss_list[-1]))

The full traceback is:
{'data': tensor([[715.9147, 679.4994, 131.4772,   9.4777,   9.4777,  13.8722,  85.8577,
           2.5333]]), 'Target': tensor([0])}

TypeError                                 Traceback (most recent call last)
<ipython-input-52-7c8c9f3a38b7> in <module>
     20
     21         # Forward pass
---> 22         output = model(X_train)
     23         # Calculating loss
     24         loss = loss_func(output, Y_train)

~\anaconda3\lib\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)
    887             result = self._slow_forward(*input, **kwargs)
    888         else:
--> 889             result = self.forward(*input, **kwargs)
    890         for hook in itertools.chain(
    891                 _global_forward_hooks.values(),

<ipython-input-39-6b9a402c220d> in forward(self, x)
     10
     11     def forward(self, x):
---> 12         x = F.relu(self.conv1(x))
     13         x = F.max_pool2d(x, 2)
     14         x = F.relu(self.conv2(x))

~\anaconda3\lib\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)
    887             result = self._slow_forward(*input, **kwargs)
    888         else:
--> 889             result = self.forward(*input, **kwargs)
    890         for hook in itertools.chain(
    891                 _global_forward_hooks.values(),

~\anaconda3\lib\site-packages\torch\nn\modules\conv.py in forward(self, input)
    397
    398     def forward(self, input: Tensor) -> Tensor:
--> 399         return self._conv_forward(input, self.weight, self.bias)
    400
    401 class Conv3d(_ConvNd):

~\anaconda3\lib\site-packages\torch\nn\modules\conv.py in _conv_forward(self, input, weight, bias)
    393                             weight, bias, self.stride,
    394                             _pair(0), self.dilation, self.groups)
--> 395         return F.conv2d(input, weight, bias, self.stride,
    396                         self.padding, self.dilation, self.groups)
    397

TypeError: conv2d(): argument 'input' (position 1) must be Tensor, not str

When I try to debug the code, the X_train and Y_train hold 'Data' & 'Target'.
I don't understand why the tensor values are not being taken by the enumerate (dataloader) in the for loop.
As otherwise, values are present in the dataloader. If each tensor row in the dataset has 'Data' and 'Target' prefixed before it, how do I remove that. Please suggest any solution.

Output:
The problem is that data is a dictionary and when you unpack it the way you did (X_train, Y_train = data) you unpack the keys while you are interested in the values.
refer to this simple example:
d = {'a': [1,2], 'b': [3,4]}
x, y = d
print(x,y) # a b

So you should change this:
X_train, Y_train = data

into this:
X_train, Y_train = data.values()


Instruction: Streamlit : TypeError: a bytes-like object is required, not 'Tensor'

Input:
I am working on a style transfer task, my model returns a tensor. recently I was saving that image using torchvision.utils
torchvision.utils.save_image(genarated_image, result_path)

now I have passed the same image to streamlit.
def image_input():
content_file = st.sidebar.file_uploader("Choose a Content Image", type=["png", "jpg", "jpeg"])
if content_file is not None:
    content = Image.open(content_file)
    content = np.array(content)  # pil to cv
    content = cv2.cvtColor(content, cv2.COLOR_RGB2BGR)
else:
    st.warning("Upload an Image OR Untick the Upload Button)")
    st.stop()

WIDTH = st.sidebar.select_slider('QUALITY (May reduce the speed)', list(range(150, 501, 50)), value=200)
content = imutils.resize(content, width=WIDTH)
generated = genarate_image(content)
st.sidebar.image(content, width=300, channels='BGR')
st.image(generated, channels='BGR', clamp=True)

But now streamlit giving me this error.

TypeError: a bytes-like object is required, not 'Tensor'

is there a way to convert tensor into a "bytes-like object" ?

Output:
It can be solved by transforming tonsor to PIL Image.
from torchvision import transforms

def trans_tensor_to_pil(tensor_img):
pil_image = transforms.ToPILImage()(tensor_img.squeeze_(0))
return pil_image