How do I set the dimensions of Conv1d correctly?
This is a toy example as I'm learning PyTorch and using it on one-dimensional time series, in this case a sine wave.
I'm trying to use Conv1d, but I get the following error:
RuntimeError: Given groups=1, weight of size [5, 1, 2], expected input[1, 994, 5] to have 1 channels, but got 994 channels instead

My 'lookback' is 5 time steps, and the shape of my data batch is [994, 5].
What am I doing wrong?
import torch;from torch.utils.data import Dataset, DataLoader
import torch.nn.functional as F;import pytorch_lightning as pl
from torch import nn, tensor

class TsDs(torch.utils.data.Dataset):
  def __init__(self, s, l=5): super().__init__();self.l,self.s=l,s
  def __len__(self): return self.s.shape[0] - 1 - self.l
  def __getitem__(self, i): return self.s[i:i+self.l], torch.log(self.s[i+self.l+1]/self.s[i+self.l])
  def plt(self): plt.plot(self.s)

class TsDm(pl.LightningDataModule):
  def __init__(self, length=5000, batch_size=1000): super().__init__();self.batch_size=batch_size;self.s = torch.sin(torch.arange(length)*0.2) + 5
  def train_dataloader(self): return DataLoader(TsDs(self.s[:3999]), batch_size=self.batch_size, shuffle=False)
  def val_dataloader(self): return DataLoader(TsDs(self.s[4000:]), batch_size=self.batch_size)

dm = TsDm()

class MyModel(pl.LightningModule):
    def __init__(self, learning_rate=0.01):
        super().__init__();self.learning_rate = learning_rate
        super().__init__();self.learning_rate = learning_rate
        self.network = nn.Sequential(nn.Conv1d(1,5,2),nn.ReLU(),nn.Linear(5,3),nn.ReLU(),nn.Linear(3,1), nn.Tanh())
        # self.network = nn.Sequential(nn.Linear(5,5),nn.ReLU(),nn.Linear(5,3),nn.ReLU(),nn.Linear(3,1), nn.Tanh())
    def forward(self, x): return self.network(x)
    def step(self, batch, batch_idx, stage):
        x, y = batch
        loss = -torch.mean(self(x)*y)
        print(loss)
        return loss
    def training_step(self, batch, batch_idx): return self.step(batch, batch_idx, &quot;train&quot;)
    def validation_step(self, batch, batch_idx): return self.step(batch, batch_idx, &quot;val&quot;)
    def configure_optimizers(self): return torch.optim.SGD(self.parameters(), lr=self.learning_rate)

mm = MyModel(0.01);trainer = pl.Trainer(max_epochs=10)
trainer.fit(mm, datamodule=dm)


There are two issues in your code:

Looking at the documentation of nn.Conv1d, your input shape should be (B, C, L). In your default case, you have L=5, the sequence length, but you need to create that extra dimension representing the feature size of a sequence element, here C=1. You can do so by changing TsDs's  __getitem__ function to:
def __getitem__(self, i):
    x = self.s[i:i+self.l] # minibatch x shaped (1, self.l)
    y = torch.log(self.s[i+self.l+1]/self.s[i+self.l]) # minibatch y shaped (1,)
    return x, y


Your convolutional layer has a stride of 1 and a size of 2, this means its output will be shaped (B, 5, L-1=4). The following layer is a fully connected layer instantiated as nn.Linear(5, 3), which means it expects (*, H_in=5) and will output (*, H_out). You can either

You can flatten the conv1d output with nn.Flatten and feed it to a bigger fully connected layer (for instance nn.Linear(20, 3).

You can use a convolutional layer with a wider kernel, if you use a kernel of 5 (your sequence length you will end up with a tensor of (B, 5, 1) which you feed to a nn.Linear(5, 3). Although this approach doesn't really scale when L is changed.

You could apply a nn.AvgPool1d to get an average representation of the sequence after the convolutional layers have been applied.


Those are just a few directions...


