Pytorch Train & Eval Different Sample Sizes
I am learning pytorch, and have the following (abbreviated) code to setup for modeling:

# define the model class for a neural net with 1 hidden layer
class myNN(nn.Module):
    def __init__(self, D_in, H, D_out):
        super(myNN, self).__init__()
        self.lin1 = nn.Linear(D_in,H)
        self.lin2 = nn.Linear(H,D_out)
    def forward(self,X):
        return torch.sigmoid(self.lin2(torch.sigmoid(self.lin1(x))))

# now make the datasets &amp; dataloaders
batchSize = 5
# Create the data class
class Data(Dataset):
    def __init__(self, x, y):
        self.x = torch.FloatTensor(x)
        self.y = torch.Tensor(y.astype(int))
        self.len = self.x.shape[0]
        self.p = self.x.shape[1]
    def __getitem__(self, index):      
        return self.x[index], self.y[index]
    def __len__(self):
        return self.len
trainData = Data(trnX, trnY)
trainLoad = DataLoader(dataset = trainData, batch_size = batchSize)
testData = Data(tstX, tstY)
testLoad = DataLoader(dataset = testData, batch_size = len(testData))

# define the modeling objects
hiddenLayers = 30
learningRate = 0.1
model = myNN(p,hiddenLayers,1)
print(model)
optimizer = torch.optim.SGD(model.parameters(), lr = learningRate)
loss = nn.BCELoss()


with trnX.shape=(70, 2), trnY.shape=(70,), tstX.shape=(30,2), and tstY.shape=(30,). The code for training is:

# train!
epochs = 1000
talkFreq = 0.2
trnLoss = [np.inf]*epochs
tstLoss = [np.inf]*epochs
for i in range(epochs):
    # train with minibatch gradient descent
    for x, y in trainLoad:
        # forward step
        yhat = model(x)
        # compute loss (not storing for now, will do after minibatching)
        l = loss(yhat, y)
        # backward step
        optimizer.zero_grad()
        l.backward()
        optimizer.step()
    # evaluate loss on training set
    yhat = model(trainData.x)
    trnLoss[i] = loss(yhat, trainData.y)
    # evaluate loss on testing set
    yhat = model(testData.x)
    tstLoss[i] = loss(yhat, testData.y)


The datasets trainData and testData have 70 and 30 observations, respectively. This is probably just a newbie problem, but when I run the training cell, it errors on the trnLoss[i] = loss(yhat, trainData.y) line with the error

ValueError: Target and input must have the same number of elements. target nelement (70) != input nelement (5)


When I inspect the output of the yhat=model(trainData.x) line, I see that yhat is a tensor with batchSize elements, despite the fact that trainData.x.shape = torch.Size([70, 2]).

How can I iteratively train the model with the minibatch gradient descent, then use the model to compute the loss &amp; accuracy on the full training &amp; testing sets? I tried setting model.train() just before the mini batch iteration, then model.eval() just before the evaluation code, to no avail.

In myNN.forward(), you are passing lower case x as input to self.lin1 while input parameter to the function is named as capital case X. Small case x is kind of global variable defined at for loop for trainload hence you are not getting any syntax error, but the value that you intend to pass is not being passed to self.lin1.

Might I also suggest that you should consider using model.eval() and with torch.no_grad() for your testing code. It is not absolutely necessary here but it will make more sense.
