Element wise calculation breaks autograd
I am using pytorch to calculate loss for a logistic regression (I know pytorch can do this automatically but I have to make it myself). My function is defined below but the cast to torch.tensor breaks autograd and gives me w.grad = None. Im new to pytorch so Im sorry.   

logistic_loss = lambda X,y,w: torch.tensor([torch.log(1 + torch.exp(-y[i] * torch.matmul(w, X[i,:]))) for i in range(X.shape[0])], requires_grad=True)


Your post isn't very clear on details and this is a monster of a one-liner. I first reworked it to make a minimal, complete, verifiable example. Please correct me if I misunderstood your intentions and please do it yourself next time.

import torch

# unroll the one-liner to have an easier time understanding what's going on
def logistic_loss(X, y, w):
    elementwise = []
    for i in range(X.shape[0]):
        mm = torch.matmul(w, X[i, :])
        exp = torch.exp(-y[i] * mm)
        elementwise.append(torch.log(1 + exp))

    return torch.tensor(elementwise, requires_grad=True)

# I assume that's the excepted dimensions of your input
X = torch.randn(5, 30, requires_grad=True)
y = torch.randn(5)
w = torch.randn(30)

# I assume you backpropagate from a reduced version
# of your sum, because you can't call .backward on multi-dimensional
# tensors
loss = logistic_loss(X, y, w).mean()
loss.mean().backward()
print(X.grad)


The simplest solution to your problem is to replace torch.tensor(elementwise, requires_grad=True) with torch.stack(elementwise). You can think of torch.tensor as a constructor for entirely new tensors, if your tensor is more of a result of some mathematical expression, you should use operations like torch.stack or torch.cat.

That being said, this code is still wildly inefficient because you do manual looping over i. Instead, you could write simply

def logistic_loss_vectorized(X, y, w):
    mm = torch.matmul(X, w)
    exp = torch.exp(-y * mm)

    return torch.log(1 + exp)


which is mathematically equivalent, but will be much faster in practice, because it allows for better parallelization due to lack of explicit looping.

Note that there is still a numerical issue with this code - you're taking a logarithm of an exponential, but the intermediate result, called exp, is likely to attain very high values, causing loss of precision. There are workarounds for that, which is why the loss functions provided by PyTorch are preferable.
