Pytorch equivalent of tf.Variable
I am trying to implement this code in pytorch:

self.scale_var = tf.Variable(
          0.1, name='scale_var',
          trainable=True,
          dtype=tf.float32,
          constraint=lambda x: tf.clip_by_value(x, 0, np.infty))


I want to have a scalar value that is trainable and would like to scale a constant with this value in the loss function. Is the below mentioned code  appropriate?

class pytorch_variable(nn.Module):
    def __init__(self):
        super(pytorch_variable,self).__init__()
        self.var = nn.Parameter(torch.tensor(0.1))

    def forward(self):
        return self.var 


What is happening right now is that the gradients flow through this, but the trainable scalar value slowly reduces to zero, decreasing by 0.001 from initial value of 0.1 (till zero because I clip the data after loss.backward() call).

In PyTorch, Variable and Tensor were merged, so you are correct that a scalar variable should just be a scalar tensor.

In isolation:

&gt;&gt;&gt; x=torch.tensor(5.5, requires_grad=True)
&gt;&gt;&gt; x.grad
&gt;&gt;&gt; x.backward(torch.tensor(12.4))
&gt;&gt;&gt; x.grad
tensor(12.4000)


0.001 is a common learning rate, so I'd suspect that's related to the rate at which your trainable variable is being updated.
