Simple linear regression in PyTorch
I know this doesn't answer you 100%, but maybe it helps. Feel free to disregard.

I just implemented this in keras and didn't have any issues

model = Sequential([
    Dense(1,activation='linear',input_shape=(1,))
])
sgd = optimizers.Adam(lr=0.001)
model.compile(optimizer='adam',
              loss='mean_squared_error',
              metrics=['mae'])
model.fit(test_df.x, test_df.y, validation_split = .2,epochs=500, batch_size=1)

plt.scatter(test_df.x,test_df.y,s=30,c='r',marker='x')
x = np.linspace(4,25,100)
y = model.get_weights()[0][0][0]*x+model.get_weights()[1][0]
plt.plot(x, y)
plt.ylim([-5,50])
plt.xlim([4.5,23])





Is it possible you read your data in incorrectly maybe? Or maybe you are referencing it incorrectly when you set your Xt and yt?