title: How to realize a polynomial regression in Pytorch / Python
body: <p>I want my neural network to solve a polynomial regression problem like y=(x*x) + 2x -3.</p>
  
  <p>So right now I created a network with 1 input node, 100 hidden nodes and 1 output node and gave it a lot of epochs to train with a high test data size. The problem is that the prediction after like 20000 epochs is okayish, but much worse then the linear regression predictions after training.</p>
  
  <pre><code>import torch
  from torch import Tensor
  from torch.nn import Linear, MSELoss, functional as F
  from torch.optim import SGD, Adam, RMSprop
  from torch.autograd import Variable
  import numpy as np
  
  
  # define our data generation function
  def data_generator(data_size=1000):
      # f(x) = y = x^2 + 4x - 3
      inputs = []
      labels = []
  
      # loop data_size times to generate the data
      for ix in range(data_size):
          # generate a random number between 0 and 1000
          x = np.random.randint(1000) / 1000
  
          # calculate the y value using the function x^2 + 4x - 3
          y = (x * x) + (4 * x) - 3
  
          # append the values to our input and labels lists
          inputs.append([x])
          labels.append([y])
  
      return inputs, labels
  
  
  # define the model
  class Net(torch.nn.Module):
      def __init__(self):
          super(Net, self).__init__()
          self.fc1 = Linear(1, 100)
          self.fc2 = Linear(100, 1)
  
  
      def forward(self, x):
          x = F.relu(self.fc1(x)
          x = self.fc2(x)
          return x
  
  
  model = Net()
  # define the loss function
  critereon = MSELoss()
  # define the optimizer
  optimizer = SGD(model.parameters(), lr=0.01)
  
  # define the number of epochs and the data set size
  nb_epochs = 20000
  data_size = 1000
  
  # create our training loop
  for epoch in range(nb_epochs):
      X, y = data_generator(data_size)
      X = Variable(Tensor(X))
      y = Variable(Tensor(y))
  
  
      epoch_loss = 0;
  
  
      y_pred = model(X)
  
      loss = critereon(y_pred, y)
  
      epoch_loss = loss.data
      optimizer.zero_grad()
  
      loss.backward()
  
      optimizer.step()
  
      print("Epoch: {} Loss: {}".format(epoch, epoch_loss))
  
  # test the model
  model.eval()
  test_data = data_generator(1)
  prediction = model(Variable(Tensor(test_data[0][0])))
  print("Prediction: {}".format(prediction.data[0]))
  print("Expected: {}".format(test_data[1][0]))
  
  </code></pre>
  
  <p>Is their a way to get way better results? I wondered if I should try to get 3 outputs, call them a, b and c, such that y= a(x*x)+b(x)+c. But I have no idea how to implement that and train my neural network.</p>