title: Weights in Neural Network
body: <p>I am reading : <a href="https://towardsdatascience.com/how-to-build-your-own-neural-network-from-scratch-in-python-68998a08e4f6" rel="nofollow noreferrer">https://towardsdatascience.com/how-to-build-your-own-neural-network-from-scratch-in-python-68998a08e4f6</a></p>
  
  <p>I saw following code: </p>
  
  <pre><code>import numpy as np
  
  def sigmoid(x):
      return 1.0/(1+ np.exp(-x))
  
  def sigmoid_derivative(x):
      return x * (1.0 - x)
  
  class NeuralNetwork:
      def __init__(self, x, y):
          self.input      = x
          self.weights1   = np.random.rand(self.input.shape[1],4) 
          self.weights2   = np.random.rand(4,1)                 
          self.y          = y
          self.output     = np.zeros(self.y.shape)
  
      def feedforward(self):
          self.layer1 = sigmoid(np.dot(self.input, self.weights1))
          self.output = sigmoid(np.dot(self.layer1, self.weights2))
  
      def backprop(self):
          # application of the chain rule to find derivative of the loss function with respect to weights2 and weights1
          d_weights2 = np.dot(self.layer1.T, (2*(self.y - self.output) * sigmoid_derivative(self.output)))
          d_weights1 = np.dot(self.input.T,  (np.dot(2*(self.y - self.output) * sigmoid_derivative(self.output), self.weights2.T) * sigmoid_derivative(self.layer1)))
  
          # update the weights with the derivative (slope) of the loss function
          self.weights1 += d_weights1
          self.weights2 += d_weights2
  
  
  if __name__ == "__main__":
      X = np.array([[0,0,1],
                    [0,1,1],
                    [1,0,1],
                    [1,1,1]])
      y = np.array([[0],[1],[1],[0]])
      nn = NeuralNetwork(X,y)
  
      for i in range(1500):
          nn.feedforward()
          nn.backprop()
  
      print(nn.output)
  </code></pre>
  
  <p>Shouldn't the weights be a 4x4 random matrix because we have 4 neurons in hidden layers and 4 input values so the total number of weight should be 16 but the following code assigns a matrix of 2x4 in the init function and creates a dot product?</p>