title: Losses are increasing in Binary classification using gradient descent optimization method
body: <p>This my program for Binary classification using gradient descent optimization method. I am not sure about my loss function. The error in my case is incresing when plotted</p>
  
  <p><a href="https://i.stack.imgur.com/xL8Im.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/xL8Im.jpg" alt="enter image description here"></a></p>
  
  <pre><code>def sigmoid_activation(x):
      return 1.0 / (1 + np.exp(-x))
  
  def predict(testX, W):
      preds= sigmoid_activation(np.dot(testX,W))
      # apply a step function to threshold (=0.5) the outputs to binary class 
  #labels
      #start your code here
      for i in range(len(preds)):
          if preds[i]&lt;0.5:
              p.append(0)
          if preds[i]&gt;=0.5:
              p.append(1) 
  
      return p
  
  epochs = 50
  alpha = 0.01
  
  (X,y)=make_moons(n_samples=1000, noise = 0.15)
  
  y=y.reshape(y.shape[0],1)
  
  X = np.c_[X, np.ones((X.shape[0]))]
  
  (trainX, testX, trainY, testY) = train_test_split(X, y,
  test_size=0.5, random_state=42)
  
  print("[INFO] training...")
  W = np.random.randn(X.shape[1], 1)
  losses = []
  for epoch in np.arange(0, epochs):
  
      #start your code here
      Z=np.dot(trainX, W)
      yhat= sigmoid_activation(Z)
  
      error=trainY-yhat
      loss = np.sum(error ** 2)
      losses.append(loss)
      gradient = trainX.T.dot(error) / trainX.shape[0]
      W= W-alpha*gradient      #moving in -ve direction
  
  
  
  
  
      # check to see if an update should be displayed
      if epoch == 0 or (epoch + 1) % 5 == 0:
          print("[INFO] epoch={}, loss={:.7f}".format(int(epoch + 1),
              loss))
  
  # evaluate our model
  print("[INFO] evaluating...")
  preds = predict(testX, W)
  print(classification_report(testY, preds))
  
  # plot the (testing) classification data
  plt.style.use("ggplot")
  plt.figure()
  plt.title("Data")
  plt.scatter(testX[:, 0], testX[:, 1], marker="o", c=testY[:,0], s=30)
  
  # construct a figure that plots the loss over time
  plt.style.use("ggplot")
  plt.figure()
  plt.plot(np.arange(0, epochs), losses)
  plt.title("Training Loss")
  plt.xlabel("Epoch #")
  plt.ylabel("Loss")
  plt.show()
  </code></pre>