title: Regularized Logistic Regression in Python (Andrew ng Course)
body: <p>I'm starting the ML journey and I'm having troubles with this coding exercise
  here is my code</p>
  
  <pre><code>import numpy as np
  import pandas as pd
  import scipy.optimize as op
  
  # Read the data and give it labels
  data = pd.read_csv('ex2data2.txt', header=None, name['Test1', 'Test2', 'Accepted'])
  
  # Separate the features to make it fit into the mapFeature function
  X1 = data['Test1'].values.T
  X2 = data['Test2'].values.T
  
  # This function makes more features (degree)
  def mapFeature(x1, x2):
      degree = 6
      out = np.ones((x1.shape[0], sum(range(degree + 2))))
      curr_column = 1
      for i in range(1, degree + 1):
          for j in range(i+1):
              out[:,curr_column] = np.power(x1, i-j) * np.power(x2, j)
              curr_column += 1
      return out
  
  
  # Separate the data into training and target, also initialize theta
  X = mapFeature(X1, X2)
  y = np.matrix(data['Accepted'].values).T
  m, n = X.shape
  cols = X.shape[1]
  theta = np.matrix(np.zeros(cols))
  
  #Initialize the learningRate(sigma)
  learningRate = 1
  
  
  # Define the Sigmoid Function (Output between 0 and 1)
  def sigmoid(z):
      return 1 / (1 + np.exp(-z))
  
  
  def cost(theta, X, y, learningRate):
      # This is require to make the optimize function work
      theta = theta.reshape(-1, 1)
      error = sigmoid(X @ theta)
      first = np.multiply(-y, np.log(error))
      second = np.multiply(1 - y, np.log(1 - error))
      j = np.sum((first - second)) / m + (learningRate * np.sum(np.power(theta, 2)) / 2 * m)
      return j
  
  
  # Define the gradient of the cost function
  def gradient(theta, X, y, learningRate):
      # This is require to make the optimize function work
      theta = theta.reshape(-1, 1)
      error = sigmoid(X @ theta)
      grad =  (X.T @ (error - y)) / m + ((learningRate * theta) / m)
      grad_no = (X.T @ (error - y)) / m
      grad[0] = grad_no[0]
      return grad
  
  
  Result = op.minimize(fun=cost, x0=theta, args=(X, y, learningRate), method='TNC', jac=gradient)
  opt_theta = np.matrix(Result.x)
  
  
  def predict(theta, X):
      sigValue = sigmoid(X @ theta.T)
      p = sigValue &gt;= 0.5
      return p
  
  p = predict(opt_theta, X)
  print('Train Accuracy: {:f}'.format(np.mean(p == y) * 100))
  </code></pre>
  
  <p>So, when the <code>learningRate = 1</code>, the accuracy should be around <code>83,05%</code> but I'm getting <code>80.5%</code> and when the <code>learningRate = 0</code>, the accuracy should be <code>91.52%</code> but I'm getting <code>87.28%</code></p>
  
  <p>So the question is <strong>What am I doing wrong? Why my accuracy is below the problem default answer?</strong></p>
  
  <p>Hope someone can guide me in the right direction. Thanks!</p>
  
  <p>P.D: Here is the dataset, maybe it can help </p>
  
  <p><a href="https://raw.githubusercontent.com/TheGirlWhiteWithBandages/Machine-Learning-Algorithms/master/Logistic%20Regression/ex2data2.txt" rel="nofollow noreferrer">https://raw.githubusercontent.com/TheGirlWhiteWithBandages/Machine-Learning-Algorithms/master/Logistic%20Regression/ex2data2.txt</a></p>