title: LASSO regression result different in Matlab and Python
body: <p>I am now trying to learn the ADMM algorithm (Boyd 2010) for LASSO regression.</p>
  
  <p>I found out a very good example on this <a href="http://www.simonlucey.com/lasso-using-admm/" rel="nofollow">page</a>.</p>
  
  <p>The matlab code is shown <a href="https://dl.dropboxusercontent.com/u/22893361/code/lasso_admm.m" rel="nofollow">here</a>.</p>
  
  <p>I tried to convert it into python language so that I could develop a better understanding. </p>
  
  <p>Here is the code:</p>
  
  <pre><code>import scipy.io as io
  import scipy.sparse as sp
  import scipy.linalg as la
  import numpy as np
  
  def l1_norm(x):
      return np.sum(np.abs(x))
  
  def l2_norm(x):
      return np.dot(x.ravel().T, x.ravel())
  
  def fast_threshold(x, threshold):
      return np.multiply(np.sign(x), np.fmax(abs(x) - threshold, 0))
  
  def lasso_admm(X, A, gamma):
      c = X.shape[1]
      r = A.shape[1]
  
      C = io.loadmat("C.mat")["C"]
  
      L = np.zeros(X.shape)
  
      rho = 1e-4
      maxIter = 200
      I = sp.eye(r)
      maxRho = 5
  
      cost = []
  
      for n in range(maxIter):
          B = la.solve(np.dot(A.T, A) + rho * I, np.dot(A.T, X) + rho * C - L)
  
          C = fast_threshold(B + L / rho, gamma / rho)
  
          L = L + rho * (B - C);  
  
          rho = min(maxRho, rho * 1.1); 
  
          cost.append(0.5 * l2_norm(X - np.dot(A, B)) + gamma * l1_norm(B))
  
      cost = np.array(cost).ravel()
  
      return B, cost
  
  data = io.loadmat("lasso.mat")
  A = data["A"]
  X = data["X"]    
  B, cost = lasso_admm(X, A, gamma)
  </code></pre>
  
  <p>I have found the loss function did not converge after 100+ iterations. Matrix B did not tend to be sparse, on the other hand, the matlab code worked in different situations.</p>
  
  <p>I have checked with different input data and compared with Matlab outputs, yet I still could not get hints.</p>
  
  <p>Could anybody take a try? </p>
  
  <p>Thank you in advance.</p>