**Problem Synopsis:**
When attempting to use the scipy.optimize.fmin\_bfgs minimization (optimization) function, the function throws a 

> 
> derphi0 = np.dot(gfk, pk)
>  ValueError: **matrices are not aligned** 
> 
> 
> 

error. According to my error checking this occurs at the very end of the first iteration through fmin\_bfgs--just before any values are returned or any calls to callback.

**Configuration:**
Windows Vista
Python 3.2.2
SciPy 0.10
IDE = Eclipse with PyDev

**Detailed Description:**
I am using the scipy.optimize.fmin\_bfgs to minimize the cost of a simple logistic regression implementation (converting from Octave to Python/SciPy). Basically, the cost function is named cost\_arr function and the gradient descent is in gradient\_descent\_arr function.

I have manually tested and fully verified that \*cost\_arr\* and \*gradient\_descent\_arr\* work properly and return all values properly. I also tested to verify that the proper parameters are passed to the \*fmin\_bfgs\* function. Nevertheless, when run, I get the ValueError: matrices are not aligned. According to the source review, the exact error occurs in the 

> 
> def line\_search\_wolfe1
>  function in # Minpack's Wolfe line and scalar searches as supplied by the scipy packages.
> 
> 
> 

Notably, if I use *scipy.optimize.fmin* instead, the *fmin* function runs to completion.

**Exact Error:**

> 
> File
>  "D:\Users\Shannon\Programming\Eclipse\workspace\SBML\sbml\LogisticRegression.py",
>  line 395, in fminunc\_opt
> 
> 
> 
> ```
> optcost = scipy.optimize.fmin_bfgs(self.cost_arr, initialtheta, fprime=self.gradient_descent_arr, args=myargs, maxiter=maxnumit, callback=self.callback_fmin_bfgs, retall=True)   
> 
> ```
> 
> File
>  "C:\Python32x32\lib\site-packages\scipy\optimize\optimize.py", line
>  533, in fmin\_bfgs old\_fval,old\_old\_fval)  
> 
>  File "C:\Python32x32\lib\site-packages\scipy\optimize\linesearch.py", line
>  76, in line\_search\_wolfe1
>  derphi0 = np.dot(gfk, pk) 
>  ValueError: matrices are not aligned
> 
> 
> 

I call the optimization function with:
optcost = scipy.optimize.fmin\_bfgs(self.cost\_arr, initialtheta, fprime=self.gradient\_descent\_arr, args=myargs, maxiter=maxnumit, callback=self.callback\_fmin\_bfgs, retall=True)

I have spent a few days trying to fix this and cannot seem to determine what is causing the **matrices are not aligned** error.

ADDENDUM: 2012-01-08
I worked with this a lot more and seem to have narrowed the issues (but am baffled on how to fix them). First, fmin (using just fmin) works using these functions--cost, gradient. Second, the cost and the gradient functions both accurately return expected values when tested in a single iteration in a manual implementation (NOT using fmin\_bfgs). Third, I added error code to optimize.linsearch and the error seems to be thrown at def line\_search\_wolfe1 in line: derphi0 = np.dot(gfk, pk).
Here, according to my tests, scipy.optimize.optimize pk = [[ 12.00921659]
 [ 11.26284221]]pk type = and scipy.optimize.optimizegfk = [[-12.00921659] [-11.26284221]]gfk type = 
Note: according to my tests, the error is thrown on the very first iteration through fmin\_bfgs (i.e., fmin\_bfgs never even completes a single iteration or update).

I appreciate ANY guidance or insights. 

My Code Below (logging, documentation removed):
Assume theta = 2x1 ndarray (Actual: theta Info Size=(2, 1) Type = )
Assume X = 100x2 ndarray (Actual: X Info Size=(2, 100) Type = )
Assume y = 100x1 ndarray (Actual: y Info Size=(100, 1) Type = )

```
def cost_arr(self, theta, X, y):

    theta = scipy.resize(theta,(2,1))         

    m = scipy.shape(X)

    m = 1 / m[1] # Use m[1] because this is the length of X
    logging.info(__name__ + "cost_arr reports m = " + str(m))         

    z = scipy.dot(theta.T, X) # Must transpose the vector theta               

    hypthetax = self.sigmoid(z)

    yones = scipy.ones(scipy.shape(y))

    hypthetaxones = scipy.ones(scipy.shape(hypthetax))

    costright = scipy.dot((yones - y).T, ((scipy.log(hypthetaxones - hypthetax)).T))

    costleft = scipy.dot((-1 * y).T, ((scipy.log(hypthetax)).T))

def gradient_descent_arr(self, theta, X, y):

    theta = scipy.resize(theta,(2,1)) 

    m = scipy.shape(X)

    m = 1 / m[1] # Use m[1] because this is the length of X

    x = scipy.dot(theta.T, X) # Must transpose the vector theta

    sig = self.sigmoid(x)

    sig = sig.T - y

    grad = scipy.dot(X,sig)

    grad = m * grad

    return grad

def fminunc_opt_bfgs(self, initialtheta, X, y, maxnumit):
    myargs= (X,y)

    optcost = scipy.optimize.fmin_bfgs(self.cost_arr, initialtheta, fprime=self.gradient_descent_arr, args=myargs, maxiter=maxnumit, retall=True, full_output=True)

    return optcost

```