minimization with a negative cost function: works in MATLAB, not in Python I'm trying to use a particular cost function (based on doubling rate of wealth) for a classification problem, and the solution works well in MATLAB. See https://github.com/acmyers/compareCostFXs

When I try to do this in Python 2.7.6 I don't get any errors, but it only returns zeros for the theta values.

Here is the cost function and optimization method I've used in Python:

def costFunctionDRW(theta, X, y):

    # Initialize useful values
    m = len(y)
    # Marginal probability of acceptance
    marg_pA = sum(y)/m
    # Marginal probability of rejection
    marg_pR = 1 - marg_pA

    # =============================================================
    pred = sigmoid(np.dot(X,theta))
    final_wealth_individual = (pred/marg_pA)*y + ((1-pred)/marg_pR)*(1-y)
    final_wealth = np.prod(final_wealth_individual)
    final_wealth = -final_wealth

    return final_wealth

result = scipy.optimize.fmin(costFunctionDRW, x0=initial_theta, \
                   args=(X_array, y_array), maxiter=1000, disp=False, full_output=True )


Any advice would be much appreciated!
The problem is with division on python 2.x.  In python 2.x, division involving two integers produces an integer result.  So 1/2==0.  Python 3.x does not have this problem, 1/2==.5.

There are two ways to avoid this.  First, you can always convert one value you a float.  So 1./2==0.5 and 1/2.==0.5.  However, you have to remember to do this everywhere, and if you forget it can lead to hard-to-find errors.

The more reliable method is to always put this at the top of your code: from __future__ import division.  This will switch python 2.x to the python 3 behavior, so 1/2==.5.  In python 3.x it does nothing, so it also makes your code python 3.x compatible in this regard.