ROC and AUC calculation Using Python 2.7 and here is my code to calculate ROC/AUC and I compare my results of tpr/fpr with threshold, it is the same result of whay scikit-learn returns.

My questions, (1) any ideas for improvements (especially for performance in my existing code) (2) any smart ideas to calculate of AUC?

import numpy as np
from sklearn import metrics


def homeMadeROC(scores, targets):
    totalPositive = sum(targets)
    totalNegative = len(targets) - totalPositive
    sortedTargets = []
    tprs = []
    fprs = []

    # order by score desc, and get related target
    for (s,t) in sorted(zip(scores, targets), reverse=True):
        sortedTargets.append(t)
    sortedScores = sorted(scores, reverse=True)
    predictedTruePositiveSofar = 0
    predictedFalsePositiveSofar = 0

    for i in range(len(sortedScores)):
        if sortedTargets[i] == 1:
            predictedTruePositiveSofar += 1
        else:
            predictedFalsePositiveSofar += 1
        tprs.append(float(predictedTruePositiveSofar)/totalPositive)
        fprs.append(float(predictedFalsePositiveSofar)/totalNegative)

    print fprs
    print tprs
    print sortedScores

if __name__ == "__main__":

    y = np.array([0, 0, 1, 1])
    pred = np.array([0.1, 0.4, 0.35, 0.8])
    fpr, tpr, thresholds = metrics.roc_curve(y, pred)
    print fpr
    print tpr
    print thresholds
    print metrics.auc(fpr, tpr)

    print "======"

    homeMadeROC([0.1, 0.4, 0.35, 0.8],[0, 0, 1, 1])
I don't know any machine learning, however, there are some ways your code can be improved.
PEP8 is Python default style guide,
and so we use lower_case_with_underscores rather than mixedCase for most names.
The major exception to this is classes that use CapitalizedWords.

If we look at your variable names:


total_positive is incorrect, it's the sum of targets, so could be called total_targets.
total_negative is even more incorrect, it's the length minus the total.
sortedTargets is correct.
tprs seems like it means something well known, so it's a good name. Also the plural is great as it's a list.
predictedTruePositiveSofar is long, instead I'd move this into it's own function and then use it once in a loop.


And so I don't like most of your names.
Try to pick good names, like tprs, for all your variables.
Naming is hard, so write a concise description of the variable in the context of the function.
But if that's too big, you may want to make a new function.

The code is mostly ok, but I'd move the initialization of your lists to where they are used.
I'd also change your two sorted calls to a single one with two zips.
To do this you need to use the * operator, to expand a list to arguments.
Which is quite well explained here.
But in short, range(1, 10, 2) is the same as range(*[1, 10, 2]).
This simplifies the creation of sorted_scores and sorted_targets.

After this I'd make a function accumulate_truth.
This will yield the amount of truthy and falsy values.
And so can move the predicted_true_positive_so_far into it's own known function.
As naming it something useful will be hard in the current context.
I'd also change this function to iterate through the list,
rather than through a range and index,
and to remove the == 1 as that can change how the function works.
This is as bool([1]) is True, but [1] == 1 is False.
So your current implementation removes some truthy values.

To simplify the creation of tprs and fprs I'd change the creation of total_positive to be a float.
This changes total_negative to a float too, and makes all the items in tprs become floats too.
Alternately you can use __future__.division.

Finally try not to merge printing and doing work in a single function.
Instead you should return tprs and fprs, and print them in the __main__.

This can result in:

from __future__ import division


def accumulate_truth(iterable):
    true = 0
    false = 0
    for i in iterable:
        if i:
            true += 1
        else:
            false += 1
        yield true, false


def home_made_ROC(scores, targets):
    total_targets = sum(targets)
    length_sub_targets = len(targets) - total_targets

    scores, targets = zip(*sorted(zip(scores, targets), reverse=True))
    tprs = []
    fprs = []
    for true, false in accumulate_truth(targets):
        tprs.append(true / total_targets)
        fprs.append(false / length_sub_targets)

    return tprs, fprs


if __name__ == '__main__':
    tprs, fprs = home_made_ROC([0.1, 0.4, 0.35, 0.8], [0, 0, 1, 1])
    print tprs
    print fprs