Scikit-learn: Getting SGDClassifier to predict as well as a Logistic Regression
The comments about iteration number are spot on. The default SGDClassifier n_iter is 5 meaning you do 5 * num_rows steps in weight space. The sklearn rule of thumb is ~ 1 million steps for typical data. For your example, just set it to 1000 and it might reach tolerance first. Your accuracy is lower with SGDClassifier because it's hitting iteration limit before tolerance so you are "early stopping"

Modifying your code quick and dirty I get:



# Added n_iter here
params = [{}, {"loss": "log", "penalty": "l2", 'n_iter':1000}]

for param, Model in zip(params, Models):
    total = 0
    for train_indices, test_indices in kf:
        train_X = X[train_indices, :]; train_Y = Y[train_indices]
        test_X = X[test_indices, :]; test_Y = Y[test_indices]
        reg = Model(**param)
        reg.fit(train_X, train_Y)
        predictions = reg.predict(test_X)
        total += accuracy_score(test_Y, predictions)

    accuracy = total / numFolds
    print "Accuracy score of {0}: {1}".format(Model.__name__, accuracy)

Accuracy score of LogisticRegression: 0.96
Accuracy score of SGDClassifier: 0.96