Partial fit multivariate SGDRegressor
Question: I am currently trying to use the SGDRegressor from scikits learn to solve a
multivariate target problem over a large dataset, X ~= (10^6,10^4). As such I
am generating the design matrix (X) in parts with the following code, where
each iteration produces a batch of size roughly (10^3,10^4):

    
    
    design = self.__iterX__(events)
    reglins = [linear_model.SGDRegressor(fit_intercept=True) for i in range(nTargets)]
    
    for X,times in design:
        for i in range(nTargets):
            reglins[i].partial_fit(X,y.ix[times].values[:,i])
    

However I get the following stack trace:

    
    
    File ".../Enthought/Canopy_64bit/User/lib/python2.7/site-    packages/sklearn/linear_model/stochastic_gradient.py", line 841, in partial_fit
        coef_init=None, intercept_init=None)
    File ".../Enthought/Canopy_64bit/User/lib/python2.7/site-packages/sklearn/linear_model/stochastic_gradient.py", line 812, in _partial_fit
        sample_weight, n_iter)
    File ".../Enthought/Canopy_64bit/User/lib/python2.7/site-packages/sklearn/linear_model/stochastic_gradient.py", line 948, in _fit_regressor
        intercept_decay)
    File "sgd_fast.pyx", line 508, in sklearn.linear_model.sgd_fast.plain_sgd (sklearn/linear_model/sgd_fast.c:8651)
        ValueError: floating-point under-/overflow occurred.
    

Looking around it seems that this can be cause by not normalizing X properly.
I understand scikits learn has a variety of functions for this, however given
that I generate X in blocks, is it enough to simply normalize each block or
would I need to figure out a way to normalize whole columns at a time?

Incidentally, is there a particular reason that the partial_fit function does
not allow multivariate targets?


Answer: You can fit one block and apply to others:

    
    
    from sklearn import preprocessing
    scaler = preprocessing.StandardScaler()
    x1 = scalar.fit_transform(X_block_1)
    xn = scalar.transform(X_block_n)
    

You can choose other normalization methods [from this page](http://scikit-
learn.org/stable/modules/preprocessing.html).

