ConvergenceWarning: Maximum Likelihood slows kernel-run-time?
Question: I use the very nicht code-object `arma_order_select_ic` in order to finde the
lowest Information Criterion for chor choosing p- and q values.

I am not sure if i do it right or if the code just stumbles upon some
mistakes...

In:

    
    
    y = indexed_df
    res = arma_order_select_ic(y, max_ar=7, max_ma=7, ic=['aic', 'bic', 'hqic'], trend='c', fit_kw=dict(method='css'))
    print res
    print ('AIC-order: {}' .format(res.aic_min_order))
    print ('BIC-order: {}' .format(res.bic_min_order))
    print ('HQIC-order: {}' .format(res.hqic_min_order)) 
    

Out:

    
    
    /Applications/anaconda/lib/python2.7/site-packages/statsmodels-0.6.1-py2.7-macosx-10.5-x86_64.egg/statsmodels/base/model.py:466: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
      "Check mle_retvals", ConvergenceWarning)
    

Plus: it prints out the three matrix-style lists (for each IC one matrix) with
the final recommendation:

    
    
    AIC-order: (7, 5)
    BIC-order: (7, 0)
    HQIC-order: (7, 0)
    

So, the whole thing seems to work.

The problem is, it takes approx 30-60 seconds as the warning is printed for
every calculation i.e. it is super slow!

I checked the relevant source code (statsmodels/base/model.py) and how to skip
printing the CovergenceWarning:

    
    
       #TODO: hardcode scale?
            if isinstance(retvals, dict):
                mlefit.mle_retvals = retvals
                if warn_convergence and not retvals['converged']:
                    from warnings import warn
                    from statsmodels.tools.sm_exceptions import ConvergenceWarning
                    warn("Maximum Likelihood optimization failed to converge. "
                         "Check mle_retvals", ConvergenceWarning)
    
            mlefit.mle_settings = optim_settings
            return mlefit
    

So i tried to delete the if part which is linked to the ConvergenceWarning but
it wont work.

This part from the same source code:

    
    
    mle_retvals : dict
        Contains the values returned from the chosen optimization method if
        full_output is True during the fit.  Available only if the model
        is fit by maximum likelihood.  See notes below for the output from
        the different methods. 
    

does not tell me where and how to change mle_retvals

How to check the mle_retvals and what to change?

Is there a way to make the ConvergenceWarning disappear to make the
calculation run faster?


Answer: The documentation in the Notes section explicitly states how you can speed
things up...See the docstring for `fit_kw` to change arguments given to the
`ARMA.fit` method. This is going to be slow for high numbers of models. It's a
naive implementation and just does a pairwise fit of them all. Try doing
`method='css'` for faster results.

I don't know why you want to change `mle_retvals`. It is in the returns
section. It's not something you change directly. You shouldn't have to delete
any source code to get things to run. That check is there to warn you that
things are going wrong. I.e., it might be the case that the models causing
these warnings are really poor models for your data.

