cytometer/stats.py
def models_coeff_ci_pval(models, extra_hypotheses=None, model_names=None):
    """
    For convenience, extract betas (coefficients), confidence intervals and p-values from a statsmodels model. Each one
    corresponds to one t-test of a hypothesis (where the hypothesis is that the coefficient ~= 0).
    This function also allows to add extra hypotheses (contrasts) to the model. For example, that the sum of two of
    the model's coefficients is ~= 0.

    * Example of a model:

    import statsmodels.api as sm
    model = sm.RLM.from_formula('weight ~ C(sex)', data=df, M=sm.robust.norms.HuberT()).fit()

    * Example of extra hypotheses:

    'Intercept + C(ko_parent)[T.MAT], C(ko_parent)[T.MAT]=C(ko_parent)[T.FKO]'

    :param models: List of statsmodels models (see example above).
    :param extra_hypotheses: (def None) String with new hypotheses to t-test in the model (see example above).
    :param model_names: (def None) List of strings with the name of each model. This will become the index in each
    output dataframe.
    :return: df_coeff, df_ci_lo, df_ci_hi, df_pval
    """
    if extra_hypotheses is not None:
        hypotheses_labels = extra_hypotheses.replace(' ', '').split(',')
    df_coeff_tot = pd.DataFrame()
    df_ci_lo_tot = pd.DataFrame()
    df_ci_hi_tot = pd.DataFrame()
    df_pval_tot = pd.DataFrame()
    for model in models:
        # values of coefficients
        df_coeff = pd.DataFrame(data=model.params).transpose()
        # values of coefficient's confidence interval
        df_ci_lo = pd.DataFrame(data=model.conf_int()[0]).transpose().reset_index()
        df_ci_hi = pd.DataFrame(data=model.conf_int()[1]).transpose().reset_index()
        # p-values
        df_pval = pd.DataFrame(data=model.pvalues).transpose()
        # extra p-values
        if extra_hypotheses is not None:
            extra_tests = model.t_test(extra_hypotheses)

            df = pd.DataFrame(data=[extra_tests.effect], columns=hypotheses_labels)
            df_coeff = pd.concat([df_coeff, df], axis='columns')

            df = pd.DataFrame(data=[extra_tests.conf_int()[:, 0]], columns=hypotheses_labels)
            df_ci_lo = pd.concat([df_ci_lo, df], axis='columns')

            df = pd.DataFrame(data=[extra_tests.conf_int()[:, 1]], columns=hypotheses_labels)
            df_ci_hi = pd.concat([df_ci_hi, df], axis='columns')

            df = pd.DataFrame(data=[extra_tests.pvalue], columns=hypotheses_labels)
            df_pval = pd.concat([df_pval, df], axis='columns')

        df_coeff_tot = pd.concat((df_coeff_tot, df_coeff))
        df_ci_lo_tot = pd.concat((df_ci_lo_tot, df_ci_lo))
        df_ci_hi_tot = pd.concat((df_ci_hi_tot, df_ci_hi))
        df_pval_tot = pd.concat((df_pval_tot, df_pval))

    df_coeff_tot = df_coeff_tot.reset_index()
    df_ci_lo_tot = df_ci_lo_tot.reset_index()
    df_ci_hi_tot = df_ci_hi_tot.reset_index()
    df_pval_tot = df_pval_tot.reset_index()
    df_coeff_tot.drop(labels='index', axis='columns', inplace=True)
    df_ci_lo_tot = df_ci_lo_tot[df_coeff_tot.columns].copy()
    df_ci_hi_tot = df_ci_hi_tot[df_coeff_tot.columns].copy()
    df_pval_tot = df_pval_tot[df_coeff_tot.columns].copy()

    if model_names is not None:
        df_coeff_tot['model'] = model_names
        df_ci_lo_tot['model'] = model_names
        df_ci_hi_tot['model'] = model_names
        df_pval_tot['model'] = model_names
        df_coeff_tot = df_coeff_tot.set_index('model')
        df_ci_lo_tot = df_ci_lo_tot.set_index('model')
        df_ci_hi_tot = df_ci_hi_tot.set_index('model')
        df_pval_tot = df_pval_tot.set_index('model')
    return df_coeff_tot, df_ci_lo_tot, df_ci_hi_tot, df_pval_tot

    For convenience, extract betas (coefficients), confidence intervals and p-values from a statsmodels model. Each one
    corresponds to one t-test of a hypothesis (where the hypothesis is that the coefficient ~= 0).
    This function also allows to add extra hypotheses (contrasts) to the model. For example, that the sum of two of
    the model's coefficients is ~= 0.

    * Example of a model:

    import statsmodels.api as sm
    model = sm.RLM.from_formula('weight ~ C(sex)', data=df, M=sm.robust.norms.HuberT()).fit()

    * Example of extra hypotheses:

    'Intercept + C(ko_parent)[T.MAT], C(ko_parent)[T.MAT]=C(ko_parent)[T.FKO]'

    :param models: List of statsmodels models (see example above).
    :param extra_hypotheses: (def None) String with new hypotheses to t-test in the model (see example above).
    :param model_names: (def None) List of strings with the name of each model. This will become the index in each
    output dataframe.
    :return: df_coeff, df_ci_lo, df_ci_hi, df_pval
    
For convenience, extract betas (coefficients), confidence intervals and p-values from a statsmodels model. Each one
corresponds to one t-test of a hypothesis (where the hypothesis is that the coefficient ~= 0).
This function also allows to add extra hypotheses (contrasts) to the model. For example, that the sum of two of
the model's coefficients is ~= 0.

Example of a model.



Example of extra hypotheses.


For convenience extract betas ( coefficients ) confidence intervals and p - values from a statsmodels model . Each one corresponds to one t - test of a hypothesis ( where the hypothesis is that the coefficient ~ = 0 ) . This function also allows to add extra hypotheses ( contrasts ) to the model . For example that the sum of two of the model ' s coefficients is ~ = 0 . Example of a model . Example of extra hypotheses .
def models_coeff_ci_pval(models, extra_hypotheses=None, model_names=None):
    if extra_hypotheses is not None:
        hypotheses_labels = extra_hypotheses.replace(' ', '').split(',')
    df_coeff_tot = pd.DataFrame()
    df_ci_lo_tot = pd.DataFrame()
    df_ci_hi_tot = pd.DataFrame()
    df_pval_tot = pd.DataFrame()
    for model in models:
        df_coeff = pd.DataFrame(data=model.params).transpose()
        df_ci_lo = pd.DataFrame(data=model.conf_int()[0]).transpose().reset_index()
        df_ci_hi = pd.DataFrame(data=model.conf_int()[1]).transpose().reset_index()
        df_pval = pd.DataFrame(data=model.pvalues).transpose()
        if extra_hypotheses is not None:
            extra_tests = model.t_test(extra_hypotheses)
            df = pd.DataFrame(data=[extra_tests.effect], columns=hypotheses_labels)
            df_coeff = pd.concat([df_coeff, df], axis='columns')
            df = pd.DataFrame(data=[extra_tests.conf_int()[:, 0]], columns=hypotheses_labels)
            df_ci_lo = pd.concat([df_ci_lo, df], axis='columns')
            df = pd.DataFrame(data=[extra_tests.conf_int()[:, 1]], columns=hypotheses_labels)
            df_ci_hi = pd.concat([df_ci_hi, df], axis='columns')
            df = pd.DataFrame(data=[extra_tests.pvalue], columns=hypotheses_labels)
            df_pval = pd.concat([df_pval, df], axis='columns')
        df_coeff_tot = pd.concat((df_coeff_tot, df_coeff))
        df_ci_lo_tot = pd.concat((df_ci_lo_tot, df_ci_lo))
        df_ci_hi_tot = pd.concat((df_ci_hi_tot, df_ci_hi))
        df_pval_tot = pd.concat((df_pval_tot, df_pval))
    df_coeff_tot = df_coeff_tot.reset_index()
    df_ci_lo_tot = df_ci_lo_tot.reset_index()
    df_ci_hi_tot = df_ci_hi_tot.reset_index()
    df_pval_tot = df_pval_tot.reset_index()
    df_coeff_tot.drop(labels='index', axis='columns', inplace=True)
    df_ci_lo_tot = df_ci_lo_tot[df_coeff_tot.columns].copy()
    df_ci_hi_tot = df_ci_hi_tot[df_coeff_tot.columns].copy()
    df_pval_tot = df_pval_tot[df_coeff_tot.columns].copy()
    if model_names is not None:
        df_coeff_tot['model'] = model_names
        df_ci_lo_tot['model'] = model_names
        df_ci_hi_tot['model'] = model_names
        df_pval_tot['model'] = model_names
        df_coeff_tot = df_coeff_tot.set_index('model')
        df_ci_lo_tot = df_ci_lo_tot.set_index('model')
        df_ci_hi_tot = df_ci_hi_tot.set_index('model')
        df_pval_tot = df_pval_tot.set_index('model')
    return df_coeff_tot, df_ci_lo_tot, df_ci_hi_tot, df_pval_tot
def models_coeff_ci_pval ( models , extra_hypotheses = None , model_names = None ) : if extra_hypotheses is not None : hypotheses_labels = extra_hypotheses . replace ( ' ' , '' ) . split ( ',' ) df_coeff_tot = pd . DataFrame ( ) df_ci_lo_tot = pd . DataFrame ( ) df_ci_hi_tot = pd . DataFrame ( ) df_pval_tot = pd . DataFrame ( ) for model in models : df_coeff = pd . DataFrame ( data = model . params ) . transpose ( ) df_ci_lo = pd . DataFrame ( data = model . conf_int ( ) [ 0 ] ) . transpose ( ) . reset_index ( ) df_ci_hi = pd . DataFrame ( data = model . conf_int ( ) [ 1 ] ) . transpose ( ) . reset_index ( ) df_pval = pd . DataFrame ( data = model . pvalues ) . transpose ( ) if extra_hypotheses is not None : extra_tests = model . t_test ( extra_hypotheses ) df = pd . DataFrame ( data = [ extra_tests . effect ] , columns = hypotheses_labels ) df_coeff = pd . concat ( [ df_coeff , df ] , axis = 'columns' ) df = pd . DataFrame ( data = [ extra_tests . conf_int ( ) [ : , 0 ] ] , columns = hypotheses_labels ) df_ci_lo = pd . concat ( [ df_ci_lo , df ] , axis = 'columns' ) df = pd . DataFrame ( data = [ extra_tests . conf_int ( ) [ : , 1 ] ] , columns = hypotheses_labels ) df_ci_hi = pd . concat ( [ df_ci_hi , df ] , axis = 'columns' ) df = pd . DataFrame ( data = [ extra_tests . pvalue ] , columns = hypotheses_labels ) df_pval = pd . concat ( [ df_pval , df ] , axis = 'columns' ) df_coeff_tot = pd . concat ( ( df_coeff_tot , df_coeff ) ) df_ci_lo_tot = pd . concat ( ( df_ci_lo_tot , df_ci_lo ) ) df_ci_hi_tot = pd . concat ( ( df_ci_hi_tot , df_ci_hi ) ) df_pval_tot = pd . concat ( ( df_pval_tot , df_pval ) ) df_coeff_tot = df_coeff_tot . reset_index ( ) df_ci_lo_tot = df_ci_lo_tot . reset_index ( ) df_ci_hi_tot = df_ci_hi_tot . reset_index ( ) df_pval_tot = df_pval_tot . reset_index ( ) df_coeff_tot . drop ( labels = 'index' , axis = 'columns' , inplace = True ) df_ci_lo_tot = df_ci_lo_tot [ df_coeff_tot . columns ] . copy ( ) df_ci_hi_tot = df_ci_hi_tot [ df_coeff_tot . columns ] . copy ( ) df_pval_tot = df_pval_tot [ df_coeff_tot . columns ] . copy ( ) if model_names is not None : df_coeff_tot [ 'model' ] = model_names df_ci_lo_tot [ 'model' ] = model_names df_ci_hi_tot [ 'model' ] = model_names df_pval_tot [ 'model' ] = model_names df_coeff_tot = df_coeff_tot . set_index ( 'model' ) df_ci_lo_tot = df_ci_lo_tot . set_index ( 'model' ) df_ci_hi_tot = df_ci_hi_tot . set_index ( 'model' ) df_pval_tot = df_pval_tot . set_index ( 'model' ) return df_coeff_tot , df_ci_lo_tot , df_ci_hi_tot , df_pval_tot
For convenience, extract betas (coefficients), confidence intervals and p-values from a statsmodels model.
For convenience extract betas ( coefficients ) confidence intervals and p - values from a statsmodels model .
"""
    For convenience, extract betas (coefficients), confidence intervals and p-values from a statsmodels model. Each one
    corresponds to one t-test of a hypothesis (where the hypothesis is that the coefficient ~= 0).
    This function also allows to add extra hypotheses (contrasts) to the model. For example, that the sum of two of
    the model's coefficients is ~= 0.

    * Example of a model:

    import statsmodels.api as sm
    model = sm.RLM.from_formula('weight ~ C(sex)', data=df, M=sm.robust.norms.HuberT()).fit()

    * Example of extra hypotheses:

    'Intercept + C(ko_parent)[T.MAT], C(ko_parent)[T.MAT]=C(ko_parent)[T.FKO]'

    :param models: List of statsmodels models (see example above).
    :param extra_hypotheses: (def None) String with new hypotheses to t-test in the model (see example above).
    :param model_names: (def None) List of strings with the name of each model. This will become the index in each
    output dataframe.
    :return: df_coeff, df_ci_lo, df_ci_hi, df_pval
    """ # values of coefficients # values of coefficient's confidence interval # p-values # extra p-values
None
models
List of statsmodels models .
List of statsmodels models .
extra_hypotheses
(def None) String with new hypotheses to t-test in the model .
( def None ) String with new hypotheses to t - test in the model .
model_names
(def None) List of strings with the name of each model. This will become the index in each
output dataframe.
( def None ) List of strings with the name of each model . This will become the index in each output dataframe .