lale/lib/aif360/datasets.py
def fetch_compas_violent_df(preprocess=False):
    """
    Fetch the `compas-two-years-violent`_ dataset, also known as ProPublica violent recidivism, from GitHub and add `fairness_info`.

    It contains information about individuals with a binary
    classification for violent recidivism, indicating whether they were
    re-arrested within two years after the first arrest. Without
    preprocessing, the dataset has 4,020 rows and 51 columns.  There
    are three protected attributes, sex, race, and age, and the disparate
    impact is 0.85.  The data includes numeric and categorical columns, with some
    missing values.

    .. _`compas-two-years-violent`: https://github.com/propublica/compas-analysis

    Parameters
    ----------
    preprocess : boolean, optional, default False

      If True,
      encode protected attributes in X as 0 or 1 to indicate privileged groups
      (1 if Female, Caucasian, or at least 25 for the corresponding sex, race, and
      age columns respectively);
      and apply one-hot encoding to any remaining features in X that
      are categorical and not protected attributes.

    Returns
    -------
    result : tuple

      - item 0: pandas Dataframe

          Features X, including both protected and non-protected attributes.

      - item 1: pandas Series

          Labels y.

      - item 3: fairness_info

          JSON meta-data following the format understood by fairness metrics
          and mitigation operators in `lale.lib.aif360`.
    """
    violent_recidivism = True
    _try_download_compas(violent_recidivism=violent_recidivism)
    if preprocess:
        # Odd finding here: "Female" is a privileged class in the dataset, but the original
        # COMPAS algorithm actually predicted worse outcomes for that class after controlling
        # for other factors. Leaving it as "Female" for now (AIF360 does this by default as well)
        # but potentially worthy of revisiting.
        # See https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm
        # and https://github.com/propublica/compas-analysis/blob/master/Compas%20Analysis.ipynb
        # (hunch is that COMPAS was trained on more biased data that is not reproduced in ProPublica's dataset)

        # Loading violent recidivism dataset using StandardDataset and default settings found in the CompasDataset
        # class since AIF360 lacks a violent recidivism dataset implementation
        df = _get_dataframe_from_compas_csv(violent_recidivism=violent_recidivism)
        default_mappings = {
            "label_maps": [{1.0: "Did recid.", 0.0: "No recid."}],
            "protected_attribute_maps": [
                {0.0: "Male", 1.0: "Female"},
                {1.0: "Caucasian", 0.0: "Not Caucasian"},
            ],
        }
        dataset = aif360.datasets.StandardDataset(
            df=df,
            label_name="two_year_recid",
            favorable_classes=[0],
            protected_attribute_names=["sex", "race"],
            privileged_classes=[[1.0], [1.0]],
            categorical_features=["age_cat", "priors_count", "c_charge_degree"],
            instance_weights_name=None,
            features_to_keep=[
                "sex",
                "age_cat",
                "race",
                "priors_count",
                "c_charge_degree",
                "two_year_recid",
            ],
            features_to_drop=[],
            na_values=[],
            custom_preprocessing=_perform_custom_preprocessing,
            metadata=default_mappings,
        )
        # above preprocessing results in a WARNING of "Missing Data: 5 rows removed from StandardDataset."
        # unclear how to resolve at the moment
        return _get_pandas_and_fairness_info_from_compas_dataset(dataset)
    else:
        return _get_pandas_and_fairness_info_from_compas_csv(
            violent_recidivism=violent_recidivism
        )

    Fetch the `compas-two-years-violent`_ dataset, also known as ProPublica violent recidivism, from GitHub and add `fairness_info`.

    It contains information about individuals with a binary
    classification for violent recidivism, indicating whether they were
    re-arrested within two years after the first arrest. Without
    preprocessing, the dataset has 4,020 rows and 51 columns.  There
    are three protected attributes, sex, race, and age, and the disparate
    impact is 0.85.  The data includes numeric and categorical columns, with some
    missing values.

    .. _`compas-two-years-violent`: https://github.com/propublica/compas-analysis

    Parameters
    ----------
    preprocess : boolean, optional, default False

      If True,
      encode protected attributes in X as 0 or 1 to indicate privileged groups
      (1 if Female, Caucasian, or at least 25 for the corresponding sex, race, and
      age columns respectively);
      and apply one-hot encoding to any remaining features in X that
      are categorical and not protected attributes.

    Returns
    -------
    result : tuple

      - item 0: pandas Dataframe

          Features X, including both protected and non-protected attributes.

      - item 1: pandas Series

          Labels y.

      - item 3: fairness_info

          JSON meta-data following the format understood by fairness metrics
          and mitigation operators in `lale.lib.aif360`.
    
Fetch the `compas-two-years-violent`_ dataset, also known as ProPublica violent recidivism, from GitHub and add `fairness_info`.
It contains information about individuals with a binary
classification for violent recidivism, indicating whether they were
re-arrested within two years after the first arrest. Without
preprocessing, the dataset has 4,020 rows and 51 columns.  There
are three protected attributes, sex, race, and age, and the disparate
impact is 0.85.  The data includes numeric and categorical columns, with some
missing values.



Parameters

preprocess : boolean, optional, default False

If True,
encode protected attributes in X as 0 or 1 to indicate privileged groups
(1 if Female, Caucasian, or at least 25 for the corresponding sex, race, and
age columns respectively);
and apply one-hot encoding to any remaining features in X that
are categorical and not protected attributes.

Returns

result : tuple

item 0: pandas Dataframe

Features X, including both protected and non-protected attributes.

item 1: pandas Series

Labels y.



JSON meta-data following the format understood by fairness metrics
and mitigation operators in `lale.lib.aif360`.
Fetch the ` compas - two - years - violent ` _ dataset also known as ProPublica violent recidivism from GitHub and add ` fairness_info ` . It contains information about individuals with a binary classification for violent recidivism indicating whether they were re - arrested within two years after the first arrest . Without preprocessing the dataset has 4 020 rows and 51 columns . There are three protected attributes sex race and age and the disparate impact is 0 . 85 . The data includes numeric and categorical columns with some missing values . Parameters preprocess : boolean optional default False If True encode protected attributes in X as 0 or 1 to indicate privileged groups ( 1 if Female Caucasian or at least 25 for the corresponding sex race and age columns respectively ) ; and apply one - hot encoding to any remaining features in X that are categorical and not protected attributes . Returns result : tuple item 0 : pandas Dataframe Features X including both protected and non - protected attributes . item 1 : pandas Series Labels y . JSON meta - data following the format understood by fairness metrics and mitigation operators in ` lale . lib . aif360 ` .
def fetch_compas_violent_df(preprocess=False):
    violent_recidivism = True
    _try_download_compas(violent_recidivism=violent_recidivism)
    if preprocess:
        df = _get_dataframe_from_compas_csv(violent_recidivism=violent_recidivism)
        default_mappings = {
            "label_maps": [{1.0: "Did recid.", 0.0: "No recid."}],
            "protected_attribute_maps": [
                {0.0: "Male", 1.0: "Female"},
                {1.0: "Caucasian", 0.0: "Not Caucasian"},
            ],
        }
        dataset = aif360.datasets.StandardDataset(
            df=df,
            label_name="two_year_recid",
            favorable_classes=[0],
            protected_attribute_names=["sex", "race"],
            privileged_classes=[[1.0], [1.0]],
            categorical_features=["age_cat", "priors_count", "c_charge_degree"],
            instance_weights_name=None,
            features_to_keep=[
                "sex",
                "age_cat",
                "race",
                "priors_count",
                "c_charge_degree",
                "two_year_recid",
            ],
            features_to_drop=[],
            na_values=[],
            custom_preprocessing=_perform_custom_preprocessing,
            metadata=default_mappings,
        )
        return _get_pandas_and_fairness_info_from_compas_dataset(dataset)
    else:
        return _get_pandas_and_fairness_info_from_compas_csv(
            violent_recidivism=violent_recidivism
        )
def fetch_compas_violent_df ( preprocess = False ) : violent_recidivism = True _try_download_compas ( violent_recidivism = violent_recidivism ) if preprocess : df = _get_dataframe_from_compas_csv ( violent_recidivism = violent_recidivism ) default_mappings = { "label_maps" : [ { 1.0 : "Did recid." , 0.0 : "No recid." } ] , "protected_attribute_maps" : [ { 0.0 : "Male" , 1.0 : "Female" } , { 1.0 : "Caucasian" , 0.0 : "Not Caucasian" } , ] , } dataset = aif360 . datasets . StandardDataset ( df = df , label_name = "two_year_recid" , favorable_classes = [ 0 ] , protected_attribute_names = [ "sex" , "race" ] , privileged_classes = [ [ 1.0 ] , [ 1.0 ] ] , categorical_features = [ "age_cat" , "priors_count" , "c_charge_degree" ] , instance_weights_name = None , features_to_keep = [ "sex" , "age_cat" , "race" , "priors_count" , "c_charge_degree" , "two_year_recid" , ] , features_to_drop = [ ] , na_values = [ ] , custom_preprocessing = _perform_custom_preprocessing , metadata = default_mappings , ) return _get_pandas_and_fairness_info_from_compas_dataset ( dataset ) else : return _get_pandas_and_fairness_info_from_compas_csv ( violent_recidivism = violent_recidivism )
Fetch the `compas-two-years-violent`_ dataset, also known as ProPublica violent recidivism, from GitHub and add `fairness_info`.
Fetch the ` compas - two - years - violent ` _ dataset also known as ProPublica violent recidivism from GitHub and add ` fairness_info ` .
"""
    Fetch the `compas-two-years-violent`_ dataset, also known as ProPublica violent recidivism, from GitHub and add `fairness_info`.

    It contains information about individuals with a binary
    classification for violent recidivism, indicating whether they were
    re-arrested within two years after the first arrest. Without
    preprocessing, the dataset has 4,020 rows and 51 columns.  There
    are three protected attributes, sex, race, and age, and the disparate
    impact is 0.85.  The data includes numeric and categorical columns, with some
    missing values.

    .. _`compas-two-years-violent`: https://github.com/propublica/compas-analysis

    Parameters
    ----------
    preprocess : boolean, optional, default False

      If True,
      encode protected attributes in X as 0 or 1 to indicate privileged groups
      (1 if Female, Caucasian, or at least 25 for the corresponding sex, race, and
      age columns respectively);
      and apply one-hot encoding to any remaining features in X that
      are categorical and not protected attributes.

    Returns
    -------
    result : tuple

      - item 0: pandas Dataframe

          Features X, including both protected and non-protected attributes.

      - item 1: pandas Series

          Labels y.

      - item 3: fairness_info

          JSON meta-data following the format understood by fairness metrics
          and mitigation operators in `lale.lib.aif360`.
    """ # Odd finding here: "Female" is a privileged class in the dataset, but the original # COMPAS algorithm actually predicted worse outcomes for that class after controlling # for other factors. Leaving it as "Female" for now (AIF360 does this by default as well) # but potentially worthy of revisiting. # See https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm # and https://github.com/propublica/compas-analysis/blob/master/Compas%20Analysis.ipynb # (hunch is that COMPAS was trained on more biased data that is not reproduced in ProPublica's dataset) # Loading violent recidivism dataset using StandardDataset and default settings found in the CompasDataset # class since AIF360 lacks a violent recidivism dataset implementation # above preprocessing results in a WARNING of "Missing Data: 5 rows removed from StandardDataset." # unclear how to resolve at the moment
preprocess