class_docstr: Base class for all structured datasets.
  
  A StructuredDataset requires data to be stored in :obj:`numpy.ndarray`
  objects with :obj:`~numpy.dtype` as :obj:`~numpy.float64`.
  
  Attributes:
      features (numpy.ndarray): Dataset features for each instance.
      labels (numpy.ndarray): Generic label corresponding to each instance
          (could be ground-truth, predicted, cluster assignments, etc.).
      scores (numpy.ndarray): Probability score associated with each label.
          Same shape as `labels`. Only valid for binary labels (this includes
          one-hot categorical labels as well).
      protected_attributes (numpy.ndarray): A subset of `features` for which
          fairness is desired.
      feature_names (list(str)): Names describing each dataset feature.
      label_names (list(str)): Names describing each label.
      protected_attribute_names (list(str)): A subset of `feature_names`
          corresponding to `protected_attributes`.
      privileged_protected_attributes (list(numpy.ndarray)): A subset of
          protected attribute values which are considered privileged from a
          fairness perspective.
      unprivileged_protected_attributes (list(numpy.ndarray)): The remaining
          possible protected attribute values which are not included in
          `privileged_protected_attributes`.
      instance_names (list(str)): Indentifiers for each instance. Sequential
          integers by default.
      instance_weights (numpy.ndarray):  Weighting for each instance. All
          equal (ones) by default. Pursuant to standard practice in social
          science data, 1 means one person or entity. These weights are hence
          person or entity multipliers (see:
          https://www.ibm.com/support/knowledgecenter/en/SS3RA7_15.0.0/com.ibm.spss.modeler.help/netezza_decisiontrees_weights.htm)
          These weights *may not* be normalized to sum to 1 across the entire
          dataset, rather the nominal (default) weight of each entity/record
          in the data is 1. This is similar in spirit to the person weight in
          census microdata samples.
          https://www.census.gov/programs-surveys/acs/technical-documentation/pums/about.html
      ignore_fields (set(str)): Attribute names to ignore when doing equality
          comparisons. Always at least contains `'metadata'`.
      metadata (dict): Details about the creation of this dataset. For
          example::
  
              {
                  'transformer': 'Dataset.__init__',
                  'params': kwargs,
                  'previous': None
              }
class_docstr_tok: ["Base","class","for","all","structured","datasets",".","A","StructuredDataset","requires","data","to","be","stored","in",":","obj",":","`","numpy.ndarray","`","objects","with",":","obj",":","`","~numpy.dtype","`","as",":","obj",":","`","~numpy.float64","`",".","Attributes",":","features","(","numpy.ndarray",")",":","Dataset","features","for","each","instance",".","labels","(","numpy.ndarray",")",":","Generic","label","corresponding","to","each","instance","(","could","be","ground-truth",",","predicted",",","cluster","assignments",",","etc.",")",".","scores","(","numpy.ndarray",")",":","Probability","score","associated","with","each","label",".","Same","shape","as","`","labels","`",".","Only","valid","for","binary","labels","(","this","includes","one-hot","categorical","labels","as","well",")",".","protected_attributes","(","numpy.ndarray",")",":","A","subset","of","`","features","`","for","which","fairness","is","desired",".","feature_names","(","list","(","str",")",")",":","Names","describing","each","dataset","feature",".","label_names","(","list","(","str",")",")",":","Names","describing","each","label",".","protected_attribute_names","(","list","(","str",")",")",":","A","subset","of","`","feature_names","`","corresponding","to","`","protected_attributes","`",".","privileged_protected_attributes","(","list","(","numpy.ndarray",")",")",":","A","subset","of","protected","attribute","values","which","are","considered","privileged","from","a","fairness","perspective",".","unprivileged_protected_attributes","(","list","(","numpy.ndarray",")",")",":","The","remaining","possible","protected","attribute","values","which","are","not","included","in","`","privileged_protected_attributes","`",".","instance_names","(","list","(","str",")",")",":","Indentifiers","for","each","instance",".","Sequential","integers","by","default",".","instance_weights","(","numpy.ndarray",")",":","Weighting","for","each","instance",".","All","equal","(","ones",")","by","default",".","Pursuant","to","standard","practice","in","social","science","data",",","1","means","one","person","or","entity",".","These","weights","are","hence","person","or","entity","multipliers","(","see",":","https",":","\/\/www.ibm.com\/support\/knowledgecenter\/en\/SS3RA7_15.0.0\/com.ibm.spss.modeler.help\/netezza_decisiontrees_weights.htm",")","These","weights","*","may","not","*","be","normalized","to","sum","to","1","across","the","entire","dataset",",","rather","the","nominal","(","default",")","weight","of","each","entity\/record","in","the","data","is","1",".","This","is","similar","in","spirit","to","the","person","weight","in","census","microdata","samples",".","https",":","\/\/www.census.gov\/programs-surveys\/acs\/technical-documentation\/pums\/about.html","ignore_fields","(","set","(","str",")",")",":","Attribute","names","to","ignore","when","doing","equality","comparisons",".","Always","at","least","contains","`","'metadata","'","`",".","metadata","(","dict",")",":","Details","about","the","creation","of","this","dataset",".","For","example",":",":","{","'transformer","'",":","'Dataset.__init__","'",",","'params","'",":","kwargs",",","'previous","'",":","None","}"]
docstr: Args:
      df (pandas.DataFrame): Input DataFrame with features, labels, and
          protected attributes. Values should be preprocessed
          to remove NAs and make all data numerical. Index values are
          taken as instance names.
      label_names (iterable): Names of the label columns in `df`.
      protected_attribute_names (iterable): List of names corresponding to
          protected attribute columns in `df`.
      instance_weights_name (optional): Column name in `df` corresponding
          to instance weights. If not provided, `instance_weights` will be
          all set to 1.
      unprivileged_protected_attributes (optional): If not provided, all
          but the highest numerical value of each protected attribute will
          be considered not privileged.
      privileged_protected_attributes (optional): If not provided, the
          highest numerical value of each protected attribute will be
          considered privileged.
      metadata (optional): Additional metadata to append.
  
  Raises:
      TypeError: Certain fields must be np.ndarrays as specified in the
          class description.
      ValueError: ndarray shapes must match.
docstr_tok: ["Args",":","df","(","pandas.DataFrame",")",":","Input","DataFrame","with","features",",","labels",",","and","protected","attributes",".","Values","should","be","preprocessed","to","remove","NAs","and","make","all","data","numerical",".","Index","values","are","taken","as","instance","names",".","label_names","(","iterable",")",":","Names","of","the","label","columns","in","`","df","`",".","protected_attribute_names","(","iterable",")",":","List","of","names","corresponding","to","protected","attribute","columns","in","`","df","`",".","instance_weights_name","(","optional",")",":","Column","name","in","`","df","`","corresponding","to","instance","weights",".","If","not","provided",",","`","instance_weights","`","will","be","all","set","to","1.","unprivileged_protected_attributes","(","optional",")",":","If","not","provided",",","all","but","the","highest","numerical","value","of","each","protected","attribute","will","be","considered","not","privileged",".","privileged_protected_attributes","(","optional",")",":","If","not","provided",",","the","highest","numerical","value","of","each","protected","attribute","will","be","considered","privileged",".","metadata","(","optional",")",":","Additional","metadata","to","append",".","Raises",":","TypeError",":","Certain","fields","must","be","np.ndarrays","as","specified","in","the","class","description",".","ValueError",":","ndarray","shapes","must","match","."]
code: def __init__(
      self,
      df,
      label_names,
      protected_attribute_names,
      instance_weights_name=None,
      scores_names=[],
      unprivileged_protected_attributes=[],
      privileged_protected_attributes=[],
      metadata=None,
  ):
      """
      Args:
          df (pandas.DataFrame): Input DataFrame with features, labels, and
              protected attributes. Values should be preprocessed
              to remove NAs and make all data numerical. Index values are
              taken as instance names.
          label_names (iterable): Names of the label columns in `df`.
          protected_attribute_names (iterable): List of names corresponding to
              protected attribute columns in `df`.
          instance_weights_name (optional): Column name in `df` corresponding
              to instance weights. If not provided, `instance_weights` will be
              all set to 1.
          unprivileged_protected_attributes (optional): If not provided, all
              but the highest numerical value of each protected attribute will
              be considered not privileged.
          privileged_protected_attributes (optional): If not provided, the
              highest numerical value of each protected attribute will be
              considered privileged.
          metadata (optional): Additional metadata to append.
  
      Raises:
          TypeError: Certain fields must be np.ndarrays as specified in the
              class description.
          ValueError: ndarray shapes must match.
      """
      if df is None:
          raise TypeError(
              "Must provide a pandas DataFrame representing "
              "the data (features, labels, protected attributes)"
          )
      if df.isna().any().any():
          raise ValueError("Input DataFrames cannot contain NA values.")
      try:
          df = df.astype(np.float64)
      except ValueError as e:
          print("ValueError: {}".format(e))
          raise ValueError("DataFrame values must be numerical.")
  
      # Convert all column names to strings
      df.columns = df.columns.astype(str).tolist()
      label_names = list(map(str, label_names))
      protected_attribute_names = list(
          map(str, protected_attribute_names)
      )
  
      self.feature_names = [
          n
          for n in df.columns
          if n not in label_names
          and (not scores_names or n not in scores_names)
          and n != instance_weights_name
      ]
      self.label_names = label_names
      self.features = df[self.feature_names].values.copy()
      self.labels = df[self.label_names].values.copy()
      self.instance_names = df.index.astype(str).tolist()
  
      if scores_names:
          self.scores = df[scores_names].values.copy()
      else:
          self.scores = self.labels.copy()
  
      df_prot = df.loc[:, protected_attribute_names]
      self.protected_attribute_names = df_prot.columns.astype(
          str
      ).tolist()
      self.protected_attributes = df_prot.values.copy()
  
      # Infer the privileged and unprivileged values in not provided
      if (
          unprivileged_protected_attributes
          and privileged_protected_attributes
      ):
          self.unprivileged_protected_attributes = (
              unprivileged_protected_attributes
          )
          self.privileged_protected_attributes = (
              privileged_protected_attributes
          )
      else:
          self.unprivileged_protected_attributes = [
              np.sort(np.unique(df_prot[attr].values))[:-1]
              for attr in self.protected_attribute_names
          ]
          self.privileged_protected_attributes = [
              np.sort(np.unique(df_prot[attr].values))[-1:]
              for attr in self.protected_attribute_names
          ]
  
      if instance_weights_name:
          self.instance_weights = df[
              instance_weights_name
          ].values.copy()
      else:
          self.instance_weights = np.ones_like(
              self.instance_names, dtype=np.float64
          )
  
      # always ignore metadata and ignore_fields
      self.ignore_fields = {"metadata", "ignore_fields"}
  
      # sets metadata
      super(StructuredDataset, self).__init__(
          df=df,
          label_names=label_names,
          protected_attribute_names=protected_attribute_names,
          instance_weights_name=instance_weights_name,
          unprivileged_protected_attributes=unprivileged_protected_attributes,
          privileged_protected_attributes=privileged_protected_attributes,
          metadata=metadata,
      )
  
imports: from collections import defaultdict
  from contextlib import contextmanager
  from copy import deepcopy
  from logging import warning
  import numpy
  import pandas
  from aif360.datasets import Dataset
label_desc: Use image node_id 1 to create a new StructuredDataset object from inherited base classes: Dataset with example: obj = StructuredDataset(df, label_names, protected_attribute_names, instance_weights_name, scores_names, unprivileged_protected_attributes, privileged_protected_attributes, metadata)