Fairness criteria through the lens of directed acyclic graphical models
Benjamin R. Baer Daniel E. Gilbert Martin T. Wells



Integration of adversarial autoencoders with residual dense convolutional networks for estimation of non-Gaussian hydraulic conductivities
    William S. Hlavacekcor1
    December 30, 2023
==========================================================================================================================================





    A substantial portion of the literature on fairness in algorithms proposes, analyzes, and operationalizes simple formulaic criteria for assessing fairness. Two of these criteria, Equalized Odds and Calibration by Group, have gained significant attention for their simplicity and intuitive appeal, but also for their incompatibility. This chapter provides a perspective on the meaning and consequences of these and other fairness criteria using graphical models which reveals Equalized Odds and related criteria to be ultimately misleading. An assessment of various graphical models suggests that fairness criteria should ultimately be case-specific and sensitive to the nature of the information the algorithm processes.






§ INTRODUCTION

The emergence of artificial intelligence’s algorithmic tools represents one of the most important social and technological developments of the last several decades. Machine learning based scoring systems now determine creditworthiness of consumers and insurance prices <cit.> and social media metrics <cit.>, algorithmic hiring platforms target job advertisements and screen résumés to calculate who should and should not be seen by human resource managers <cit.>, and predictive analytics are deployed as sentencing tools in the criminal justice system <cit.>. Big data's algorithmic tools have come to play a decisive role in many aspects of our lives. However, there is concern that these algorithmic tools may lack fairness and exacerbate existing social inequalities <cit.>.

One might imagine that because algorithms are inherently procedural, ensuring fairness should be a simple matter of not explicitly using race or gender as features <cit.>. This notion of fairness has been called Fairness Through Unawareness, and it is easy to see why it is insufficient. First of all, other features will generally redundantly encode sensitive variables <cit.>. We could trivially skirt around Fairness Through Unawareness by including variables which are close proxies for gender or race like hair length or name, but even less suspicious and eminently predictive features such as zip code, language usage, or GPA will allow an algorithm to partially infer an individual's sensitive characteristics and make generalizations on those bases. Furthermore, including information about an individual's sensitive characteristics can actually serve to make a predictive algorithm more fair, especially when there are interaction effects between sensitive characteristics and other features. The area of algorithmic fairness constitutes an attempt to move beyond Fairness Through Unawareness and develop a link between the mathematical properties of algorithms and our philosophical and intuitive notions of fairness. Unfortunately, there is little consensus on the philosophical bedrock upon which algorithmic fairness should rest <cit.>.

Much of the literature on fairness in algorithms has been influenced by a controversy surrounding the Northpointe COMPAS algorithm, an algorithm for predicting criminal recidivism. <cit.> analyzed the output of the algorithm and determined that its predictions were unfair due to the fact that the rate of false positives and false negatives differed significantly between racial groups. In response, Northpointe published a rejoinder arguing the criteria used by <cit.> to assess fairness were nonstandard, and a proper analysis reveals that the predictions made by the COMPAS algorithm are in fact calibrated by race <cit.>. 

Beyond merely inspiring interest in the study of algorithmic fairness, this controversy may have influenced the early direction of the field. One significant branch of the field is concerned with the development, study, comparison, and implementation of simple fairness criteria, much like the balanced-odds criterion implicit in <cit.> or the calibration criterion used in <cit.>. These fairness criteria have largely been tailored to the classification setting.

Furthermore, <cit.> had access to limited information in assessing the COMPAS algorithm; the authors were able to acquire the COMPAS scores for 11,575 pretrial defendants, along with information about their criminal histories, race, and whether that defendant in fact went on to reoffend <cit.>. However, these authors did not have access to the precise features used by the COMPAS algorithm nor the specifications of the COMPAS algorithm itself. Therefore, the authors assessed the fairness of the COMPAS algorithm by examining its false positive and false negative rates across races, which can be calculated using only the COMPAS predictions, the races of the defendants, and whether they reoffended.

Other commonly considered fairness-apt data sets have a similar form; we often desire to assess whether a predictive algorithm is fair using only information about the predictions, the observed outcomes, and the race or gender of the subjects. Perhaps for this reason, much of the early work on algorithmic fairness has centered around so-called oblivious fairness criteria, which assess algorithms only on the basis of their outputs compared to the ground truth. The three central oblivious criteria are most often called Demographic Parity, Equalized Odds, and Calibration by Group, although it is common to encounter these and related concepts under a host of names.

Two prominent strains of criticism have emerged which cast doubt on the utility of simple one-size-fits-all metrics for the fairness of algorithms. The first criticism concerns obliviousness; even alongside the introduction of Equalized Odds, <cit.> note that intuitively fair and intuitively unfair algorithmic procedures can appear identical if we only compare the algorithm's output to the ground truth. Indeed, many of our intuitive notions of fairness have to do with the nature of the information used to make a prediction, rather than the outcome.

A second strain of criticism concerns incompatibility between the three oblivious fairness criteria, and thus their lack of universality. Most notably, <cit.> and <cit.> proved that Calibration and Equalized Odds could not simultaneously be achieved; this recast the disagreement between ProPublica and Northpointe as a philosophical rather than statistical debate. 

Various review papers have been written which tie together the outpouring of early ideas in fairness in algorithms. In this paper, we do not intend to exhaustively catalogue the world of fairness criteria: instead we will focus on a small number of basic criteria which have received significant attention, similar to <cit.>. For a comprehensive map of fairness measures and their relationships to one another, see <cit.>. <cit.> elucidates the incompatibility of Calibration and Equalized Odds using visualizations of outcome distributions. These authors argue that the problem of infra-marginality suggests that Equalized Odds is a poor criterion for fairness.

We agree. We are concerned that work which generalizes and operationalizes Equalized Odds may further obscure the criterion's underlying flaws. The purpose of this article is to provide an alternate source of intuition about fairness criteria using probabilistic directed acyclic graphical models. Graphical models have been used to motivate and expose fairness criteria in other works, especially those which work with explicitly causal criteria for fairness. We believe that graphical models provide an invaluable source of intuition even in non-causal scenarios, and themselves reveal the weakness of Equalized Odds.

Using Bayesian networks, we can view fairness criteria in a way which is easily generalized beyond classification settings. Considering generalizations as defined in <cit.> of Demographic Parity, Equalized Odds, and Calibration helps to expose certain fundamental aspects of these criteria which the classification setting obscures. 

In Section <ref>, we provide a brief overview of probabilistic directed graphical models and the associated causal theory. In Section <ref>, we define the three oblivious fairness criteria and their generalizations. In Section <ref>, we discuss two graphical scenarios and the implications of various fairness criteria therein. In Section <ref>, we review the incompatibility between Equalized Odds and Calibration and give a graphical view of the problem with Equalized Odds. This motivates a modified class of criteria which we call Separation by Signal. Finally, in Section  <ref>, we discuss the relationship between causality and fairness.



§ GRAPHICAL MODELS



A directed acyclic graph (DAG) 𝒢 is a pair {𝐕,𝐄} where 𝐕 = {V_1,…,V_n} is a set of nodes and 𝐄 is a set of directed edges, each pointing from one node to another. The acyclic property of DAGs requires that the edges in 𝐄 never form a directed path leading from one variable back to itself. Let the parents of V_i, Pa(V_i), refer to the set of nodes which share an edge with V_i such that the edge is pointing to V_i. 



 §.§ Probabilistic Directed Acyclic Graphical Models



A probabilistic directed acyclic graphical (PDAG) model, sometimes known as a Bayesian Network, is a pair {𝒢,𝒫} where 𝒢 is a DAG and 𝒫 is a probability distribution over the nodes of 𝒢 <cit.>. Each node V_1,…,V_n in 𝒢 represents a random variable, and these random variables are jointly governed by the probability distribution 𝒫. In this paper, we will consider only PDAG models which satisfy the Markov Condition. A PDAG model {𝒢,𝒫} satisfies the Markov Condition only if the probability distribution 𝒫 can be factored into the conditional distributions of each node given its parents. That is,

    𝒫(V_1,…,V_n) = ∏_i=1^n 𝒫(V_i |Pa(V_i)).

Node V_1 is considered an ancestor of node V_2 if there is a directed path leading from V_1 to V_2. In that case, node V_2 is a descendent of V_1. A node is a root if it has no ancestors and a leaf if it has no descendants. The random variables associated with root nodes we call exogenous because their distribution does not depend on any of the other modelled variables.

In discussing PDAG models, three common relationships between nodes are of particular interest. Nodes V_1 and V_3 are confounded by node V_2 if V_2 is an ancestor of both V_3 and V_1. In this case, we say there is a backdoor path between V_1 and V_3. If V_1 is an ancestor of V_2 and V_2 is an ancestor of V_3, then V_2 is a mediator of the relationship between V_1 and V_3. Finally, if V_1 and V_2 are both ancestors of V_3, then V_3 is said to be a collider for V_1 and V_2. See Figure <ref> for a depiction of these relationships. 



We can determine certain marginal and conditional independence relationships between the random variables V_1,…,V_n using the structure of the DAG. The nodes V_i and V_j are d-separated if the structure of the DAG implies that V_i and V_j are (marginally) independent, i.e., V_i  V_j. A set 𝒮 of nodes d-separates V_i and V_j if the structure of the DAG implies that V_i and V_j are independent given the variables in 𝒮, i.e. V_i  V_j |𝒮. Specifically, under the Markov Condition, the nodes V_i and V_j are d-separated given S if S blocks all paths between V_i and V_j. A connected sequence of edges between two nodes is considered a path regardless of the edges' orientation. A path is blocked if either:

    
  * it contains a mediator or confounder V_k where V_k ∈𝒮, or
    
  * it contains a collider V_k where V_k ∉𝒮 and if V_l is a descendent of V_k, V_l ∉𝒮.

Thus, conditioning on colliders (or their descendents) actually unblocks paths and can induce dependency between marginally independent variables. See Figure <ref> for examples. Note that while d-separation implies conditional independence, d-connection or lack of d-separation does not necessarily imply dependence. Therefore, it is sometimes useful to make the assumption that a PDAG model is faithful, which means that every conditional d-connection relationship in the graph implies dependence between those variables. 

Unfaithfulness can occur because whenever there exist multiple paths leading from V_i to V_j, the dependencies along those paths can cancel each other out. For example, consider a PDAG model associated with the DAG in Figure <ref>. Let V_2 = 3V_1 + ϵ_2, V_2 = 2V_3 = V_1 + ϵ_3, and V_4 = -2V_2 + 3V_3 + ϵ_4. Then V_4 = 3ϵ_3 - 2ϵ_2 + ϵ_4, thus V_4 is independent of V_1, despite the fact that V_1 and V_4 are d-connected. Thus this PDAG model is unfaithful. Note, however, it is unusual for path effects to precisely cancel each other except when variables are carefully constructed to do so.







 §.§ Causality



Strictly speaking, the directed edges in PDAG models need not have any causal interpretation, as long as they are consistent with the conditional dependencies in 𝒫. However, DAGs are not fully determined by their associated probability distributions: a given probability distribution is usually consistent with multiple DAGs with differently oriented edges. Thus it is natural to use a PDAG model to convey causal meaning, so that an edge pointing from V_i to V_j then means that V_i has a causal effect on V_j. When PDAG models are used in the context of causal inference, they are often called Structural Causal Models and the associated DAG may be called a Causal Graph.

Pearl's theory of causality addresses two types of causal questions: questions about the effects of manipulating variables and questions about counterfactual states of affairs <cit.>. We will focus on inferences about variable manipulations. Pearl uses a construct called the do() operator to express do-statements such as 𝒫(V_1 = v_1 |do(V_2 = v_2)). This statement can be interpreted as “the probability that V_1 = v_1 when we intervene to set V_2 = v_2.” Formally, to intervene on the variable V_2 by setting it to v_2 means to construct an alternate PDAG model in which the edges between V_2 and its ancestors are deleted and the distribution of the root V_2 is set to be a point mass at v_2.  

These do-statements can sometimes be resolved into equivalent see-statements using Pearl's three rules of do-calculus, which are consequences of the Markov Condition. See-statements are expressions which may involve various conditional probabilities, but do not contain the do-operator. Thus, see-statements can be evaluated using only information about the joint probability distribution 𝒫 of the variables in the original PDAG. Note that in some cases, we may include unobserved variables in a PDAG model. We will indicate that a variable is unobserved in a DAG with a dotted outline as in Figure <ref>. When do-statements cannot be resolved into see-statements depending only on observed variables, they are called unidentifiable. 



For an accessible and more complete introduction to PDAG models and Pearl's causal theory, see <cit.>. The purpose of our use of PDAG models in this paper is mostly to provide intuition regarding sets of variables with various conditional dependency relationships. However, in Section <ref> we will discuss certain aspects of fairness which require a properly causal treatment. 



§ THREE CRITERIA FOR FAIRNESS



We will review the definitions of three prominent fairness criteria: Independence, Separation, and Sufficiency, which we will examine through the lens of PDAG models in this paper. Let Y be a response, an outcome of interest measured for an individual. For example, Y could be whether an individual will repay a loan, or whether she will click on an advertisement. Let A be a sensitive characteristic, a categorical variable indicating that individual's class with respect to a fairness-apt feature such as race or gender. Let R be a prediction, an estimated response for that individual.

If we select an individual at random from the population, the quantities Y, A, and R can be modelled as random variables. We are concerned with assessing whether R is a fair prediction. The three fairness criteria we examine in this paper are oblivious criteria, which means they assess only the joint distribution of the tuple (A,R,Y) <cit.>. In other words, these criteria are not concerned with the functional form of R or the information upon which it is based; they treat R as a black box.

Most of the work that has been done on fairness criteria for machine learning has considered the classification setting, in which Y is a categorical (and often binary) variable. Therefore it is no surprise that each of the three fairness criteria defined in this section were first introduced as criteria for assessing classifiers. These fairness criteria for the classification setting are known as Demographic Parity, Equalized Odds, and Calibration by Group. However, <cit.> offers sensible generalizations of these three criteria to settings with arbitrary, possibly continuous R and Y. Here we will introduce both the original and generalized versions of each of the three criteria.  



 §.§ Demographic Parity and Independence



As a starting point for assessing the fairness of a prediction algorithm, we may ask whether the algorithm is making systematically different predictions for different groups. Suppose A is a categorical sensitive characteristic taking values in the set 𝔸. Considering only the binary classification case for the moment, suppose that the response Y ∈{0,1} and the prediction R ∈{0,1}. 


    A prediction R satisfies Demographic Parity if 
    P(R = 1 | A = a) = P(R = 1 | A = a')
 for every sensitive characteristic a,a' ∈𝔸.

In the binary case, this is equivalent to the statement R  A. Therefore, the natural generalization of Demographic Parity as suggested by <cit.> is as follows. For arbitrary random variables R, A and Y,


    A prediction R satisfies Independence if R  A.

This is a strong criterion in the sense that it requires that no aspect of the distribution of R depend on A. A weaker form of Independence could require only that the expected value and possibly the variance of R not depend on A. See <cit.> for methods for achieving full or partial Independence in predictions.



 §.§ Equalized Odds and Separation



The Independence criterion does not take the response Y into account; that is, it enforces equality of the distributions of the prediction R across the protected characteristic A even when the distribution of the response Y may differ across protected classes. For binary classifiers, <cit.> argues:

    Demographic Parity is seriously flawed on two counts. First, it doesn't ensure fairness. The notion permits that we accept the qualified applicants in one demographic, but random individuals in another, so long as the percentages of acceptance match. This behavior can arise naturally, when there is little or no training data available for one of the demographics. Second, demographic parity often cripples the utility that we might hope to achieve, especially in the common scenario in which an outcome to be predicated, e.g. whether the loan will be defaulted, is correlated with the protected attribute. 

In light of this, <cit.> proposes an alternative criterion for fairness. Suppose Y ∈{0,1} and the prediction R ∈{0,1}. 


    A prediction R satisfies Equalized Odds if 
    P(R = 1 | Y = y, A = a) = P(R = 1 | Y = y, A = a')
 for every sensitive characteristic a,a' ∈𝔸 and response y ∈{0,1}.


<cit.> argues that when Y and A are not independent, Y itself does not satisfy Demographic Parity, and therefore nor would a “perfect” classifier R=Y. On the other hand, a “perfect” classifier R=Y will always satisfy Equalized Odds. Thus unless we are attempting to modify our predictions as a form of affirmative action, Equalized Odds seems to have an advantage over Demographic Parity. In Section <ref> we argue that this intuition regarding “perfect” classifiers is an artifact of the discrete classification setting and its motivation has less force in arbitrary regression settings. Thus we will consider a general form of Equalized Odds offered by <cit.>.


    A prediction R satisfies Separation if R  A | Y.

Like Independence, this is a strong criterion, and can be relaxed by requiring only that the conditional expectation (R | Y) and possibly the conditional variance Var(R | Y) do not depend on A. See <cit.> for expositions of Separation-like criteria and methods for enforcing them. 



 §.§ Calibration by Group and Sufficiency



Calibration itself is not a fairness concept; it is a popular criterion for assessing an aspect of the performance of predicted probabilities <cit.>. Consider the case where the response Y ∈{0,1} is binary and the predicted probability that Y=1 is P ∈ [0,1]. 


    A predicted probability P satisfies Calibration if P(Y=1 | P = p) = p for any p ∈ [0,1].

This would suggest that classifications generated by a calibrated predicated probability are trustworthy in the sense that a practitioner has no incentive to make post-hoc adjustments to compensate for known biases in ranges of the prediction.

A related criterion, which is directly relevant to fairness, is whether a predicted probability is calibrated across subpopulations, that is, whether a given predicted probability has the same meaning when generated for individuals from different subpopulations. Suppose Y ∈{0,1} and P ∈ [0,1]. 


    A predicted probability P satisfies Calibration by Group if P(Y = 1 | P = p, A = a) = p for each sensitive attribute a ∈𝔸 and probability p ∈ [0,1].

Calibration by Group is intuitively appealing because if it is not satisfied, some individuals' predictions must deviate from the model-grounded truth in a manner depending on their group membership. That is, a predicted probability P which satisfies Calibration by Group has equal performance across the sensitive attribute. Indeed, Calibration by Group is a combination of Calibration and lack of dependence on the sensitive attribute A. The lack of dependence can be isolated, in terms of a prediction R ∈{0, 1}, through the following definition.


    A prediction rule R ∈{0, 1} satisfies Predictive Parity if P[Y = 1 | R = r, A = a] = P[Y=1 | R=r] where the prediction r ∈{0,1} and the sensitive characteristic a ∈𝔸.

Predictive Parity was discussed and coined by <cit.>. <cit.> present a natural generalization of predictive parity for a not necessarily binary Y and R.


    A prediction R satisfies Sufficiency if Y  A | R.




§ FAIRNESS CRITERIA IN TWO SCENARIOS



To supplement the basic motivations of these three fairness criteria, we will discuss their implications in two prediction scenarios. We will find that these various criteria do not represent equally viable choices with different subjective implications, but rather that certain criteria are operational in certain scenarios, and seemingly meaningless in others. Independence has clear use cases and represents an assumption about the relationships between the sensitive characteristic and the response, or else a desire to impose a regime of special intervention in favor of a particular group. Sufficiency, on the other hand, serves more as a measure of the extent to which a prediction takes advantage of all of the information relevant in predicting the response. And finally, Separation does have meaningful use cases in esoteric scenarios such as Scenario 2. However, in most scenarios of interest, Separation has counterproductive and destructive implications.



 §.§ Scenario 1: Loan Repayment


We wish to predict whether an individual will repay a loan. Suppose we model the situation using the PDAG in Figure <ref>.
    


We consider three features with different conditional dependency structures encoded by this PDAG model. The applicant's hair color X_1 is a descendant of her race A and is not a mediator of the effect of A on the whether she repays the loan, Y. Perhaps for this reason, it seems intuitively unfair (and is illegal in some places <cit.>) to determine an applicant's loan premium based on her hair color. However, in some cases it may be tempting to do so because there is a backdoor path connecting X_1 and Y, thus X_1 and Y are statistically marginally dependent. The nature of X_1 illustrates one shortcoming of Fairness Through Unawareness, which demands we do not use A itself as an input into our prediction. It would be no violation of Fairness Through Unawareness to use X_1 alone to predict Y. However, if we do so, we are merely taking advantage of the backdoor path through A; in other words, we are using a noisy version of A to predict Y rather than A itself. 

The applicant's credit rating X_2 is a mediator between A and Y. Therefore, it is statistically dependent on race, but is also predictive of whether the applicant will repay the loan even after taking race into consideration. While an applicant's credit rating is a natural feature for predicting loan repayment, it redundantly encodes race to some extent. Finally, the interest rate of the loan X_3 influences Y but is marginally independent of race. Therefore, X_3 itself seems to be an innocuous prediction. 

We now assess the implications of the Independence, Separation, and Sufficiency criteria in this context.



  §.§.§ Independence



The Independence criterion requires that our prediction R is marginally independent of the applicant's race A. Thus, assuming that our PDAG model is faithful, we can achieve Independence only by positioning R such that it is unconditionally d-separated from A. The applicant's hair color X_3 is itself d-separated from A, therefore any prediction R which is a descendant only of X_3 will always satisfy Independence. 



However, a prediction R which descends from either the applicant's hair color X_1 or credit rating X_2 would fail to satisfy Independence. Nonetheless, there may remain valuable information within X_2 which may help us predict Y while maintaining Independence. We can extract this information by constructing a model which can be represented by the PDAG in Figure <ref>.



By decomposing X_2 into a component which depends on A and an exogenous component ϵ_X_2 which is marginally independent of A, we can construct a prediction R which uses more information but is still d-separated from A and thus satisfies Independence. While X_2 is observed in our model, this exogenous component ϵ_X_2 is unobserved, and must be recovered in a model-specific manner. Consider a simple case, in which X_2 | A ∼𝒩(μ_A,1). Then ϵ_X_2 = (X_2 - μ_A) ∼𝒩(0,1) would satisfy the conditional independencies encoded by this PDAG model, and thus we may safely allow our prediction R to depend on ϵ_X_2.

In the context of this scenario, satisfying Independence while using information about X_2 entails that we use a version of X_2 which is de-meaned by race. That is, we would use as a feature an individual's credit rating relative only to others of the same race. 

This procedure may seem justifiable if credit ratings are themselves racially biased and thus an inaccurate indicator of an individual's likelihood of repaying the loan. However this is untrue by assumption in our model, because Y is conditionally independent of A given X_2. Thus in this case, the procedure of de-meaning credit ratings by race to satisfy Independence should be interpreted as a special modification of the prediction R to benefit a particular (likely disadvantaged) group. If credit ratings are in fact racially biased, we may use a modification of the model in Scenario 1.



In this modified scenario, X_2 is a racially biased credit rating which unfairly modifies information about an applicant's true driver of default risk, ϵ_X_2. Under these assumptions, we can achieve Independence without sacrificing any predictive accuracy. 

Finally, in Scenario 1, no information about an applicant's hair color X_1 can be productively used while maintaining Independence. Any exogenous components which resulted from a decomposition of X_1 would be independent of Y; in this scenario, Independence bars us from considering an applicant's hair color entirely. 



  §.§.§ Separation



The Separation criterion requires that our prediction R is independent of an applicant's race A conditional on whether she does in fact repay the loan, Y. We have seen that Independence is a strong criterion that requires that R has no dependency on A despite the fact that Y itself is dependent on A. In Scenario 1, we may desire a criterion which allows us to take into account more information about the applicant's credit rating X_2 (because X_2 has a direct and perhaps causal relationship with Y). Nonetheless, we may we still desire that our criterion prohibits the use of spurious information like hair color, X_1. 

However, in this scenario, Separation does no such thing. To see this clearly, we will consider all prediction rules which are descendants of only one feature. Let R_j denote an arbitrary prediction rule which depends on only the feature X_j, for each j = 1, 2, 3.



Any prediction R_1 depending on only X_1 violates Separation as expected, since Y does not d-separate R_1 and A. But this is also the case for R_2: Y does not block the path between R_2 and A. Even more surprisingly, because Y is a collider, conditioning on Y actually unblocks the path between R_3 and A. Therefore even though the interest rate of the loan X_3 was eligible for use under Independence due to its independence with A, it cannot be used under Separation. Extracting components from the features X_1, X_2 or X_3 is also futile; any graphical descendent or ancestor of these features would still be d-connected to A given Y.  

In general, Separation prevents us from constructing any predictions which are descendants of X_1, X_2 or X_3 in a faithful PDAG model. We can, however, induce a violation of faithfulness to force independence between R and A. To do so, we must let R be a direct descendent of A. For example, considering for simplicity a prediction using information about credit rating, X_2, suppose we the PDAG model contains the following Gaussian linear model:

    Y | X_2    ∼𝒩(β X_2, σ^2), 
    
        X_2 | A    ∼𝒩(μ_A,σ_A^2).

As throughout this paper, assume that all of the model parameters, β, σ^2, μ_A, and σ_A^2 are known. Then, by basic properties of the multivariate normal distribution <cit.>, 

    X_2 | Y, A    ∼𝒩((1-ρ_A)μ_A + ρ_A Y/β, σ^2 ρ_A ),

where ρ_A = β^2σ_A^2/β^2σ_A^2 + σ^2. 
As we may have anticipated due to the structure of the PDAG model, the mean and variance of X_2 given Y still depend on race A. However, we can use this conditional distribution to construct the optimal linear prediction R which cancels out these dependencies by explicitly taking into account the race A. It is:

    R(X_2) = β(X_2 - (1-ρ_A)μ_A/ρ_A) + Z,

where Z ∼𝒩(0,(c - 1/ρ_A)σ^2) is an independent source of noise, and c = max{1/ρ_a} across all races, a ∈𝔸. Thus, R's dependencies are encoded by the new PDAG model which is not faithful, depicted in Figure <ref>.



Note that R(X_2) | Y ∼𝒩(Y,cσ^2), and this distribution does not depend on A. However, this prediction rule R is suspicious perhaps most notably because it requires that the addition of Harrison Bergeron-esque noise to to the predictions for certain individuals in order to achieve parity in prediction error across groups. The inclusion of additional noise is similar to a result found in <cit.> for discrete classifiers. A depiction of the prediction R above is in Figure <ref>.



Thus, in Scenario 1, Separation does not seem to be a natural criterion for fairness because it suggests only counterproductive procedures for constructing estimators. However, we will show that in Scenario 2, Separation sometimes has the power to discriminate between subjectively different prediction rules.



  §.§.§ Sufficiency



The Sufficiency criterion demands that whether an applicant will repay her loan Y is independent of her race A conditional on the prediction R. Because R will always be a graphical descendent of some subset of the features, X_1, X_2, and X_3, we can see from Figure <ref> that we cannot achieve Sufficiency merely by carefully choosing R's arguments. Conditioning on R will never d-separate Y from A.

However, we can see that the applicant's credit rating X_2 d-separates Y from A, and therefore any invertible function R(X_2) will satisfy Sufficiency. 
This argument will not generally work when the prediction rule R is a function of more than one feature, though, since it is not generally possible to invert such a function. Indeed, in order to construct a prediction rule R(X_2, X_3) which satisfies Separation, the prediction R must explicitly block the path between A and Y. This can be done when the response Y only depends on the features X_2 and X_3 through a parameter θ(X_2, X_3): in this case, the predictor R = θ satisfies Sufficiency. Of course, the parameter θ which controls the way in which the probability of loan repayment depends on the loan interest rate X_3 and an applicant's credit rating X_2 is not known in practice, so this exact predictor is not available for use. This argument further shows that, in this case, the only way that prediction rules which are estimated from training data can be Sufficient is through their recovery of the signal.

In general, suppose we can represent our PDAG model as

where 𝒢_A is a subgraph containing node A, 𝒢_Y is a subgraph containing node Y, and θ is some (possibly deterministic) function of the nodes of 𝒢_A. Then θ𝒢_A itself satisfies Sufficiency, and our goal is to construct a prediction R which contains all of the information in θ.




 §.§ Scenario 2: Job Advertisement


In this scenario, modelled after a similar scenario from <cit.>, we are looking to serve advertisements for a programming job to web users who are likely to be programmers. To predict whether or not the user is a programmer, we use information about his or her browsing history. For a real life example of issues in fairness which may arise from serving job advertisements, see <cit.>. Suppose we model the relationship between measurements on an individual user using the PDAG in Figure <ref>.



We assume X_1, a variable indicating whether a user has visited , has no relationship to whether or not the individual is a programmer except by virtue of the information it encodes about gender. The novelty of this scenario is that we observe X_2, a variable indicating whether a user has visited , which we assume is a graphical descendant of whether a user is a programmer, Y. According to this model, a user's gender affects the likelihood that he or she is a programmer, but a programmer has a certain likelihood of visiting  regardless of his or her gender.

Therefore if we do not wish to target users for this employment advertisement based on gender, X_1 is a suspicious candidate, but X_2 may reasonably be considered fair game. We now examine the implications of the oblivious criteria in this scenario.



  §.§.§ Independence



As in Scenario 1, if the prediction R depends on whether a user has visited , X_1, R will violate Independence, and any information in X_1 which is independent of a user's gender A will also be independent of whether he or she is a programmer, Y. 
Furthermore, because X_2 is a descendant of Y with no backdoor connection to A, any component of whether the user has visited , X_2, which is independent of A will also be independent of Y. Therefore, we cannot construct any non-trivial predictions R in Scenario 2 which satisfy Independence. 



  §.§.§ Separation



On the other hand, Scenario 2 is where Separation shines. Consider again estimators which depend on only one feature: let R_1 denote an arbitrary prediction rule which depends on only X_1, and, likewise, let R_2 denote an arbitrary prediction rule which depends on only X_2.



A prediction R_1 will fail to satisfy Separation because conditioning on whether a user is a programmer, Y, does not d-separate whether he or she has visited , X_1, and his or her gender, A. Clearly, this will be the case regardless of what information we extract from X_1. However, any prediction which is a descendent only of whether the user has visited , X_2, will in fact satisfy Separation, because conditioning on Y blocks the path between X_2 and A.

Thus we can interpret Separation as a criterion which encourages us to use information which depends on A only through the response, Y. However, it is not clear that there are many situations in which we observe features which behave as graphical descendants of the response. We are generally interested in using features which temporally precede the observation of Y; usually X causes Y and not the other way around. 

In fact Scenario 2, which was designed to illustrate a possible use of the Separation criterion, is unrealistic. A modification to Scenario 2 which is more realistic can be modelled with the PDAG in Figure <ref>.



Here, whether or not a user is a programmer is actually an unobserved variable, U, and is relevant to us only because it determines the likelihood of the observable event that the user clicks on our advertisement, Y. From this more realistic model we can see that Y no longer d-separates R_2 from A. Thus to the extent that Y is not identical to U, Separation will not hold.



  §.§.§ Sufficiency



In contrast to Scenario 1, we cannot construct a non-trivial prediction satisfying Sufficiency. Neither X_1 nor X_2 are mediators of the effect of A on Y. The only way our prediction R could block the direct path between A and Y would be for R to perfectly encode the information in A.




§ UNDERSTANDING SEPARATION


Among the three oblivious criteria of fairness discussed, we are most skeptical of Separation. As mentioned in Section <ref>, there is a significant literature focused on applying and generalizing the criterion. However, unlike the other criteria, Separation has fundamental limitations, which we now explore.




In Section <ref>, we found that predictions R which are a function of features that are descendants of the response Y will satisfy Separation, so we will now focus on other cases. In particular, we will focus on an arrangement of the features, sensitive characteristic, and response which we feel is most likely to occur in practice. We assume the DAG is arranged so that the sensitive characteristic A is a root, the response Y is a leaf, and none of the features are descendants of the response. We feel that this DAG is ubiquitous since predictions are often made about the future, so that the features will need to be causal ancestors of the response. An example of such an arrangement is provided in Scenario 1. To be concrete, we will focus on a typical example of a DAG which encodes dependencies between the features, visualized in Figure <ref>.


We will let A be a root node which may be an ancestor of any of the features, X_1, …, X_p. Furthermore, features may be ancestors or descendants of each other, but all of the features are ancestors of the response, Y.

We will now argue that Separation will tend not to hold by examining the structure of the typical DAG, which is visualized with a prediction R in the leftmost graph of Figure <ref>.

There, conditioning on a leaf of the graph, the response Y, will not generally block paths from the root, the sensitive characteristic A, to the prediction R, which depends on the mediators, the features X. This is due to a nondegeneracy of the graph: the response Y will be influenced by exogenous random noise, say ϵ, in addition to a signal through the features X_1, …, X_p. Due to the interference of the noise ϵ, the information in the response Y is fundamentally different than the information in the features, which leads to the impossibility of blocking, and the failure of Separation.

This argument is easiest to see in the case of the visualized Figure <ref>, but holds more generally. Whenever there is at least one feature which is an ancestor of the response Y: the exogenous noise ϵ will still interfere with the response Y, leading to an inability to recover the information in the feature. Specializing the conclusion of this argument to the binary case uncovers a peculiarity in Equalized Odds, despite that Equalized Odds is derived from common measures for summarizing the accuracy of a classifier. 



 §.§ Incompatibility between Separation and Sufficiency



Measures of fairness were beginning to be intensely studied and debated when <cit.> and <cit.> established the surprising result that Calibration and Equalized Odds cannot simultaneously hold in all but degenerate settings. Specifically, it was established:


    Consider the binary setting, where the response Y ∈{0,1} and the prediction R ∈{0,1}. When R ≠ Y, the prediction R can only satisfy both Equalized Odds and Calibration by Group if P(Y=1 | A = a) = P(Y=1 | A=a') for all a,a' ∈𝔸, i.e., the mean response does not differ between levels of the protected characteristic A.


The theorem has been generalized beyond the binary case to hold for Separation and Sufficiency. <cit.> provide an argument using undirected graphs, which we now reproduce. In <ref>, we assumed that R ≠ Y. In the general case, we similarly, but more generally, assume that all events in the joint distribution of (A, R, Y) have positive probability. This assumption makes it so that there's no degeneracy between the random variables. Consider an undirected graphical model of the variables A, R and Y.  
    


Separation requires that R  A | Y, so that there can be no edge between A and R. Similarly, Sufficiency requires that Y  A | R, so that there can be no edge between A and Y. We indicate the impossibility of an edge with a dashed line. Therefore, it must hold that A  Y, since there can be no path drawn connecting A and Y. This establishes the result: under the non-degeneracy assumption, if Separation and Sufficiency both hold, it must be that A  Y. The condition A  Y is a generalization of the condition in <ref> that the mean response does not differ between levels of the protected characteristic A. 

These results cast the disagreement between ProPublica <cit.> and Northpointe <cit.> in a new light. On one hand, the failure of the COMPAS algorithm to achieve equal error rates between groups does not seem to be an objective form of unfairness if it is mathematically impossible for a calibrated classifier to do so. On the other hand, there remains the question of whether disparate impact caused by unbalanced error rates is sufficient cause to dispense with calibration. 

In binary classification settings, it is common to characterize the performance of a classifier using the false positive and false negative rate. These quantities specify the distribution of R given Y. However, as we have seen using the example of a simple linear regression model in Section <ref>, attempting to enforce parity between quantities conditional on Y can lead to counterproductive procedures. Furthermore, as we have argued, even when Calibration by Group does not hold, in general there is no reason to expect conditioning on Y to block the paths between R and A. Thus we believe that the choice between Calibration by Group and Equalized Odds is not a mere subjective trade-off; instead we find Separation to be a fundamentally unhelpful fairness criterion. 



 §.§ Parity by Signal



The original motivation of Equalized Odds <cit.> was to overcome limitations of Independence. In addition to the stringency of the criterion, the authors argue that a limitation of the criterion is that the response Y itself does not satisfy Independence whenever there is dependence between Y and the sensitive characteristic A. This is undesirable, they write, since the response Y is an “ideal [prediction], which can hardly be considered discriminatory as it represents the actual outcome.”

This line of reasoning seems to obscure a crucial point. When the probability distribution of the response Y depends on the features X_1, …, X_p only through a signal parameter θ(A,X_1, …, X_p), which we could take without loss of generality to be the conditional mean E[Y | A,X_1, …, X_p] when θ is one-dimensional, the signal θ will not be an allowable prediction under Separation.

This follows by the same reasoning as in Section <ref>: in non-degenerate settings, conditioning on the response Y will not block the dependence between any of its ancestors. This is a significant limitation since the discrepancy between the response Y and the signal θ is generally unique to each individual and cannot be predicted. Indeed, no prediction rule R can achieve zero prediction error when the response Y has an exogenous noise component. Therefore, in practicality, the perfect prediction R is the signal θ, not the response Y. With this in mind, we explicitly define a new measure of fairness.

    Represent Y = F_θ(A,X_1,…,X_p)(ϵ), where ϵ is exogenous noise, that is, ϵ A, X_1, …, X_p, and F is a function indexed by a signal parameter θ. Then a prediction R satisfies Parity by Signal when R is conditionally independent of the sensitive attribute A given the signal θ, i.e. R  A |θ.

Another way to view this definition of fairness is that the predictions for similar people do not unnecessarily depend on the sensitive characteristic, where similar people are defined to be those whose features contribute—via the signal θ(A,X_1, …, X_p)—in the same way to the outcome. This is related to the measure of fairness described by <cit.>, wherein Separation by Signal would be considered as utilizing a perfect similarity metric. In Scenario 1, an optimal prediction rule which satisfies Separation was presented, and it was found to be unusual: however, in that same scenario, the true mean does indeed satisfy Parity by Signal.

This definition of fairness is not without its limitations. Evaluating whether a prediction R satisfies Separation by Signal requires the signal θ, which is generally not known in practice. Above we demonstrated that Separation by Signal is a useful device to develop understanding, but a close variant of it can also be made operational. Separation by Signal compares the prediction R to the signal θ, but, instead, R could be compared to another prediction S which we believe to be more accurate than R.

    A prediction R satisfies Parity by S if R is conditionally independent of the sensitive attribute A given S, i.e. R  A | S.

Notice that a prediction R satisfies Separation by Signal if and only if R satisfies Separation by θ. 

We can interpret a variety of fairness-testing procedures as a form of testing for Parity by S. For example, in the context of testing whether various police precincts exhibit racial bias in contraband searches, <cit.> develop a threshold test which is a test for a kind of Parity by Signal. They are in the binary setting and consider the prediction R(X_1, …, X_n) = 𝕀[ p_A(X_1, …, X_p) > t_A ] to be that an individual is carrying contraband when the probability p_A(X_1, …, X_p) of the individual carrying contraband is larger than a threshold t_A and to be that an individual is not carrying contraband otherwise. They develop Bayesian tests for whether the threshold t_A depends on the race A, since they argue that a fundamental form of unfairness occurs when minorities are ruled against more stringent thresholds. Due to the prediction R depending only on the probability p_A and the threshold t_A, this is a test for whether R satisfies Parity by p_A.

In the above example, the threshold test sought to determine whether there was a specific form of bias in police officers' decisions to search for contraband. This is an example of testing subjective human predictions, with some modeling assumptions. However, a Separation by S criterion can also be desirable to hold for a prediction R even when both R and S are generated by machine algorithms.
Consider cases in which we believe that a model is unfair due to misspecification; perhaps this model is missing necessary features or fails to model interactions between the sensitive characteristic and other features in a way which leads the predictions generated by the algorithm to disparately impact certain groups. (See in particular Scenario 1: The Red Car in <cit.>.) Specifically, suppose that the prediction R(X) = β^T X and the prediction S(X) = β_A^T X, where each coefficient β_A differs based on the sensitive characteristic A. In this case, a likelihood ratio test <cit.> between these models would be a test for whether R satisfies Separation by S.



§ CAUSAL CONSIDERATIONS



In various discussions of Independence and Separation-like criteria, authors propose generalizations which enforce parity only after conditioning on certain features <cit.>. Consider the following generalizations of Independence. Suppose X is some subvector of the features X_1, …, X_p.


    A prediction R satisfies Conditional Independence with respect to X if R  A | X.

Notice that Conditional Independence with respect to (X_1, …, X_p) always holds when the prediction R is a deterministic function of the features X_1, …, X_p. This reinforces that the purpose of Conditional Independence is to study the influence of a subvector X of the features. Notice also the connection to Parity by Signal and Parity by a prediction S, discussed in Section <ref>, which involve evaluating independence conditional on a specific function of the features.

Conditional Independence criteria themselves convey little information about the underlying desires of the practitioner. However, causal reasoning can provide principled methods for developing fairness criteria which may ultimately be expressed as Conditional Independence criteria. Here, we discuss two scenarios in which Conditional Independence criteria can be derived using causal reasoning. For a variety of perspectives on the role of Pearl's causality theory in fairness, see <cit.> and <cit.>.



 §.§ Scenario 3: College Admissions



For the purpose of college admissions, we wish to predict whether a student will drop out before completing his or her degree. Suppose we model the situation using the PDAG in Figure <ref>.


Suppose that the admissions committee wishes to use all of the relevant information about student performance contained in the student's SAT score X_2, but otherwise wishes to ignore the student's race-laden socioeconomic status X_1, despite the fact that socioeconomic factors do have a direct effect on a student's probability of dropping out. In the language of <cit.>, this means that X_2 is a resolving variable. The Independence criterion would not allow us to use all of the information in X_2, since we would have to extract the component of X_2 which is independent of the student's race A. Separation or Parity by Signal, on the other hand, would not allow us to fully make use of X_2 while entirely excluding information in X_1.

In fact we actually desire for A to have no direct effect on R that is not mediated by X_2. We can express this criterion using the formula for Controlled Direct Effect (CDE) as:

    𝒫(R |do(A = a),do(X_2 = x)) = 𝒫(R |do(A = a'),do(X_2 = x))

for all a,a' ∈𝔸 and all x in the range of X_2. In this case, this do-expression is identifiable and simplifies to the expression R  A | X_2. However, in cases when a resolving variable is itself confounded with other variables, the do-expressions in (<ref>) may be unidentifiable or require some do-calculus to resolve into observational expressions.  



 §.§ Scenario 4: Insurance Prices


In the scenarios we have discussed so far, we model the sensitive characteristic A as an exogenous variable. Thus A has always been a root in our PDAG models, and the total causal effects of A on endogenous entities such as R and Y coincide with the observed effects. However, when A is an endogenous variable, there may be backdoor paths from A to our predictions or response. Consider the following scenario, which makes clear the need for causal reasoning in fairness.

We wish to predict whether an individual is likely to have a car accident for the purpose of determining her insurance premium. Suppose we model the relevant variables using the PDAG in Figure <ref>.



We may deem it unfair to use an individual's religion, A, as a factor in our prediction, R. However, there is a backdoor path between A and Y, and thus A and Y are marginally dependent. For the same reason, any prediction R which is a non-trivial descendent of the individual's driving record X will fail to satisfy Independence. Of course, as in all of the ubiquitous cases when the features are ancestors of the response, conditioning on Y will not black the path between R and A either, thus our prediction will also fail to satisfy Separation.

In this case, we need not consider X to be a resolving variable. Whatever dependence results between A and R is spurious. We are interested in ensuring that A has no Total Effect (TE) on R, which we can express as:

    𝒫(R |do(A = a)) = 𝒫(R |do(A = a')).

 Note that this is a special case of the Counterfactual Fairness criterion in <cit.>, although these authors do not explicitly consider cases in which A is endogenous. The consequence of this criterion is that we can freely construct predictions R which are based on traffic tickets, X, or other inferred aspects of personality, U.



§ CONCLUSION



In Scenarios 1 through 4, PDAGs have proven to be fertile ground for developing intuition about the three basic oblivious criteria for fairness: Independence, Separation, and Sufficiency. In general, constructing a PDAG model relating the sensitive characteristic, features, and response is a clarifying exercise, because it allows us to more directly connect our senses about what is intuitively fair to the implications of the decisions we make in specifying an algorithm. In contrast to the project of constructing statistically optimal estimators, a fundamental concern in constructing fair estimators is blocking the use of information which is subjectively unacceptable. Here, PDAGs and d-separation are natural tools.

In contrast, the oblivious fairness criteria alone are limited, because their behavior is opaque and sensitive to the particularities of the scenario. Enforcing Independence between the sensitive characteristic and the prediction was seen to have wildly different implications in scenarios when the response was dependent on race and when it was not. In the former case, Independence prohibited 
discrimination which could not be statistically justified, and in the latter case, Independence was an intervention in favor of adversely affected groups.

Sufficiency can be achieved by blocking all paths through which information can flow between the sensitive characteristic and the response. Generally this is only possible by accurately recovering the signal. That is, it was shown that Sufficiency can be achieved by appropriately choosing the features through which information flows from race to the response and appropriately choosing a prediction that blocks that flow of information. 

Separation naturally allows for the use of features which are descendants of the response, but exhibits strange behavior whenever there are features which are non-descendants of the response, even when those features are independent of the sensitive characteristic. For non-degenerate and faithful PDAG models, Separation will not hold since the response is comprised of not only the signal, but also the noise, which obscures the information about the signal in the response. In Scenario 1, a violation of faithfulness was induced to produce an optimal prediction rule that turned out to be highly inappropriate, for some groups even requiring the addition of further random noise.

For the most part, these measures have been found wanting. 

We join the recent consensus that the assessment of fairness in algorithms should not start and end with the use of a singular criterion. The constraints we wish to impose on predictions should be sensitive to each scenario, and PDAG models can help to explore them.

More generally, we notice that there is little consensus on the underlying philosophical principles that should provide the foundation for the quantification of fairness. While much work in fairness seems to be framed around preventing unfairness like that allegedly exhibited by the COMPAS algorithm, there is no consensus that the COMPAS algorithm was ever unfair. 
We worry that as constructs from fairness are taken out of context and treated as black boxes for mathematical study and elaboration, the implicit underlying notions of fairness will be obscured.
We hope that fairness research can be grounded in clear, practical examples of the ways algorithms can be unfair. 




§ ACKNOWLEDGEMENTS



Thanks to David Kent for helpful conversations.


plainnat

