---
abstract: 'Instrumental variable (IV) studies seek to emulate randomized encouragement designs where patients are assigned a random nudge towards a particular treatment. Unfortunately, IV studies may fall short of their experimental ideal due to hidden bias affecting both the proposed instrument and the outcomes. While sensitivity analyses have been developed for IV designs most proceed under an assumption of effect homogeneity, unlikely to hold for many applications. In our case study on the efficacy of surgical versus non-surgical management for two gastrointestinal tract conditions, whether or not surgery will be of greatest benefit strongly depends upon patient-specific physiology. The validity of our proposed instrument, surgeons' tendencies to operate, is plausible but not ironclad, necessitating a sensitivity analysis allowing for effect variation. We present a new sensitivity analysis accommodating arbitrary patterns of non-identified effect heterogeneity. With binary outcomes, we prove that the conventional sensitivity analysis based on McNemar's test is itself asymptotically valid with heterogeneous effects. We then demonstrate that the identified components of effect heterogeneity can, when suitably exploited, improve the performance of a sensitivity analysis through providing less conservative standard error estimators. We further highlight the importance of heterogeneity in determining the power of a sensitivity analysis.'
address:
- '$^{1}$ Operations Research and Statistics Group, MIT Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA '
- ' $^{2}$ Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA'
- '$^{3}$Center for Surgery and Health Economics, Department of Surgery, Hospital of the University of Pennsylvania, Philadelphia, PA'
- '$^4$ University of Pennsylvania, 3400 Spruce St, Philadelphia, PA'
author:
- 'Colin B. Fogarty$^{1}$'
- 'Kwonsang Lee$^{2}$'
- 'Rachel R. Kelz$^{3}$'
- 'Luke J. Keele$^{4}$'
bibliography:
- 'er\_bib.bib'
title: Biased Encouragements and Heterogeneous Effects in an Instrumental Variable Study of Emergency General Surgical Outcomes
---

Introduction
============

Emergency general surgery conditions and operative care {#sec:motiv_example}
-------------------------------------------------------

Emergency general surgery (EGS) and trauma surgery are considered different forms of acute care. EGS refers to medical emergencies where the injury is most often endogenous (a burst appendix) while trauma care refers to injuries that are exogenous (a gunshot wound). Correspondingly, there are a set of medical conditions known as EGS conditions that are distinct from trauma injuries [@gale2014public; @shafi2013emergency]. EGS conditions affect an estimated 3-4 million patients per year and account for more than 800,000 operations in the United States alone [@gale2014public; @scott2016use; @havens2015excess; @ogola2015financial]. Operative management is often the primary course of treatment for EGS conditions. Surgical treatment, however, carries additional risk of complications and adverse reactions to anesthesia. For many EGS conditions, non-operative care including observation, minimally invasive procedures and supportive care may also be effective, with a lower risk of complication. Given these competing paths for treatment, a critical clinical question faced by the acute care community is determining the overall efficacy of surgical interventions for EGS conditions relative to alternative courses of treatment.

We focus on the effectiveness of operative care for two common and related EGS conditions affecting the lower gastrointestinal tract: ulcerative colitis and diverticulitis. Diverticula are marble-sized pouches that can form in the lining of the lower part of the large intestine. Diverticulitis occurs when one or more diverticula tear and become inflamed or infected, which causes severe abdominal pain, fever, nausea and a marked change in bowel habits. It may be managed with rest, changes to diet and antibiotics, but may ultimately require surgery. Ulcerative colitis is a form of inflammatory bowel disease that causes long-lasting inflammation and ulcers in the lining of the large intestine and rectum. The condition most often begins gradually and can become worse over time and lead to life-threatening complications. Like diverticulitis, colitis may be managed through diet and medications, but may require surgery to remove sections of the intestine.

One challenge in studying the effectiveness of surgery for these EGS conditions is that randomized comparisons between surgical and non-surgical care are generally judged to violate the principle of clinical equipoise. Evidence on the effectiveness of procedures for EGS conditions must instead be collected from observational studies, which introduces potential biases into resulting analyses due to the non-random determination of the assigned treatment. Unsurprisingly, patients who undergo emergency operative treatment are often the sickest patients [@shafi2013emergency]. While statistical adjustment for measures of patient frailty may render treated and control groups comparable, such adjustments are often insufficient due to both inadequacies in observed covariates and the potential presence of hidden biases [@keele2018icubeds]. Faced with such biases, one alternative is to search for "natural experiments" where an intervention is judged to be allocated in an as-if random fashions. While exploiting these "gifts of nature" [@Rosenzweig:2000 pg. 872] requires considerable care, results from natural experiments are often compelling despite the absence of active randomization [@Angrist:2010; @Dunning:2012].

Physician preference as an instrument in comparative effectiveness research {#sec:intro}
---------------------------------------------------------------------------

The instrumental variable (IV) method is one form of natural experiment exploiting haphazard nudges towards treatment exposure. An IV is a variable that is associated with the treatment of interest, but affects outcomes only indirectly through its impact on treatment assignment [@Angrist:1996]. One frequently proposed instrument in comparative effectiveness research is a physicians's preference for a specific course of treatment. Here, it is assumed that for a patient of a given level of disease severity, physicians may differ in their preferred course of treatment for primarily idiosyncratic reasons. Thus, when a patient receives care from a physician with a specific set of treatment preferences, it is assumed they are being exposed to an as-if randomized encouragement towards that treatment. @brookhart2006evaluating first used this type of IV in the context of drug prescriptions. See @davies2013cox for examples and @brookhart2007preference for an overview.

We consider surgeon's preference for operative management as a candidate instrument for whether a specific patient with a diagnosis of colitis or diverticulitis receives surgery after admission to the emergency department. We measure each surgeon's preference for operative management with data on how they manage care for EGS conditions. For each surgeon in our data, we have information on whether or not a surgeon operated for patients with 51 acute EGS conditions. Following @keeleegsiv2018, we use this information to construct a measure of a surgeon's tendency to operate (TTO) by calculating the percentage of times a surgeon operates when presented with an EGS condition. For the surgeons in the data, TTO–expressed as percentage–varied from less than 5% to 95%. Consider two patients in the emergency department with diverticulitis with similar baseline characteristics. One might receive care from a surgeon that tends to operate (higher TTO), while another might receive care from a surgeon that is more likely to recommend non-surgical management (lower TTO). Being assigned to a patient with a higher or lower tendency to operate may then be viewed nudge towards or away from surgery [@keeleegsiv2018]. Since these are emergency admissions, it is unlikely that patients themselves are selecting their surgeon. Instead, this suggests that assignment to surgeon with a specific TTO is a plausibly haphazard process, with clear impact on whether or not a patient receives operative care.

Our data set merges the American Medical Association (AMA) Physician Masterfile with all-payer hospital discharge claims from New York, Florida and Pennsylvania in 2012-2013. The study population consists of all patients admitted for inpatient care emergently, urgently, or through the emergency department during that time period in those states. The data include an identifier for each patient's surgeon, along with patient sociodemographic and clinical characteristics including indicators for frailty, severe sepsis or septic shock, and 31 comorbidities based on Elixhauser indices [@elixhauser1998comorbidity]. The data also include measures of the surgeon's age and years experience. Surgeons were excluded from the cohort if they did not perform at least 5 operations for one of the 51 specific EGS conditions per year within the two-year study time-frame. All told, this resulted in $N=44,082$ patients presenting with either colitis or diverticulitis. We use three different measures for health outcomes. We also investigate whether the presence of sepsis (an inflammatory response to infection that can result in organ malfunction) acts as a modifier for the effect of surgery. Our primary outcome was the presence of any in-hospital complication (infection, bleeding) as measured by a binary variable. Secondary outcomes are hospital length-of-stay; and in-hospital mortality within 30 days after admission. Mortality was treated as a secondary outcome, as deaths for patients with a colitis or diverticulitis diagnosis are rare. To our knowledge, this is the first study based on IV methods that provides evidence on the effectiveness of operative management for these two conditions.

Probing the robustness of an instrumental variable analysis
-----------------------------------------------------------

In order for a surgeon's tendency to operate to be a valid instrument, one must assume that there are no lurking variables which simultaneously influence the value of TTO for the surgeon assigned to a patient and the patient's health outcomes. Suppose instead that surgeons with higher TTO are precisely the more skilled surgeons. Were this the case, and if patients most in the most dire straits were assigned to the most skilled surgeons, then TTO's status as an instrumental variable would be undermined and the resulting estimate of surgery's efficacy in managing colitis and diverticulitis may be biased. While the data include measures of surgeon age and experience, these are imperfect measures of surgical skill, such that even after adjusting for these measures a critic of the study may be unconvinced that bias has been removed.

Such suspicions of residual bias motivate sensitivity analyses for IV studies. A sensitivity analysis asks how strong the impact of a hidden bias would have to be on the instrumental variable to alter the substantive conclusions from the study. @Rosenbaum:1996 first proposed a sensitivity analysis for the possibility of hidden bias in instrumental variable studies. Existing methods for sensitivity analysis in an IV design share a common limitation: they are derived under the assumption of a pre-specified model of effects for each individual in the observational study. By far the most prevalent model for effects under these approaches is that of proportional doses - i.e. that the difference in potential outcomes under encouragement to treatment versus encouragement to control is proportional to the difference in treatment actually received under encouragement to treatment versus encouragement to control [@Rosenbaum:2002a; @Imbens:2005b]. This model is reviewed in §\[sec:propdose\]. In our example, with respect to length of stay, this would imply the hospital length of stay that would have been observed if assigned to a surgeon with a high versus low TTO would be proportional to the difference in whether or not surgery would actually be performed on that individual when assigned to a high versus low TTO surgeon, and further that the same proportionality constant holds across all individuals. This would imply that all patients who would undergo surgery if and only if assigned to a surgeon with a high TTO have the same treatment effect. Furthermore, with binary outcomes such as our complication outcome, the only plausible value for this effect would be zero.

The effect homogeneity implied by this model seems at odds with the possibility of essential heterogeneity. Essential heterogeneity refers to the possibility that individuals select treatments based on expected idiosyncratic gains or losses from treatments. For example, essential heterogeneity would be present if patients opt for or against surgery based on their belief of which treatment would furnish the largest benefit. Were patients correct in their beliefs, one would then expect surgery to have a positive effect relative to non-surgical alternatives on those selecting into surgery, and a negative effect on those who received the non-surgical alternatives. Many argue that essential heterogeneity is a key feature of the IV framework [@heckman2006understanding], and a suspicion that essential heterogeneity may be present would call into question the utility of a sensitivity analysis conducted under the proportional dose model.

The roles of effect heterogeneity in randomization-based sensitivity analysis
-----------------------------------------------------------------------------

Treatment effect heterogeneity refers to variation in the effects of an intervention across individuals. While some of this variation may be ascribed to intrinsically random variation, a portion of it may well be predictable on the basis of observed covariates known as *effect modifiers*. Heterogeneity in effects is plausible if not certain in our application, as the degree of risk associated with surgery is dependent upon to patient-specific physiology. Presence of sepsis upon presentation to a surgeon is just one factor that might influence whether surgery would be beneficial relative to non-surgical alternatives.

Treatment effect heterogeneity has both identified and nonidentified components. Using our investigation of EGS outcomes as a case study, we illustrate the varied roles that these components can play in the analysis of matched instrumental variable studies. In addressing the potentially pernicious nonidentified aspects of effect heterogeneity, we present a sensitivity analysis for matched IV studies that is exact under the proportional dose model but is asymptotically correct for a much weaker null hypothesis on a parameter known as the effect ratio [@Baiocchi:2010; @kang2016full], the ratio of the average effects of encouragement on the potential outcomes to encouragement on the treatment received without specifying the individual-level treatment effects. This method obviates concerns that the nonidentified components of effect heterogeneity may conspire with hidden bias to render attempts at sensitivity analysis insufficient. We compare our methods with existing approaches for sensitivity analysis in matched IV studies justified under more restrictive models of effects. In the particular case of binary outcomes, we prove that the suggested method for sensitivity analysis of @Baiocchi:2010 using McNemar's test, justified therein only under Fisher's null of no treatment effect, is a special case of our method and hence also provides an asymptotically correct sensitivity analysis for the null of no effect ratio while allowing for patient-specific heterogeneity.

Next, we use our clinical application to EGS outcomes to highlight the roles that the identified aspects of effect heterogeneity can play in improving the performance of a sensitivity analysis. We develop improved standard error estimates for use within a sensitivity analysis with heterogeneous effects. These standard errors are valid regardless of the truth of an underlying model for effect modification as a function of observed covariates, but provide more substantial gains should the posited model of effects motivating them prove correct. By further comparing results of our analysis between the sepsis and non-sepsis subgroups, we provide an illustration of the role that latent heterogeneity can play in the resulting power of a sensitivity analysis as originally discussed in @Rosenbaum:2005b. In an analysis assuming no hidden bias, the power of a test of no treatment effect when one is actually present tends to one in the limit so long as the effect is identified. Contrary to this, a sensitivity analysis' limiting power under non-local alternatives depends not upon the magnitude of the treatment effect itself, but rather upon its magnitude *relative to* the degree of heterogeneity within matched sets. Perhaps surprisingly, a sensitivity analysis may be less able to discriminate bias from treatment effect in a subgroup with a larger treatment effect than a subgroup with a smaller treatment effect even asymptotically should the subgroup with the larger effect also have a larger degree of heterogeneity. That is to say, if subgroup 1 has a larger treatment effect than subgroup 2, a sensitivity analysis may nonetheless have limiting power 0 to detect the effect in subgroup 1, but limiting power 1 in subgroup 2. Exploiting the identified components can substantially benefit the resulting sensitivity analysis, while the unidentified components are rendered innocuous by the methods developed here.

A matched instrumental variable study and its experimental ideal
================================================================

A near-far match
----------------

It is well documented that stronger instruments provide both more informative effect estimates and estimates that are more resistant to hidden bias, providing motivation to actively strengthen an instrument as part of the analysis [@Small:2008]. One approach for strengthening an instrument is a near-far matched design [@Baiocchi:2010]. Near-far matching creates matched pairs that are similar in terms of observed covariates but highly dissimilar on the values of the instrument. We use a form of near-far matching that combines refined covariate balance with near-far matching [@Pimentel:2015a; @keeleicumatch2019]. This allows us to both balance a large number of nominal covariates viewed to be of clinical importance while maintaining a large distance between pairs on the instrumental variable.

Each matched pair is exactly matched on both hospital and an indicator of sepsis, such that across-hospital differences in quality of care and fundamental differences in physiology between septic and non-septic patient cannot bias the analysis. For the remaining covariates, we minimized the total of the within-pair distances on covariates as measured by the Mahalanobis distance and applied a caliper to the propensity score. We further applied near-fine balance constraints for three indicators for surgical volume at the hospital at the time of admission and a dummy variable for whether a patient had a specific disability. Fine balance constrains two groups to be balanced on a particular variable without restricting matching on the variable within individual pairs [@Rosenbaum:2007b]. A near-fine balance constraint returns a finely balanced match when one is feasible, and otherwise minimizes the deviation from fine balance [@Yang:2012]. As a result, the marginal distribution for these three indicators will be exactly or nearly exactly the same across levels of the IV. Finally, we also applied optimal subsetting to discard matched pairs with high levels of imbalance on covariates [@Rosenbaum:2011]. After matching, we calculated the standardized difference for each covariate, which is the mean difference between matched patients divided by the pooled standard deviation before matching. In the match, we attempted to produce absolute standardized differences of less than 0.10, a commonly recommended threshold [@Rosenbaum:2010].

Table \[tab.bal\] contains balance statistics before and after matching on selected covariates. In general, balance even before matching is quite good: none of the absolute standardized differences exceed 0.20, and only six exceed 0.10. Matching moved most absolute standardized differences below 0.1, suggesting that matching was successful in attenuating potential biases due to discrepancies in observed surgeon characteristics and available data on patient physiology. From the perspective of observed covariates, Table \[tab.bal\] suggests that surgeons with high versus low TTO appear to have been assigned to patients in a haphazard, asystematic manner.

Notation for randomized encouragement designs and instrumental variable studies
-------------------------------------------------------------------------------

We now introduce notation for the experimental design that near-far matching emulates. To assist the reader, essential notation and definitions introduced here and elsewhere are summarized in §A of the appendix. There are $2n$ individuals partitioned into $n$ matched pairs. Each pair contains one individual encouraged to take the treatment, denoted $Z_{ij}=1$, and one individual not encouraged to take the treatment ($Z_{ij}=0$). In our study, the patient assigned to the surgeon with the higher value of the instrumental variable $TTO$ in each pair is the patient for whom $Z_{ij}=1$. Individual $ij$ has two potential responses: one under encouragement to treatment, $y_{ij}(1)$, and the other under no encouragement, $y_{ij}(0)$. Further, individual $ij$ has two values for the level of treatment actually received: that under encouragement, $d_{ij}(1)$, and under no encouragement, $d_{ij}(0)$. In this work we will consider binary values for $d_{ij}(z)$ for $z=0,1$, such that $d_{ij}(z)$ represents whether or not an individual would actually take the treatment when assigned encouragement level $z$. That said, the methods we develop readily extend to the case of continuous $d_{ij}(z)$. The observed outcome and exposure are $Y_{ij} = y_{ij}(Z_{ij})$ and $D_{ij} = d_{ij}(Z_{ij})$ respectively. Let $\mathcal{F} = \{y_{ij}(1), y_{ij}(0), d_{ij}(1), d_{ij}(0), \mathbf{x}_{ij}, u_{ij}: i=1, \ldots ,n; j=1, 2\}$, where $\mathbf{x}_{ij}$ represent the observed covariates for each individual and $u_{ij}$ is an unobserved covariate $0 \leq u_{ij} \leq 1$. While a univariate hidden variable constrained to the unit interval may appear restrictive, in fact there always exists a hidden variable between 0 and 1 such that if the practitioner had access to it, adjustment for it would be sufficient for identifying a causal effect [@Rosenbaum:2017 §9, footnote 15]. We consider finite-population inference moving forwards, which will be reflected by conditioning upon $\mathcal{F}$ in the probabilistic statements that follow. Hence, inferential statements will pertain to parameters defined in the observed study population, without dependence upon the existence of a hypothetical superpopulation. Let $\mathbf{Z} = (Z_{i1},Z_{i2},...,Z_{n2})^T$, and let boldface similarly denote other vector quantities such as $\mathbf{u} = (u_{i1},u_{i2},...,u_{n2})^T$.

Patients in our study are placed into pairs using observed covariates $\mathbf{x}_{ij}$ such that $\mathbf{x}_{i1}\approx \mathbf{x}_{i2}$. Despite this, paired subjects may not be equally likely to be exposed to a high versus low value of the instrument due to discrepancies on an unobserved covariate, i.e. $u_{i1} \neq u_{i2}$. Let $\Omega = \{\mathbf{z}: z_{i1}+z_{i2}=1\}$ be the set of the $2^n$ possible values $\mathbf{z}$ of $\mathbf{Z}$, and let $\mathcal{Z}$ denote the event $\{\mathbf{Z}\in \Omega\}$. In finite-population causal inference the only source of randomness is the assignment of the encouragement $Z_{ij}$. Were $\mathbf{Z}$ chosen uniformly from $\Omega$ as in a randomized experiment, we would have $\text{Pr}(\mathbf{Z}=\mathbf{z}\mid \mathcal{F}, \mathcal{Z}) = 1/|\Omega|$ for each $\mathbf{z}\in \Omega$, where $|A|$ is the number of elements of a finite set $A$.

A model for biased encouragements
---------------------------------

While assignment of patients to surgeons of varying tendencies to operate may have appeared haphazard on the basis of observed covariates, matching can do nothing to preclude hidden biases from influencing effect estimates and hypothesis tests. We employ the model for a sensitivity analysis from @Rosenbaum:2002 [§4], which parameterizes the impact of hidden biases on the probabilities of encouragement within a matched set. Let $\pi_{ij} = {\text{Pr}}(Z_{ij}=1 \mid \mathcal{F})$. The model proposes that for individuals $j$ and $k$ in the same matched set $i$, $\pi_{ij}$ and $\pi_{ik}$ may differ in the odds ratio scale by at most $\Gamma \geq 1$, $$\frac{1}{\Gamma} \leq \frac{\pi_{ij}(1-\pi_{ik})}{\pi_{ik}(1-\pi_{ij})} \leq \Gamma.
    \label{eqn:sensi}$$ Returning attention to the paired structure by conditioning upon $\mathcal{Z}$, is equivalent to assuming $1/(1+\Gamma) \leq \text{Pr}(Z_{ij}=1\mid \mathcal{F}, \mathcal{Z}) \leq \Gamma/(1+\Gamma)$ for $i=1, \ldots ,n; j=1, 2$. In the context of our study, equation (\[eqn:sensi\]) suggests that before matching and without conditioning on the matched structure, patients with similar observed covariates could differ in their odds ratio of being assigned to a surgeon with a high TTO by at most $\Gamma$. Conditioning on the matched structure, the probability that a given patient in a matched set was assigned to a surgeon with high TTO given that exactly one of the two patients in that matched set were assigned to a high TTO patient falls between $1/(1+\Gamma)$ and $\Gamma/(1+\Gamma)$. $\Gamma=1$ corresponds to a randomized encouragement design [@Holland:1988], while pushing $\Gamma$ beyond one allows unmeasured confounding to increasingly bias the encouragement to treatment.

Existing modes of inference for the effect ratio
================================================

The effect ratio and its interpretation
---------------------------------------

We consider inference for an estimand known as the effect ratio, $$\lambda = \frac{\sum_{i=1}^{n}\sum_{j=1}^{2} \{y_{ij}(1) - y_{ij}(0)\}}{\sum_{i=1}^{n}\sum_{j=1}^{2} \{d_{ij}(1) - d_{ij}(0)\}},$$ where it is assumed that $\sum_{i=1}^{n}\sum_{j=1}^{2} \{d_{ij}(1) - d_{ij}(0)\} > 0$. The effect ratio is the ratio of two average treatment effects. In our application, the treatment effect in the numerator is the effect of being assigned to a surgeon with high versus low TTO on any one of the health outcomes of interest. The treatment effect in the denominator is the effect of surgeon TTO on whether or not a patient has surgery. Under perfect compliance, $d_{ij}(1)-d_{ij}(0) = 1$ for all individuals and $\lambda$ would be the sample average treatment effect for the individuals in our matched study.

@Angrist:1996 discuss how additional assumptions about the treatment actually received and the potential responses justify alternative interpretations for the effect ratio. When $d_{ij}(z)$ are binary for $z=0,1$, individuals fall into four different classes: compliers, always-takers, never-takers, and defiers. Compliers are patients who are exposed to surgery if they were encouraged by the IV but would not otherwise have surgery. Always-takers have surgery regardless of assigned IV status, and never-takers are never exposed to surgery regardless of IV status. Defiers are patients who are only exposed to the treatment (surgery) if not encouraged or are unexposed if encouraged. Suppose one is willing to assume that (1) $d_{ij}(1)\geq d_{ij}(0)$, known as monotonicity; and (2) $d_{ij}(1)-d_{ij}(0)=0 \Rightarrow y_{ij}(1)-y_{ij}(0) = 0$, the exclusion restriction. Assumption (1) implies that there are no defiers, while assumption (2) implies that the encouragement only affects the potential outcomes through altering the treatment actually received. Under these assumptions, $\lambda$ may be interpreted as the sample average treatment effect among compliers, i.e. the individuals who would undergo surgery if and only if assigned to a surgeon with a higher preference for surgery. @small2017instrumental describe a relaxation of monotonicity called stochastic monotonicity which may be more appropriate for IV studies using physician preference as an instrument. Under stochastic monotonicity, $\lambda$ instead refers to a weighted average treatment effect where individuals more strongly influenced by the encouragement receive larger weight. Inferences for the effect ratio $\lambda$ are valid regardless of these assumptions when interpreted solely as a ratio of two treatment effects, but one is entitled to additional interpretations should further assumptions be made.

Assuming validity of the instrumental variable, i.e. that (\[eqn:sensi\]) holds at $\Gamma=1$, @Baiocchi:2010 provide a large-sample method for constructing confidence intervals and performing inference for the effect ratio under effect heterogeneity. Using the method of @Baiocchi:2010, Table \[tab.out\] shows estimated effect ratios for septic and non-septic patients on our three surgical outcomes: mortality, presence of a complication after the administered course of treatment, and hospital length of stay. To contextualize the effect ratio in this case, the effect ratio for length of stay for septic patients estimates the ratio of the average effects of being assigned to a high versus low tendency to operate surgeon on (a) the length of stay in the hospital to (b) whether or not the patient actually underwent surgery. Assuming monotonicity and the exclusion restriction, the estimand can further be interpreted as the average effect of *surgery* on hospital length of stay for the subset of individuals who would undergo surgery if and only if assigned to a surgeon with a higher preference for surgery. Under these assumptions, we find that among compliers who are non-septic, the estimated difference in the percentage of complications had all patients undergone surgery versus had all patients undergone non-surgical management is 12% (favoring non-surgical alternatives). For septic patients complying with their encouragement, this risk difference is 25% (favoring non-surgical alternatives). While the method yielding the estimates in Table \[tab.out\] is valid under heterogeneous effect under the assumption of no hidden bias, lurking variables such as innate surgeon ability or unobserved patient health characteristics may have resulted in bias in these IV estimates. We now describe existing methods for sensitivity analysis in instrumental variable studies, and note their apparent inability to accommodate arbitrary patterns of effect heterogeneity.

          Septic Patients    Non-septic Patients
  -- -- ------------------- ---------------------
               25.3                 12.0
         \[ 19.3 , 31.4 \]    \[ 8.3 , 15.7 \]
                6.8                  4.1
          \[ 4.4 , 9.2 \]      \[ 3.4 , 4.8 \]
               1.96                 0.59
         \[ -1.9 , 5.9 \]     \[ -0.5 , 1.6 \]

  : Estimated effect of surgery on EGS patient outcomes based the effect ratio. Point estimates for binary outcomes are risk differences expressed as percentages. Brackets contain 95% confidence intervals attained by inverting the test of @Baiocchi:2010.<span data-label="tab.out"></span>

Sensitivity analysis for the effect ratio with effects proportional to dose {#sec:propdose}
---------------------------------------------------------------------------

@Rosenbaum:1996 and @Rosenbaum:2002 [§5.4] develop methods for exact sensitivity analyses in paired instrumental variable studies under the proportional dose model, which states $$H_{prop}^{(\lambda_0)} : y_{ij}(1) - y_{ij}(0) = \lambda_0 \{d_{ij}(1) - d_{ij}(0)\}, i=1, \ldots ,n; j=1, 2.$$ This naturally extends the model of constant effects commonly employed in randomized experiments to the case of randomized encouragement designs. If an individual complies with their encouraged treatment then $y_{ij}(1)-y_{ij}(0)=\lambda_0$, while if an individual defies their encouragement then $y_{ij}(1)-y_{ij}(0) = -\lambda_0$ as $d_{ij}(1)-d_{ij}(0) = -1$. If the encouragement does not influence the treatment received, $y_{ij}(1)-y_{ij}(0)$ is set to zero and hence the exclusion restriction holds. Under this model, a sensitivity analysis proceeds by finding the worst-case inference for a particular $\Gamma$ in (\[eqn:sensi\]). One then iteratively increases the value of $\Gamma$ until the null hypothesis can no longer be rejected. @Baiocchi:2010 describe how McNemar's test can be used for the particular null hypothesis $H_{prop}^{(0)}$ with binary outcomes.

The proportional dose model may be considered overly restrictive for certain instrumental variable studies. In particular, it implies that all individuals who would comply with their assigned encouragement have an identical treatment effect $\lambda$, which precludes essential heterogeneity among other forms of effect heterogeneity. Furthermore, for binary outcomes, the *only* value of $\lambda$ for which the model is plausible is $\lambda=0$, amounting to an assumption of Fisher's sharp null that the encouragement has no effect for any individual. Given the apparent limitations of the proportional dose model in the presence of heterogeneous treatment effects, a more attractive form of sensitivity analysis might instead focus on the weaker null hypothesis $$\begin{aligned}
H_{weak}^{(\lambda_0)}: \lambda = \lambda_0.\end{aligned}$$ The hypothesis $H_{weak}^{(\lambda_0)}$ is a composite hypothesis, testing all possible combinations of potential outcomes that satisfy $\lambda = \lambda_0$. This hypothesis does not require any further pre-specification of the individual level potential outcomes or doses received. The proportional dose hypothesis $H_{prop}^{(\lambda_0)}$ is simply one element of $H_{weak}^{(\lambda_0)}$, and need not yield the supremum $p$-value over the composite null when conducting a sensitivity analysis. See @Baiocchi:2010 for more on the relationship between $H_{prop}^{(\lambda_0)}$ and $H_{weak}^{(\lambda_0)}$.

Sensitivity analysis for the effect ratio
=========================================

A valid approach with heterogeneous effects {#sec:senshet}
-------------------------------------------

We now develop a sensitivity analysis for testing $H_{weak}^{(\lambda_0)}$ in paired instrumental variable designs. Observe that under the null hypothesis, $(2n)^{-1}\sum_{i=1}^n\sum_{j=1}^2[y_{ij}(1)-y_{ij}(0) - \lambda_0\{d_{ij}(1)-d_{ij}(0)\}] = 0$. Define $\zeta^{(\lambda_0)}_{i} = (Z_{i1}-Z_{i2})\{Y_{i1}-Y_{i2} - \lambda_0(D_{i1}-D_{i2})\}$ as the encouraged-minus-non encouraged differences in the terms $Y_{ij}-\lambda_0D_{ij}$, which may be thought of as the observed outcome adjusted for the treatment level received for each individual $ij$. Observe that at $\Gamma=1$, $E(\zeta_i^{(\lambda_0)}\mid {{\mathcal{F}}}, {{\mathcal{Z}}}) = (1/2)\sum_{j=1}^2[y_{ij}(1)-y_{ij}(0) - \lambda_0\{d_{ij}(1)-d_{ij}(0)\}]$, and that $n^{-1}\sum_{i=1}^n\zeta_i^{(\lambda_0)}$ hence forms an unbiased estimating equation for $\lambda_0$ in a randomized encouragement design. Further observe that under $H_{prop}^{(\lambda_0)}$, the absolute value $|\zeta_i^{(\lambda_0)}|$ is fixed across randomizations, with only its sign varying. This observation underpins the use of permutation-based methods for sensitivity inference described in @Rosenbaum:2002; however, for other elements of $H_{weak}^{(\lambda_0)}$, the value of $|\zeta_i^{(\lambda_0)}|$ may itself be random over ${{\mathbf{z}}}\in \Omega$.

Suppose one wants to conduct a sensitivity analysis for whether the effect ratio equals $\lambda_0$ at level of unmeasured confounding $\Gamma$ without assuming proportional doses. Define $L_{\Gamma i}$ as $$\begin{aligned}
L_{\Gamma i}= \zeta^{(\lambda_0)}_{i} - \left(\frac{\Gamma-1}{1+\Gamma}\right)|\zeta^{(\lambda_0)}_{i}|.\end{aligned}$$ Based upon the $n$ random variables $L_{\Gamma i}$, define $\text{se}(\bar{L}_\Gamma)$ to be the conventional standard error for a paired design, $$\begin{aligned}
\text{se}(\bar{L}_\Gamma)^2 = \frac{1}{n(n-1)}\sum_{i=1}^n(L_{\Gamma i} - \bar{L}_\Gamma)^2,\end{aligned}$$ and consider using as a test statistic with a greater-than alternative $T(\mathbf{Z}, \mathbf{Y}-\lambda_0\mathbf{D}) =  \bar{L}_\Gamma/\text{se}(\bar{L}_\Gamma)$.

With our test statistic defined, attention now shifts towards the construction of a reference distribution for the sensitivity analysis. When testing with a greater-than alternative, the sensitivity analysis focuses bounding the upper-tail probability for the employed test statistic. Let $V_{\Gamma i}$ take values $\pm 1$ with ${{\text{pr}}}(V_{\Gamma i} = 1\mid {{\mathcal{F}}}, {{\mathcal{Z}}}) = \Gamma/(1+\Gamma)$, and define the random variable $B_{\Gamma i}$ as $$\begin{aligned}
B_{\Gamma i} &= V_{\Gamma i}|\zeta^{(\lambda_0)}_{i}| - \left(\frac{\Gamma-1}{1+\Gamma}\right)|\zeta^{(\lambda_0)}_{i}|.\end{aligned}$$ The construction of $B_{\Gamma i}$ is motivated by the proportional dose model: under $H_{prop}^{(\lambda_0)}$, $B_{\Gamma i}$ stochastically dominates $L_{\Gamma i}$ if (\[eqn:sensi\]) holds at $\Gamma$. Despite the fact that we are testing the weaker null $H_{weak}^{(\lambda_0)}$ consider using as a reference distribution the distribution of $\bar{B}_\Gamma/\text{se}(\bar{B}_\Gamma)$ given $|\boldsymbol\zeta^{(\lambda_0)}|$. In other words, we employ the randomization distribution of the test statistic $\bar{L}_\Gamma/\text{se}(\bar{L}_\Gamma)$ imagining that the proportional dose model held, and setting the assignment probabilities equal to those that stochastic bound $L_{\Gamma i}$ under that model. The decision whether or not to reject the null hypothesis in a sensitivity analysis at $\Gamma$ using the studentized statistic $\bar{L}_\Gamma/\text{se}(\bar{L}_\Gamma)$ is $$\begin{aligned}
\label{eq:procedure}
\varphi^{(\lambda_0)}(\alpha,\Gamma) &= {\mathbbm{1}}\{\bar{L}_\Gamma/\text{se}(\bar{L}_\Gamma) \geq G_\Gamma^{-1}(1-\alpha)\},\end{aligned}$$ where $G_\Gamma^{-1}(q) = \inf \{k : G_\Gamma(k) \geq q\}$ is the $q$th quantile of the distribution of $\bar{B}_\Gamma/\text{se}(\bar{B}_\Gamma)$ given $|\boldsymbol\zeta^{(\lambda_0)}|$. See Algorithm 2 of @Fogarty:2019 for more on constructing $G_\Gamma(\cdot)$, with his $\hat{\tau}_i$ replaced by $\zeta_i^{(\lambda_0)}$.

Suppose that $H_{weak}^{(\lambda_0)}$ is true and that (\[eqn:sensi\]) holds at $\Gamma$. Then, under mild regularity conditions and for $\alpha \leq 0.5$,$$\begin{aligned}
\underset{n\rightarrow \infty}{\lim} E(\varphi^{(\lambda_0)}(\alpha,\Gamma)\mid {{\mathcal{F}}}, {{\mathcal{Z}}})     & \leq \alpha.\end{aligned}$$\[prop:stu\]

The procedure $\varphi^{(\lambda_0)}(\alpha,\Gamma)$ provides an asymptotically valid sensitivity analysis for the effect ratio $\lambda$. Under the additional assumptions of monotonicity and the exclusion restriction, $\varphi^{(\lambda_0)}(\alpha,\Gamma)$ would also be a valid sensitivity analysis for the sample average treatment effect among compliers. The proof of Proposition \[prop:stu\] mirrors that of Theorem 2 in @Fogarty:2019 and is sketched in the appendix. For testing the null hypothesis on the effect ratio without assuming proportional doses, regularity conditions are required to ensure that $n\times \text{se}(\bar{L}_\Gamma)^2$ converges in probability to a limiting value, and that a central limit theorem applies to $\sqrt{n}\bar{L}_\Gamma$.

Analogous arguments to those presented in @Fogarty:2019 [§4.3] imply that one attains a sensitivity analysis that is asymptotically correct under $H_{weak}^{(\lambda_0)}$ and finite-sample exact under $H_{prop}^{(\lambda_0)}$ by replacing $\bar{L}_\Gamma/\text{se}(\bar{L}_\Gamma)$ and $\bar{B}_\Gamma/\text{se}(\bar{B}_\Gamma)$ with $\max\{0, \bar{L}_\Gamma/\text{se}(\bar{L}_\Gamma)\}$ and $\max\{0, \bar{B}_\Gamma/\text{se}(\bar{B}_\Gamma)\}$. Recalling that we have motivated our test with a greater than alternative in mind, this truncation is unlikely to matter in practice. A sensitivity analysis based upon $\bar{L}_\Gamma/\text{se}(\bar{L}_\Gamma)$ would only fail to be finite-sample exact should the critical value generated by the employed reference distribution fall below 0. In the limit, this would only occur if the investigator desired inference at a level $\alpha \geq 0.5$, a rare choice for most any application.

At $\Gamma=1$, our method is asymptotically equivalent to that of @Baiocchi:2010. The test statistic proposed in that work for inference at $\Gamma=1$ is precisely $\bar{L}_1/\text{se}(\bar{L}_1)$, and the methods differ solely in the reference distribution employed. @Baiocchi:2010 use the Normal distribution, while our approach uses a reference distribution generated by biased permutations of the observed data under the assumption of proportional doses in order to maintain exactness under that model for effects. Under suitable regularity conditions our reference distribution $G_\Gamma(k)$ converges in probability to $\Phi(k)$, the CDF of a standard Normal, pointwise at all points $k$. Their method can be viewed as a large-sample approximation of our method at $\Gamma=1$, and replacing our reference distribution $G_\Gamma(k)$ at $\Gamma > 1$ with the standard Normal $\Phi(k)$ would provide the natural extension of their large-sample approach for inference on the effect ratio to a sensitivity analysis.

Binary responses: Equivalence with McNemar's test and testing nonzero nulls
---------------------------------------------------------------------------

After introducing the effect ratio and developing large-sample inference for it at $\Gamma = 1$, @Baiocchi:2010 suggest conducting a sensitivity analysis based on McNemar's test while restricting attention to the narrower null hypothesis $H_{prop}^{(0)}$. One may be concerned that the sensitivity analysis using McNemar's test would be misleading if instead $H_{weak}^{(0)}$ held, such that the effect ratio equalled zero but Fisher's sharp null did not hold. As we now demonstrate, such concerns are not warranted. In fact, McNemar's test is equivalent to the test $\varphi^{(0)}(\alpha,\Gamma)$ presented in the previous section, and hence also provides a sensitivity analysis that is both finite-sample exact for $H_{prop}^{(0)}$ and asymptotically correct for $H_{weak}^{(0)}$.

\[thm.equiv\] Suppose that outcomes are binary and one employs a sensitivity analysis based on McNemar's test statistic, $$\begin{aligned}
T_{M}(\mathbf{Z}, \mathbf{Y}) &= \sum_{i=1}^nZ_{ij}Y_{ij},\end{aligned}$$ using its worst-case distribution under the assumption of $H^{(0)}_{prop}$ as described in @Rosenbaum:1987 and @Baiocchi:2010. Denote the resulting candidate level$-\alpha$ sensitivity analysis by $\varphi_M(\alpha, \Gamma)$. Then, for any observed vector ${{\mathbf{Z}}}\in \Omega$ and any vector of binary responses ${{\mathbf{Y}}}$, $$\begin{aligned}
\varphi_M(\alpha, \Gamma) = \varphi^{(0)}(\alpha,\Gamma).\end{aligned}$$ That is, the two sensitivity analyses are equivalent, furnishing identical $p$-values for all $\Gamma$.

If (\[eqn:sensi\]) holds at $\Gamma$, outcomes are binary, and $H^{(0)}_{weak}$ holds, $$\begin{aligned}
\underset{n\rightarrow \infty}{\lim} E(\varphi_M(\alpha,\Gamma)\mid {{\mathcal{F}}}, {{\mathcal{Z}}})    & \leq \alpha\end{aligned}$$

The proof of Proposition \[thm.equiv\] is presented in the appendix, and requires showing that over elements of $\Omega$, $\bar{L}_\Gamma/\text{se}(\bar{L}_\Gamma)$ is a strictly increasing function of McNemar's statistic $T_M({{\mathbf{Z}}}, {{\mathbf{Y}}}) = \sum_{i=1}^nZ_{ij}Y_{ij}$, the number of encouraged individuals for whom an event occurred. The statistics then have perfect rank correlation over ${{\mathbf{z}}}\in\Omega$, implying identical $p$-values for any data set. Because $\varphi^{(0)}(\alpha,\Gamma)$ is valid under the weak null, the sensitivity analyses for binary outcomes presented in @Baiocchi:2010, motivated under Fisher's sharp null, is also an asymptotically correct sensitivity analysis for the effect ratio equaling zero in the presence of effect heterogeneity. Indeed, *any* sensitivity analysis conducted using McNemar's test statistic is not only exact for Fisher's sharp null, but is also asymptotically correct for the null of no average effect in the study population. Sensitivity analyses using McNemar's tests are ubiquitous in paired observational studies with binary outcomes, and Proposition \[thm.equiv\] provides a useful fortification for those analyses should a critic be concerned about effect heterogeneity.

The equivalence with McNemar's test pertains to the particular test that the effect ratio equals zero. Observe that the procedure $\varphi^{(\lambda_0)}(\alpha,\Gamma)$ remains asymptotically valid for testing $H_{weak}^{(\lambda_0)}$ for $\lambda_0\neq 0$ even with binary outcomes. Owing to this, the test $\varphi^{(\lambda_0)}(\alpha,\Gamma)$ can be inverted to provide asymptotically valid sensitivity intervals for the effect ratio with binary outcomes.

Application to the study of surgical outcomes for EGS patients
--------------------------------------------------------------

Using these developments, we now perform a sensitivity analysis for our study. As previously discussed, at $\Gamma=1$ our IV analysis assumes that patients are assigned to surgeons in an as-if random fashion after accounting for observed covariates. Since our data do not truly stem from a randomized encouragement design, we cannot rule out the presence of hidden bias from IV–outcome confounders. We assess whether our results are sensitivity to this form of hidden bias. In Table \[tab.sens\], we summarize the sensitivity analysis using the sensitivity value – the value of $\Gamma$ at which the estimate is no longer statistically significant at the 0.05 level [@Zhao:2019].

                    Septic Patients   Non-septic Patients
  ---------------- ----------------- ---------------------
  Complication           1.56                1.59
  Length of Stay         1.64                2.36
  Mortality              1.00                1.00

  : Sensitivity analysis for the estimated effect of surgery on EGS patient outcomes. Cell entries are sensitivity values when testing at $\alpha=0.05$, the largest values of $\Gamma$ for which the null of zero effect ratio is rejected [@Zhao:2019].<span data-label="tab.sens"></span>

Table \[tab.sens\] illustrates that when effect modification is present, hypothesis tests for different subgroups may vary not only in their degree of statistical significance assuming $\Gamma=1$, but also in their sensitivity to bias should the test at $\Gamma=1$ reject. For related results, see also @hsu2013effect and @lee2018discovering. We find that is true with respect to sepsis. While the point estimates are larger for the septic patients as displayed in Table \[tab.out\], the $\Gamma$ changepoint values are larger in the non-septic group for both complication and hospital length of stay. Specifically, in the IV analysis, for septic patients the changepoint value is 1.64, while it is 2.36 for non-septic patients. To put this in perspective, for septic patients an unobserved covariate would have to increase the odds of treatment by a surgeon with a high TTO by a factor of 1.64 within matched pairs in order to overturn our finding of a positive effect ratio at $\alpha = 0.05$. In contrast, for non-septic patients an unobserved covariate would have to increase the odds of treatment by a high TTO surgeon by a factor of 2.36 in order to overturn the finding of a positive effect ratio among this subgroup.

Sepsis is but one of many patient-level health characteristics which might reasonably influence whether or not surgery will be more effective than non-surgical alternatives. For example, the age of a patient and the number of comorbidities may well influence which mode of treatment should be preferred. We next explore how accounting for additional heterogeneity in the matched pair outcomes that is predictable on the basis of effect modifiers may further decrease reported sensitivity to hidden bias. After doing so, we present theoretical results to help explain the pattern in Table \[tab.sens\] of smaller treatment effects being considerably less sensitive to hidden bias.

Improved standard errors for sensitivity analysis by exploiting effect modification {#sec:var}
===================================================================================

Conservativeness of finite population sensitivity analysis under effect heterogeneity
-------------------------------------------------------------------------------------

Proposition \[prop:stu\] tells us that rejecting the null hypothesis $H_{weak}^{(\lambda_0)}$ through the test $\varphi^{(\lambda_0)}(\alpha,\Gamma)$ provides an asymptotically valid sensitivity analysis, in the sense that if (\[eqn:sensi\]) holds at $\Gamma$ then the test will commit a Type I error with probability at most $\alpha$ in the limit. In practice, however, it may well be the case that $\varphi^{(\lambda_0)}(\alpha,\Gamma) << \alpha$ under $H_{weak}^{(\lambda_0)}$, resulting in a conservative testing procedure.

To see why, imagine that we had access to $E(\bar{L}_\Gamma\mid {{\mathcal{F}}}, {{\mathcal{Z}}})$ and ${\text{sd}}(\bar{L}_\Gamma\mid {{\mathcal{F}}}, {{\mathcal{Z}}})$, the *true* expectation and standard deviation of the conditional distribution $\bar{L}_\Gamma \mid {{\mathcal{F}}}, {{\mathcal{Z}}}$. In this case, rejecting the null hypothesis when the test statistic $\{\bar{L}_\Gamma - E(\bar{L}_\Gamma\mid {{\mathcal{F}}}, {{\mathcal{Z}}})\}/{\text{sd}}(\bar{L}_\Gamma\mid {{\mathcal{F}}}, {{\mathcal{Z}}})$ exceeds $\Phi^{-1}(1-\alpha)$ would, under mild conditions ensuring that a central limit theorem holds for $\bar{L}_\Gamma$, result in a test statistic with asymptotic level exactly equal to $\alpha$. Unfortunately, this test statistic cannot be deployed in practice. For $\Gamma > 1$ the expectation $E(\bar{L}_\Gamma\mid {{\mathcal{F}}}, {{\mathcal{Z}}})$ cannot be computed due to its dependence on both the vector of unmeasured confounders $\mathbf{u}$ and the missing potential outcomes, which are not imputed under the weak null $H_{weak}^{(\lambda_0)}$. Further, for any value of $\Gamma$, ${\text{sd}}(\bar{L}_\Gamma\mid {{\mathcal{F}}}, {{\mathcal{Z}}})$ is also unknown when effects are heterogeneous. By using the test statistic $\bar{L}_\Gamma/\text{se}(\bar{L}_\Gamma)$, our procedure overcomes these difficulties by means of the bounds $E(\bar{L}_\Gamma\mid {{\mathcal{F}}}, {{\mathcal{Z}}}) \leq 0$ and (2) ${\text{var}}(\bar{L}_\Gamma\mid {{\mathcal{F}}}, {{\mathcal{Z}}}) \leq E\{{\text{se}}(\bar{L}_\Gamma)^2\mid {{\mathcal{F}}}, {{\mathcal{Z}}}\}$. See Lemmas 2 and 3 of @Fogarty:2019 for proofs of these inequalities.

That $E(\bar{L}_\Gamma\mid {{\mathcal{F}}}, {{\mathcal{Z}}})$ is unknown under $H_{weak}^{(\lambda_0)}$ is a feature unique to a sensitivity analysis when $\Gamma>1$. In a randomized encouragement design ($\Gamma=1)$, it is known that $E(\bar{L}_\Gamma\mid {{\mathcal{F}}}, {{\mathcal{Z}}}) = 0$ under the null even if effects are heterogeneous. For continuous outcomes, the bound ${{E}}(\bar{L}_\Gamma\mid {{\mathcal{F}}}, {{\mathcal{Z}}}) \leq 0$ is sharp for $\Gamma > 1$, in that the upper bound is actually attained at the worst-case distribution for $\bar{L}_\Gamma$ under the proportional dose model. For this reason, using an expectation smaller than zero when performing a sensitivity analysis is not justified without additional assumptions about the treatment effects and the pattern of hidden bias. While we cannot generally hope to improve upon the bound $E(\bar{L}_\Gamma\mid {{\mathcal{F}}}, {{\mathcal{Z}}})\leq 0$ without risking an anti-conservative procedure, we now illustrate that improvements on $\text{se}(\bar{L}_\Gamma)$ *can* be attained while preserving the asymptotic level of the resulting sensitivity analysis.

A general construction of valid standard errors for sensitivity analysis
------------------------------------------------------------------------

Extending Lemma 3 of @Fogarty:2019 to encouragement designs, we have $$\begin{aligned}
E\{\text{se}(\bar{L}_\Gamma)^2\mid {{\mathcal{F}}}, {{\mathcal{Z}}}\} &= {\text{var}}(\bar{L}_\Gamma \mid {{\mathcal{F}}}, {{\mathcal{Z}}}) + \frac{1}{n(n-1)}\sum_{i=1}^n(\mu_{\Gamma i} - \bar{\mu}_\Gamma)^2,\end{aligned}$$ where $\mu_{\Gamma i} = E(L_{\Gamma i}\mid {{\mathcal{F}}}, {{\mathcal{Z}}})$ is the true, but unknowable, expectation of $L_{\Gamma i}$ and $\bar{\mu}_\Gamma = n^{-1}\sum_{i=1}^n\mu_{\Gamma i}$. The bias in $\text{se}(\bar{L}_\Gamma)$ thus depends upon the degree of heterogeneity in the expectations $\mu_{\Gamma i}$. We can interpret the magnitude of the bias as the mean squared error estimate from a regression of $\mu_{\Gamma i}$ on an intercept column $\mathbf{1}_{n}$, a vector containing $n$ ones. Imagine now that, based on covariate information at our disposal, we were able to account for some of the variation in $\mu_{\Gamma i}$ through covariance adjustment. This would reduce the mean squared error, and hence the degree of bias. We now present a general construction for standard error estimators that make use of this idea, resulting in less conservative estimate of $\text{se}(\bar{L}_\Gamma)$ while nonetheless providing an asymptotically valid sensitivity analysis for any value of $\Gamma$.

Let $\mathbf{Q}$ be an $n \times p$ matrix with $p < n$ that is constant across all ${{\mathbf{z}}}\in \Omega$, and let $\mathbf{H} = {{\mathbf{Q}}}({{\mathbf{Q}}}^T {{\mathbf{Q}}})^{-1} {{\mathbf{Q}}}^T$ be the orthogonal projection of $\mathbb{R}^n$ onto the column space of ${{\mathbf{Q}}}$. Let $h_{ik}$ be the $\{i, k\}$ element of ${{\mathbf{H}}}$ and let $\tilde{L}_{\Gamma i} = L_{ \Gamma i}/\sqrt{1-h_{ii}}$. Define the new variance estimator $\text{se}(\bar{L}_\Gamma; {{\mathbf{Q}}})^2$ as $$\begin{aligned}
    \text{se}(\bar{L}_\Gamma; {{\mathbf{Q}}})^2 = \frac{1}{n^2} \mathbf{\tilde{L}_{\Gamma}}^T (\mathbf{I} -\mathbf{H}) \mathbf{\tilde{L}_{\Gamma}},
    \label{eqn:new_var}\end{aligned}$$ where ${{\mathbf{I}}}$ is the $n \times n$ identity matrix and $\mathbf{\tilde{L}_\Gamma} = (\tilde{L}_{\Gamma 1}, \ldots, \tilde{L}_{\Gamma n})^T$.

For any value of $\Gamma$, $$\begin{aligned}
    E\{\text{se}(\bar{L}_\Gamma; {{\mathbf{Q}}})^2 \mid \mathcal{F}, \mathcal{Z}\} - {\text{var}}(\bar{L}_\Gamma \mid \mathcal{F}, \mathcal{Z}) = \frac{1}{n^2} \bm{\tilde{{\mu}}_\Gamma}^T ({{\mathbf{I}}}- {{\mathbf{H}}}) \bm{\tilde{{\mu}}_\Gamma} \geq 0.
    \end{aligned}$$ where $\bm{\tilde{{\mu}}_\Gamma} = (\tilde{\mu}_{\Gamma 1}, \ldots, \tilde{\mu}_{\Gamma n})^T$, and $\tilde{\mu}_{\Gamma i} = \mu_{\Gamma i}/\sqrt{1-h_{ii}}$. \[prop:var1\]

The proof of Proposition \[prop:var1\] follows that of Proposition 1 in @Fogarty:2018. The result states that regardless of the true value of $\Gamma$ for which (\[eqn:sensi\]) holds and regardless of the form for the matrix ${{\mathbf{Q}}}$, $\text{se}(\bar{L}_\Gamma, {{\mathbf{Q}}})^2$ will be conservative in expectation for ${\text{var}}(\bar{L}_\Gamma \mid \mathcal{F}, \mathcal{Z})$ so long as ${{\mathbf{Q}}}$ does not vary across $\Omega$.

A regression-based standard error
---------------------------------

Setting ${{\mathbf{Q}}}= \mathbf{1}_n$ recovers the usual standard error estimator, i.e, $\text{se}(\bar{L}_\Gamma, \mathbf{1}_n) = \text{se}(\bar{L}_\Gamma)$ as defined in §\[sec:senshet\]. Guided by the form of the bias, we see that ${{\mathbf{Q}}}$ should be chosen to predict the individual-level expectations $\tilde{\mu}_{\Gamma i} \approx \mu_{\Gamma i}$. One choice would be to let ${{\mathbf{Q}}}= (\mathbf{1}, \bar{\mathbf{X}})$ be a $n \times (k+1)$ matrix where the $q$th column of $\bar{\mathbf{X}}$ is $(\bar{x}_{1q}, \ldots, \bar{x}_{nq})^T$, and where $\bar{x}_{iq} = (x_{i1q} + x_{i2q})/2$, the average of the $q$th covariate's values in the $i$th pair. The $i$th row of $\bar{\mathbf{X}}$ then contains the average of the $k$ covariates within pair $i$, reflecting a hope that the heterogeneous expectations $\mu_{\Gamma i}$ may be linear in the within-pair covariate averages. Under mild regularity conditions, it can be shown that the modified variance estimator $\text{se}(\bar{L}_\Gamma; {{\mathbf{Q}}}_{reg})$ with ${{\mathbf{Q}}}_{reg}= (\mathbf{1}_n, \bar{\mathbf{X}})$ is asymptotically never worse than ${\text{se}}(\bar{L}_\Gamma)$, *regardless* of whether or not $\bm{\mu_\Gamma}$ is truly linear in the within-pair covariate averages $\bar{\mathbf{X}}$.

Take ${{\mathbf{Q}}}_{reg} = (\mathbf{1}_n, \bar{\mathbf{X}})$. If $\bm{\mu_\Gamma}$ is not a constant vector, then under regularity conditions, $$\begin{aligned}
    \frac{\text{se}(\bar{L}_\Gamma; {{\mathbf{Q}}}_{reg})^2 - {\text{var}}(\bar{L}_{\Gamma} \mid \mathcal{F}, \mathcal{Z})}{{\text{se}}(\bar{L}_\Gamma; \mathbf{1}_n)^2 - {\text{var}}(\bar{L}_{\Gamma} \mid \mathcal{F}, \mathcal{Z})}&\xrightarrow{p} 1-R^2,
    \end{aligned}$$ where $R^2$ is the coefficient of determination from a regression of $\bm{\mu_\Gamma}$ on $(\bm{1_n}, \bm{\bar{X}})$. If $\bm{\mu_\Gamma}$ is a constant vector, then $\text{se}(\bar{L}_\Gamma; \bm{1_n})^2/{\text{var}}(\bar{L}_\Gamma\mid {{\mathcal{F}}}, {{\mathcal{Z}}}) \overset{p}{\rightarrow} 1$ and $\text{se}(\bar{L}_\Gamma; {{\mathbf{Q}}}_{reg})^2/{\text{var}}(\bar{L}_\Gamma\mid {{\mathcal{F}}}, {{\mathcal{Z}}}) \overset{p}{\rightarrow} 1$. \[prop:var2\]

Sufficient conditions for Proposition \[prop:var2\] are presented along with its proof in the supplementary materials. Under the same conditions, the conclusion of Proposition \[prop:var2\] would hold replacing $\bm{\tilde{L}_\Gamma}$ with $\bm{L_\Gamma}$, and with $1/n^2$ replaced by $1/\{n(n-k-1)\}$ in (\[eqn:new\_var\]). In that case, the resulting standard error estimator would simply be the $RMSE$ from a regression of $\bm{L_\Gamma}$ on $(\bm{1_n}, \bm{\bar{X}})$, divided by $\sqrt{n}$.

A nonparametric approach: pairing the pairs through nonbipartite matching {#sec:pop}
-------------------------------------------------------------------------

While the regression-based standard error does not re quire a properly specified linear model for its validity, the gains are dependent on the extent to which $\bm{\mu_\Gamma}$ is linear in ${{\mathbf{Q}}}$. While averages of polynomial terms can be included within $\bar{{{\mathbf{X}}}}$, alternative nonparametric approaches may be preferred. Here we illustrate that a variant of the approach first considered in @Abadie:2008 can also be employed in a finite population sensitivity analysis.

Suppose that $n$ is even for simplicity, and consider "pairing the pairs" through the use of optimal nonbipartite matching based on Mahalanobis distances between the average values of the within-pair covariates $\bar{\mathbf{x}}_i$. This results in $n/2$ "pairs of pairs," where matched pairs have been matched to other matched pairs with similar average values for $\bar{{{\mathbf{x}}}}_i$. For each pair $i$, let $\mathcal{J}(i)$ be the index of the pair that was matched with the $i$th pair such that $\mathcal{J}(\mathcal{J}(i)) = i$. Consider the variance estimator $$\begin{aligned}
\label{eq:pop}
\text{se}(\bar{L}_\Gamma; {{\mathbf{Q}}}_{PoP})^2 &= \frac{1}{2n^2}\sum_{i=1}^n(L_{\Gamma i} - L_{\Gamma \mathcal{J}(i)})^2,\end{aligned}$$which simply squares the differences in the terms $L_{\Gamma i}$ within each pair of pairs

$$\begin{aligned}
E\{\text{se}(\bar{L}_\Gamma; {{\mathbf{Q}}}_{PoP})^2\mid {{\mathcal{F}}}, {{\mathcal{Z}}}\}- {\text{var}}(\bar{L}_\Gamma\mid {{\mathcal{F}}}, {{\mathcal{Z}}}) &= \frac{1}{2n^2}\sum_{i=1}^n(\mu_{\Gamma i} - \mu_{\Gamma \mathcal{J}(i)})^2. \end{aligned}$$

While a proof unique to this estimator follows by expanding the square, the result can also be obtained through Proposition \[prop:var1\]. Let ${{\mathbf{Q}}}_{PoP}$ be the $n \times (n/2)$ matrix with the $q$th column containing membership indicators for the $q$th of $n/2$ pairs of pairs. This design matrix yields $h_{ii} = 1/2$, such that $\tilde{L}_{\Gamma i} = \sqrt{2}L_{\Gamma i}$. Observe that as the average values $\bar{{{\mathbf{x}}}}_i$ are fixed over ${{\mathbf{z}}}\in \Omega$, the pairing of pairs encoded within ${{\mathbf{Q}}}_{PoP}$ is also constant over the randomization distribution. The residual vector $({{\mathbf{I}}}- {{\mathbf{H}}}_{PoP})\bm{\tilde{L}_{\Gamma}}$ has $\sqrt{1/2}(L_{\Gamma i} - L_{\Gamma \mathcal{J}(i)})$, in its $i$th entry, and the analogous form holds for the residual vector $({{\mathbf{I}}}- {{\mathbf{H}}}_{PoP})\bm{\tilde{\mu}_{\Gamma}}$. Summing the squares of these residuals and dividing by $n^2$ shows that Proposition \[prop:var1\] recovers both the estimator and the correct bias formula.

An odd number of pairs can readily be accommodated through a host of corrections. One approach is inspired by the output of the `nonbimatch` function within the `nbpMatching` package in `R` [@Lu:2011], a common package for nonbipartite matching. When given an odd number of elements, `nonbimatch` introduces a "ghost" element has zero distance between all other elements, and then proceeds with the optimization problem. The pair matched to the ghost is, in reality, not matched to any other pair. We can then match the unmatched pair to the most similar pair of pairs, creating $(n-1)/2-1$ pairs of pairs and one triple of pairs. One could then proceed with ${{\mathbf{Q}}}_{PoP}$ as the $n\times (n-1)/2$ matrix, where the first $(n-1)/2-1$ columns contain indicators for pairs of pairs membership, and the final column contains binary indicators for membership in the lone triple. By Proposition \[prop:var1\], the estimator $\text{se}(\bar{L}_\Gamma ; {{\mathbf{Q}}}_{PoP})^2$ is a conservative estimate for ${\text{var}}(\bar{L}_\Gamma \mid {{\mathcal{F}}}, {{\mathcal{Z}}})$.

Comparative improvement and fundamental limitations of bias reduction for finite-population standard errors {#sec:limit}
-----------------------------------------------------------------------------------------------------------

@Abadie:2008 consider a superpopulation formulation wherein the covariates ${{\mathbf{x}}}$ are viewed as fixed, but the potential outcomes are random. Under this generative framework, they show that under suitable regularity conditions, $n\times \text{se}(\bar{L}_1; {{\mathbf{Q}}}_{PoP})^2$ is consistent for $n\times{\text{var}}(\bar{L}_1\mid {{\mathbf{x}}}, {{\mathcal{Z}}})$ in randomized experiments with perfect compliance. Note however that even at $\Gamma=1$, the target of estimation in our framework is instead ${\text{var}}(\bar{L}_1\mid {{\mathcal{F}}}, {{\mathcal{Z}}})$, and that ${{\mathcal{F}}}$ includes the potential outcomes. That inference conditions upon the potential outcomes results in what @Ding:2019 refer to as an *idiosyncratic* component of heterogeneity which cannot be explained by observed covariates. As a result, $n\times \text{se}(\bar{L}_\Gamma; {{\mathbf{Q}}}_{PoP})^2$ will not generally be consistent for $n\times{\text{var}}(\bar{L}_\Gamma \mid {{\mathcal{F}}}, {{\mathcal{Z}}})$. Instead, it will have a positive asymptotic bias determined by the limiting value of $(2n)^{-1}\sum_{i=1}^n(\mu_{\Gamma i} - \mu_{\Gamma \mathcal{J}(i)})^2$ given ${{\mathcal{F}}}$ and ${{\mathcal{Z}}}$, which generally cannot be driven to zero as it depends on idiosyncratic features of the observed study population. Nonetheless, by pairing the pairs one may reasonably expect that the explainable portion of the variation in $\mu_{\Gamma i}$ has been captured in the limit so long as the underlying function is sufficiently well-behaved, whereas the method based on linear regression may fail to entirely do so under misspecification.

To develop intuition, imagine for a moment that the terms $\mu_{\Gamma i} = E(\bar{L}_{\Gamma i}\mid {{\mathcal{F}}}, {{\mathcal{Z}}})$ are themselves drawn from a distribution conditional upon ${{\mathbf{x}}}_i$ with expectation $\eta({{\mathbf{x}}}_i)$ and variance $\sigma^2({{\mathbf{x}}}_i)$. Let $\varepsilon({{\mathbf{x}}}_i) = \mu_{\Gamma i} - \eta({{\mathbf{x}}}_i)$ such that $E\{\varepsilon({{\mathbf{x}}}_i) \mid {{\mathbf{x}}}_i\} = 0$. Under suitable regularity conditions, the limiting bias in $n\times \text{se}(\bar{L}_\Gamma; {{\mathbf{Q}}}_{reg})^2$ as an estimator of $ n\times {\text{var}}(\bar{L}_\Gamma\mid {{\mathcal{F}}}, {{\mathcal{Z}}})$ is $$\begin{aligned}
n\{\text{se}(\bar{L}_\Gamma;{{\mathbf{Q}}}_{reg})^2 - {\text{var}}(\bar{L}_\Gamma\mid {{\mathcal{F}}}, {{\mathcal{Z}}})\} &\overset{p}{\rightarrow} \underset{n\rightarrow \infty}{\lim}n^{-1} \{\bm{\eta}({{\mathbf{x}}}) + \bm{\varepsilon}({{\mathbf{x}}})\}^T({{\mathbf{I}}}- {{\mathbf{H}}}_{reg})\{\bm{\eta}({{\mathbf{x}}}) + \bm{\varepsilon}({{\mathbf{x}}})\}
    \\&= n^{-1} \underset{n\rightarrow \infty}{\lim}\bm{\eta}({{\mathbf{x}}})^T({{\mathbf{I}}}- {{\mathbf{H}}}_{reg})\bm{\eta}({{\mathbf{x}}}) + n^{-1}\underset{n\rightarrow \infty}{\lim}\sum_{i=1}^n\varepsilon^2({{\mathbf{x}}}_i).\end{aligned}$$ The term $n^{-1}\underset{n\rightarrow \infty}{\lim}\sum_{i=1}^n\varepsilon^2({{\mathbf{x}}}_i)$ in the expression is the idiosyncratic effect variation described in @Ding:2019, and appears because inference is being conducted conditional upon ${{\mathcal{F}}}$ with the potential outcomes fixed, rather than conditional upon only ${{\mathbf{x}}}_i$ while viewing the potential outcomes as random. The bias would only depend upon this idiosyncratic term were $\eta({{\mathbf{x}}}_i)$ linear in ${{\mathbf{x}}}_i$, but an additional positive factor persists under misspecification. For the pairs of pairs estimator, under suitable moment conditions given in @Abadie:2008, we instead have $$\begin{aligned}
n\{\text{se}(\bar{L}_\Gamma;{{\mathbf{Q}}}_{PoP})^2 - {\text{var}}(\bar{L}_\Gamma\mid {{\mathcal{F}}}, {{\mathcal{Z}}})\} &\overset{p}{\rightarrow} \underset{n\rightarrow \infty}{\lim}(2n^{-1}) \sum_{i=1}^n\{\eta({{\mathbf{x}}}_i) - \eta({{\mathbf{x}}}_{\mathcal{J}(i)}) + \varepsilon({{\mathbf{x}}}_i) - \varepsilon({{\mathbf{x}}}_{\mathcal{J}(i)})\}^2\\
    &= \underset{n\rightarrow \infty}{\lim}(2n^{-1}) \sum_{i=1}^n\{\eta({{\mathbf{x}}}_i) - \eta({{\mathbf{x}}}_{\mathcal{J}(i)})\}^2 + n^{-1}\underset{n\rightarrow \infty}{\lim}\sum_{i=1}^n\varepsilon^2({{\mathbf{x}}}_i),\end{aligned}$$ Under additional Lipschitz and boundedness conditions on $\eta({{\mathbf{x}}}_i)$ along with an assumption that $n^{-1}\sum_{i=1}^n||{{\mathbf{x}}}_i - {{\mathbf{x}}}_{\mathcal{J}(i)}||^2$ tends to zero, arguments akin to those in @Abadie:2008 imply that $\underset{n\rightarrow \infty}{\lim}(2n)^{-1}\sum_{i=1}^n\{\eta({{\mathbf{x}}}_i) - \eta({{\mathbf{x}}}_{\mathcal{J}(i)})\}^2 = 0$. Under these conditions, it would then follow that $\underset{n\rightarrow \infty}{\text{plim}}\; \text{se}(\bar{L}_\Gamma;{{\mathbf{Q}}}_{PoP})/\text{se}(\bar{L}_\Gamma;{{\mathbf{Q}}}_{reg}) \leq 1$, such that $\text{se}(\bar{L}_\Gamma ; {{\mathbf{Q}}}_{PoP})$ would provide a less conservative standard error in the limit than $\text{se}(\bar{L}_\Gamma;{{\mathbf{Q}}}_{reg})$.

Improved sensitivity analysis and an omnibus test
=================================================

An improved sensitivity analysis for sample average effects {#sec:new}
-----------------------------------------------------------

The standard errors in §\[sec:var\] can be used to improve the power of the sensitivity analysis for IV while maintaining the asymptotic level of the test. Consider conducting a sensitivity analysis at $\Gamma$ using $\bar{L}_\Gamma/\text{se}(\bar{L}_\Gamma; {{\mathbf{Q}}})$ as a test statistic for some matrix ${{\mathbf{Q}}}$ instead of $\bar{L}_\Gamma/\text{se}(\bar{L}_\Gamma)$. Let $G_{\Gamma}(\cdot; {{\mathbf{Q}}})$ be the distribution of $\bar{B}_\Gamma/\text{se}(\bar{B}_\Gamma; {{\mathbf{Q}}})$ given $|\bm{\zeta}^{(\lambda_0)}|$, where $\bar{B}_\Gamma$ is defined as in §\[sec:senshet\]. The modified test is $$\begin{aligned}
\varphi_{{\mathbf{Q}}}^{(\lambda_0)}(\alpha,\Gamma ) &= {\mathbbm{1}}\{\bar{L}_\Gamma/\text{se}(\bar{L}_\Gamma; {{\mathbf{Q}}}) \geq G_\Gamma^{-1}(1-\alpha; {{\mathbf{Q}}})\}.\end{aligned}$$

Suppose (\[eqn:sensi\]) holds at $\Gamma$ and that $H_{weak}^{(\lambda_0)}$ is true. Then, under regularity conditions, $$\begin{aligned}
\underset{n\rightarrow\infty}{\lim}E\{\varphi_{{\mathbf{Q}}}^{(\lambda_0)}(\alpha,\Gamma )\} \leq \alpha,\end{aligned}$$ such that the procedure yields a valid level$-\alpha$ sensitivity analysis for the effect ratio with heterogeneous effects. Furthermore, for ${{\mathbf{Q}}}= {{\mathbf{Q}}}_{reg}$ or ${{\mathbf{Q}}}= {{\mathbf{Q}}}_{PoP}$ and regardless of whether or not the null holds, then under regularity conditions $$\begin{aligned}
\underset{n\rightarrow\infty}{\lim}E\{\varphi_{{\mathbf{Q}}}^{(\lambda_0)}(\alpha,\Gamma )\} \geq  \underset{n\rightarrow\infty}{\lim}E\{\varphi^{(\lambda_0)}(\alpha,\Gamma )\},\end{aligned}$$ such that the new procedure is both less conservative under the null and more powerful under the alternative than the procedure employing the conventional standard error estimator.

We leave a sketch of the proof along with a discussion of sufficient conditions for the appendix, but intuition can be formed from the discussion in the previous section. The magnitude of the improvement will depend upon the extent to which $\text{se}(\bar{L}_\Gamma;{{\mathbf{Q}}})$ improves upon the conventional standard error estimator, $\text{se}(\bar{L}_\Gamma)$. In understanding the differences between $\varphi_{{\mathbf{Q}}}^{(\lambda_0)}$ and $\varphi^{(\lambda_0)}$, it is useful to consider how one would use Monte-Carlo simulation to approximate the test. Algorithm \[alg:stu\] contains the required steps for any matrix ${{\mathbf{Q}}}$, and ${{\mathbf{Q}}}=\bm{1_n}$ returns the test in (\[eq:procedure\]) with the conventional standard error.

\[h\]

1.  Form a matrix ${{\mathbf{Q}}}$ and its hat matrix ${{\mathbf{H}}}= {{\mathbf{Q}}}({{\mathbf{Q}}}^T{{\mathbf{Q}}})^{-1}{{\mathbf{Q}}}^T$

2.  In the $m$th of $M$ iterations:

    1.  Generate $V_{\Gamma i}\overset{iid}{\sim} 2\times Bernoulli\left(\frac{\Gamma}{1+\Gamma}\right)-1$ for each $i$.

    2.  Compute $B_{\Gamma i} = V_{\Gamma i}|\zeta^{(\lambda_0)}_i| - \left(\frac{\Gamma-1}{1+\Gamma}\right)|\zeta^{(\lambda_0)}_i|$ for each $i$.

    3.  Compute $\bar{B}_\Gamma = n^{-1}\sum_{i=1}^nB_{\Gamma i}$.

    4.  Form $\tilde{B}_{\Gamma i} = B_{\Gamma i}/\sqrt{1-h_{ii}}$ for each $i$

    5.  Compute $\text{se}(\bar{B}_\Gamma;{{\mathbf{Q}}})$ as $$\begin{aligned}
        \text{se}(\bar{B}_\Gamma;{{\mathbf{Q}}}) = \sqrt{\frac{1}{n^2}\mathbf{\tilde{B}_\Gamma}^T(\mathbf{I}-{{\mathbf{H}}})\mathbf{\tilde{B}_\Gamma}}.
         \end{aligned}$$

    6.  Compute $A^{(m)}_\Gamma = \bar{B}_\Gamma/\text{se}(\bar{B}_\Gamma; {{\mathbf{Q}}})$; store this value across iterations.

3.  Approximate the bound on the greater-than $p$-value by $$\begin{aligned}
     \hat{p}_{val} &= \frac{1 + \sum_{m=1}^M{\mathbbm{1}}\{A^{(m)}_\Gamma\geq \bar{L}^{obs}_\Gamma/\text{se}(\bar{L}^{obs}_\Gamma ; {{\mathbf{Q}}})\}}{1+M}\end{aligned}$$

Understanding performance in a simulation study
-----------------------------------------------

We now conduct a simulation study to further illustrate the potential improvements that standard errors exploiting effect modification may provide. In each simulated data set there are $n$ pairs, each matched exactly on a $k\geq 5$ dimensional vector of covariates ${{\mathbf{x}}}_i$. In the $m$th of $M$ iterations, the pair-specific covariate vector is drawn such that each component is $iid$ Uniform on the interval \[0,1\]. The treatments actually received $d_{ij}(z)$ ($z=0,1$) are binary. We assume that there are no defiers, that the exclusion restriction holds, and that individuals $ij$ are assigned status as compliers ($d_{ij}(1)=1, d_{ij}(0) = 0$), never-takers ($d_{ij}(1) = d_{ij}(0)=0)$, and always-takers $(d_{ij}(1)=d_{ij}(0) = 1)$ independently with probability $p_C = 0.75$, $p_N = 0.125$, and $p_C = 0.125$ respectively. We then use the functional form for the potential outcomes suggested in @Fogarty:2018 [§7.1], suitably modified to include potential noncompliance. For $z=0,1$,

$$\begin{aligned}
y_{ij}(z) &=\begin{cases} a\left(10\sin(\pi x_{i1}x_{i2}) + 20(x_{i3}-1/2)^2 + 10\exp(x_{i4}) +5(x_{i5}-1/2)^3 + \varepsilon_{ij}\right) & d_{ij}(z) = 1\\
 10\sin(\pi x_{i1}x_{i2}) + 20(x_{i3}-1/2)^2 + 10\exp(x_{i4}) + 5(x_{i5}-1/2)^3 + \varepsilon_{ij} & d_{ij}(z) = 0\nonumber \end{cases}\\
\varepsilon_{ij}&\overset{iid}{\sim}\mathcal{N}(0,1).\nonumber\end{aligned}$$

We proceed with two different values of $a$ in the above model: $a=1$ and $a=2$. At $a=1$ the proportional dose model described in §\[sec:propdose\] holds at $\lambda=0$, and there is no effect modification. At $a=2$ effect modification is present and is nonlinear in the observed covariates.

The effect ratio in simulation $m$ is $\lambda_m$, and is determined once the potential outcomes and dosages received are simulated. We imagine that there is no hidden bias, such that (\[eqn:sensi\]) holds at $\Gamma=1$. We then proceed with inference for the null hypothesis $\lambda = \lambda_m$ at $\Gamma=1$ with a two-sided alternative, and we set $\alpha=0.1$. As $H_{weak}^{(\lambda_m)}$ is true and that the sensitivity model holds at $\Gamma=1$, our proposed methods for sensitivity analysis using $\varphi_{{\mathbf{Q}}}^{(\lambda_m)}(0.1,1)$ should commit a Type I error with probability at most 0.1. We compare three choices for the matrix ${{\mathbf{Q}}}$ used to form the variance estimator in (\[eqn:new\_var\]):

1.  *Intercept*. Including only the intercept column, recovering the usual variance estimator for a paired design.

2.  *Linear.* Including a constant column and the covariates ${{\mathbf{x}}}_{i}$.

3.  *Pairs of Pairs*. Form pairs of pairs with similar covariate values through nonbipartite matching as described in §\[sec:pop\], and estimate the variance by (\[eq:pop\]).

In each iteration we also construct 90% confidence intervals at $\Gamma=1$ for $\lambda_m$ through inverting the test $\varphi_{{\mathbf{Q}}}^{(\lambda_0)}$, and compare the widths of the intervals generated by the three choices for ${{\mathbf{Q}}}$. In addition to varying the value of $a$, we also vary the sample size between $n=100$ and $n=2500$, and the number of covariates between $k=5$ and $k=10$. Observe that the potential outcomes depend only on the first five covariates, such that the setting with $k=10$ includes five irrelevant covariates.

  ------------- ---------- -------- ------- ----------- ------- ----------- ------- -----------
                                                                                    
                                                                                    
                                     Size    CI Length   Size    CI Length   Size    CI Length
  Prop. Dose    $n=100$    $k=5$     0.101     0.636     0.101     0.636     0.102     0.640
  Prop. Dose    $n=100$    $k=10$    0.099     0.637     0.099     0.637     0.101     0.641
  Prop. Dose    $n=2500$   $k=5$     0.101     0.125     0.098     0.125     0.100     0.125
  Prop. Dose    $n=2500$   $k=10$    0.102     0.125     0.097     0.125     0.105     0.125
  Effect Mod.   $n=100$    $k=5$     0.002     2.648     0.027     1.848     0.024     1.927
  Effect Mod.   $n=100$    $k=10$    0.003     2.646     0.028     1.845     0.012     2.135
  Effect Mod.   $n=2500$   $k=5$     0.002     0.514     0.026     0.364     0.044     0.337
  Effect Mod.   $n=2500$   $k=10$    0.002     0.514     0.029     0.363     0.025     0.372
  ------------- ---------- -------- ------- ----------- ------- ----------- ------- -----------

  : \[tab:effmodsim\] Simulated performance of our testing procedure with different standard error estimators. The desired Type I error rate in all settings is $\alpha=0.1$.

Table \[tab:effmodsim\] contains the results from the simulation study. We first observe that under the proportional dose model, the estimated size for all three procedures was roughly 0.1 for each combination of covariate dimension and sample size. This should not come as surprise, as under the proportional dose model all three procedures are guaranteed to be finite-sample exact since they are randomization tests; discrepancies from 0.1 in the table are the result of simulation error. The confidence interval lengths are also comparable within each combination of $k$ and $n$ since all three standard errors times $\sqrt{n}$ are consistent for $\sqrt{n}\times {\text{sd}}(\bar{L}_\Gamma \mid {{\mathcal{F}}}, {{\mathcal{Z}}})$ under proportional doses.

When effect modification is present, we observe that in all simulations the true Type I error rates from the resulting methods fall well below the desired size. While the covariates ${{\mathbf{x}}}_i$ are effect modifiers, the simulation study contains idiosyncratic, irreducible effect heterogeneity in the form of the $\varepsilon_{ij}$ terms. As described in §\[sec:limit\], this induces conservativeness in the resulting inference. We see that by using covariate information the regression-based and pairs of pairs standard errors both outperform the conventional standard errors, which results in less conservative inference and narrower confidence intervals. Despite the fact that the effect modification is nonlinear in the observed covariates, the regression-based standard error outperforms the pairs of pairs procedure in every simulation setting except for ($n=2500, k=5$). This is due to the difficulties faced by nonparametric estimation in low sample, high covariate dimension regimes. Note that this does not corrupt the procedure based on the pairs of pairs standard error in terms of invalid inference, but it does limit its efficacy. This reflects the bias-variance tradeoff: in small samples and/or with a large number of covariates, it may be the case that a misspecified linear model produces better results than the nonparametric pairs of pairs approach due to improved stability of the estimated functional form.

An omnibus test for effect heterogeneity in instrumental variable studies {#sec:omni}
-------------------------------------------------------------------------

@Fogarty:2018 [§6] describes how the discrepancy between standard errors involving covariate information and the conventional standard error estimator for a paired design can be used to form an exact test for the null hypothesis of no effect modification. In the context of an instrumental variable design, this amounts to a test of the null hypothesis that the proportional dose model holds for some $\lambda_0$, $$\begin{aligned}
 \label{eq:prop}
H_0: y_{ij}(1)-y_{ij}(0) = \lambda_0 \{d_{ij}(1)-d_{ij}(0)\} \text{ for all }i,j,\text{  for some }\lambda_0,\end{aligned}$$ against the alternative that it does not hold for all $i,j$ and for any shared value of $\lambda_0$.

Suppose that $\Gamma=1$, and for a given value $\lambda_0$ consider as a test statistic the $F$-statistic comparing a regression of $\zeta_i^{(\lambda_0)}$ on (a) a design matrix incorporating covariates, such as ${{\mathbf{Q}}}_{reg}$ or ${{\mathbf{Q}}}_{PoP}$; to (b) the null model containing only an intercept column. If effect modification existed on the basis of the observed covariates, one would expect the regression incorporating covariates to reduce the sum of squared error relative to a model containing only an intercept, which would in turn inflate the $F$-statistic. Call the resulting statistic $F({{\mathbf{Z}}}, \bm{\zeta}^{(\lambda_0)})$. If the proportional dose model holds at $\lambda_0$, then $|\zeta_i^{(\lambda_0)}|$ would be fixed across randomizations with only its sign varying. This would allow for computation of a randomization-based $p$-value for the proportional dose model holding at $\lambda_0$: $$\begin{aligned}
\label{eq:pval}
p^{(\lambda_0)} &= \frac{1}{2^n}\sum_{{{\mathbf{z}}}\in \Omega}{\mathbbm{1}}\{F({{\mathbf{z}}}, \bm{\zeta}^{(\lambda_0)}) \geq f^{(\lambda_0)}\},\end{aligned}$$ where $f^{(\lambda_0)}$ is the observed value of the test statistic.

The value of $\lambda_0$ is a nuisance parameter for testing (\[eq:prop\]). Under perfect compliance, i.e. the typical paired experiment, Proposition 3 of @Fogarty:2018 shows that $F({{\mathbf{Z}}}, \bm{\zeta}^{(\lambda_0)})$ is pivotal with respect to the particular value of $\lambda_0$, such all values for $\lambda_0$ yield the same $p$-value, and any value for $\lambda_0$ could be used to test (\[eq:prop\]). Unfortunately, this does not hold in the presence of noncompliance, meaning that $p^{(\lambda_0)}$ varies as a function of $\lambda_0$. To overcome this, we employ the approach of @Berger:1994 to create a test of the hypothesis in (\[eq:prop\]). By the lemma of @Berger:1994 [§2], a valid $p$-value for (\[eq:prop\]) is $$\begin{aligned}
\label{eq:pbeta}
p_\beta &=  \underset{\lambda_0 \in CI_{1-\beta}}{\sup}p^{(\lambda_0)} + \beta.\end{aligned}$$ where $CI_{1-\beta}$ is *any* $100(1-\beta)\%$ confidence interval for $\lambda_0$ under the assumption of the proportional dose model. This permits the use of randomization-based confidence intervals derived in @Imbens:2005b and @Rosenbaum:2002 [§5.4] formed using test statistics other than the difference in means. However, for the purposes of improving the power of the test when effect modification is present, the confidence intervals described in §\[sec:new\] based on improved standard errors may be preferred. Through employing a randomization distribution, the resulting procedure yields an exact test of (\[eq:prop\]) for any sample size. See @ding2015randomization for other examples of randomization-based tests of treatment effect heterogeneity.

An omnibus test of effect heterogeneity is of special interest for instrumental variable studies. Even if one assumes monotonicity and the exclusion restriction, the effect ratio merely attests to the sample average treatment effect among compliers, rather than the average treatment effect for the study population. Some authors argue that a key weakness of IV designs is that they only identify this more local estimand [@deaton2010instruments; @swanson2014think; @swanson2017challenging]. For an IV design to provide an estimate of the SATE, the investigator must invoke an effect homogeneity or no-interaction assumption [@robins1994correcting; @Hernan:2006]. If we fail to reject the null hypothesis in (\[eq:prop\]), there is no evidence to reject the proportional dose model or to suggest that effect heterogeneity is present. If effects are roughly homogeneous, the IV estimand may be a good proxy for the sample average treatment effect in the study population.

Application to the EGS study
----------------------------

Table \[tab.sens2\] contains a comparison of 95% confidence intervals calculated at $\Gamma=1$ and sensitivity values within the septic and non-septic subgroups for the hospital length of stay outcome using three standard errors: the conventional standard error for a paired design, the regression-based standard error, and one formed by pairing together pairs on the basis of observed covariates. In our application, the different standard error estimators did not materially alter the confidence interval lengths. We found that the $\Gamma$ changepoints were also similar across standard errors. Results for the complication and mortality outcomes indicated a similar trend and are omitted here.

  ------------------------------------- --------------------- -------------------
                                           Septic Patients    
  (lr)<span>2-2</span> Standard Error      95% Conf. Int.      Sensitivity Value
  Conventional                            \[ 3.87 , 8.71 \]          1.64
  Regression-based                        \[ 4.03 , 8.69 \]          1.62
  Pair of Pairs                           \[ 4.01 , 8.66 \]          1.64
                                         Non-septic Patients  
  (lr)<span>2-2</span> Standard Error      95% Conf. Int.      Sensitivity Value
  Conventional                            \[ 3.36 , 4.81 \]          2.36
  Regression-based                        \[ 3.36 , 4.81 \]          2.38
  Pair of Pairs                           \[ 3.37 , 4.82 \]          2.37
  ------------------------------------- --------------------- -------------------

  : Confidence intervals and sensitivity values for hospital length of stay with different standard errors. <span data-label="tab.sens2"></span>

This lack of improvement suggests that there is little evidence for substantive effect modification within the sepsis and non-sepsis subgroups on the basis of other patient characteristics. To formally test this, we use the test of the proportional dose model described in §\[sec:omni\]. We set $\beta=0.01$, and construct a $99\%$ confidence interval for $\lambda_0$ by inverting the test $\varphi^{(\lambda_0)}_{{{\mathbf{Q}}}_{PoP}}$ at $\Gamma=1$. We then maximize (\[eq:pval\]) over $\lambda_0$ in this confidence interval, and form $p_{0.01}$ through (\[eq:pbeta\]). For septic patients, the resulting $p$-value for the complication outcome is 0.65, and the $p$-value is 0.10 for the length of stay outcome. Within the non-septic patients, the $p$-values are 0.67 and 0.77 for the complication and length of stay outcomes respectively. Therefore, we fail to reject the null hypothesis that the proportional dose model holds in both subgroups, finding little evidence that the IV estimates vary as a function of over 90 additional observed covariates.

Heterogeneity and design sensitivity in IV studies
==================================================

Larger but brittler effect estimates
------------------------------------

While there was no evidence for effect modification within the septic and non-septic subgroups on the basis of observed covariates, the effect ratio estimates within the septic group were larger than those in the non-septic subgroup for all three outcomes as presented in Table \[tab.out\]. As Table \[tab.sens\] illustrates, the results from the sensitivity analysis are reversed for length of stay and complication. For the complication outcome, we were able to reject the null of no effect ratio until $\Gamma=1.56$ and $\Gamma=1.59$ for the septic and non-septic groups respectively. For the length of stay variable, the discrepancy is even more substantial: for two individuals in the same matched set, a hidden variable would need to produce discrepancies in the odds that the individuals were assigned to a surgeon with a higher TTO by a factor of 1.64 in the septic group to explain away the finding of no effect ratio, while in the septic-group the odds would need to differ by a factor 2.36. Despite the septic subgroup having the larger estimated effects, the estimates for the *non-septic* group proved to be more robust to hidden bias. We now demonstrate how this phenomenon is reflective of the importance of reduced within-pair homogeneity for reducing sensitivity to hidden bias.

A favorable reality unknown to the practitioner {#sec:favorable}
-----------------------------------------------

Imagine that $Z_{ij}$ were actually a valid instrument and that, furthermore, the true value for the effect ratio exceeded its hypothesized value. Given that our instrument is not randomly assigned, a critic could always counter that a large effect estimate may be attributable not to a treatment effect, but rather to bias from lurking variables invalidating the proposed instrument. Even in this favorable situation of a valid instrument and a positive treatment effect, we would hope that our inferences would prove robust to moderate degrees of hidden bias to protect ourselves against such criticism. Theoretical calculations within this section proceed under this favorable setting: there is truly no hidden bias, there is truly an effect, but the practitioner is blind to this fact and hence hopes that her inferences perform well under the stress of a sensitivity analysis. In this way, we assess our method's ability to discriminate between (1) no treatment effect and hidden bias; and (2) a treatment effect without hidden bias.

Until this point inference has been performed conditional upon ${{\mathcal{F}}}$, and a generative model for ${{\mathcal{F}}}$ has been neither required nor assumed. For the calculations in this section, it is convenient to assume a superpopulation model. Because ${{\mathcal{F}}}$ will be viewed as random, the procedures exploiting effect modification presented in §\[sec:new\] no longer provide valid tests for the effect ratio; see @Fogarty:2018 [§5] for more on the relationship between superpopulation formulations and validity of standard errors using covariate information. We thus focus attention in the sensitivity analysis given in (\[eq:procedure\]) using the conventional standard error estimate, $\varphi^{(\lambda_0)}$, which remains valid under the superpopulation setting considered here. Following @Small:2008, we imagine that $\zeta_i^{(\lambda_0)}$ is generated as $$\begin{aligned}
\label{eq:gen2}
\zeta_i^{(\lambda_0)} &= \epsilon_i + S_i(\lambda - \lambda_0),\end{aligned}$$ where $\epsilon_i = (Z_{i1}-Z_{i2})\{(Y_{i1}-Y_{i2}) - \lambda(D_{i1} - D_{i2})\}$ are the encouraged-minus-non encouraged differences in responses adjusted according to the treatment actually taken, and $S_i = (Z_{i1}-Z_{i2})(D_{i1}-D_{i2})$ are the encouraged-minus-non encouraged differences in the treatment received, reflecting the strength of the instrument. Note that this generative model does *not* imply the proportional dose model, such that the individual-level effects are allowed to be heterogeneous.

We assume that $\epsilon_i$ are $iid$ from a symmetric distribution with mean zero and finite variance $\sigma^2$. The treatments actually received are assumed binary. We assume that there are no defiers, that the exclusion restriction holds, and that individuals $ij$ are assigned status as compliers, never-takers and always-takers independently with probability $p_C$, $p_N$, and $p_A$ respectively. This results in ${{\text{pr}}}(S_i=1) = p_C + p_Ap_N$, ${{\text{pr}}}(S_i=-1) = p_Ap_N$ and ${{\text{pr}}}(S_i=0) = 1-p_C-2p_Ap_N$. The terms $\zeta_i^{(\lambda_0)}$ are $iid$ draws from a mixture distribution with three components, namely $$\begin{aligned}
\label{eq:mixture}
\zeta_i^{(\lambda_0)} & = \begin{cases} \epsilon_i + (\lambda - \lambda_0) & \text{w.p. } p_C + p_Ap_N\\
\epsilon_i & \text{w.p. } 1 - p_C - 2p_Ap_N\\
\epsilon_i - (\lambda-\lambda_0)& \text{w.p. } p_Ap_N.\end{cases}\end{aligned}$$ The parameter $\lambda$ represents the true treatment effect among compliers, while $\lambda_0$ is its value under the null.

Design sensitivity
------------------

In a sensitivity analysis, bias dominates variance in large samples. In fact, under mild regularity conditions there is a number $\tilde{\Gamma}$, called the <span>design sensitivity</span>, such that the power of a sensitivity analysis tends to 1 if $\Gamma < \tilde{\Gamma}$ and tends to 0 if $\Gamma > \tilde{\Gamma}$ as $n\rightarrow \infty$ in the favorable situation of no bias in treatment assignment [@Rosenbaum:2004b]. The design sensitivity can be used as a means of appraising competing design strategies, comparing different test statistics, and assessing the impact of different assumptions within a sensitivity analysis. Larger values for $\tilde{\Gamma}$ indicate reduced sensitivity of inferences to unmeasured confounding in large samples.

For any value of $\Gamma$, our procedure employs the random variable $\bar{L}_\Gamma = n^{-1}\sum_{i=1}^n\{\zeta_i^{(\lambda_0)} - (\Gamma-1)/(1+\Gamma)|\zeta_i^{(\lambda_0)}|\}$. Under the favorable situation being considered this random variable has expectation $E(\zeta_i^{(\lambda_0)}) - (\Gamma-1)/(1+\Gamma)E|\zeta_i^{(\lambda_0)}|$, where $$\begin{aligned}
\label{eq:expec}
E(\zeta_i^{(\lambda_0)}) &= p_C(\lambda-\lambda_0),\\
E|\zeta_i^{(\lambda_0)}| &= (p_C + 2p_Ap_N)E|\epsilon_i + (\lambda-\lambda_0)| + (1-p_C - 2p_Ap_N)E|\epsilon_i|\nonumber\end{aligned}$$ are expectations with respect to the mixture distribution (\[eq:mixture\]). The second line uses symmetry of $\epsilon_i$ about zero. In a sensitivity analysis, we replace this true expectation with 0, the worst-case expectation for $\bar{L}_\Gamma$ under the null hypothesis. Under mild conditions, the design sensitivity is simply the value of $\Gamma$ that solves $E(\zeta_i^{(\lambda_0)})- (\Gamma-1)/(1+\Gamma)E|\zeta_i^{(\lambda_0)}| = 0$.

\[prop:designsens\] Suppose that the paired differences $\zeta_i^{(\lambda_0)}$ are drawn $iid$ from the mixture distribution (\[eq:designsens\]), and that $\epsilon_i$ are drawn $iid$ from a symmetric, mean zero distribution with finite variance. Then, the design sensitivity is $$\begin{aligned}
\label{eq:designsens}
\tilde{\Gamma}&=  \frac{E|\zeta_i^{(\lambda_0)}| + E(\zeta_i^{(\lambda_0)})}{E|\zeta_i^{(\lambda_0)}| -E(\zeta_i^{(\lambda_0)})}.\end{aligned}$$

Under $iid$ draws from a distribution with finite variance the strong law of large numbers applies to $n\times \text{se}(\bar{L}_\Gamma)^2$. Therefore, Proposition 2 of @Rosenbaum:2013 holds, and Corollary 1 of @Rosenbaum:2013 yields the formula for the design sensitivity.

Despite not resulting in a valid level$-\alpha$ sensitivity analysis in the superpopulation setting considered here, an analogous proof shows that $\varphi_{{{\mathbf{Q}}}}^{(\lambda_0)}$ also has the design sensitivity given in (\[eq:designsens\]). This is yet another reflection of bias trumping variance in observational studies considering hidden bias. In the limit, sensitivity to hidden bias is determined by the extent to which a test statistic can control the worst-case bias at $\Gamma$, and discrepancies in variance become irrelevant. So while exploiting effect modification can provide improvements in power under the assumption of no hidden bias even in the limit for finite-population causal inference, there is a sense in which improvements provided by exploiting effect modification for the changepoint value for $\Gamma$ reported by a sensitivity analysis are confined to small and moderate sample sizes.

The formula for the design sensitivity would remain unchanged had we used the test statistic $\bar{L}_\Gamma$ instead of the studentized statistic $\bar{L}_\Gamma/\text{se}(\bar{L}_\Gamma)$. The sensitivity analysis based on the unstudentized $\bar{L}_\Gamma$ is common in practice and is called the permutational $t$-test [@Rosenbaum:2007d], applied here to the adjusted paired differences $(Z_{i1}-Z_{i2})\{(Y_{i1}-Y_{i2}) - \lambda_0(D_{i1}-D_{i2})\}$. The studentization is essential for maintaining the correct Type I error rate in the presence of heterogeneous effects [@Fogarty:2019 Theorem 4], and through Proposition \[prop:designsens\] we see that it does so without sacrificing performance relative to the non-studentized $\bar{L}_\Gamma$ in the favorable setting of no bias.

Making sense of our sensitivity analysis {#sec:powersim}
----------------------------------------

The formula for design sensitivity in (\[eq:designsens\]) demonstrates the dependence of a sensitivity analysis's limiting power on various components of the data generating process. While the magnitude of the effect relative to its postulated value matters, so too does the proportion of compliers relative to always-takers and never-takers. The variance of $\epsilon_i$, reflecting the degree of within-pair heterogeneity, plays an important role through its influence on $E|\epsilon_i|$ and $E|\epsilon_i + \lambda - \lambda_0|$. These results help explain the phenomenon observed in our data, where larger treatment effect estimates in the septic subgroup were less robust to hidden bias than the smaller estimates in the non-septic subgroup.

Consider the length of stay outcome variable, and suppose for the purpose of illustration that the estimated effect ratios are actually the true values of $\lambda$, such that $\lambda_{S} = 6.8$ for septic patients and $\lambda_{NS} = 4.1$ for the non-septic patients. Suppose further that the standard deviation of $\zeta_i^{(\lambda_S)}$ and $\zeta_i^{(\lambda_{NS})}$ in our sample reflect the true standard deviations for the distribution of $\epsilon_i$ in the septic and non-septic subgroups, yielding $\sigma_S =25.3$ and $\sigma_{NS} = 8.9$. Table \[tab:DS\] gives the design sensitivities from (\[eq:designsens\]) using these values for $\lambda$ and $\sigma$ assuming both Normal and Laplace distributions for $\epsilon_i$, and varying the proportion of compliers. First, the role of instrument strength is pivotal for the design sensitivity of an instrumental variable study. In keeping with @Small:2008, we observed that the weaker the instrument, the lower the design sensitivity becomes, and results from IV studies with weak instruments are generally highly sensitive to hidden bias. Furthermore, we see that for a given level of compliance and regardless of distribution, the parameter settings motivated by the non-septic subgroup, with a *lower* effect ratio but lower heterogeneity, outperforms the septic subgroup in terms of design sensitivity. This illustrates that in the limit, a smaller treatment effect may well prove more robust to hidden bias when the data exhibit lower heterogeneity.

<span>c c c c c c c c c c c c</span> & &\
(lr)<span>2-6</span> (lr)<span>7-11</span> & 100% & 75% & 50% & 25% & 10%& 100% & 75% & 50% & 25% & 10%\
\
Septic & & & &&& && & &\
$\lambda = 6.8;\;\; \sigma = 25.3$\
\
Non-septic & & & &&& && & &\
$\lambda = 4.1;\;\; \sigma = 8.9$\

In small and moderate samples, discrepancies in the standard errors of test statistics play a larger role in the power of a sensitivity analysis. Using a simulation study, we now compare the role that the effect size relative to both the standard error and to individual-level variability plays in an observational study. To do so, we maintain the parameter values from the septic and non-septic groups and fix a common percentage of compliers, always-takers, and never-takers, but simulate from the generative model (\[eq:gen2\]) using different sample sizes for the septic and non-septic groups. The ratio of the variances of $\epsilon_i$ in the septic and non-septic subgroups is $\sigma^2_S/\sigma^2_{NS}\approx 8$, and we set $n_S/n_{NS}$, the ratio of the sample sizes for the septic and non-septic groups, to also equal 8. We do this so that the variances of the sample averages of the $\epsilon_i$ terms in both groups, $\sigma^2_S/n_S$ and $\sigma^2_{NS}/n_{NS}$, are approximately equal. The equality of these variances for the sample mean may appear to put inference within the non-septic subgroup at a disadvantage: as $\lambda_S > \lambda_{NS}$, inference at $\Gamma=1$ for the null of no effect ratio would be more powerful in the septic group than in the non-septic group. Does this disadvantage carry over to a sensitivity analysis at $\Gamma > 1$?

In each simulation, $n_S$ and $n_{NS}$ observations are drawn from the mixture distribution in (\[eq:designsens\]), with parameters $(\lambda_{S}, \sigma^2_S)$ and $(\lambda_{NS}, \sigma^2_{NS})$ and with $\epsilon_i$ normally distributed. We assume there are no defiers, set the probability of compliance to $p_C = 0.75$, and set $p_A=p_N = (1-p_C)/2$ for always-takers and never-takers. After drawing the samples, we conduct a sensitivity analysis at $\Gamma$, and record whether or not we correctly reject the null of zero effect ratio for each subgroup. We conduct this simulation study for $n_S=200, 1000, 2000$, $10,000$, with corresponding values $n_{NS} = 25, 125, 250, 1250$, and over a range of values for $\Gamma$. For each combination of sample sizes and values of $\Gamma$, we run 10,000 simulations. Through this simulation, we highlight the different roles that sample size can play for inference both assuming no hidden bias ($\Gamma=1$) and in a sensitivity analysis.

![Sample size, heterogeneity, and the power of a sensitivity analysis. The four plots show the power of the test against the null $\lambda = 0$ in the absence of unmeasured confounding as a function of $\Gamma$ for $I = 200, 1000, 2000$, and $10000$. The septic subgroup has $n_S = I$ pairs with a larger effect ratio and larger degree of heterogeneity, while the non-septic has $n_{NS} = I/8$ pairs, a smaller effect ratio and a smaller degree of heterogeneity. Design sensitivities are $\tilde{\Gamma}_S = 1.65$ and $\tilde{\Gamma}_{NS}=2.33$ respectively. <span data-label="fig:power"></span>](powerSepsis.pdf)

Figure \[fig:power\] presents the results of the simulation study. Each plot shows the performance of the sensitivity analyses in the septic and non-septic subgroups, varying $n_S$ while maintaining $n_S/n_{NS} = 8$. For instance, the upper-left panel shows the performance in a sensitivity analysis as a function of $\Gamma$ with $n_S = 200$, and $n_{NS} = 25$. Assuming no hidden bias ($\Gamma=1$), the test in the septic subgroup has higher power as $\lambda_S > \lambda_{NS}$ and $\sigma^2_{NS}/n_{NS} \approx \sigma^2_{S}/n_S$. As $\Gamma$ increases, we see that despite the smaller effect size the test in the non-septic subgroup begins to outperform that in the septic group, owing to the fact that $\sigma^2_{NS} < \sigma^2_S$. As $n_S$ increases with $n_S/n_{NS}$ fixed, we see that the larger sample size in the septic-group provides a smaller and smaller benefit. In sufficiently large samples, at $\Gamma=1$ the tests in both subgroups reject the null of no effect ratio with near certainty. As $\Gamma$ increases in this regime, the performance of the tests in both subgroups are dictated to a larger and larger degree by the respective design sensitivities: $\tilde{\Gamma}_{S} = 1.65$, and $\tilde{\Gamma}_{NS} = 2.33$. This highlights the importance of reduced heterogeneity of responses within pairs for increasing robustness to hidden bias, echoing the findings of @Rosenbaum:2005b.

Summary
=======

Assuming the validity of surgeon's tendency to operate as an instrument, our effect estimates indicate that surgery has an adverse, statistically significant, effect on both hospital length of stay and presence of a complication within both the septic and non-septic subgroups, with a larger estimated effect within the septic group. While the effect estimates for 30-day mortality indicate that surgery may also have an adverse effect, these results were not statistically significant at $\alpha=0.05$ even assuming no hidden bias ($\Gamma=1$). The complication outcome was robust to a moderate degree of hidden bias: for both septic and non-septic patients, two individuals with the same observed covariates would have to differ in their odds of being assigned to a high versus low TTO surgeon by a factor of roughly 1.6 to overturn the findings of the study. For septic patients a similar odds ratio discrepancy would be required to overturn the finding of an adverse effect on hospital length of stay, while for non-septic patients the odds would have to differ by nearly 2.4. This helps to frame the debate about what arguments against the validity of TTO as an instrument would actually matter: if a pattern of hidden bias could not influence the odds of assignment of surgeons to patients by this extent, it could not explain away the finding of an adverse effect.

The above conclusions required innovations in sensitivity analyses for instrumental variable designs. Given surgery's suspected variation in efficacy due to baseline patient characteristics we developed a sensitivity analysis that does not require effect homogeneity. Instead, we show how a sensitivity analysis for the effect ratio may proceed under essential heterogeneity, and indeed any pattern of effect heterogeneity. We then developed variance estimators that can result in improved sensitivity analyses when effects vary as a function of observed covariates. Further, we showed how these improved variance estimators can form the basis of an exact test for whether effect homogeneity holds, helpful in considering whether certain interpretations of an IV estimator are warranted. Motivated by the finding that treatment effect estimates in the non-septic group were more robust to hidden bias despite being of a smaller estimated magnitude, we showcased the role of strong instruments and reduced within-pair effect variation for design sensitivity and the power of a sensitivity analysis.

Review
======

<span>ll</span> Notation & Description\
$Z_{ij}$ & Encouragement to treatment (1 yes, 0 no)\
$d_{ij}(1)$ & $d_{ij}(0)$ & Potential exposures for individual $ij$, $i=1, \ldots n$; $j=1, 2$\
$y_{ij}(1)$ & $y_{ij}(0)$ & Potential outcomes\
$Y_{ij}$ & $D_{ij}$ & Observed outcome and exposure\
$\lambda$ & Effect ratio:\
& $\frac{\sum_{i=1}^{n} \sum_{j=1}^{2} \{ y_{ij}(1) - y_{ij}(0)\}}{\sum_{i=1}^{n} \sum_{j=1}^{2} \{ d_{ij}(1) - d_{ij}(0)\}} $\
$H_{prop}^{(\lambda_0)}$ & Null hypothesis under the proportional dose model\
& $y_{ij}(1) - y_{ij}(0) = \lambda_0 \{d_{ij}(1) - d_{ij}(0)\}\;\;\forall \;\;ij$\
$H_{weak}^{(\lambda_0)}$ & Neyman's weak null hypothesis on the effect ratio\
&$\lambda = \lambda_0$\
\
\
\
\
\
\
\
\
\[0.2cm\]\
\
\

Preliminaries {#sec:A1}
-------------

For each $\lambda_0$, we define a new variable $y_{ij}^*(z) = y_{ij}(z) - \lambda_0 d_{ij}(z) (z=0,1)$ which can be thought as the potential outcome adjusted for the dosage/exposure level received for each individual $ij$. The adjusted potential outcomes $y_{ij}^*(1), y_{ij}^*(0)$ are treated as if they are the potential outcomes usually defined in the literature. Define $\tau_{ij} = y_{ij}^*(1) - y_{ij}^*(0)$ and $\bar{\tau}_{i} = (\tau_{i1} + \tau_{i2})/2$. Then, $\tau_{ij}$ can be considered as the effect of the treatment on the adjusted outcome for individual $ij$, and $\bar{\tau_i}$ can be considered as the average treatment effect within pair $i$. Recall that $\zeta_i^{(\lambda_0)} = (Z_{i1} - Z_{i2})\{Y_{i1} - Y_{i2} - \lambda_0 (D_{i1} -D_{i2})\}$, which can be represented by $(Z_{i1} - Z_{i2})(Y_{i1}^* - Y_{i2}^*)$ where $Y_{ij}^* = y_{ij}^*(1) Z_{ij} + y_{ij}^*(0) (1-Z_{ij})$ is the observed "adjusted" outcome. This quantity can be understood as the treated-minus-control paired "adjusted" outcome difference. Thus, $\zeta_i^{(\lambda_0)}$ would be an unbiased estimator for $\bar{\tau}_i$ in a randomized encouragement design, but may exhibit bias in the presence of unmeasured confounders. Further define $$\eta_{i} = \frac{y_{i1}^*(1) + y_{i1}^*(0)}{2} - \frac{y_{i2}^*(1) + y_{i2}^*(0)}{2},$$ which is the difference in the averages of the potential outcomes for individuals within a pair. Also, recall that our sensitivity analysis model assumes, for each $\Gamma$, $$1 - \theta_\Gamma \leq \pi_{i} = {\text{Pr}}(Z_{i1}=1 \mid \mathcal{F}, \mathcal{Z}) \leq \theta_\Gamma\;\;\; i=1, \ldots, n$$ where $\theta_\Gamma = \Gamma/(1+\Gamma)$.

Asymptotics in the forthcoming discussions have in mind a single population ${{\mathcal{F}}}$ of increasing size; randomness stems only from the assignments ${{\mathbf{Z}}}$. The statement that $H_{weak}^{(\lambda_0)}$ holds in the limit should, in reality, reflect the changing value of the effect ratio $\lambda_{0n}$ as the finite population $\mathcal{F}_n$ grows. We will suppress both this dependence and conditioning upon ${{\mathcal{F}}}$ and ${{\mathcal{Z}}}$ for readability.

Regularity conditions
---------------------

There exist constants $C>0$, $\mu_m$ and $\mu_a$ such that as $n \to \infty$, $$\begin{aligned}
        n^{-1} \sum_{i=1}^{n} |\eta_i| > C, \quad n^{-1}\sum_{i=1}^{n} \eta_i^2 > C, &\\
        n^{-2} \sum_{i=1}^{n}\eta_i^2 \to 0, \quad n^{-2} \sum_{i=1}^{n}\eta_i^4 \to 0, \quad n^{-2} \sum_{i=1}^{n}\bar{\tau}_i^4 \to 0,& \\
        n^{-1} \sum_{i=1}^{n} (2\pi_i -1) \eta_i \to \mu_m, \quad n^{-1} \sum_{i=1}^{n} \pi_i |\bar{\tau}_i + \eta_i| + (1-\pi_i)|\bar{\tau}_i - \eta_i| \to \mu_a &
    \end{aligned}$$ \[eqn:condi1\]

There exists a constant $\nu^2 > 0$ such that $$\begin{aligned}
        n^{-1} \sum_{i=1}^{n} \pi_i (\bar{\tau}_i + \eta_i)^2 + (1-\pi_i)(\bar{\tau}_i - \eta_i)^2 \to \nu^2.
    \end{aligned}$$ \[eqn:condi2\]

Proofs
======

Proposition 1
-------------

Given Conditions \[eqn:condi1\] and \[eqn:condi2\], Proposition 1 follows immediately from Theorems 1 and 2 of @Fogarty:2019. We present a sketch of the proof here, the sketch slightly diverging from the proof in @Fogarty:2019 as here we merely establish asymptotic Type I error control under the weak null.

The first component of the proof ignores the reference distribution $G_\Gamma (\cdot)$, instead focusing on the large-sample reference distribution $\Phi(\cdot)$, the CDF of the standard Normal. Under Conditions \[eqn:condi1\] - \[eqn:condi2\]:

1.  Under $H_{weak}^{(\lambda_0)}$, the random variable $\sqrt{n}\bar{L}_\Gamma$ converges in distribution to a Normal random variable, whose expectation and standard deviation are unknown due to their dependence on both the unknown potential outcomes and the unmeasured confounders.

2.  Under $H_{weak}^{(\lambda_0)}$ and if the sensitivity model holds at $\Gamma$, $E(\sqrt{n}\bar{L}_\Gamma)\leq 0$, and ${\text{var}}(\sqrt{n}\bar{L}_\Gamma) \leq nE\{\text{se}(\bar{L}_\Gamma)^2\}$, which tends to a limit as $n\rightarrow \infty$. That is, the unknown expectation is upper bounded by zero, and there exists a sample-based standard error whose expectation upper bounds the unknown variance.

3.  The estimator $\text{se}(\bar{L}_\Gamma)^2$ converges in probability to its expectation, which is never smaller than $\text{var}(\bar{L}_\Gamma)$. For any $\epsilon >0$, we have that $\lim_{n\rightarrow\infty} {{\text{pr}}}(n\times \text{se}(\bar{L}_\Gamma)^2 + \epsilon \leq {\text{var}}(\sqrt{n}\bar{L}_\Gamma)\} \rightarrow 0$ Fixing $\alpha \leq 0.5$ and sending $\epsilon$ to zero, $$\begin{aligned}
    \underset{n\rightarrow\infty}{\lim}{{\text{pr}}}\left\{\bar{L}_\Gamma \geq \text{se}(\bar{L}_\Gamma) \Phi^{-1}(1-\alpha)\right\} \leq \underset{n\rightarrow\infty}{\lim}{{\text{pr}}}\left\{\bar{L}_\Gamma \geq \text{sd}(\bar{L}_\Gamma) \Phi^{-1}(1-\alpha)\right\} \leq \alpha.\end{aligned}$$

The second component of the proof involves showing that $\Phi^{-1}(1-\alpha)$ may be replaced by $G^{-1}(1-\alpha; \bm{1_n})$, the reference distribution generated by Algorithm 1 in the main text. Showing this uses a technique from @hoe52 for assessing the limiting behavior of a permutation distribution, along with the variant of Slutsky's theorem for randomization distributions from @chu13. To generate a reference distribution, we use $B_{\Gamma i}$ that stochastically dominates $L_{\Gamma i}$ under $H_{prop}^{(\lambda_0)}$, of the form $$\begin{aligned}
B_{\Gamma i} = V_{\Gamma i} |\zeta_i^{(\lambda_0)}| - \left(\frac{\Gamma -1}{\Gamma+1} \right) |\zeta_i^{(\lambda_0)}|,\end{aligned}$$ where $V_{\Gamma i}=\pm 1$ are independent across pairs, with ${{\text{pr}}}(V_{\Gamma i} = 1) = \theta_\Gamma$. Let $(\sqrt{n}\bar{B}_\Gamma, \sqrt{n}\bar{B}'_\Gamma)$ be averages of $B_{\Gamma i}$ and $B'_{\Gamma i}$ formed with iid vectors $(\bm{V_\Gamma}$, $\bm{V'_\Gamma})$, such that $(\sqrt{n}\bar{B}_\Gamma, \sqrt{n}\bar{B}'_\Gamma)$ are identically distributed with covariance zero. We have $$\begin{aligned}
E(B_{\Gamma i}) &= 0\\
{\text{var}}(B_{\Gamma i}) &= E\{{\text{var}}(B_{\Gamma i}\mid |\zeta_i^{(\lambda_0)}|\} + {\text{var}}\{E(B_{\Gamma i}\mid |\zeta_i^{(\lambda_0)}|\}\\
&= 4\theta_\Gamma(1-\theta_\Gamma)\{\pi_i(\bar{\tau}_i + \eta_i)^2 + (1-\pi_i)(\bar{\tau}_i - \eta_i)^2\},\end{aligned}$$ where $\theta_\Gamma, \pi_i, \eta_i$, and $\tau_i$ are defined as in Section \[sec:A1\]. Further, under Conditions \[eqn:condi1\] and \[eqn:condi2\], $(\sqrt{n}\bar{B}_\Gamma, \sqrt{n}\bar{B}'_\Gamma)^T$ tends in distribution to a multivariate normal, with mean vector zero, correlation zero, and equal variances $\nu_{\Gamma}^2 = 4 \theta_\Gamma (1-\theta_\Gamma) \nu^2$ where $\nu^2$ is defined in Condition \[eqn:condi2\]. We also have that $\sqrt{n}\text{se}(\bar{B}_\Gamma)$ converges in probability to $\nu_\Gamma$, such that $(\bar{B}_\Gamma/\text{se}(\bar{B}_\Gamma), \bar{B}'_\Gamma/\text{se}(\bar{B}'_\Gamma))^T$ converge in distribution to iid standard Normals. Recall that $G(\cdot; \bm{1}_n)$ is the CDF of $\bar{B}_\Gamma/\text{se}(\bar{B}_\Gamma)$. Combining the result of @hoe52, also given in Theorem 15.2.2 of @leh05, with Slutsky's theorem for randomization distributions [@chu13] yields that $\Phi^{-1}(1-\alpha)$ may be replaced by $G^{-1}(1-\alpha; \bm{1}_n)$ while preserving the asymptotic level of the procedure, proving the result.

Proposition 2
-------------

Let $\mathcal{D} = \{i: Y_{i1}+Y_{i2} = 1\}$, $\mathcal{C}_0 = \{i:Y_{i1}+Y_{i2}=0\}$, $\mathcal{C}_1 = \{i: Y_{i1}+Y_{i2} = 2\}$, and $\theta_\Gamma = \Gamma/(1+\Gamma)$. Under the sharp null, for any ${{\mathbf{z}}}\in \Omega$, McNemar's test statistic can be written as $T_{\mathcal{D}}$ + $|\mathcal{C}_1|$, where $T_{\mathcal{D}} = T_{\mathcal{D}}({{\mathbf{z}}}, {{\mathbf{Y}}}) = \sum_{i\in \mathcal{D}}\sum_{j=1}^2z_{ij}Y_{ij}$.

We now show that the studentized test statistic $\bar{L}_\Gamma/\text{se}(\bar{L}_\Gamma)$ is a monotone increasing function of $T_{\mathcal{D}}$ over ${{\mathbf{z}}}\in \Omega$ under the sharp null. First observe that under the sharp null, for any ${{\mathbf{z}}}\in \Omega$ $$\begin{aligned}
n\bar{L}_\Gamma &= \sum_{i=1}^n\left\{(z_{i1}-z_{i2})(Y_{i1}-Y_{i2}) - (2\theta_\Gamma-1)|Y_{i1}-Y_{i2}|\right\}\\
&= \sum_{i \in {{\mathcal{D}}}}(z_{i1}-z_{i2})(Y_{i1}-Y_{i2}) - (2\theta_\Gamma-1)|{{\mathcal{D}}}|\\
&= \sum_{i \in {{\mathcal{D}}}}\left\{z_{i1}Y_{i1} + z_{i2}Y_{i2} - z_{i1}(1-Y_{i1}) - z_{i2}(1-Y_{i2})\right\} - (2\theta_\Gamma-1)|{{\mathcal{D}}}|\\
&= 2T_{{\mathcal{D}}}+ |{{\mathcal{D}}}| - (2\theta_\Gamma-1)|{{\mathcal{D}}}| = 2(T_{{\mathcal{D}}}- \theta_\Gamma|{{\mathcal{D}}}|),\end{aligned}$$

Meanwhile, $\sum_{i=1}^nL_{\Gamma i}^2$ can be expressed as $$\begin{aligned}
\sum_{i=1}^nL_{\Gamma i}^2 &= \{1+(2\theta_\Gamma-1)^2\}\sum_{i=1}^n(Y_{i1}-Y_{i2})^2 - 2(2\theta_\Gamma-1)\sum_{i=1}^n(Z_{i1}-Z_{i2})(Y_{i1}-Y_{i2})|Y_{i1}-Y_{i2}|\\
&= \{1+(2\theta_\Gamma-1)^2\}|{{\mathcal{D}}}| - 2(2\theta_\Gamma-1)(2T_{{\mathcal{D}}}+ |{{\mathcal{D}}}|)\\
&= \{1-(2\theta_\Gamma-1)^2\}|{{\mathcal{D}}}| - 2(2\theta_\Gamma-1)\{2(T_{{\mathcal{D}}}- \theta_\Gamma|{{\mathcal{D}}}|)\},\end{aligned}$$ and $n(n-1)\text{se}(\bar{L}_\Gamma)^2$ as $$\begin{aligned}
n(n-1)\text{se}(\bar{L}_\Gamma)^2 = \{1-(2\theta_\Gamma-1)^2\}|{{\mathcal{D}}}| - 2(2\theta_\Gamma-1)\{2(T_{{\mathcal{D}}}- \theta_\Gamma|{{\mathcal{D}}}|)\} - n^{-1}\{2(T_{{\mathcal{D}}}- \theta_\Gamma|{{\mathcal{D}}}|)\}^2.\end{aligned}$$ Let $a = n^{-1}$, $b=2(2\theta_\Gamma-1)$, $c=\{1-(2\theta_\Gamma-1)^2\}|{{\mathcal{D}}}|$, and $x = 2(T_{{\mathcal{D}}}- \theta_\Gamma|{{\mathcal{D}}}|)$. We then have $$\begin{aligned}
\frac{\bar{L}_\Gamma}{\text{se}(\bar{L}_\Gamma)}\propto \frac{x}{\sqrt{c - bx - ax^2}},\end{aligned}$$ and to complete the proof it is sufficient to show that this function has a positive first derivative with respect to $x$ over its domain, $2\theta_\Gamma |{{\mathcal{D}}}|\leq x \leq (2+ 2\theta_\Gamma)|{{\mathcal{D}}}|$.

Differentiating yields $$\begin{aligned}
\frac{\partial}{\partial x}\frac{\bar{L}_\Gamma}{\text{se}(\bar{L}_\Gamma)} \propto \frac{\sqrt{c-bx - ax^2} + \frac{bx/2 + ax^2}{\sqrt{c-bx-ax^2}}}{c-bx-ax^2},\end{aligned}$$ and we want to show that this is always positive, which amounts to showing $c \geq (b/2)x$. Maximizing the right hand side over $x$, this means showing $\{1-(2\theta_\Gamma-1)^2\}|{{\mathcal{D}}}| \geq (2\theta_\Gamma-1)(2|{{\mathcal{D}}}| - 2\theta_\Gamma|{{\mathcal{D}}}|)$, which does hold. $\bar{L}_\Gamma/\text{se}(\bar{L}_\Gamma)$ is monotone increasing in $2(T_{{\mathcal{D}}}- \theta_\Gamma|{{\mathcal{D}}}|)$, and hence also in $T_{{\mathcal{D}}}+ |{{\mathcal{C}}}_1|$, which is McNemar's test. The test statistics have perfect rank correlation over ${{\mathbf{z}}}\in \Omega$, and hence are equivalent.

Proposition 3
-------------

Recall that $\tilde{L}_{\Gamma i} = L_{\Gamma i}/\sqrt{1-h_{ii}}$, $\tilde{\mu}_{\Gamma i} = E(\tilde{L}_{\Gamma i})$ and $\tilde{\bm{\mu}}_{\Gamma} = E(\tilde{\mathbf{L}}_\Gamma)$. Define $\bm\Lambda$ as the covariance matrix for $\tilde{L}_{\Gamma i} $, and note that $\bm{\Lambda}$ is a diagonal matrix, and the $i$-th diagonal element of $\bm\Lambda$ is $\Lambda_ii = {\text{var}}(\tilde{L}_{\Gamma i}) = {\text{var}}(L_{\Gamma i})/(1-h_{ii})$

Using results on expectations of quadratic forms, $E\{\text{se}(\bar{L}_\Gamma; {{\mathbf{Q}}})^2\}$ can be computed as $$\begin{aligned}
    E\{\text{se}(\bar{L}_\Gamma; {{\mathbf{Q}}})^2 \mid \mathcal{F}, \mathcal{Z}\} &= \frac{1}{n^2} E \left( \mathbf{\tilde{L}_{\Gamma}}^T (\mathbf{I} -\mathbf{H}) \mathbf{\tilde{L}_{\Gamma}} \right) \\
    &= \frac{1}{n^2} \left[ {\text{tr}}\{({{\mathbf{I}}}- {{\mathbf{H}}}) \bm{\Lambda}\} + \bm{\tilde{{\mu}}_\Gamma}^T ({{\mathbf{I}}}- {{\mathbf{H}}}) \bm{\tilde{{\mu}}_\Gamma}  \right] \\
    &= \frac{1}{n^2} \sum_{i=1}^{n} (1-h_{ii}) \Lambda_{ii} + \frac{1}{n^2}  \bm{\tilde{{\mu}}_\Gamma}^T ({{\mathbf{I}}}- {{\mathbf{H}}}) \bm{\tilde{{\mu}}_\Gamma} \\
    &= \frac{1}{n^2} \sum_{i=1}^{n}  {\text{var}}({L}_{\Gamma i} \mid \mathcal{F} , \mathcal{Z}) + \frac{1}{n^2}  \bm{\tilde{{\mu}}_\Gamma}^T ({{\mathbf{I}}}- {{\mathbf{H}}}) \bm{\tilde{{\mu}}_\Gamma} \\
    &=  {\text{var}}(\bar{L}_{\Gamma}) + \frac{1}{n^2}  \bm{\tilde{{\mu}}_\Gamma}^T ({{\mathbf{I}}}- {{\mathbf{H}}}) \bm{\tilde{{\mu}}_\Gamma} \geq {\text{var}}(\bar{L}_{\Gamma}) \end{aligned}$$ The last inequality holds because the matrix $({{\mathbf{I}}}- {{\mathbf{H}}})$ is positive semidefinite.

Proposition 4
-------------

Define $\mathbf{H}_{\mathbf{Q}_{reg}} = \mathbf{Q}_{reg}(\mathbf{Q}_{reg}^T \mathbf{Q}_{reg})^{-1} \mathbf{Q}_{reg}^{T}$ and $\mathbf{H}_{\mathbf{1}_n} = \mathbf{1}_n(\mathbf{1}_n^T\mathbf{1}_n)^{-1} \mathbf{1}_n$. We prove the result for a general matrix ${{\mathbf{Q}}}$ with a fixed number of columns for all $n$. Specifically, under suitable regularity conditions will prove $$n \times {\text{se}}(\bar{L}_{\Gamma}; {{\mathbf{Q}}})^2 - {\text{var}}(\sqrt{n} \bar{L}_{\Gamma}) \overset{p}{\to} {\underset{n \to \infty}{\lim}}\frac{1}{n} {\bm\mu_{\Gamma}^T} ({{\mathbf{I}}}- {{\mathbf{H_Q}}}) \bm\mu_{\Gamma}.$$ The proof follows that of Theorem 1 in @Fogarty:2018. The following regularity conditions are considered to prove Proposition 4.

Assume that ${{\mathbf{Q}}}$ is a $(n \times k)$ matrix with $k$ fixed. The following conditions hold for each $\Gamma \geq 1$.

-   (Bounded Fourth Moments).

    There exists a $C_1 < \infty$ such that $\frac{1}{n} \sum_{i=1}^{n} {{E}}(L_{\Gamma i}^4) < C_1$ and $\frac{1}{n} \sum_{i=1}^{n} Q_{iq}^4 < C_1$ for $q=1, \ldots, k$. For the specification of ${{\mathbf{Q}}}_{reg} = (\mathbf{1}, \bar{\mathbf{X}})$, the term $Q_{iq}^4$ can be replaced by $\bar{x}_{iq}^4$.

-   (Existence of Population Moments).

    -   $\frac{1}{n} \sum_{i=1}^{n} \mu_{\Gamma i}$, $\frac{1}{n} \sum_{i=1}^{n} \mu_{\Gamma i}^2$, and ${\text{var}}(\sqrt{n} \bar{L}_{\Gamma}) = \frac{1}{n} \sum_{i=1}^{n} {\text{var}}(L_{\Gamma i})$ converge to finite limits as $n \to \infty$.

    -   $\frac{1}{n} \sum_{i=1}^{n} \mu_{\Gamma i} Q_{iq}$ converges to a finite limit for $q=1, \ldots, k$ as $n \to \infty$.

    -   $\frac{1}{n} {{\mathbf{Q}}}^T {{\mathbf{Q}}}$ converges to a finite, invertible matrix as $n \to \infty$.

\[eqn:condi3\]

Define $\kappa_{\Gamma q} = \underset{n \to \infty}{\lim} \frac{1}{n} \sum_{i=1}^{n} \mu_{\Gamma i} Q_{iq}$. This is the limit of the sum of all elements in the $q$th column of $\bm{\mu}_{\Gamma}^T {{\mathbf{Q}}}$. Let $\bm\kappa_{\Gamma}$ as the vector of length $k$ containing limits $\kappa_{\Gamma q}$, $q=1, \ldots, k$. Also, define $\Sigma_{{{\mathbf{Q}}}} = \underset{n \to \infty}{\lim} n^{-1} {{\mathbf{Q}}}^T {{\mathbf{Q}}}$.

Consider the following lemma:

Under Condition \[eqn:condi3\],

1.  $n^{-1} \sum_{i=1}^{n} L_{\Gamma i} Q_{iq}$ converges in probability to ${\underset{n \to \infty}{\lim}}n^{-1} \sum_{i=1}^{n} \mu_{\Gamma i} Q_{iq}$.

2.  $h_{ii} \to 0$

3.  $n^{-1} \sum_{i=1}^{n} L_{\Gamma i}^2 \overset{p}{\to} {\underset{n \to \infty}{\lim}}n^{-1} 
            \sum_{i=1}^{n} \{\mu_{\Gamma i}^2 + {\text{var}}(L_{\Gamma i})\}$

\[lem1\]

\[Proof of Lemma 1\]\
Proof of (1). Since ${{E}}(L_{\Gamma i}) = \mu_{\Gamma i}$, ${{E}}(n^{-1} \sum_{i=1}^{n} L_{\Gamma i} Q_{iq}) \rightarrow {\underset{n \to \infty}{\lim}}n^{-1} \sum_{i=1}^{n} \mu_{\Gamma i} Q_{iq}$. The variance ${\text{var}}(n^{-1} \sum_{i=1}^{n} L_{\Gamma i} Q_{iq})$ goes to zero, $$\begin{aligned}
        {\text{var}}(n^{-1} \sum_{i=1}^{n} L_{\Gamma i} Q_{iq}) &= \frac{1}{n^2} \sum_{i=1}^{n} Q_{iq}^2{\text{var}}(L_{\Gamma i})\\
        & \leq  \frac{1}{n^2} \left\{\sum_{i=1}^{n} {\text{var}}(L_{\Gamma i})^2 \right\}^{1/2} \left\{ \sum_{i=1}^{n} (Q_{iq})^4 \right\}^{1/2} \\
        & \leq  \frac{1}{n} \left\{ \frac{1}{n} \sum_{i=1}^{n} {{E}}(L_{\Gamma i}^4) \right\}^{1/2} \left\{ \frac{1}{n} \sum_{i=1}^{n} (Q_{iq})^4 \right\}^{1/2} \to 0
    \end{aligned}$$

Proof of (2). Since $h_{ii}$ can be represented by $Q_{i}^T ({{\mathbf{Q}}}^T {{\mathbf{Q}}})^{-1} Q_i$ and $Q_i$ does not vary with $n$, from $\Sigma_{{{\mathbf{Q}}}} = \underset{n \to \infty}{\lim} n^{-1} {{\mathbf{Q}}}^T {{\mathbf{Q}}}$, we have $${\underset{n \to \infty}{\lim}}h_{ii} = {\underset{n \to \infty}{\lim}}n^{-1} Q_i^T \Sigma_{{{\mathbf{Q}}}}^{-1} Q_i = 0.$$

Proof of (3). This is a straightforward application of Chebyshev's inequality. Observe that ${\text{var}}(n^{-1} \sum_{i=1}^{n} L_{\Gamma i}^2) \leq n^{-2} \sum_{i=1}^{n} {{E}}(L_{\Gamma i}^4) < n^{-1} C_1\rightarrow 0$, such that average $n^{-1}\sum_{i=1}^nL^2_{\Gamma i}$ converges in probability to its expectation.

For any ${{\mathbf{Q}}}$, $n \times {\text{se}}(\bar{L}_{\Gamma}; {{\mathbf{Q}}})^2$ is $$\begin{aligned}
    n \times {\text{se}}(\bar{L}_{\Gamma}; {{\mathbf{Q}}})^2 &= \frac{1}{n} \left( \tilde{\mathbf{L}}_{\Gamma}^T ({{\mathbf{I}}}- {{\mathbf{H_Q}}}) \tilde{\mathbf{L}}_{\Gamma} \right) \\
    &=  \frac{1}{n} \left( \tilde{\mathbf{L}}_{\Gamma}^T \tilde{\mathbf{L}}_{\Gamma} -   \tilde{\mathbf{L}}_{\Gamma}^T {{\mathbf{Q}}}({{\mathbf{Q}}}^T {{\mathbf{Q}}})^{-1} {{\mathbf{Q}}}^{T}  \tilde{\mathbf{L}}_{\Gamma} \right).\end{aligned}$$ From Lemma \[lem1\], the above expression converges in probability to $$\begin{aligned}
    \frac{1}{n} \tilde{\mathbf{L}}_{\Gamma}^T \tilde{\mathbf{L}}_{\Gamma} &\overset{p}{\rightarrow} {\underset{n \to \infty}{\lim}}\frac{1}{n} \bm{\mu}_{\Gamma}^T \bm\mu_{\Gamma} + {\underset{n \to \infty}{\lim}}{\text{var}}(\sqrt{n} \bar{L}_{\Gamma}) \\
    \frac{1}{n} \tilde{\mathbf{L}}_{\Gamma}^T {{\mathbf{Q}}}({{\mathbf{Q}}}^T {{\mathbf{Q}}})^{-1} {{\mathbf{Q}}}^{T}  \tilde{\mathbf{L}}_{\Gamma} &\overset{p}{\rightarrow} \bm\kappa_{\Gamma}^T \Sigma_{{{\mathbf{Q}}}}^{-1} \bm\kappa_{\Gamma} = {\underset{n \to \infty}{\lim}}\frac{1}{n} \bm\mu_{\Gamma}^T {{\mathbf{H_Q}}}\bm\mu_{\Gamma}.\end{aligned}$$ Therefore, we have $$n \times {\text{se}}(\bar{L}_{\Gamma}; {{\mathbf{Q}}})^2 - {\text{var}}(\sqrt{n} \bar{L}_{\Gamma}) \overset{p}{\rightarrow}  {\underset{n \to \infty}{\lim}}\frac{1}{n} \bm{\mu}_{\Gamma}^T \bm\mu_{\Gamma} -  {\underset{n \to \infty}{\lim}}\frac{1}{n} \bm\mu_{\Gamma}^T {{\mathbf{H_Q}}}\bm\mu_{\Gamma} =  {\underset{n \to \infty}{\lim}}\frac{1}{n} \bm\mu_{\Gamma}^T ({{\mathbf{I}}}-{{\mathbf{H_Q}}}) \bm\mu_{\Gamma}.$$

Also, since the coefficient $R^2$ in a regression of $\bm{\mu}_\Gamma$ on ${{\mathbf{Q}}}_{reg}$ has the representation $$R^2 = 1 - \frac{\bm\mu_{\Gamma}^T (1- {{\mathbf{H}}}_{\mathbf{Q}_{reg}}) \bm\mu_{\Gamma}}{\bm\mu_{\Gamma}^T (1- {{\mathbf{H}}}_{\mathbf{1}_{n}})\bm\mu_{\Gamma} },$$ we have $$\frac{n \times {\text{se}}(\bar{L}_{\Gamma}; {{\mathbf{Q}}}_{reg})^2 - {\text{var}}(\sqrt{n} \bar{L}_{\Gamma})}{n \times {\text{se}}(\bar{L}_{\Gamma}; {\mathbf{1}_n})^2 - {\text{var}}(\sqrt{n} \bar{L}_{\Gamma})} \overset{p}{\rightarrow} 1- R^2.$$

Proposition 5
-------------

Assume that the pair index $i$ is re-ordered according to the pairs of pairs. For instance, for the $q$th of $n/2$ pairs of pairs, we assume that $i=2q-1$ and $i=2q$, $q=1, \ldots, n/2$, are in the $q$th pair of pairs. Then, the $n \times (n/2)$ matrix ${{\mathbf{Q}}}_{PoP}$ and the $n \times n$ matrix ${{\mathbf{H_Q}}}_{PoP}$ are represented by $${{\mathbf{Q}}}_{PoP} = \begin{bmatrix}
1 & & & \\
1 & & & \\
 & 1 & & \\
 & 1 & & \\
 & & \vdots & \\
 & & & 1 \\
 & & & 1 \\
\end{bmatrix}, \quad {{\mathbf{H_Q}}}_{PoP} = \begin{bmatrix}
1/2 & 1/2 & & & & & \\
1/2 & 1/2 & & & & & \\
& & 1/2 & 1/2 & & &\\
& & 1/2 & 1/2 & & &\\
& & & &\ddots & &\\
& & & & & 1/2 & 1/2 \\
& & & & & 1/2 & 1/2  \\
\end{bmatrix}.$$

Using this representation of ${{\mathbf{Q}}}$, the vector $({{\mathbf{I}}}- {{\mathbf{H_Q}}}_{PoP}) \tilde{\mathbf{L}}_{\Gamma}$ contains residuals. The $i$th component of this vector is $(L_{\Gamma i} - L_{\Gamma \mathcal{J}(i)})/\sqrt{2} $, where $\mathcal{J}(i)$ is index of the pair that was matched with to $i$th pair such that $\mathcal{J}(\mathcal{J}(i)) = i$. Since $({{\mathbf{I}}}- {{\mathbf{H_Q}}}_{PoP})$ is idempotent, the variance estimator ${\text{se}}(\bar{L}_{\Gamma}; {{\mathbf{Q}}}_{PoP})^2$ can be represented by the equation (5) in the main manuscript. Also, from Proposition 3, the bias ${{E}}[{\text{se}}(\bar{L}_{\Gamma}; {{\mathbf{Q}}}_{PoP})^2 \mid \mathcal{F}, \mathcal{Z}] - {\text{var}}(\bar{L}_{\Gamma} \mid \mathcal{F}, \mathcal{Z})$ is $$\begin{aligned}
    \frac{1}{n^2} \tilde{\bm{\mu}}_{\Gamma}^T ({{\mathbf{I}}}- {{\mathbf{H_Q}}}_{PoP}) \tilde{\bm{\mu}}_{\Gamma} & = \frac{1}{n^2} \left\{({{\mathbf{I}}}- {{\mathbf{H_Q}}}_{PoP}) \tilde{\bm{\mu}}_{\Gamma}\right\}^T ({{\mathbf{I}}}- {{\mathbf{H_Q}}}_{PoP}) \tilde{\bm{\mu}}_{\Gamma} \\
    & = \frac{1}{2n^2} \sum_{i=1}^{n} (\mu_{\Gamma i} - \mu_{\Gamma \mathcal{J}(i)})^2.\end{aligned}$$ The last equality stems from the fact that the $i$th component of $({{\mathbf{I}}}- {{\mathbf{H_Q}}}_{PoP}) \tilde{\bm\mu}_{\Gamma}$ is $(\mu_{\Gamma i} - \mu_{\Gamma \mathcal{J}(i)})/\sqrt{2} $.

Proposition 6
-------------

Recall that, for some matrix ${{\mathbf{Q}}}$, the new testing procedure is defined as $$\varphi_{{{\mathbf{Q}}}}^{(\lambda_0)} (\alpha, \Gamma) =  {\mathbbm{1}}\{\bar{L}_\Gamma/\text{se}(\bar{L}_\Gamma; {{\mathbf{Q}}}) \geq G_\Gamma^{-1}(1-\alpha; {{\mathbf{Q}}})\}$$ where $G_{\Gamma}( \cdot; {{\mathbf{Q}}})$ is the distribution of $\bar{B}_{\Gamma}/{\text{se}}(\bar{B}_{\Gamma}; {{\mathbf{Q}}})$ given $|\bm{\zeta^{(\lambda_0)}}|$ and is itself random over ${{\mathbf{z}}}\in \Omega$.

As in the proof of Proposition 1, we first establish $$\begin{aligned}
    \underset{n\rightarrow\infty}{\lim}{{\text{pr}}}\{\bar{L}_\Gamma/\text{se}(\bar{L}_\Gamma; {{\mathbf{Q}}}) \geq \Phi^{-1}(1-\alpha)\} \leq \alpha,\end{aligned}$$ This holds under Condition \[eqn:condi3\] for ${{\mathbf{Q}}}_{reg}$ (or, more generally, for matrices ${{\mathbf{Q}}}_{reg}$ whose number of columns do not grow with $n$), as under those conditions the standard errors have limits in probability. For ${{\mathbf{Q}}}_{PoP}$, observe that the number of columns is $n/2$, such that Proposition 3 does not apply. It is sufficient to show that for any $\epsilon > 0$, $$\begin{aligned}
\underset{n\rightarrow\infty}{\lim}{{\text{pr}}}\{n\times \text{se}(\bar{L}_\Gamma;{{\mathbf{Q}}}_{PoP})^2 \leq  n\times {\text{var}}(\bar{L}_\Gamma) - \epsilon\} = 0\end{aligned}$$ The estimator $n\times \text{se}(\bar{L}_\Gamma; {{\mathbf{Q}}}_{PoP})^2$ has expectation $$\begin{aligned}
n\times E\{\text{se}(\bar{L}_\Gamma; {{\mathbf{Q}}}_{PoP})^2\}&= n\times {\text{var}}(\bar{L}_\Gamma) + \frac{1}{2n} \sum_{i=1}^{n} (\mu_{\Gamma i} - \mu_{\Gamma \mathcal{J}(i)})^2.\end{aligned}$$ The term $(2n)^{-1} \sum_{i=1}^{n} (\mu_{\Gamma i} - \mu_{\Gamma \mathcal{J}(i)})^2$ is nonnegative, and Conditions \[eqn:condi1\]-\[eqn:condi2\] imply that $n\times {\text{var}}(\bar{L}_\Gamma)$ tends to a (positive) limit. We have $$\begin{aligned}
{\text{var}}\{n\times \text{se}(\bar{L}_\Gamma;{{\mathbf{Q}}}_{PoP})^2\} \leq 16 n^{-2}\sum_{i=1}^nE(L_{\Gamma i}^4),\end{aligned}$$ which tends to zero under part (1) of Condition \[eqn:condi3\]. Through Chebyshev's inequality, we then have for any $\epsilon > 0$ $$\begin{aligned}
&{{\text{pr}}}\left\{n\times\text{se}(\bar{L}_\Gamma;{{\mathbf{Q}}}_{PoP})^2 - n\times {\text{var}}(\bar{L}_\Gamma) \leq -\epsilon\right\}\\ & \leq {{\text{pr}}}\left\{n\times\text{se}(\bar{L}_\Gamma;{{\mathbf{Q}}}_{PoP})^2 - n\times {\text{var}}(\bar{L}_\Gamma) - (2n)^{-1} \sum_{i=1}^{n} (\mu_{\Gamma i} - \mu_{\Gamma \mathcal{J}(i)})^2\leq -\epsilon\right\}\\
& \leq {{\text{pr}}}\left\{\left|n\times\text{se}(\bar{L}_\Gamma;{{\mathbf{Q}}}_{PoP})^2 - n\times {\text{var}}(\bar{L}_\Gamma) - (2n)^{-1} \sum_{i=1}^{n} (\mu_{\Gamma i} - \mu_{\Gamma \mathcal{J}(i)})^2\right|\geq \epsilon\right\}\\
&\leq {\text{var}}\{n\times \text{se}(\bar{L}_\Gamma;{{\mathbf{Q}}}_{PoP})^2\}/\epsilon^2\rightarrow 0,\end{aligned}$$ establishing $$\begin{aligned}
    \underset{n\rightarrow\infty}{\lim}{{\text{pr}}}\{\bar{L}_\Gamma \geq \text{se}(\bar{L}_\Gamma ; {{\mathbf{Q}}}) \times \Phi^{-1}(1-\alpha)\} \} \leq \underset{n\rightarrow\infty}{\lim}{{\text{pr}}}\{\bar{L}_\Gamma \geq \text{sd}(\bar{L}_\Gamma ) \times \Phi^{-1}(1-\alpha)\} \} \leq \alpha\end{aligned}$$ with both ${{\mathbf{Q}}}= {{\mathbf{Q}}}_{reg}$ and ${{\mathbf{Q}}}= {{\mathbf{Q}}}_{PoP}$.

Next, we show that $\Phi^{-1}(1-\alpha)$ may be replaced with $G^{-1}(1-\alpha;{{\mathbf{Q}}})$ for either of these choices for ${{\mathbf{Q}}}$. The proof differs from the analogous portion of Proposition 1 only in that we must instead establish that $$\begin{aligned}
 n \times {\text{se}}(\bar{B}_{\Gamma}; {{\mathbf{Q}}})^2 \overset{p}{\rightarrow} \nu_{\Gamma}^2\end{aligned}$$ for ${{\mathbf{Q}}}\neq \bm{1_n}$. This can be proved by using ${{E}}\{se(\bar{B}_{\Gamma}; {{\mathbf{Q}}})^2\} = {\text{var}}(\bar{B}_{\Gamma})$ for *any* ${{\mathbf{Q}}}$ and that ${\text{var}}\{n \times {\text{se}}(\bar{B}_{\Gamma}; {{\mathbf{Q}}})^2\} \to 0$. Recall that ${\text{se}}(\bar{B}_{\Gamma}; {{\mathbf{Q}}})^2 = \frac{1}{n^2} \tilde{{{\mathbf{B}}}}_{\Gamma}^T({{\mathbf{I}}}- {{\mathbf{H_Q}}}) \tilde{{{\mathbf{B}}}}_{\Gamma}$ where $\bar{B}_{\Gamma i} = \frac{B_{\Gamma i}}{\sqrt{1-h_{ii}}}$. Define $\Lambda_{\Gamma}$ be the covariance matrix of $\tilde{\mathbf{B}}_{\Gamma}$, and $\Lambda_{\Gamma ii}$ be the $i$th diagonal element of $\Lambda_{\Gamma}$. We have ${{E}}(B_{\Gamma i}) = 0$ and $\Lambda_{\Gamma ii} = \frac{1}{1-h_{ii}} {\text{var}}(B_{\Gamma_i})$.

${{E}}\{{\text{se}}(\bar{B}_{\Gamma}; {{\mathbf{Q}}})^2\} = {\text{var}}(\bar{B}_{\Gamma})$

${{E}}\{{\text{se}}(\bar{B}_{\Gamma}; {{\mathbf{Q}}})^2\}$ can be computed by $$\begin{aligned}
{{E}}[{\text{se}}(\bar{B}_{\Gamma}; {{\mathbf{Q}}})^2] &= \frac{1}{n^2} {{E}}[\tilde{{{\mathbf{B}}}}_{\Gamma}^T({{\mathbf{I}}}- {{\mathbf{H_Q}}}) \tilde{{{\mathbf{B}}}}_{\Gamma}] \\
&= \frac{1}{n^2} {\text{tr}}[({{\mathbf{I}}}- {{\mathbf{H_Q}}}) \Lambda_{\Gamma}] + {{E}}(\tilde{{{\mathbf{B}}}}_{\Gamma})^T ({{\mathbf{I}}}- {{\mathbf{H_Q}}}) {{E}}(\tilde{{{\mathbf{B}}}}_{\Gamma}) \\
&= \frac{1}{n^2} {\text{tr}}\{({{\mathbf{I}}}- {{\mathbf{H_Q}}}) \Lambda_{\Gamma}\}\\
&= \frac{1}{n^2} \sum_{i=1}^{n} {\text{var}}(B_{\Gamma i}) = {\text{var}}(\bar{B}_{\Gamma})\end{aligned}$$

The second to last line follows from $E(B_{\Gamma i}) = 0$ for all $i$, as the random variable $B_{\Gamma i}$ behaves as though the proportional dose model holds under the worst-case assignment probabilities. For ${{\mathbf{Q}}}_{reg}$, by using the fact that $h_{ii} \to 0$ and assuming $\frac{1}{n} \sum_{i=1}^{n} {{E}}(B_{\Gamma i}^4)$ is bounded, $n\times {\text{var}}\{{\text{se}}(\bar{B}_{\Gamma}; {{\mathbf{Q}}}_{reg})^2\} \to 0$ through an analogous argument to that in Proposition 4. Assuming $n^{-1}\sum_{i=1}^nE(\bar{B}_{\Gamma i})$ is bounded also gives that $n\times {\text{var}}\{{\text{se}}(\bar{B}_{\Gamma}; {{\mathbf{Q}}}_{PoP})^2 \} \to 0$, such that $n \times {\text{se}}(\bar{B}_{\Gamma}; {{\mathbf{Q}}}_{PoP})^2 \overset{p}{\rightarrow} \nu_{\Gamma}^2$. With this established, the proof that the quantile $G^{-1}(1-\alpha ; {{\mathbf{Q}}})$ with ${{\mathbf{Q}}}= {{\mathbf{Q}}}_{reg}$ or ${{\mathbf{Q}}}_{PoP}$, may be employed is analogous to Proposition 1.

To prove that $\varphi_{{{\mathbf{Q}}}}^{(\lambda_0)}(\alpha,\Gamma)$ with ${{\mathbf{Q}}}= {{\mathbf{Q}}}_{reg}$ or ${{\mathbf{Q}}}= {{\mathbf{Q}}}_{PoP}$ is both less conservative under the null and more powerful under the alternative than the test using the conventional standard error, it suffices to show that for any $\epsilon > 0$ and for either of these choices ${{\mathbf{Q}}}$, $$\begin{aligned}
\underset{n\rightarrow\infty}{\lim}{{\text{pr}}}[n\times\left\{\text{se}(\bar{L}_\Gamma; \bm{1_n})^2 - \text{se}(\bar{L}_\Gamma ; {{\mathbf{Q}}})^2\right\} \leq -\epsilon]  &= 0\end{aligned}$$

Observe that $$\begin{aligned}
&{{\text{pr}}}[n\left\{\text{se}(\bar{L}_\Gamma; \bm{1_n})^2 - \text{se}(\bar{L}_\Gamma ; {{\mathbf{Q}}})^2\right\} \leq -\epsilon]\\
&={{\text{pr}}}\left(n\left[\text{se}(\bar{L}_\Gamma; \bm{1_n})^2 - E\{\text{se}(\bar{L}_\Gamma; \bm{1_n})^2\} +  E\{\text{se}(\bar{L}_\Gamma; \bm{1_n})^2\}- \text{se}(\bar{L}_\Gamma ; {{\mathbf{Q}}})^2\right] \leq -\epsilon\right)\\
&\leq {{\text{pr}}}\left(n\left[\text{se}(\bar{L}_\Gamma; \bm{1_n})^2 - E\{\text{se}(\bar{L}_\Gamma; \bm{1_n})^2\}\right] \leq -\epsilon\right) +  {{\text{pr}}}\left(n\left[E\{\text{se}(\bar{L}_\Gamma; \bm{1_n})^2\}- \text{se}(\bar{L}_\Gamma ; {{\mathbf{Q}}})^2\right] \leq -\epsilon\right)\\\end{aligned}$$ The first probability tends to zero, as under Conditions \[eqn:condi1\]-\[eqn:condi2\] the conventional standard error converges in probability to its expectation. For ${{\mathbf{Q}}}_{reg}$, the second probability also tends to zero under Condition \[eqn:condi3\] through a proof analogous to Proposition 4. For ${{\mathbf{Q}}}_{PoP}$, we need conditions to further ensure that $$\begin{aligned}
\underset{n\rightarrow\infty}{\lim}\left\{\frac{1}{n}\sum_{i=1}^n(\mu_{\Gamma i} - \bar{\mu}_\Gamma)^2 - \frac{1}{2n}\sum_{i=1}^n(\mu_{\Gamma i} - \mu_{\Gamma \mathcal{J}(i)})^2\right\} \geq 0.\end{aligned}$$ In a sense, the required condition is that pairing pairs on the basis of observed covariates is effectively creating pairs of pairs with similar expectations. Observe that if we randomly paired the pairs without any consideration of similarity on observed covariates, the difference in these two terms would *equal* zero. Regularity conditions are needed to preclude pairings of pairs yielding a worse alignment of expectations than what would be expected under random pairings of pairs. The sufficient condition reflecting this natural requirement takes the form $$\begin{aligned}
\underset{n\rightarrow\infty}{\lim}\frac{1}{2n}\sum_{i=1}^n(\mu_{\Gamma i} - \bar{\mu}_\Gamma)(\mu_{\Gamma \mathcal{J}(i)} - \bar{\mu}_\Gamma) \geq 0.\end{aligned}$$ See @Abadie:2008 for a discussion of sufficient conditions in a superpopulation formulation where the potential outcomes are viewed as random but the covariates as fixed, wherein natural connections between our sufficient condition and Lipschitz conditional expectation and conditional variance functions are explored.

{'source': 'arXiv_out/1909'}