Title: Rank conditional coverage and confidence intervals in high dimensional problems,
Abstract: Confidence interval procedures used in low dimensional settings are often
inappropriate for high dimensional applications. When a large number of
parameters are estimated, marginal confidence intervals associated with the
most significant estimates have very low coverage rates: They are too small and
centered at biased estimates. The problem of forming confidence intervals in
high dimensional settings has previously been studied through the lens of
selection adjustment. In this framework, the goal is to control the proportion
of non-covering intervals formed for selected parameters.
In this paper we approach the problem by considering the relationship between
rank and coverage probability. Marginal confidence intervals have very low
coverage rates for significant parameters and high rates for parameters with
more boring estimates. Many selection adjusted intervals display the same
pattern. This connection motivates us to propose a new coverage criterion for
confidence intervals in multiple testing/covering problems --- the rank
conditional coverage (RCC). This is the expected coverage rate of an interval
given the significance ranking for the associated estimator. We propose
interval construction via bootstrapping which produces small intervals and have
a rank conditional coverage close to the nominal level. These methods are
implemented in the R package rcc.