Title: Sampling-based vs. Design-based Uncertainty in Regression Analysis,
Abstract: Consider a researcher estimating the parameters of a regression function
based on data for all 50 states in the United States or on data for all visits
to a website. What is the interpretation of the estimated parameters and the
standard errors? In practice, researchers typically assume that the sample is
randomly drawn from a large population of interest and report standard errors
that are designed to capture sampling variation. This is common practice, even
in applications where it is difficult to articulate what that population of
interest is, and how it differs from the sample. In this article, we explore an
alternative approach to inference, which is partly design-based. In a
design-based setting, the values of some of the regressors can be manipulated,
perhaps through a policy intervention. Design-based uncertainty emanates from
lack of knowledge about the values that the regression outcome would have taken
under alternative interventions. We derive standard errors that account for
design-based uncertainty instead of, or in addition to, sampling-based
uncertainty. We show that our standard errors in general are smaller than the
infinite-population sampling-based standard errors and provide conditions under
which they coincide.