Title: Post hoc inference via joint family-wise error rate control,
Abstract: We introduce a general methodology for post hoc inference in a large-scale
multiple testing framework. The approach is called "user-agnostic" in the sense
that the statistical guarantee on the number of correct rejections holds for
any set of candidate items selected by the user (after having seen the data).
This task is investigated by defining a suitable criterion, named the
joint-family-wise-error rate (JER for short). We propose several procedures for
controlling the JER, with a special focus on incorporating dependencies while
adapting to the unknown quantity of signal (via a step-down approach). We show
that our proposed setting incorporates as particular cases a version of the
higher criticism as well as the closed testing based approach of Goeman and
Solari (2011). Our theoretical statements are supported by numerical
experiments.