Title: Deterministic parallel analysis: An improved method for selecting the number of factors and principal components,
Abstract: Factor analysis and principal component analysis (PCA) are used in many
application areas. The first step, choosing the number of components, remains a
serious challenge. Our work proposes improved methods for this important
problem. One of the most popular state-of-the-art methods is Parallel Analysis
(PA), which compares the observed factor strengths to simulated ones under a
noise-only model. This paper proposes improvements to PA. We first de-randomize
it, proposing Deterministic Parallel Analysis (DPA), which is faster and more
reproducible than PA. Both PA and DPA are prone to a shadowing phenomenon in
which a strong factor makes it hard to detect smaller but more interesting
factors. We propose deflation to counter shadowing. We also propose to raise
the decision threshold to improve estimation accuracy. We prove several
consistency results for our methods, and test them in simulations. We also
illustrate our methods on data from the Human Genome Diversity Project, where
they significantly improve the accuracy.