Many psychology applications assess measurement invariance of a construct (e.g., depression) over time. These applications are often characterized by few time points (e.g., 3), but high rates of dropout. Although such applications routinely assume that the dropout mechanism is ignorable, this assumption may not always be reasonable. In the presence of nonignorable dropout, fitting a conventional longitudinal factor model (LFM) to assess longitudinal measurement invariance can yield misleading inferences about the level of invariance, along with biased parameter estimates. In this article we develop pattern mixture longitudinal factor models (PM-LFMs) for quantifying uncertainty in longitudinal invariance testing due to an unknown, but potentially nonignorable, dropout mechanism. PM-LFMs are a kind of multiple group model wherein observed missingness patterns define groups, LFM parameters can differ across these pattern-groups subject to identification constraints, and marginal inference about longitudinal invariance is obtained by pooling across pattern-groups. When dropout is nonignorable, we demonstrate via simulation that conventional LFMs can indicate longitudinal noninvariance, even when invariance holds in the overall population; certain PM-LFMs are shown to ameliorate this problem. On the other hand, when dropout is ignorable, PM-LFMs are shown to provide results comparable to conventional LFMs. Additionally, we contrast PM-LFMs to a latent mixture approach for accommodating nonignorable dropoutwherein missingness patterns can differ across latent groups. In an empirical example assessing longitudinal invariance of a harsh parenting construct, we employ PM-LFMs to assess sensitivity of results to assumptions about nonignorable missingness. Software implementation and recommendations for practice are discussed.