Structured Latent Factor Analysis for Large-scale Data: Identifiability, Estimability, and Their Implications
Yunxiao Chen Xiaoou Li Siliang Zhang
stat.ME

Robust functional estimation in the multivariate partial linear model
    Michael Levine
    
=====================================================================






Latent factor models are widely used to measure unobserved latent traits in social and behavioral sciences, including psychology, education, and marketing. When used in a confirmatory manner, design information is incorporated as zero constraints on corresponding parameters, yielding structured (confirmatory) latent factor models.
In this paper, we study how such design information affects the identifiability and the estimation of a structured latent factor model.
Insights are gained through both asymptotic and non-asymptotic analyses. Our asymptotic results are established under a regime where both the number of manifest variables and the sample size diverge, motivated by applications to large-scale data. Under this regime, we define the structural identifiability of the latent factors and establish necessary and sufficient conditions that ensure structural identifiability.  In addition, we propose an estimator which is shown to be consistent and rate optimal when structural identifiability holds. Finally, a non-asymptotic error bound is derived for this estimator, through which the effect of design information is further quantified. Our results shed lights on the design of large-scale measurement in education and psychology and have important implications on measurement validity and reliability.

	
KEY WORDS:
High-dimensional latent factor model, confirmatory factor analysis, identifiability of latent factors, structured low-rank matrix, large-scale psychological measurement



§ INTRODUCTION



Latent factor models are one of the major statistical tools for multivariate data analysis
that have received many applications in  social and behavioral sciences <cit.>.
Such models capture and interpret the common dependence among multiple manifest variables through the introduction of low-dimensional latent factors, where the latent factors are often given substantive interpretations. For example, in educational and psychological measurement, the manifest variables may be a test-taker's responses to test items, and the latent factors are often interpreted as his/her cognitive abilities and psychological traits, respectively. In marketing, the manifest variables may be an audience's ratings on movies, and the latent factors may be interpreted as his/her preferences on multiple characteristics of movies.

In many applications, latent factor models are used in a confirmatory manner, where design information on the relationship between manifest variables and latent factors is specified a priori.
This is translated into a zero constraint on a loading parameter of the model. We call such a model  a structured latent factor model. For example, consider a
mathematics test consisting of J items that measures students' abilities on calculus and algebra. Then a structured latent factor model with two factors may be specified to model its item response data, where the two factors may be
interpreted as the calculus and algebra factors, respectively.
The relationship between the manifest variables and the two factors is coded by a J× 2 binary matrix. The jth row being (1, 0), (0, 1), and (1, 1) means that the jth items can be solved using only calculus skill, only algebra skill, and both, respectively.


Identifiability is an important property of a structured latent factor model for ensuring the substantive interpretation of latent factors.
When the model is not identifiable, certain latent factors cannot be uniquely extracted from data and thus their substantive interpretations may not be valid.
In particular, if no design information is incorporated, then it is well known that a latent factor model is not identifiable due to rotational indeterminacy, in which case one can simultaneously rotate the latent factors and the loading matrix of the model, without changing the distribution of data. It is thus important to study the relationship between design information of manifest variables and identifiability of latent factors. 
That is, 
when is a latent factor identifiable and when is not? 

A related problem is the estimation of a structured latent factor model. Under what notion of consistency, can a latent factors be consistently estimated? In that case, what is the convergence rate? These problems will be investigated in this paper. Similar identifiability problems have been considered in the context of linear factor models <cit.> and restricted latent class models <cit.>.


This paper considers the identifiability and estimability of structured latent factor models.
We study this problem under  a generalized latent factor modeling framework, which includes latent factor models for continuous, categorical, and count data.

Unlike many existing works that take an empirical Bayes framework treating the latent factors as random variables, we treat them as unknown model parameters. This treatment has been taken in many works on latent factor models <cit.>. Under this formulation, the identification of a latent factor becomes equivalent to the identification of the direction of a vector in ℝ^∞, where each entry of the infinite-dimensional vector corresponds to a sample (e.g., a test-taker or a costumer) in a population. As will be shown in the sequel, to identify such a direction, we need an infinite number of manifest variables (e.g., measurements). Under such an asymptotic regime, we provide a necessary and sufficient condition on the design of manifest variables for the identifiability of latent factors. This condition can be very easily interpreted from a repeated measurement perspective. In the rest of the paper, this notation of identifiability is called structural identifiability.  Comparing with existing works <cit.>, the current development applies to a more general family of latent factor models, and avoids distribution assumptions on the latent variables by adopting the double asymptotic regime that the sample size N and the number of manifest variables J simultaneously grow to infinity.


Under the double asymptotic regime and when structural identifiability holds, we propose an estimator and establish its consistency and convergence rate. Under suitable conditions, this convergence rate is shown to be optimal through an asymptotic lower bound for this estimation problem. 
Finally, a non-asymptotic error bound is derived for this estimator, which complements the asymptotic results by quantifying the estimation accuracy under a given design with finite N and J.
To establish these results,
we prove useful probabilistic error bounds and develop perturbation bounds on the intersection of linear subspaces which are of independent value for the theoretical analysis of low-rank matrix estimation.








The rest of the paper is organized as follows.
In Section <ref>, we introduce a generalized latent factor modeling framework, within which our research questions are formulated. In Section <ref>,
we discuss the structural identifiability for latent factors,
establish the relationship between structural identifiability and estimability,
and provide an estimator for which asymptotic results and a non-asymptotic error bound are established.





Further implications of our theoretical results on large-scale measurement
are provided in Section <ref> and extensions of our results to more complex settings are discussed in
Section <ref>.
A new  perturbation bound on linear subspaces is presented in Section <ref> that is
key to our main results. Numerical results are presented in Section <ref> that contain
two simulation studies and an application to a personality assessment dataset.
Finally, concluding remarks are provided in Section <ref>.
The proofs of all the technical results are provided as supplementary material.
































































































































§ STRUCTURED LATENT FACTOR ANALYSIS




 §.§ Generalized Latent Factor Model


Suppose that there are N individuals (e.g., N test-takers) and J manifest variables (e.g. J test items). Let Y_ij be a random variable denoting the ith individual's value on the jth manifest variable and let y_ij be its realization.  For example, in educational tests, Y_ijs could be binary responses from the examinees, indicating whether the answers are correct or not.
We further assume that each individual i is associated with a K-dimensional latent vector, denoted as _i = (θ_i1, ..., θ_iK)^⊤ and each manifest variable j is associated with K parameters _j = (a_j1, ..., a_jK)^⊤. 
We give two concrete contexts. Consider an educational test of mathematics, with
K = 3 dimensions of “algebra",  “geometry", and “calculus". Then θ_i1, θ_i2, and θ_i3
represent individual i's proficiency levels on algebra, geometry, and calculus, respectively.
In the measurement of  Big Five  personality factors <cit.>,  K = 5 personality factors are considered, including “openness to experience", “conscientiousness", “extraversion", “agreeableness", and “neuroticism". Then θ_i1, ..., θ_i5 represent individual i's levels on the continuums of the five personality traits.
The manifest parameter _js can be understood as the regression coefficients when regressing
Y_ijs on _is, i = 1, ..., N.



In many applications of latent factor models, especially in psychology and education, 
the estimations of _is and _js are both  of interest. 



Our development is under a generalized latent factor model framework <cit.>, which extends
the generalized linear model framework <cit.> to latent factor analysis.

Specifically, we assume that the distribution of Y_ij given _i and _j is a member of the exponential family with natural parameter

    m_ij = _j^⊤_i = a_j1θ_i1 + ⋯ a_jKθ_iK,

 and possibly a scale (i.e. dispersion) parameter ϕ. More precisely, the density/probability mass function takes the form:

    f(y |_j, _i, ϕ) = exp(ym_ij - b(m_ij)/ϕ + c(y, ϕ)),

where b(·) and c(·) are pre-specified functions that depend on the member of the exponential family. Given
_i and _j, i = 1, ..., N and j = 1, ..., J, we assume that all Y_ijs are independent. Consequently, the likelihood function,
in which _is and _js are treated as fixed effects, can be written as

    L(_1, ..., _N, _1, ..., _J, ϕ) = ∏_i=1^N∏_j=1^J exp(y_ij m_ij - b(m_ij)/ϕ + c(y_ij, ϕ)).

This likelihood function is known as the joint likelihood function in the literature of latent variable models <cit.>.


We remark that
in the existing literature of latent factor models,
there is often an intercept term indexed by j in the specification of (<ref>),
which can be easily realized under our formulation
by constraining θ_i1 = 1, for all i = 1, 2, ..., N. In that case, a_j1 serves as the intercept term.
This framework provides a large class of models, including linear factor models for continuous data, as well as logistic and Poisson models for multivariate binary and count data, as special cases. These special cases are listed below.








  
  * Linear factor model: 
    Y_ij|_i, _j ∼ N(_j^⊤_i, σ^2),
 where the  scale parameter ϕ = σ^2.
  
  * Multidimensional Item Response Theory (MIRT) model: 
    Y_ij|_i, _j ∼ Bernoulli(exp(_j^⊤_i)/1+exp(_j^⊤_i)),
 where the scale parameter ϕ = 1.
  
  * Poisson factor model: 
    Y_ij|_i, _j ∼ Poisson(exp(_j^⊤_i)),
 where the scale parameter ϕ = 1.














































The analysis of this paper is based on the joint likelihood function (<ref>) where both _is and _js are treated as fixed effects, though in the literature the person parameters _i are often treated as random effects and integrated out in the likelihood function <cit.>.
This fixed effect point of view allows us to straightforwardly treat the measurement problem as an estimation problem.



For ease of exposition, we assume the scale parameter is known in the rest of the paper, while pointing out that
it is straightforward to extend all the results to the case where it is unknown. 





















 §.§ Confirmatory Structure


In this paper, we consider a confirmatory setting where the relationship between the manifest variables and the latent factors is known a priori. Suppose that there are J manifest variables and K latent factors. Then the confirmatory information is recorded by a J× K matrix, denoted by Q = (q_jk)_J× K, whose entries take value zero or one. In particular, q_jk = 0 means that manifest variable j is not directly associated with latent factor k. Such design information is often available in many applications of latent factor models, including in education, psychology, economics, and marketing <cit.>.


The design matrix Q incorporates domain knowledge into the statistical analysis by imposing zero constraints on model parameters. For the generalized latent factor model, the loading parameter a_jk is constrained to zero if q_jk is zero.
The constraints induced by the design matrix play an important role in the identifiability and the interpretation of the latent factors.
Intuitively, suitable zero constraints on loading parameters will anchor the latent factors by preventing rotational indeterminacy,
a major problem for the identifiability of latent factor models. In the rest of this paper, we formalize this intuition by studying how the design matrix Q affects the identifiability and estimability of generalized latent factor models.


 

















 
































 §.§ A Summary of Main Results


In this paper, we investigate how design information given by _js affects the quality of measurement. This problem is tackled through both asymptotic analysis and a non-asymptotic error bound, under the generalized latent factor modeling framework.

Our asymptotic analysis focuses on the identifiability and estimability of the latent factors, under a setting where both N and J grow to infinity.


To define identifiability, consider a population of people where N = ∞
and a universe of manifest variables where J = ∞.
A latent factor k is a hypothetical construct, defined by the person population.
More precisely, it is determined by
the individual latent factor scores of the entire person population, denoted by
(θ_1k^*, θ_2k^* ...) ∈ℝ^ℤ_+, where θ_ik^* denotes the true latent factor score of person i on latent factor k and ℝ^ℤ_+ denotes the set of vectors with countably infinite real number components.

The identifiability
of the kth latent factor then is equivalent to the identifiability of
a vector in ℝ^ℤ_+ under the distribution of an infinite dimensional random matrix, {Y_ij: i = 1, 2, ..., j = 1, 2, ...}. This setting is natural in the context of large-scale measurement where both N and J are large.



Under the above setting, this paper addresses three research questions.
First, how should the identifiability of latent factors be suitably formalized? Second, under what design
are the latent factors identifiable? Third, what is the relationship between the identifiability and estimability? In other words,  whether and to what extend can we recover the scores of an identifiable latent factor from data?


We further provide a non-asymptotic error bound to complement the asymptotic results. For finite N and J, the effect of design information on the estimation of a latent factor
is reflected by a multiplying coefficient in the bound. 





























 §.§ Preliminaries


In this section, we fix some notations used throughout this paper.



  
Notations.


  
  * : the set of all positive integers.
  
  * ^: the set of vectors with countably infinite real number components.
  
  
  * ^×{1, ..., K}: the set of all the real matrices with countably infinite rows and K columns.
  
  * {0, 1}^×{1, ..., K}: the set of all the binary matrices with countably infinite rows and K columns.
  
  * Θ: the parameter matrix for the person population, Θ∈^×{1, ..., K}.
  
  * A: the parameter matrix for the manifest variable population, A ∈^×{1, ..., K}.
  
  * Q: the design matrix for the manifest variable population, Q ∈{0, 1}^×{1, ..., K}.
  
  * 0: the vector or matrix with all components being 0.
  
  * P_Θ,A: the probability distribution of (Y_ij, i,j∈ℤ_+), given person and item parameters Θ and A.
  
  * 𝐯_[1:m]: the first m components of  a vector 𝐯.
  
  * W_[S_1,S_2]: the submatrix of a matrix W formed by rows S_1 and columns S_2,   where S_1, S_2 ⊂.
  
  * W_[1:m, k]: the first m components of the k-th column of a matrix W.
  
  * W_[k]: the k-th column of a matrix W.
  
  * ‖𝐯‖: the Euclidian norm of a vector 𝐯.
  
  * sin∠(𝐮, 𝐯): the sine of the angle between two vectors,

    sin∠(𝐮, 𝐯) =  √(1 - (𝐮^⊤𝐯)^2/‖𝐮‖^2 ‖𝐯‖^2),
 
        where 𝐮, 𝐯∈^m, 𝐮, 𝐯≠0. We point out that the angles between vectors are assumed to belong to [0,π] and sin∠(𝐮, 𝐯)≥ 0 for all vectors 𝐮, 𝐯.
        
  
  * ‖ W‖_F: the Frobenius norm of a matrix W = (w_ij)_m× n, ‖ W‖_F ≜√(∑_i=1^m ∑_j = 1^n w_ij^2).
  
  * ‖ W‖_2: the spectral norm of matrix W, i.e., the largest singular value of matrix.
  
  * σ_1(W)≥σ_2(W)≥...≥σ_n(W): the singular values of a matrix W∈^m× n, in a descending order.
  
  * γ(W): a function mapping from ^× n to , defined as

    γ(W) ≜lim inf_m→∞σ_n(W_[1:m,1:n])/√(m).

  
  * |S|: the cardinality of a set S.
  
  * N∧ J: the minimum value between N and J.

 













































































§ MAIN RESULTS




 §.§ Structural Identifiability

We first formalize the definition of structural identifiability.
For two vectors with countably infinite components
𝐰=(w_1,w_2,...)^⊤, 𝐳=(z_1,z_2,...)^⊤∈ℝ^ℤ_+,
we define


    ∠(𝐰,𝐳)=lim sup_n→∞sin∠(𝐰_[1:n], 𝐳_[1:n]),




which quantifies the angle between two vectors 𝐰 and 𝐳 in ℝ^ℤ_+. In particular, we say the angle between 𝐰 and 𝐳 is zero when  ∠(𝐰, 𝐳) is zero.



Consider the kth latent factor, where k ∈{1, ..., K},
and  a nonempty parameter space ⊂^×{1, ..., K}×^×{1, ..., K} for (Θ, A).
We say the k-th latent factor is structurally identifiable in the parameter space  if for any (Θ,A), (Θ',A')∈, P_Θ,A=P_Θ',A' implies ∠(Θ_[k],Θ'_[k])=0.

We point out that the parameter space , which will be specified later in this section, is essentially determined by the design information q_jks.
As will be shown shortly,
a good design imposes suitable constraints on the parameter space, which further ensures the structure identifiability of the latent factors.

This definition of identifiability avoids the consideration of the scale of the latent factor, which is not uniquely determined as the distribution of data only depends on {_i^⊤_j: i,j∈}.
Moreover, the sine measure is a canonical way to quantify the distance between two linear spaces that has been used in, for example, the well-known sine theorems  for matrix perturbation <cit.>.
As will be shown in the sequel, this definition of structural identifiability naturally leads to a relationship between identifiability and estimability and has  important implications on psychological measurement.

We now characterize the structural identifiability under suitable regularity conditions.
We consider a design matrix Q for the manifest variable population, where
Q∈{0,1}^×{1, ..., K}.

	Our first regularity assumption is about the stability of the Q matrix.

	
  A1 The limit

    p_Q(S)=lim_J→∞|{j:  q_jk = 1,  k ∈ S   q_jk = 0,  k ∉ S, 1≤ j≤ J}|/J
 exists for any subset
S⊂{1,...,K}. In addition, p_Q(∅)=0.






The above assumption requires that the frequency of manifest variables associated with and only with latent factors in S converges to a limit proportion p_Q(S).


In addition, p_Q(∅)=0 implies that there are few irrelevant manifest variables.
We point out that this assumption is adopted mainly to simplify the statements in Theorem <ref> and Theorem <ref>. As discussed later, this assumption is automatically satisfied if _js are  generated under a stochastic design. This assumption will be further relaxed in Section <ref> where a non-asymptotic error bound is established.

We also make the following assumption on the generalized latent factor model, which is satisfied under most of the widely used models,  including the linear factor model, MIRT model, and the Poisson factor model listed above.

		
  A2 The natural parameter space {ν: |b(ν)|<∞}=.



Under the above assumptions, Theorem <ref> provides a necessary and sufficient condition on the design matrix Q for the structural identifiability of the kth latent factor.

This result is established within the parameter space
 _Q⊂ℝ^ℤ_+×{1, ..., K}×ℝ^ℤ_+×{1, ..., K},
 
    _Q= 	_Q^(1)×_Q^(2).

Here, we define
 
    _Q^(1)={Θ∈ℝ^ℤ_+×{1, ..., K}: _i≤ C  and γ(Θ)>0
     	}

 and
 
    _Q^(2)={
    A∈ℝ^ℤ_+×{1, ..., K}:  _j≤ C,  A_[R_Q(S),S^c]=0 S ⊂{1, ..., K},
     and γ(A_[R_Q(S),S])>0  for all  S,  p_Q(S)>0
     	},












where C is a positive constant, the γ function is defined in (<ref>), and






	
    R_Q(S)={
    	j: q_jk' = 1,  k' ∈ S   q_jk' = 0,  k' ∉ S
    	}


denotes the set of manifest variables that are associated with and only with latent factors in S.
Discussions on the parameter space are provided after the statement of Theorem <ref>.




Under Assumptions A1 and A2, the k-th latent factor is structurally identifiable
in _Q
if and only if

    {k}=⋂_k∈ S, p_Q(S)>0 S,

where we define ⋂_k∈ S, p_Q(S)>0 S=∅ if p_Q(S)=0 for all S that contains k.


The following proposition guarantees that the parameter space is nontrivial.

	For any Q satisfying A1, _Q≠∅.

We further remark on the parameter space _Q.
First, _Q requires some regularities on each _i and _j (i.e., _i≤ C,_j≤ C) and the A-matrix satisfying the constraints imposed by Q (A_[R_Q(S),S^c]=0) for all S. It further requires that there is enough variation among people, quantified by
γ(Θ)>0, where the γ function is defined in (<ref>). Note that this requirement is mild, in the sense that
if _is are independent and identically distributed (i.i.d.) with a strictly positive definite covariance matrix, then  γ(Θ)>0 a.s., according to the strong law of large numbers. Furthermore, γ(A_[R_Q(S),S])>0  for  S  p_Q(S)>0 requires that each type of manifest variables (categorized by S)
contains sufficient information if appearing frequently (p_Q(S)>0). Similar to the justification for Θ,
γ(A_[R_Q(S),S])>0 can
also be justified by considering that _js are i.i.d. following a certain distribution for j ∈ R_Q(S).


We provide an example to facilitate the understanding of Theorem <ref>. Suppose that assumptions A1 and A2 hold.
If K=2 and p_Q({
	1})= p_Q({1,2})=1/2, then the second latent factor is not structurally identifiable, even if it is associated with infinitely many  manifest variables. In addition, having many manifest variables with a simple structure ensures the structural identifiability of a latent factor. That is, if p_Q({k}) > 0, then the kth factor is structurally identifiable.

Finally, we briefly discuss Assumption A1.


The next proposition implies that if we have a stochastic design where Q has i.i.d. rows, then Assumption A1 is satisfied almost surely.


Assume that the design matrix Q has i.i.d. rows and _i≠ (0,...,0) for all i=1,2,...

Then, Assumption A1 is satisfied almost surely. Moreover, let

    w_S=P(q_1k=1  for all  k∈ S,  and q_1k=0  for all  k∉ S)

for each S⊂{1,...,K}.
Then, Q satisfies Assumption A1 with p_S(Q)=w_S for all S⊂{1,...,K} almost surely.








































































































 §.§ Identifiability and Estimability

It is well known that for a fixed dimensional parametric model with i.i.d. observations, the
identifiability of a model parameter is necessary for the existence of a consistent estimator.
We extend this result to the infinite-dimensional parameter space under the current setting.
We start with a generalized definition for the consistency of estimating a latent factor.
An estimator given N individuals and J manifest variables is denoted by
(Θ̂^(N,J),Â^(N,J)), which only depends on Y_[1:N,1:J] for all N,J∈ℤ_+.









The sequence of estimators
{
(Θ̂^(N,J),Â^(N,J)), N,J∈ℤ_+
} is said to
consistently estimate the latent factor k if




    sin∠(Θ̂^(N,J)_[k],Θ_[1:N,k])P_Θ,A→ 0,  as  N,J→∞,

for all (Θ,A)∈_Q.


The next proposition establishes the necessity of the structural identifiability of a latent factor on its estimability.

	If latent factor k is not structurally identifiable in _Q, then there does not exist a consistent estimator for latent factor k.




The above results of identifiability and estimability are all established under
a double asymptotic regime that both N and J grow to infinity, which
is suitable for large-scale applications where both N and J are large. It differs from the classical asymptotic setting <cit.> where J is fixed and N grows to infinity.
The classical setting treats the latent factors as random effects and focuses on the identifiability of the loading parameters.
There are several reasons for adopting the double asymptotic regime. First, the double asymptotic regime allows us to directly focus on the
identifiability and the estimation of factor scores _i, which is of more interest in many applications (e.g, psychological and educational measurement) than the loading parameters. To consistently estimate the factor scores, naturally, we need the number of measurements J to grow to infinity. Second, having a varying J allows us to take the number of manifest variables into consideration in the research design and data collection  when measuring certain traits with substantive meaning through a latent factor model. Third, even if we only focus on the loading parameters, the classical asymptotic regime does not lead to consistency on the estimation of the loading parameters.

For example, for the MIRT model for binary data, one cannot consistently estimate the loading parameters, when J is fixed and N grows to infinity, unless we assume _is to be i.i.d. samples from a certain parametric distribution and use this parametric assumption in the estimation procedure. Finally, this double asymptotic regime is not entirely new. In fact, this regime has been adopted in, for example, <cit.>, <cit.>, and <cit.> among others,
for the asymptotic analysis of linear and nonlinear factor models.


















 §.§ Estimation and Its Consistency



We further show that the structural identifiability and estimability are equivalent under our setting. For ease of exposition, let Q∈{0, 1}^×{1, ..., K} be a design matrix satisfying Assumption A1.
In addition, let (Θ^*,A^*)∈_Q be the true parameters for the person and the manifest variable populations. We provide
an estimator (Θ̂^(N, J),Â^(N, J))
such that

    sin∠ (Θ̂_[k]^(N, J), Θ_[1:N,k]^*)  P_Θ^*,A^*→ 0,    N, J, →∞,

when Q satisfies (<ref>) which leads to
the structural identifiability of latent factor k

according to Theorem <ref>.
Specifically, we consider the following estimator

    (Θ̂^(N, J),Â^(N, J))∈    - l(_1, ..., _N, _1, ..., _J),
    
    s.t.    _i≤ C', _j≤ C',
    
        _j ∈𝒟_j, i = 1, ..., N, j = 1, ..., J,


where l(_1, ..., _N, _1, ..., _J)= ∑_i=1^N∑_j=1^Jy_ij(_i^⊤_j) -b(_i^⊤_j),
C' is any constant greater than C in the definition of _Q,
and 𝒟_j = {∈^K: a_jk= 0  q_jk = 0} imposes the constraint on _j. Note that maximizing l(_1, ..., _N, _1, ..., _J) is equivalent to maximizing the joint likelihood (<ref>), due to the natural exponential family form.



















The next theorem provides an error bound on (Θ̂^(N, J),Â^(N, J)).




	Under assumptions A1-A2 and (Θ^*, A^*)∈_Q, there exists a constant  (independent of N and J, depending on the function b, the constant C', and K) such that,
	
    1/√(NJ)EΘ̂^(N, J)(Â^(N, J))^⊤-Θ^*_[1:N, 1:K] A^*⊤_[1:J, 1:K]_F≤/√(N∧ J).

Moreover, if Q satisfies
(<ref>) and thus latent factor k is structurally identifiable,
then there exists >0 such that

    E sin∠(Θ^*_[1:N, k],Θ̂_[k]^(N, J)) ≤/√(N∧ J).

	

Proposition <ref>, and Theorems <ref> and <ref> together imply that the structural identifiability and estimability over _Q are equivalent, which is
 summarized in the following corollary.
	
		Under Assumptions A1 and A2, there exists an estimator (Θ̂^(N, J),Â^(N, J)) such that
		lim_N,J→∞sin∠(Θ̂_[k]^(N, J),Θ_[1:N,k])=0 in P_Θ,A for all (Θ,A)∈_Q if and only if the design matrix Q satisfies (<ref>).
	



The estimator (<ref>) is closely related to the joint likelihood estimator in the literature of econometrics and psychometrics.
When J is fixed and N grows to infinity, this estimator is shown to be inconsistent, due to the simultaneous growth of the sample size and the parameter space <cit.>. However, if N and J simultaneously grow to infinity, <cit.> shows that the joint maximum likelihood estimator is consistent under the Rasch model, a
simple unidimensional item response theory model.  Then under a general family of multidimensional item response theory models and under an exploratory factor analysis setting, an estimator similar to (<ref>) is considered in <cit.> but without the zero constraints given by the Q-matrix.  As a result, the estimator obtained in <cit.> is rotationally indeterminant.










	
   	The error bound (<ref>) holds even when one or more latent factors are not structurally identifiable. In particular, (<ref>) holds when removing the constraint _j ∈𝒟_j from (<ref>), which corresponds to the exploratory factor analysis setting where no design matrix Q is pre-specified <cit.>. 
   In that case, the best one can achieve is to recover the linear space spanned by the column vectors of Θ^* and similarly
the linear space spanned by the column vectors of A^*. To make sense of such exploratory factor analysis results, one needs an additional rotation step to find an approximately sparse estimate of A^* <cit.>.



	


The proposed estimator (<ref>) and its error bound are related to exact low-rank matrix completion <cit.> and approximate low-rank matrix completion <cit.>, where a  bound similar to (<ref>) can typically be derived.
The key differences are (a)
the research on matrix completion is only interested in the estimation of Θ^* A^*⊤, while the current paper focuses on the estimation of Θ^* that is a fundamental problem of psychological measurement and
(b) our results are derived under a more general family of models.





To further evaluate the efficiency of the proposed estimator,

we provide the following lower bounds. 

Suppose that Assumptions A1 and A2 hold.
For N, J ∈, let M̅^(N,J) be an arbitrary estimator which maps data Y_[1:N,1:J] to ℝ^N× J.


Then there exists >0 and N_0,J_0>0 such that for N≥ N_0, J≥ J_0, there exists (Θ^*,A^*)∈𝒮_Q such that

    P_Θ^*,A^*(1/√(NJ)M̅^(N,J)-Θ^*_[1:N, 1:K] (A^*_[1:J, 1:K])^⊤_F≥/√(N∧ J))≥1/2.

Moreover, let Θ̅^(N,J)_[k] be an arbitrary estimator which maps data Y_[1:N,1:J] to ℝ^N.

Then for each A^*∈𝒮_Q^(2) defined in (<ref>), there exists >0 and N_0,J_0>0 such that for N≥ N_0, J≥ J_0, there exists Θ^*∈𝒮_Q^(1) such that

    P_Θ^*,A^*(sin∠(Θ̅^(N, J)_[k],Θ^*_[1:N, k])≥/√(J))≥1/2.







Based on the asymptotic upper bounds and lower bounds obtained in Theorems <ref> and <ref>, we have the following findings. First, the proposed estimator Θ̂^(N, J)(Â^(N, J))^⊤ is rate optimal for the estimation of Θ^*_[1:N, 1:K] (A^*_[1:J, 1:K])^⊤, because the upper bound (<ref>) matches the lower bound (<ref>). Second, for estimating Θ^*_[1:N,k], the proposed estimator Θ̂_[k] is rate optimal when
lim sup_N, J→∞ J/N  < ∞. This assumption on the rate J/N seems reasonable in many
applications of confirmatory generalized latent factor models, as N is typically larger than J.
Finally, if N=o(J), then the asymptotic upper bound (<ref>) and lower bound (<ref>) do not match, in which case the proposed estimator may not be rate optimal. In particular, when N=o(J), it is known that the lower bound for the estimation of Θ^*_[1:N,k] can be achieved under a linear factor model <cit.>. However, this lower bound may not be achievable under a generalized latent factor model for categorical data, such as the MIRT model for binary data.
This problem is worth further investigation.


We end this section by providing an alternating minimization algorithm (Algorithm <ref>) for solving the optimization program (<ref>), which is computationally efficient through our parallel computing implementation using Open Multi-Processing
<cit.>.
Specifically, to handle the constraints,
we adopt a projected gradient descent algorithm <cit.> for solving (<ref>) and (<ref>) in each iteration, where the projections have closed-form solutions.  Similar algorithms have been considered in other works, such as
<cit.>, <cit.> and <cit.>,
for solving optimization problems with respect to low-rank matrices. Following Corollary 2 of <cit.>, we obtain Lemma <ref> below on the convergence property of the proposed algorithm.




Any limit point of the sequence {_i^(l),_j^(l), i=1,...,N, j=1,...,J, l=1,2,...} obtained from Algorithm <ref> is a critical point of the optimization program (<ref>).




















































































 §.§ A Non-asymptotic Error Bound


We further provide a non-asymptotic error bound as a complement to the asymptotic results. Through this error bound,  the effect of design information on the estimation of latent factors is quantified for finite N and J, without requiring N and J to grow to infinity.





We introduce the following definition on collections of manifest varible types.

	For a given k⊂{1,...,K}, we say a collection of subsets of {1,...,K}, 𝒜, is feasible  for the kth dimension if
	
    {k}=⋂_S∈𝒜 S.





The concept of feasible collection is closely related to the necessary and sufficient condition (<ref>) for the structural identifiability of the kth factor. 


Given the concept of feasible collection of manifest variable types, we then define an index as follows,

    σ_N,J=min{max_𝒜: 𝒜 is feasiblemin_S∈𝒜σ_|S|( A^*_[R_Q(S)∩{1,...,J},S] )
    	/√(J),σ_K(Θ^*_[1:N,1:K])/√(N)},

which serves as a measure of the signal strength on dimension k.
We elaborate on this index. First, for a given feasible collection 𝒜, the first quantity in the large brackets of (<ref>), min_S∈𝒜σ_|S|( A^*_[R_Q(S)∩{1,...,J},S] )/√(J), measures the amount of information contained in manifest variables associated with the set of factors S∈𝒜. In particular, if for any feasible collection 𝒜, there exists an S ∈𝒜 such that manifest variables of type S do not exist, then this quantity becomes zero and thus σ_N, J = 0 as the second term in the brackets of (<ref>) is nonnegative. In that case, the manifest variables essentially contain no information about the kth dimension. Second, the second term in the brackets, σ_K(Θ^*_[1:N,1:K])/√(N), measures the co-linearity of different dimensions of Θ^*_[1:N,1:K].
In summary, σ_N,J is a nonnegative index, with
a larger value suggesting that Θ^*_[1:N,k] is easier to estimate.
An error bound is then established in the next theorem.



Suppose that Assumption A2 holds and _i^*≤ C, _j^*≤ C for all 1≤ i≤ N and 1≤ j≤ J.
Then, there is a constant , independent of N and J, such that
	
    1/√(NJ)EΘ̂^(N, J)(Â^(N, J))^⊤-Θ^*_[1:N, 1:K] A^*⊤_[1:J, 1:K]_F≤/√(N∧ J).


Moreover,  if σ_N,J> 4^1/2(N∧ J)^-1/2,  then there exists a monotone decreasing function (·):R_+→ R_+ (independent of N,J and may depend on C', K and function b) such that for N≥ N_0 and J≥ J_0,

    E sin∠(Θ^*_[1:N, k],Θ̂_[k]^(N, J)) ≤(σ_N,J) /√(N∧ J).

The precise forms of the constant  and the function (·) are given in the Appendix.




The above theorem does not require N and J to grow to infinity and thus is referred to as a non-asymptotic result.
Comparing with Theorem <ref>, the assumptions of Theorem <ref> are weaker.
That is,
Theorem <ref> does not require any limit-type assumptions as required in Theorem <ref>, including Assumption A1, the requirement (Θ^*,A^*)∈𝒮_Q, and condition (<ref>).
Instead, it quantifies the effect of
design information on the estimation of latent factors 
under a non-asymptotic setting, only requiring the signal level σ_N,J is higher than the noise level 4^1/2(N∧ J)^-1/2. According to the proposition below, this assumption can be implied by the limit-type assumptions of Theorem <ref> and thus is weaker.


If Assumption A1 and condition (<ref>) are satisfied and (Θ^*,A^*)∈𝒮_Q, then
	
    lim inf_N,J→∞σ_N,J>0.







































§ FURTHER IMPLICATIONS


In this section, we discuss the implications of the above results in the applications of generalized latent factor models to large-scale educational and psychological measurement. In such applications, each manifest variable typically corresponds to an item in a test, each
individual is a test-taker,  and both N and J are large. The latent factors are often interpreted as individuals' cognitive abilities, psychological traits, etc., depending on the context of the test. It is often of interest to
estimate the latent factor scores _i.





 §.§ On the Design of Tests.

According to Theorems <ref> and <ref>, the key to the structural identifiability and consistent estimation of factor k is

    {k}=⋂_k∈ S, p_Q(S)>0 S,

which provides insights on the measurement design. First, it implies that the “simple structure" design advocated in psychological measurement is a safe design. Under the simple structure design, each manifest variable is associated with one and only one factor. If each latent factor k is associated with many manifest variables that only measure factor k, or more precisely p_Q({k}) > 0, (<ref>) is satisfied.

Second, our result implies that a simple structure is not necessary for a good measurement design. A latent factor can still be identified even when it is always measured together with some other factors. For example, consider the Q-matrix in Table <ref>. Under this design,
all three factors satisfy (<ref>) even when there is no item measuring a single latent factor.

Third, (<ref>) is not satisfied when  there exists a k' ≠ k and k'∈⋂_k∈ S, p_Q(S)>0 S.
That is,
almost all manifest variables that are associated with factor k are also associated with factor k', in the asymptotic sense.
Consequently, one cannot distinguish factor k from factor k', making factor k structurally unidentifiable. We point out that in this case, factor k' may still be structurally identifiable, as p_Q({k'}) > 0 is still possible.


Finally, (<ref>) is also not satisfied when ⋂_k∈ S, p_Q(S)>0 S=∅.
 It implies that the factor k is not structurally identifiable when the factor is not measured by a sufficient number of manifest variables.











 §.§ Properties of Estimated Factor Scores




  
A useful result.

Let (Θ^*,A^*)∈_Q be the true parameters for the person and the manifest variable populations. We start with a lemma connecting sine angle consistency and L_2 consistency.

Let ,'∈ R^n with 𝐰≠0 and 𝐰'≠0 and c=sign(cos∠(,')).
Then,

    /-c'/'^2=2-2√(1-sin^2∠(,')).


Combining Theorem <ref> and Lemma <ref>, we have the 
following corollary which

establishes a relationship between the true person parameters and their estimates.
This result is the key to the rest of the results in this section.


  Under Assumption A1-A2 and (<ref>) is satisfied for some k,
  then 

there exists a sequence of random variables c_N,J∈{-1,1}, such that
		
    Θ^*_[1:N, k]/Θ^*_[1:N, k]-c_N,JΘ̂^(N, J)_[k]/Θ̂^(N, J)_[k]P_Θ^*, A^*→ 0,    N, J, →∞.




 Corollary <ref> follows directly from (<ref>). It provides an alternative view on how Θ̂_[k]^(N, J) approximates Θ^*_[1:N, k]. Since the likelihood function depends on Θ_[1:N, 1:K] and A_[1:J, 1:K] only through Θ_[1:N, 1:K] A_[1:J, 1:K]^⊤, the scale of Θ_[1:N, k] is not identifiable even when it is structurally identifiable. This phenomenon is intrinsic to latent variable models <cit.>. Corollary <ref> states
 
 that Θ^*_[1:N, k]  and Θ̂_[k]^(N, J)  are close in Euclidian distance after properly normalized.
 
 The
  normalized vectors Θ^*_[1:N, k]/‖Θ^*_[1:N, k]‖ and c_N,JΘ̂_[k]^(N, J)/‖Θ̂_[k]^(N, J)‖ are both of unit length. The value of c_N,J depends on the angle between Θ^*_[1:N, k] and Θ̂_[k]^(N, J). Specifically, c_N,J = 1 if cos∠(Θ^*_[1:N, k], Θ̂_[k]^(N, J)) > 0 and
 c_N,J = -1 otherwise.  In practice, especially in psychological measurement,  c_N,J  can typically be determined by additional domain knowledge.







  
On the distribution of person population. In psychological measurement, the distribution of true factor scores
is typically of interest, which may provide an overview of the population on the constructs being measured.
Corollary <ref> implies the following proposition on the empirical distribution of the factor scores.


Suppose assumptions A1-A2 are satisfied
and furthermore (<ref>) is satisfied  for factor k.  We normalize
θ_ik^* and θ̂_ik^(N, J) by

    v_i=√(N)θ_ik^*/Θ^*_[1:N, k] and v̂_i= c_N,J√(N)θ̂_ik^(N, J)/Θ̂_[k]^(N, J),    i=1,...,N,

where c_N,J is defined and discussed in Corollary <ref>.
Let F_N and F̂_N,J be the empirical measures of v_1,...,v_N and v̂_1,...,v̂_N, respectively.





Then,
			
    Wass(F_N,F̂_N,J) P_Θ^*,A^*→  0,    N, J, →∞,
		where Wass(·,·) denotes the Wasserstein distance between two probability measures
			
    Wass(μ,ν)=sup_h is 1-Lipschitz|∫ h dμ-∫ hdν|.





We point out that the normalization in (<ref>) is reasonable. Consider a random design setting where
θ^*_iks are i.i.d. samples from some distribution with a finite second moment. Then F_N converges weakly to the distribution of η / √(Eη^2), where η is a random variable following the same distribution.
Proposition <ref> then implies that when factor k is structurally identifiable and both N and J are large,
the empirical distribution of θ̂_1k^(N, J), θ̂_2k^(N, J), ..., θ̂_Nk^(N, J)
approximates the empirical distribution of θ^*_1k, θ^*_2k, ..., θ^*_Nk accurately, up to a scaling. Specifically, for any 1-Lipschitz function h,
∫ h(x)F̂_N, J(dx) is a consistent estimator for ∫ h(x)F_N(dx) according to the definition of Wasserstein distance. Furthermore, Corollary <ref> states that under the regularity conditions, lim_N→∞∑_i=1^N (v_i - v̂_i)^2/N = 0, implying that   ∑_i=1^N 1_{(v_i - v̂_i)^2 ≥ϵ}/N = 0, for all ϵ > 0. That is, most of the v̂_is will fall into a small neighborhood of the corresponding v_is.




  
On ranking consistency. The estimated factor scores may also be used to rank individuals along a certain trait. In particular, in educational testing, the ranking provides an ordering of the students' proficiency in a certain ability (e.g., calculus, algebra, etc.). Our results also imply the validity of the ranking along a latent factor when it is structurally identifiable and N and J are sufficiently large. More precisely, we have the following proposition.



Suppose assumptions A1-A2 are satisfied
and furthermore (<ref>) is satisfied  for factor k.
Consider v_i and v̂_i, the normalized versions of θ_ik^* and θ̂_ik^(N, J) as defined in (<ref>).
In addition, assume that there exists a constant κ_R such that for any sufficiently small ϵ>0 and sufficiently large N,
		
    ∑_i≠ i' I{| v_i-v_i'|≤ϵ}/N(N-1)/2≤κ_R ϵ.

		Then,
		
    τ(𝐯,𝐯̂)/N(N-1)/2P_(Θ^*,A^*)→  0,    N, J, →∞,

		where τ(𝐯,𝐯̂)=∑_i≠ i'I(v_i>v_i',v̂_i<v̂_i')+I(v_i<v_i',v̂_i>v̂_i') is the number of inconsistent pairs according to the ranks of 𝐯 = (v_1, ..., v_N) and 𝐯̂ = (v̂_1, ..., v̂_N).


We point out that (<ref>) is a mild regularity condition on the empirical distribution F_N. It requires that the probability mass under F_N does not concentrate in any small ϵ-neighborhood, which further implies that the pairs of individuals who are difficult to distinguish along factor k, i.e., (i, i')s that v_i and v_i' are close, take only a small proportion among all the (N-1)N/2 pairs. In fact, it can be shown that (<ref>) is true with probability tending to 1 as N grows to infinity, when θ_ik^*s are i.i.d. samples from a distribution with a bounded density function.
Proposition <ref> then implies that if we rank the individuals using v̂_i (assuming c_N,J can be consistently estimated based on other information), the proportion of incorrectly ranked pairs converges to 0.
Note that  τ(𝐯,𝐯̂) is known as the Kendall's tau distance <cit.>, a widely used measure for ranking consistency.




  
On classification consistency. Another common practice of utilizing estimated factor scores is to classify individuals into two or more groups along a certain construct. 
For example, in an educational mastery test, it is of interest to classify examinees into “mastery" and “nonmastery" groups according their proficiency in a certain ability <cit.>. In measuring psychopathology, it is common to classify respondents into “diseased" and “non-diseased" groups based on a mental health disorder. We justify the validity of making classification based on the estimated factor score.



Suppose assumptions A1-A2 are satisfied
and furthermore (<ref>) is satisfied  for factor k.
Consider v_i and v̂_i, the normalized versions of θ_ik^* and θ̂_ik^(N, J) as defined in (<ref>).
Let τ_-<τ_+ be the classification thresholds, then

    ∑_i=1^N I{v̂_i≥τ_+, v_i≤τ_-  } + I{v̂_i≤τ_-, v_i ≥τ_+ }/NP_Θ^*,A^*→  0,    N, J, →∞,.






Considering two pre-specified thresholds τ_- and τ_+ is the well-known indifference zone formulation of educational mastery test <cit.>. In that context, examinees with v_i ≥τ_+ are classified into the “mastery" group and those with v_i≤τ_- are classified into the “nonmastery" group.
The interval  (τ_-, τ_+) is known as the indifference zone, within which no decision is made.
Proposition <ref> then implies that when factor k is structurally identifiable, the classification error tends to 0 as both N and J grow to infinity.











































§ EXTENSIONS




 §.§ Generalized Latent Factor Models with Intercepts

As mentioned in Section <ref>, intercepts can be easily incorporated into the generalized latent factor model by restricting θ_i1=1. Then, a_j1s are the intercept parameters and q_j1=1 for all j.

Consequently,  for any S satisfying p_Q(S) > 0, 1∈ S and thus
the latent factors 2-K are not structurally identifiable according to Theorem <ref>.

Interestingly, these factors are still structurally identifiable if we
restrict to the following parameter space

    _Q,-
    	=
    	{   
    	(Θ,A)∈_Q:
    lim_N→∞1/N1_N^⊤Θ_[1:N,m]=0  for  m≥ 2,  and θ_i1=1  for  i∈},

which requires that Θ_[k] and Θ_[1] are asymptotically orthogonal, for all k ≥ 2.


Under Assumptions A1-A2, and assuming that q_j1=1 for all j∈ and K≥ 2,
then
	the kth latent factor is structurally identifiable in _Q,- if and only if
	
    {1, k}=⋂_k∈ S, p_Q(S)>0 S,

	for k≥ 2.
	


The next proposition guarantees that _Q,- is also non-empty.

	For all Q satisfying A1 and q_j1=1 for all j∈, and in addition C>1, then _Q,-≠∅.



	
		When having intercepts in the model, similar results of consistency and a non-asymptotic error bound can be established for the estimator


    (Θ̂^(N, J),Â^(N, J))∈    - l(_1, ..., _N, _1, ..., _J),
    
    s.t.    _i≤ C', _j≤ C', _j ∈𝒟_j, 
       θ_i1 = 1, ∑_i'=1^N θ_i'k = 0,
        i = 1, ..., N, j = 1, ..., J, k = 2, ..., K.








 §.§ Extension to Missing Values


Our estimator can also handle missing data which are often encountered in practice.
Let Ω = (ω_ij)_N× J be the indicator matrix of nonmissing values, where
ω_ij = 1 if Y_ij is observed and ω_ij = 0 if Y_ij is missing. When data are completely missing at random, the joint likelihood function becomes

    L^Ω(_1, ..., _N, _1, ..., _J, ϕ) = ∏_i,j: ω_ij = 1exp(y_ij m_ij - b(m_ij)/ϕ + c(y_ij, ϕ))

and our estimator becomes

    (Θ̂^(N, J),Â^(N, J))∈    - l^Ω(_1, ..., _N, _1, ..., _J),
    
    s.t.    _i≤ C', _j≤ C',
    
        _j ∈𝒟_j, i = 1, ..., N, j = 1, ..., J,

where l^Ω(Θ A^⊤)= ∑_i, j: ω_ij = 1 y_ijm_ij-b(m_ij).
Moreover, results similar to Theorem <ref> can be established even when having missing data.
Specifically, we assume

  
  A3 ω_ijs in Ω are independent and identically distributed Bernoulli random variables with

    P(ω_ij = 1) = n/NJ.


This assumption implies that data are completely missing at random and only about n entries of (Y_ij)_N× J
are observed. We have the following result.


Under assumptions A1-A3 and (Θ^*, A^*)∈_Q, there exists >0 such that,

	
    1/√(NJ) E(Θ̂^(N, J)(Â^(N, J))^⊤-Θ^*_[1:N, 1:K] (A_[1:N, 1:K]^*)^⊤_F)
    ≤   max{√(N∨ J/n),(NJ)^1/2/n^3/4}.

Moreover, if Q  satisfies
(<ref>) and thus latent factor k is structurally identifiable,
then  there exists a constant , such that
	
    E sin∠(Θ^*_[1:N, k],Θ̂_[k]^(N, J)) ≤max{√(N∨ J/n),(NJ)^1/2/n^3/4}.




Results similar to (<ref>) have also
been derived in the matrix completion literature <cit.> under specific statistical models with an underlying low rank structure.
Proposition <ref> extends the existing results on matrix completion to a generalized latent factor model.














§ A USEFUL PERTURBATION BOUND ON LINEAR SUBSPACE

The standard approach  (see, e.g., <cit.>) for bounding the error of the maximum likelihood estimator is by making use of the strong/weak  convexity of the log-likelihood function.
However, in the generalized latent factor model, the log-likelihood function is not convex in (Θ_[1:N,1:K],A_[1:J,1:K]). Thus, the standard approach is not applicable for proving (<ref>) in Theorem <ref> and similar results on the estimation of Θ_[1:N,k] in Theorem <ref> and Proposition <ref>.

New technical tools are developed to handle this problem. Specifically, there are two major steps for proving Theorem <ref>.
In the first step, we establish (<ref>), which bounds the error on the estimation of Θ_[1:N, 1:K]^* A^*⊤_[1:J, 1:K] as a whole. This step extends the error bound for exact low-rank matrix completion <cit.> under the generalized latent variable model setting. A small  estimation error of Θ_[1:N, 1:K]^* A^*⊤_[1:J, 1:K] implies that the linear space spanned by the column vectors of Θ_[1:N, 1:K]^* can be recovered up to a small perturbation.
In the second step, given the result from the first step and design information, we show that the direction of the vector Θ_[1:N, k]^* can be recovered up to a small perturbation. This step is technically challenging and is tackled by a new perturbation bound for the intersection of linear spaces. This perturbation bound, as introduced below, may be of independent value for the theoretical analysis of low-rank matrix estimation.



Let (W) denote the column space of a matrix W. Under the conditions of Theorem <ref>, the result of (<ref>)
combined with the Davis-Kahan-Wedin sine theorem <cit.> allows us to bound
sin∠((Θ^*_[1:N, S]),(Θ̂^(N, J)_[S])), for any S satisfying p_Q(S) > 0,
where ∠(L,M) denotes the largest principal angle between two linear spaces L and M, i.e.,
sin∠(L,M) = max_𝐮∈ M, 𝐮≠ 0min_𝐯∈ L, 𝐯≠0sin∠(𝐮,𝐯).
Our strategy is to bound 
    sin∠(Θ^*_[1:N, k],Θ̂_[k]^(N, J)) = sin∠((Θ^*_[1:N, k]),(Θ̂_[k]^(N, J)))
 by sin∠((Θ^*_[1:N, 1:K]),(Θ̂^(N, J))) under the assumptions of Theorem <ref>.
Note that (Θ^*_[1:N, k]) = ⋂_k∈ S, p_Q(S) > 0(Θ^*_[1:N, S]) and similarly
(Θ̂_[k]^(N, J)) = ⋂_k∈ S, p_Q(S) > 0(Θ̂_[S]^(N, J)).
Consequently, it remains to show that
if the linear spaces are perturbed slightly,
then their intersection does not change much.
To this end, we establish a new perturbation bound on the intersection of general linear spaces in the next proposition.


Let L, M, L', M' be linear subspaces of a finite dimensional vector space. Then,

    _L'∩ M'-_L∩ M≤  8 max{α(θ_min,+(L,M)),α(θ_min,+( L', M'))
          }(_L-_L'+_M-_M'),

where  we define θ_min,+(L,M) as the smallest positive principal angle between L and M (defined as 0 if all the principal angles are 0), _M denotes the orthogonal projection onto a linear space M, and α(θ)=2(1+cosθ)/(1-cosθ)^3.
 
Here, the norm · could be any unitary invariant, uniformly generated and normalized matrix norm.
In particular, if we take · to be the spectral norm ·_2, then we have

    sin∠(L'∩ M',L∩ M)
    ≤    8 max{α(θ_min,+(L,M)),α(θ_min,+( L', M'))
          }(sin∠(L,L')+sin∠(M,M')).



We refer the readers to <cit.> for more details on principal angles between two linear spaces and on matrix norms. In particular, the spectral norm is a unitary invariant, uniformly generated and normalized matrix norm.
The result in Proposition <ref> holds for all linear subspaces.

The right-hand side in (<ref>) is finite if and only if θ_max,+(L,M)≠ 0 and θ_max,+(L',M')≠0.

In our problem, L=(Θ_[1:N,S_1]) and M=(Θ_[1:N,S_2]) for S_1,S_2⊂{1,...,K} and S_1≠ S_2. The next lemma further bounds
α(θ_min,+(L,M)) when L and M are column spaces of a matrix,  which is a key step in proving (<ref>).


Let W∈^N× K∖{0} for some positive integer N and K, and S_1,S_2⊂{1,...,K} be such that S_1∖ S_2≠∅ and S_2∖ S_1 ≠∅, then

    cos (θ_min,+((W_[S_1]),(W_[S_2])))≤ 1-σ_|S_1∪ S_2|^2(W_[S_1∪ S_2])/W_2^2.

















































































§ NUMERICAL EXPERIMENTS




 §.§ Simulation Study I
 We first verify Theorem <ref> and its implications when all latent factors are structurally identifiable. Specifically, we consider K = 5 under the three models discussed in Section <ref>, including the linear, the MIRT and the Poisson models. Two design structures are considered, including (1) a simple structure, where p_Q({k}) = 1/5, k = 1, ..., 5 and (2) a mixed structure, where p_Q(S)= 1/5, S = {1, 2, 3}, {2, 3, 4}, {3, 4, 5}, {4, 5, 1}, and {5, 1, 2}. The true person parameters
_i^*s and the true manifest parameters
_j^*s
are generated i.i.d. from distributions over the ball {∈^K: ‖‖≤ 2.5}, respectively, (i.e., C = 2.5 in S_Q). Under these settings, all the latent factors are structurally identifiable.



For each model and each design structure, a range of J values are considered and we let N  = 25J. Specifically, we consider J = 100, 200, ..., 1000 for the linear, the MIRT, and the Poisson models.  For each combination of a model, a design structure, and a J value, 50 independent datasets are generated. For each dataset, we apply Algorithm <ref> to solve (<ref>), where C' = 1.2C.


Results are shown in Figures <ref>-<ref>.


Figure <ref> shows the trend of 1/NJΘ̂^(N, J)(Â^(N, J))^⊤-Θ^*_[1:N, 1:K] A^*⊤_[1:J, 1:K]_F^2 (y-axis) when J increases (x-axis), where each panel corresponds to a model. This result verifies (<ref>) in Theorem <ref>.
According to these plots, the normalized squared Frobenius norm between
Θ^*_[1:N, 1:K] A^*⊤_[1:J, 1:K] and its estimate Θ̂^(N, J)(Â^(N, J))^⊤ decays towards zero, as J and N increase.
Figures <ref>-<ref> present results based on the first latent factor and we point out that
the results based on the other latent factors are almost the same.

Figure <ref> is used to verify (<ref>)
in Theorem <ref>, showing the pattern that sin∠(Θ^*_[1:N, k],Θ̂_[k]^(N, J)) decreases as J and N increase.


Moreover, Figure <ref> provides evidence on the result of Proposition <ref>.
Displayed in Figure <ref> are the histograms of v_is and v̂_is, respectively,
based on a randomly selected dataset when J = 1000 under the Poisson model and the simple structure.
According to this figure, little difference is observed between the empirical distribution of v_is and that of v̂_is. Similar results are observed for other datasets under all these three models when J and N are large.








Finally,
Figures <ref> and <ref> show results of ranking and classification based on the estimated factor scores, whose theoretical results are given in Propositions <ref> and <ref>.
The y-axes of the two figures show the normalized Kendall's tau distance in (<ref>)
and the classification error in (<ref>), respectively.
Specifically, τ_- and τ_+ are chosen as
0.14 and 0.43 which are the 55% and 65% quantiles of the v_is.
From these plots, both the ranking and the classification errors tend to zero, as J and N grow large.



 §.§ Simulation Study II

We then provide an example, in which a latent factor is not identifiable. Specifically, we consider K = 2 and the same latent factor models as in Study I.
The design structure is given by p_Q({1}) = 1/2 and p_Q({1, 2}) = 1/2.
The true person parameters
_i^*s and the true manifest parameters
_j^*s
are generated i.i.d. from distributions over the ball {∈^K: ‖‖≤ 3}, respectively, (i.e., C = 3 in S_Q). Under these settings, the first latent factor is structurally identifiable and the second factor is not.
For each model, we consider J = 100, 200, ..., 1000. The rest of the simulation setting is the same as Study I.


Results are shown in Figures <ref> and <ref>. First, Figure <ref> presents the patten that 1/NJΘ̂^(N, J)(Â^(N, J))^⊤-Θ^*_[1:N, 1:K] A^*⊤_[1:J, 1:K]_F^2
decays to 0 when J increases, even when a latent factor is not structurally identifiable. This is consistent with the first part of Theorem <ref>. Second, Figure <ref> shows the
trend of sin∠(Θ^*_[1:N, k],Θ̂_[k]^(N, J)) as J increases. In particular,
the value of sin∠(Θ^*_[1:N, k],Θ̂_[k]^(N, J)) stays above 0.3 for most of the data sets for the factor which is structurally unidentifiable, while it still decays towards 0 for the identifiable one.




 §.§ Real Data Example



As an illustration, we apply the proposed method to analyze a personality assessment dataset based on an International Personality Item Pool (IPIP) NEO personality inventory <cit.>. This inventory is a public-domain version of the widely used NEO personality inventory <cit.>, which is designed to measure the Big Five personality factors (K = 5), including Neuroticism (N), Agreeableness (A), Extraversion (E), Openness to experience (O),  and  Conscientiousness (C).
The dataset contains 20,993 individuals and 300 items. We use a subset of the dataset which contains 7,325 individuals who have answered all 300 items. The measurement design matrix has a simple structure, which is a safe design according to our identifiability theory. Under this design, each item only measures one personality factor and each factor is measured by 60 items.
All the items are on a five-category rating scale, where reverse-worded items were reversely recorded
(1 → 5, 2 → 4, 4 → 2, 5 → 1) at the time the respondent completed the inventory.
In this analysis we dichotomize them by merging categories {1, 2, 3} and {4, 5}, respectively, and then fit the MIRT model. 










The results are shown in Figures <ref> through <ref> and Tables <ref> and Table <ref>. In Figure <ref>, the histograms of Θ̂_[k]^(N, J) are given, for k = 1, ..., 5, which correspond to the N, A, E, O, and C factors, respectively.  As we can see, the estimated factor scores are quite normally distributed, especially for the first three factors. For the O and C factors, the distributions of the estimated factor scores are slightly right skewed. In Table <ref>, the correlations between the factors are calculated using the estimated factor scores. The correlations between the factors are relatively small, which  
are largely consistent with the existing findings in the literature of Big Five personality <cit.>. Figure <ref> shows the box plot of the 60 unconstrained loadings for each of the five factors, according to which all the estimated loadings take value between 0 and 3 and the majority of them take value between 0.5 and 2. Table <ref> lists the content of the top five items with the highest estimated loadings for each factor. These items are indeed representative of the Big Five factors.







§ CONCLUDING REMARKS



In this paper, we study how design information affects the identifiability and estimability of a general family of structured latent factor models, through both asymptotic and non-asymptotic analyses.



In particular,  under a double asymptotic regime where both the numbers of individuals and manifest variables grow to infinity, we define the concept of structural identifiability for latent factors.

Then

necessary and sufficient conditions are established for the structural identifiability of a given latent factor. Moreover, an estimator is proposed that can consistently recover all the structurally identifiable latent factors. A non-asymptotic error bound is developed
to characterize the effect of design information on the estimation of latent factors, which complements the asymptotic results. In establishing these results, new perturbation bounds on the intersection of linear subspaces, as well as some other technical tools, are developed, which may be of independent theoretical interest. As shown in Section <ref>,
our results have significant implications on the use of generalized latent factor models in large-scale educational and psychological measurement.


There are many future directions along the current work. First, it is of interest to develop methods, such as information criteria, for model comparison based on the proposed estimator. These methods can be used to
select a design matrix Q that best describes the data structure when there are multiple candidates,
or to determine the underlying latent dimensions.
Second, the current results may be further generalized by considering more general latent factor models beyond the exponential family. For example,
we may establish similar identifiability and estimability results when the distribution of Y_ij is a
more complicated function or even an unknown function of _i and _j.



§ ACKNOWLEDGEMENT

We would like to thank the editors and two referees for their helpful and constructive comments. We would also like to thank Prof. Zhiliang Ying for his valuable comments.
Xiaoou Li's research is partially supported by the NSF grant DMS-1712657. Yunxiao Chen's research is supported in part by NAEd/Spencer postdoctoral fellowship.


apalike
xx

Andersen1970andersen1970asymptotic
Andersen, E. B.  1970.
Asymptotic properties of conditional maximum-likelihood estimators,
  Journal of the Royal Statistical Society. Series B (Methodological)
  32: 283–301.

Anderson2003anderson2003introduction
Anderson, T. W.  2003.
An introduction to multivariate statistical analysis, Wiley,
  Hoboken, NJ.

Anderson  Rubin1956anderson1956statistical
Anderson, T. W.  Rubin, H.  1956.
Statistical inference in factor analysis, in J. Neyman (ed.),
  Proceedings of the third Berkeley symposium on mathematical statistics
  and probability, Vol. 5, University of California Press, Berkeley, CA,
  pp. 111–150.

Bai  Li2012bai2012statistical
Bai, J.  Li, K.  2012.
Statistical analysis of factor models of high dimension, The
  Annals of Statistics 40: 436–465.

[Bartholomew et al.]Bartholomew, Knott   Moustaki2011bartholomew2011latent
Bartholomew, D. J., Knott, M.  Moustaki, I.  2011.
Latent variable models and factor analysis: A unified approach,
  Wiley, Hoboken, NJ.

[Bartroff et al.]Bartroff, Finkelman   Lai2008bartroff2008modern
Bartroff, J., Finkelman, M.  Lai, T. L.  2008.
Modern sequential analysis and its applications to computerized
  adaptive testing, Psychometrika 73: 473–486.

Bhaskar  Javanmard2015bhaskar20151
Bhaskar, S. A.  Javanmard, A.  2015.
1-bit matrix completion under exact low-rank constraint, 49th
  Annual Conference on Information Sciences and Systems (CISS), IEEE,
  pp. 1–6.

[Bi et al.]Bi, Qu, Wang  Shen2017bi2017group
Bi, X., Qu, A., Wang, J.  Shen, X.  2017.
A group-specific recommender system, Journal of the American
  Statistical Association 112: 1344–1353.

[Bing et al.]Bing, Bunea, Ning   Wegkamp2017bing2017adaptive
Bing, X., Bunea, F., Ning, Y.  Wegkamp, M.  2017.
Adaptive estimation in structured factor models with applications to
  overlapping clustering, arXiv preprint:1704.06977 .

Cai  Zhou2013cai2013max
Cai, T.  Zhou, W.-X.  2013.
A max-norm constrained minimization approach to 1-bit matrix
  completion, The Journal of Machine Learning Research 14: 3619–3647.

Candès  Plan2010candes2010matrix
Candès, E. J.  Plan, Y.  2010.
Matrix completion with noise, Proceedings of the IEEE 98: 925–936.

[Chen et al.]Chen, Li  Zhang2019chen2017joint
Chen, Y., Li, X.  Zhang, S.  2019.
Joint maximum likelihood estimation for high-dimensional exploratory
  item factor analysis, Psychometrika 84: 124–146.

Costa  McCrae1985costa1985neo
Costa, P. T.  McCrae, R. R.  1985.
The NEO personality inventory, Psychological Assessment
  Resources, Odessa, FL.

Dagum  Menon1998dagum1998openmp
Dagum, L.  Menon, R.  1998.
OpenMP: An industry standard API for shared-memory programming,
  Computational Science & Engineering, IEEE 5: 46–55.

[Davenport et al.]Davenport, Plan, van den Berg   Wootters2014davenport20141
Davenport, M. A., Plan, Y., van den Berg, E.  Wootters, M.
  2014.
1-bit matrix completion, Information and Inference 3: 189–223.

Davis1963davis1963rotation
Davis, C.  1963.
The rotation of eigenvectors by a perturbation, Journal of
  Mathematical Analysis and Applications 6: 159–173.

Digman1997digman1997higher
Digman, J. M.  1997.
Higher-order factors of the Big Five, Journal of Personality
  and Social Psychology 73: 1246– 1256.

Gatignon2003gatignon2003statistical
Gatignon, H.  2003.
Statistical analysis of management data, Springer, New York,
  NY.

Ghosh1995ghosh1995inconsistent
Ghosh, M.  1995.
Inconsistent maximum likelihood estimators for the Rasch model,
  Statistics & Probability Letters 23: 165–170.

Goldberg1993goldberg1993structure
Goldberg, L. R.  1993.
The structure of phenotypic personality traits., American
  Psychologist 48: 26–34.

Grayson  Marsh1994grayson1994identification
Grayson, D.  Marsh, H. W.  1994.
Identification with deficient rank loading matrices in confirmatory
  factor analysis: Multitrait-multimethod models, Psychometrika 59: 121–134.

Grippo  Sciandrone2000grippo2000convergence
Grippo, L.  Sciandrone, M.  2000.
On the convergence of the block nonlinear Gauss–Seidel method
  under convex constraints, Operations Research Letters 26: 127–136.

Gu  Xu2018gu2018sufficient
Gu, Y.  Xu, G.  2018.
The sufficient and necessary condition for the identifiability and
  estimability of the DINA model, Psychometrika .
To appear.

Haberman1977haberman1977maximum
Haberman, S. J.  1977.
Maximum likelihood estimates in exponential response models, The
  Annals of Statistics 5: 815–841.

Holland1990holland1990sampling
Holland, P. W.  1990.
On the sampling theory roundations of item response theory models,
  Psychometrika 55: 577–601.

Johnson2014johnson2014measuring
Johnson, J. A.  2014.
Measuring thirty facets of the Five Factor Model with a 120-item
  public domain inventory: Development of the IPIP-NEO-120, Journal of
  Research in Personality 51: 78–89.

Kendall  Gibbons1990kendall1990rank
Kendall, M. G.  Gibbons, J. D.  1990.
Rank correlation methods, Oxford University Press, New York,
  NY.

Lord1980lord1980applications
Lord, F. M.  1980.
Applications of item response theory to practical testing
  problems, Routledge, New York, NY.

McCullagh  Nelder1989mccullagh1989generalized
McCullagh, P.  Nelder, J. A.  1989.
Generalized Linear Models, CRC Press, Boca Raton, FL.

Neyman  Scott1948neyman1948consistent
Neyman, J.  Scott, E. L.  1948.
Consistent estimates based on partially consistent observations, Econometrica 16: 1–32.

Owen  Wang2016owen2016bi
Owen, A. B.  Wang, J.  2016.
Bi-cross-validation for factor analysis, Statistical Science
  31: 119–139.

Parikh  Boyd2014parikh2014proximal
Parikh, N.  Boyd, S.  2014.
Proximal algorithms, Foundations and Trends® in
  Optimization 1: 127–239.

Reckase2009reckase2009multidimensional
Reckase, M.  2009.
Multidimensional item response theory, Springer, New York, NY.

Shapiro1985shapiro1985identifiability
Shapiro, A.  1985.
Identifiability of factor analysis: Some results and open problems,
  Linear Algebra and its Applications 70: 1–7.

Skrondal  Rabe-Hesketh2004skrondal2004generalized
Skrondal, A.  Rabe-Hesketh, S.  2004.
Generalized latent variable modeling: Multilevel, longitudinal,
  and structural equation models, CRC Press, Boca Raton, FL.

Stewart  Sun1990stewart1990matrix
Stewart, G.  Sun, J.  1990.
Matrix perturbation theory, Academic Press, Cambridge, MA.

Thompson2004thompson2004exploratory
Thompson, B.  2004.
Exploratory and confirmatory factor analysis: Understanding
  concepts and applications, American Psychological Association, Washington,
  DC.

[Udell et al.]Udell, Horn, Zadeh   Boyd2016udell2016generalized
Udell, M., Horn, C., Zadeh, R.  Boyd, S.  2016.
Generalized low rank models, Foundations and
  Trends® in Machine Learning 9: 1–118.

Wedin1972wedin1972perturbation
Wedin, P.  1972.
Perturbation bounds in connection with singular value decomposition,
  BIT Numerical Mathematics 12: 99–111.

Xu2017Xu2017
Xu, G.  2017.
Identifiability of restricted latent class models with binary
  responses, The Annals of Statistics 45: 675–707.

Xu  Zhang2016xu2016identifiability
Xu, G.  Zhang, S.  2016.
Identifiability of diagnostic classification models, Psychometrika 81: 625–649.

[Zhu et al.]Zhu, Shen  Ye2016zhu2016personalized
Zhu, Y., Shen, X.  Ye, C.  2016.
Personalized prediction and sparsity pursuit in latent factor models,
  Journal of the American Statistical Association 111: 241–252.





