Abstract
Abstract. We consider least squares estimators for linear regression models with long-memory dependence, continuous time, and nonlinear inequality constraints imposed on the parameter. We study the solution of the problem of minimization of the least squares functional in the linear regression with a given (long) radius of dependence and nonlinear inequality constraints imposed on the parameter. We prove that the solution being appropriately centered and normalized converges in distribution to the solution of the quadratic programming problem. The latter solution is non-Gaussian in contrast to known results for long-memory dependence without constraints for which an analogous transform of the solution of the minimization problem is asymptotically Gaussian in many typical cases.
Introduction
The asymptotic distribution of normalized least squares estimators for linear regression models with long-memory dependence (strong or long-range dependence, in other words) is studied in the paper for the case where nonlinear inequality constraints are imposed on the parameter. There are many examples of applications of such models in practice, since much of the data observed in various fields of science and technology is known to possess the long-memory dependence structure and since a number of problems in the control of industrial processes as well as problems of modelling economic processes lead to regression models with a priori known inequality constraints imposed on the parameters of the models. The presence of inequality constraints in a model allows one to obtain a better fit of the regression model to the data and to improve the quality of the estimators of the parameters of the model.
Introduction
The asymptotic behavior of least squares estimators for long-memory dependence models (for both discrete and continuous cases) without constraints imposed on the parameters is studied by many authors. Taqqu [16] and Dobrushin and Major [5] proved the noncentral limit theorem that characterizes such models. The further asymptotic theory of least squares estimators for long-memory dependence models without constraints is due to Yajima [18] , Künsch, Beran, and Hampel [10] , Dahlhaus [4] , Leonenko and Bensic [12] , Leonenko [11] , Ivanov and Leonenko [7] - [9] .
Introduction
On the other hand, regression models with independent or weakly dependent errors that involve constraints imposed on the parameters are studied in papers by Korkhin [2] ,
The basic model. First assumptions
We consider the estimator β = (β 1 , . . . , β n ) in the linear regression model with continuous time (1) y
(t) = β g(t) + η(t), 0 ≤ t ≤ T,
and nonlinear constraints
(t) = β g(t) + η(t), 0 ≤ t ≤ T,
where β is an unknown parameter, g(t) = [g 1 (t), . . . , g n (t)] , h(β) = [h 1 (β), . . . , h r (β)] are known functions, and η(t), t ∈ R, is a measurable mean square continuous second order stationary random process with zero mean and covariance B η (t), t ∈ R.
(t) = β g(t) + η(t), 0 ≤ t ≤ T,
In what follows we consider the case where the process η(t) is subordinated to a long-memory Gaussian process; namely, we assume the following (see, for example, Taqqu [16] ).
Let η(t) = G(ε(t))
, t ∈ R, be a random process where G is an arbitrary real-valued measurable nonrandom function and let ε(t), t ∈ R, be a Gaussian random process with zero mean and covariance B ε (t) = (1+t
Let η(t) = G(ε(t))
The model described above is rather general. Below we provide some examples. Denote by Φ(·) the standard Gaussian distribution function and let U be the uniform Parametric regression models (1) with discrete or with continuous time and longmemory errors without constraints (2) are considered by Dobrushin and Major [5] , Yajima [18] , Künsch, Beran, and Hampel [10] , Dahlhaus [4] , and Leonenko [11] (also see the references therein). On the other hand, regression models (1) with independent or weakly dependent errors and constraints (2) are considered by Korkhin [2] and Knopov [1] . If Assumption 1 holds, then it is well known that the function G admits the following expansion in the Hilbert space L 2 (R, φ(u) du) in terms of the Chebyshev-Hermite polynomials with the unit leading coefficient
Let η(t) = G(ε(t))
where
Let η(t) = G(ε(t))
are the Chebyshev-Hermite polynomials; that is,
Let η(t) = G(ε(t))
and φ(u) is the probability density of ε(0):
Let η(t) = G(ε(t))
have the derivatives of the first and second order that are bounded in a neighborhood of the true value β 0 ; 2) h j (β 0 ) = 0, j ∈ {1, . . . , q}, h j (β 0 ) < 0, j ∈ {q + 1, . . . , r}; 3) there exist β * such that h(β * ) < 0; 4) the vectors ∇h j (β 0 ), j ∈ {1, . . . , q}, be linearly independent; 5) h j (β), j = 1, . . . , r, be convex.
Let η(t) = G(ε(t))
7. For j = 1, . . . , n, there are functionsḡ j (t), t ∈ [0, 1], that are square integrable in t ∈ [0, 1] and such that
Let η(t) = G(ε(t))
Note that Assumption 4 follows from Assumptions 7, since σ m = (σ lj )) n l,j=1 is a positive definite matrix, where
8.
For αm < 1 and j = 1, . . . , n,
8.
andḡ j (t) is defined in Assumption 7.
8.
Let Assumptions 1, 3, 7, and 8 hold and consider random variables
8.
. . is the multiple stochastic Wiener-Itô integral, W is a complex-valued Gaussian white noise, and the hyperplanes λ i = ±λ j , i, j = 1, . . . , m, i = j, are excluded from the integration domain in the above multiple stochastic integral. Note that the random variables Q jm possess finite second moments.
8.
It follows from Assumption 5 that the limits
9.
Assume that β T (a solution of the minimization problem) is a consistent estimator.
9.
License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use
REGRESSION WITH NONLINEAR CONSTRAINTS AND LONG-MEMORY DEPENDENCE 125
Remark 2.1. Assumptions 2-5 and 7-8 were proposed by Ivanov and Leonenko [8] for a more general setting. We use these assumptions in order to apply the central and noncentral limit theorems for processes with long-memory dependence proved in the paper [8] .
REGRESSION WITH NONLINEAR CONSTRAINTS AND LONG-MEMORY DEPENDENCE 125
Remark 2.2. Assumption 9 holds if, for example, the assumptions of Theorem 3.3 of [3] (or Theorem 3 of [14] ) hold. Below we use this assumption for J = {β : h(β) ≤ 0}. According to the result mentioned above, the estimator β T , being a solution of the minimization problem, is a consistent estimator of the parameter β. Moreover β T is a strong consistent estimator; that is, P lim T →∞ β T − β 0 p = 0 = 1. However, weak consistency (the convergence in probability) is enough for our purposes (recall that weak consistency is assumed in Assumption 9).
REGRESSION WITH NONLINEAR CONSTRAINTS AND LONG-MEMORY DEPENDENCE 125
Remark 2.3. Assumptions 6, 3) and 6, 5) imply that the gradients are linearly independent, that is,
REGRESSION WITH NONLINEAR CONSTRAINTS AND LONG-MEMORY DEPENDENCE 125
Assumptions 2-5 hold for the model of polynomial regression. Assumptions 7-8 hold, for example, in the case of n = 1, g(t) = log(e + t), and the quadratic function h(β).
REGRESSION WITH NONLINEAR CONSTRAINTS AND LONG-MEMORY DEPENDENCE 125
Now we are ready to state our main result.
REGRESSION WITH NONLINEAR CONSTRAINTS AND LONG-MEMORY DEPENDENCE 125
3. The statement of the main result Theorem 3.1. Let Assumptions 1, 2, 5, 6, and 9 hold. We also assume that Assumption 4 holds for m = 1. Let a random vector β T be a solution of the following minimization problem:
Then the random vector
U T = B(T ) −1/2 T −1/2 d T ( β T − β 0 ) converges
in distribution as T → ∞ to the random vector U that is a solution of the following quadratic programming problem:
where R 0 is defined in (6) and Q is a Gaussian random vector with zero mean and covariance matrix C 
in distribution as T → ∞ to the random vector U that is a solution of the following quadratic programming problem:
Then the random vector
in distribution as T → ∞ to the random vector U that is a solution of the following quadratic programming problem:
where R 0 and Q m are defined in (6) and (9), respectively.
in distribution as T → ∞ to the random vector U that is a solution of the following quadratic programming problem:
License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use Remark 3.1. The most studied case is that of m = 1. Note that the random vector Q m is Gaussian in this case.
in distribution as T → ∞ to the random vector U that is a solution of the following quadratic programming problem:
The asymptotic behavior of the normalized least squares estimator of the regression coefficients β is studied, for example, in [4] and [8] for models with long-memory dependence and without constraints (2) . Note that a more general case of the so-called M -estimators is considered in [8] . The M -estimators are asymptotically Gaussian if m = 1 and non-Gaussian if m ≥ 2. The asymptotic distribution of these estimators in the problem with nonlinear constraints (2) is a solution of the quadratic programming problem (11) and almost always is non-Gaussian even in the case of m = 1 (despite the cases discussed below). This is a new phenomenon that did not appear in the regression models earlier.
in distribution as T → ∞ to the random vector U that is a solution of the following quadratic programming problem:
The normalized least squares estimators are asymptotically non-Gaussian for m ≥ 2 for both models without constraints (see, for example, Ivanov and Leonenko [8] ) and in models of constrained regression.
in distribution as T → ∞ to the random vector U that is a solution of the following quadratic programming problem:
Consider some particular cases of the function h(β) for m = 1.
in distribution as T → ∞ to the random vector U that is a solution of the following quadratic programming problem:
. . , r. Assumption 6, 3) does not hold in this case. The inequality constraints in (11) are satisfied for all β and therefore one should solve the problem (11) without constraints. The minimum is attained at
in distribution as T → ∞ to the random vector U that is a solution of the following quadratic programming problem:
0 Q , which is a Gaussian vector. Therefore the normalized least squares estimators are asymptotically Gaussian in this case. Similar reasoning proves that least squares estimators are asymptotically non-Gaussian if m ≥ 2.
in distribution as T → ∞ to the random vector U that is a solution of the following quadratic programming problem:
2) h j (β) < 0, j = 1, . . . , r. All assumptions of Theorem 3.1 hold in this case. Note that q = 0 in Assumption 6, 2). Constraints (11) do not make sense at all (they are not active, in other words) and, similarly to case 1), we find that U = R −1 0 Q . Again the asymptotic distribution of the normalized least squares estimators is Gaussian. As above, the asymptotic distribution is non-Gaussian if m ≥ 2. We see that the models with constraints considered in the cases 1) and 2) above are asymptotically equivalent to those without constraints.
Auxiliary results
In what follows we need some results of the paper by Korkhin [2] . We prove some analogs of Korkhin's lemmas for continuous time and for stochastic processes with longmemory dependence (Korkhin [2] considers the case of discrete time and independent errors).
Auxiliary results
The following lemma provides some properties of random matrices. First we introduce the notion of the modal matrix. The following result contains some properties of a solution of a quadratic programming problem to be studied below.
Lemma 4.2. The quadratic programming problem
where β, Q, h i ∈ R n and where R is a positive definite matrix of size n × n possesses a solution β(Q) that is continuous in Q.
Lemma 4.2. The quadratic programming problem
Now we obtain some auxiliary results (analogs of the Korkhin lemma). First we consider the case of m = 1.
Lemma 4.2. The quadratic programming problem
The following relations are consequences of the necessary condition for the existence of an extremum of the problem (10):
Lemma 4.2. The quadratic programming problem
where λ iT are Lagrange coefficients. Relations (14)- (15) hold for β T , since h i (β), i ∈ I, are convex and there exists β
Lemma 4.2. The quadratic programming problem
We have p lim T →∞ ∇S T (β 0 ) = 0, since the averages of a long-memory process converge to zero. This implies that (17) p lim
Lemma 4.2. The quadratic programming problem
since β is consistent. We derive from (15) and Assumption 6, 1) that
Lemma 4.2. The quadratic programming problem
The gradients ∇h i ( β T ), i ∈ I β , are linearly independent by Assumptions 6, 3) and 5). Thus the system of equations (14) uniquely determines λ iT , i ∈ {1, . . . , q} ∩ I β (note that λ iT = 0, i ∈ {1, . . . , q} \ ({1, . . . , q} ∩ I β ) according to equalities (15) ). The right hand side of system (14) converges in probability to zero in view of relations (17), (18), (20), and Assumptions 6, 1). Then we get from Assumptions 6, 4) that
Lemma 4.2. The quadratic programming problem
Therefore we have proved the following result. converge in probability to zero as T → ∞.
Lemma 4.2. The quadratic programming problem
It follows from Lemma 4.3 that T we obtain from relations (14)- (16) that
Lemma 4.2. The quadratic programming problem
where H(β) is the r × n matrix whose row i is ∇h i (β), i ∈ {1, . . . , r}, and where
Lemma 4.2. The quadratic programming problem
, where L 1T and L 2T are the vectors whose components are l iT , i ∈ {1, . . . , q}, and l iT , i ∈ {q + 1, . . . , r}, respectively; H 1 (β) is the q × n matrix whose row i is ∇h i (β), i ∈ {1, . . . , q}; and H 2 (β) is the (r − q) × n matrix whose row i is ∇h i (β), i ∈ {q + 1, . . . , r}.
Lemma 4.2. The quadratic programming problem
Taking into account this notation we obtain from (24) and (26) that
Lemma 4.2. The quadratic programming problem
According to (2) , 
. . , q},
License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use
REGRESSION WITH NONLINEAR CONSTRAINTS AND LONG-MEMORY DEPENDENCE 129
where (31)
REGRESSION WITH NONLINEAR CONSTRAINTS AND LONG-MEMORY DEPENDENCE 129
Here
REGRESSION WITH NONLINEAR CONSTRAINTS AND LONG-MEMORY DEPENDENCE 129
is the Hessian of size n × n, and
REGRESSION WITH NONLINEAR CONSTRAINTS AND LONG-MEMORY DEPENDENCE 129
Multiplying both sides of equality (30) by
REGRESSION WITH NONLINEAR CONSTRAINTS AND LONG-MEMORY DEPENDENCE 129
T we rewrite this expression in the matrix form
REGRESSION WITH NONLINEAR CONSTRAINTS AND LONG-MEMORY DEPENDENCE 129
where Ψ 1 ( β T ) is the matrix of size r × q whose row i is ψ i ( β T ), i ∈ {1, . . . , q}, and W 1T is the q-dimensional vector with components w iT , i ∈ {1, . . . , q}.
REGRESSION WITH NONLINEAR CONSTRAINTS AND LONG-MEMORY DEPENDENCE 129
The above results imply the following three lemmas.
Lemma 4.4. Let functions g i (t), t ∈ [0; T ], be bounded and let estimators β T be consistent. Then
Proof. Relation (33) follows from (19) and (25), since h i (β 0 ) < 0, i ∈ {q + 1, . . . , r}, and 
Proof. Equality (27) implies that −R
Using (36) and the representation of the matrices A T in terms of the matrix B T we have
Now we derive from (28) and (32) that
Since A T is a symmetric matrix, we have
Now we derive from (28) and (32) that
where C is an orthogonal matrix (that is,
Now we derive from (28) and (32) that
. . , ν nT ), and ν iT is the ith eigenvalue of the matrix A T . Since R T and R 0 are nonsingular matrices, Assumption 4 implies that
Now we derive from (28) and (32) that
Since β T is consistent, Assumptions 6, 1) and relations (20) and (31) imply that
Now we derive from (28) and (32) that
Then we derive from (36) and (39) that
Now we derive from (28) and (32) that
This equality and Assumption 6, 4) prove that A is a positive definite matrix. Now we establish from Lemma 4.1 that
Now we derive from (28) and (32) that
where 
From (38) and (43) we obtain
Then (37) implies that
From (38) and (43) we obtain
where
From (38) and (43) we obtain
According to the limit theorem proved in Ivanov and Leonenko [8] (Theorem 1 therein) for long-memory processes,
From (38) and (43) we obtain
if m = 1, where Q T and σ 1 are defined in (26) and (5), respectively, and Q is a zero mean Gaussian random vector with covariance matrix C Relations (45), (41), and (42) together with the latter two equalities imply that the limit distribution of K T coincides with the distribution of the random variable
From (38) and (43) we obtain
by (44). Since the limit distribution of K T exists, for an arbitrary δ > 0 there are numbers T 2 > 0 and ε 1 > 0 such that
From (38) and (43) we obtain
The latter relation and (47) yield
From (38) and (43) we obtain
Then for an arbitrary δ > 0 there exists a number ε 1 > 0 such that
From (38) and (43) we obtain
It follows from (43) that
From (38) and (43) we obtain
where
From (38) and (43) we obtain
Relations (41) and (42) imply that
From (38) and (43) we obtain
Thus given δ > 0 there exists a number ε > 0 such that
From (38) and (43) we obtain
Note that inequalities (48) and (50) hold simultaneously for T 0 = max(T 3 , T 4 ). Consider the inequality
From (38) and (43) we obtain
Then we get from (48) and (50) that
From (38) and (43) we obtain
This together with (49) implies (34). 
From (38) and (43) we obtain
Proof. Taking into account (35) we have
From (38) and (43) we obtain
T Q T ≥ ε 3 3 for some number ε 3 > 0 and for all T > 0. We estimate the terms on the right hand side of (53). Since p lim
From (38) and (43) we obtain
0 , we have for given numbers δ > 0 and ε 1 > 0 that
From (38) and (43) we obtain
in the latter inequality where ε > 0 is arbitrary. Then
From (38) and (43) we obtain
Multiply both sides of the inequality under the sign of probability by R
From (38) and (43) we obtain
in inequality (51). It follows from (34) and (51) that
From (38) and (43) we obtain
We obtain from (54) and the latter inequality that
From (38) and (43) we obtain
Now we derive from relations (20) and (33) that
From (38) and (43) we obtain
Thus given ε 2 > 0 and δ 1 > 0 there exists a number T 3 > 0 such that
From (38) and (43) we obtain
Using (46) and (39) we prove that, for a given δ 1 > 0, there are two numbers ε 4 > 0 and T 4 > 0 such that
From (38) and (43) we obtain
Putting ε 3 = 3ε 4 = 3ε 2 on the right hand side of (53) we obtain inequality (52) for ε = ε 3 and T 0 = max(T 1 , T 2 , T 3 , T 4 ) from (53), (55), (56), and the latter inequality where δ = δ 1 .
Proof of the main result
Now we evaluate the limit of the sequence of random variables U T . Consider the following convex programming problem:
Proof of the main result
where
Proof of the main result
and Ω is the sampling space. The other symbols in (57) are
Proof of the main result
where the functions h i2 and θ 1 are defined as in (31).
Proof of the main result
A vector U * T (ω) considered at a fixed ω is understood as a solution of the problem (57). This means that U * T (ω), ω ∈ Ω, is a random vector. Since R T (ω) is a symmetric matrix, the necessary conditions for the extremum in the problem (57) are given by
Proof of the main result
In turn necessary conditions for the extremum in the problem (14) , (15) given (25) and (27) can be rewritten as follows:
Proof of the main result
where
Proof of the main result
according to (58). This together with (60) and (61) implies that the vectors U T (ω) and L T (ω) form a solution of the system of equations (60); that is, U T (ω) satisfies the necessary conditions for the extremum in the problem (57). Now we are ready to prove the main result.
Proof of the main result
Proof of the main result. Consider the following quadratic programming problem:
Proof of the main result
Denote byŨ T (ω) a solution of problem (62). According to Lemma 4.2,Ũ T is a continuous function of Q T (ω), namelỹ
Proof of the main result
On the other hand, U (ω) = κ(Q(ω)) by (61). Thus
Proof of the main result
Since the functions h i (β), i ∈ {1, . . . , r}, are convex and s i (X) ≥ 0, X ∈ R n , the set O T is convex. Therefore (57) yields
Proof of the main result
Since the matrix R 0 is positive definite, the functionφ T (X) is strongly convex in X for every fixed ω. As U T (ω) ∈ O we have
Proof of the main result
for some constant µ > 0 (see [2] ). Thus (64)
Proof of the main result
for an arbitrary ε > 0 where ε 1 = µε 2 /2. We estimate the probabilities on the right hand side of inequality (64). Using Lemma 4.6 we obtain from (57) and (62) that (65)
Proof of the main result
for T > T 0 and for arbitrary ε 2 > 0 and δ > 0 where b > 0 is some number.
Proof of the main result
Analogously, it follows from (63) that
Proof of the main result
Now we estimate the second term in (64). We prove the inequality (67)
Proof of the main result
Inequality (67) is equivalent to the following one:
Proof of the main result
(68)
Proof of the main result
The latter inequality is obvious. Now consider the first term on the right hand side of (67). If R T is a positive definite matrix, then problem (57) has a unique solution, since O T is a convex set and Assumption 6 holds. The above results together with (60) and (61) imply that U *
Proof of the main result
where Λ iT > 0, i ∈ {1, . . . , n}, is the i-th eigenvalue of R T , whence we derive In the latter chain of inequalities we use r times an obvious bound P(A ∩ B) ≥ P(A) + P(B) − 1.
Proof of the main result
Since the function h i2 is continuous and bounded at the point β = β 0 , we obtain from (59) and (63) that As an application of (67) we have
Proof of the main result
by (72) and (68) for ε 2 = ε 1 /2. Substituting estimates (73) and (66) to the right hand side of (64) for ε 2 = ε 1 /2 we obtain P{ U T −Ũ T 2 < ε 2 } > 1−2δ−ς, T > max(T 1 , T 2 , T 6 ). The proof for the case of m ≥ 2 is similar. The difference is that the noncentral limit theorem (Theorem 3 of [8] ) is substituted for the central limit theorem (Theorem 1 of [8] ). The assumptions of Theorem 3 of [8] follow from the assumptions of our Theorem 3.1, so Theorem 3 of [3] can be used in our proof, indeed.
Concluding remarks
Note that the random vector (9) is Gaussian for m = 1 and non-Gaussian for m ≥ 2. The solution of the quadratic programming problem (11) is non-Gaussian even in the case of m = 1 in contrast to the results of [8] . It is proved that the least squares estimators 

{'paper_id': '118515726', 'title': 'Asymptotic distributions of least squares estimators of the coefficients in the model of linear regression with nonlinear constraints and long-memory dependence', 's2_url': 'https://api.semanticscholar.org/CorpusID:118515726', 'source': 'S2ORC'}