Fitting MA(q) Models in the Closed Invertible Region
arXiv:1611.04907v1 [] 15 Nov 2016

Y. Zhanga.∗ and A.I. McLeodb.1
a

Acadia University and b The University of Western Ontario

Preprint. Zhang, Y. and McLeod, A.I. (2006). Fitting MA(q) Models in the Closed Invertible
Region, Statistics and Probablity Letters, 76, 1331-1334.
Abstract
The use of reparameterization in the maximization of the likelihood function of the MA (q)
model is discussed. A general method for testing for the presence of a parameter estimate on
the boundary of an MA(q) model is presented. This test is illustrated with a brief simulation
experiment for the MA(q) for q = 1, 2, 3, 4 in which it is shown that the probability of an
estimate being on the boundary increases with q.
Key Words: Admissible region for the autoregressive-moving average time series; ARMA
model reparameterization; Numerical maximum likelihood estimation.
∗

Corresponding Author: A.I. McLeod

Department of Statistical and Actuarial Sciences
The University of Western Ontario
London, Ontario N6A 5B7
aimcleod@uwo.ca
1

A.I. McLeod supported by NSERC Discovery Grant.

1

1. Introduction
The MA (q) model with mean zero may be written, zt = θ(B)at , where θ(B) = 1−θ1 B −
. . . θq B q and at is Gaussian white noise with variance σa2 . The model is said to be invertible
if all roots of θ(B) = 0 lie outside the unit circle (Box et al., 1991, §3.3.1). MA (q) models,
with roots inside the unit circle, may be reparameterized so that the roots are outside the
unit circle (Brockwell and Davis, 1991, p.88). When using an exact maximum likelihood
algorithm, such as the innovations algorithm (Brockwell and Davis, 1991, §8.6), some parameter estimates may lie on the non-invertible boundary (Kang, 1975). Cryer and Ledolter
(1981) proved algebraically that there is a positive probability of the maximum likelihood
estimator being on the unit boundary in the MA (1) case. The use of reparameterization to
obtain maximum likelihood estimates over the closed invertible region is discussed in §2.
A noninvertible MA (q) model means that at cannot be expressed in terms of past observations. For this reason, in most situations many time series analysts would prefer an
invertible model. Additionally, with a noninvertible model care must be taken in the algorithms used to compute the residuals and forecasts. Statistical inference for the parameters
in a noninvertible model also becomes more difficult. For these reasons it is recommended
that a model be tested to determine if it has a parameter estimate on the noninvertible
boundary. In those cases where a parameter estimate on the boundary is found, there are
a number of remedies. One simple approach would be to consider a different type of time
series model. For example, in §3 it is noted that in the mixed ARMA (p, q) case, where
p > 0, maximum likelihood estimates on the noninvertible boundary are less likely. Another
approach would be to use mean likelihood estimation (McLeod and Quenneville, 2000) or
a Bayesian approach (Marriott and Smith, 1992). In addition to guaranteeing invertibility, these estimation techniques have the same first-order asymptotic efficiency as maximum
likelihood and the mean-square error of the parameters estimates is usually less (McLeod
and Quenneville, 2000). In §4 we present a convenient test for the presence of an estimate
on the noninvertible boundary. It should be pointed out that one cannot simply constrain
the maximum likelihood estimates to be inside the invertible region since from the results of
2

Cryer and Ledolter (1981), estimates on the boundary will cause the constrained likelihood
approach to either fail by converging to an estimate outside the admissible region or else, if
a penalty function approach is used for the maximization, the estimates will be very close
to the boundary. Neither of these situations is desirable.
2. Reparameterization For Constrained Maximum Likelihood Estimation
Let Dθ denote the invertible region for an MA (q) in the parameter space (θ1 , . . . , θq ).
As noted by Monahan (1984), the reparameterization discussed by Barndorff-Neilsen and
Schou (1973) may be extended for use with the ARMA (p, q). For simplicity we discuss the
MA (q) case but it is easy to extend our results to the moving-average parameters in the
ARMA (p, q). Monahan (1984) defined a transformation, B : (ζ1 , . . . , ζq ) −→ (θ1 , . . . , θq ).
For brevity, let ζ = (ζ1 , . . . , ζq ), θ = (θ1 , . . . , θq ) and θi = Bi (ζ), i = 1, . . . , q. This transformation may be computed using the recursion,
θi,k = θi,k−1 + ζk θk−i,k−1,

i = 1, ..., k − 1; k = 1, . . . , q,

(1)

where θi,q = θi , ζi = θi,i , i = 1, . . . , q. The invertible region for the transformed parameters ζ,
denoted by Dζ , is simply the interior of the unit cube, |ζi | < 1, i = 1, . . . , q. Barndorff-Neilsen
and Schou (1973) showed that B is 1:1 and onto as well as continuously differentiable with
a continuously differentiable inverse inside the invertible region. However, since the determinant Jacobian of the transformation B is zero on the non-invertible boundary (BarndorffNeilsen and Schou, 1973, p.414), the transformation is not 1:1 there and consequently the inverse function is not well defined. For example in the MA (2) case, B(ζ1 , ζ2) = (ζ1 (1−ζ2 ), ζ2 )
and so B(ζ1 , 1) = (0, 1).
Denote the likelihood function of the MA (q) by L(θ). Then the reparameterized likelihood function may be written L(B(ζ)), where ζ is constrained, |ζi | ≤ 1, i = 1, . . . , q.
Standard minimization algorithms, such as those implemented in Mathematica (Wolfram,
2005), may be used to obtain the maximum likelihood estimates over the closed invertible
region.
3

The transformation B is continuous and differentiable in the closed invertible region. In
order to use the transformation B for maximum likelihood estimation in the case where the
estimates may lie on the boundary of Dθ , it is necessary that each point on the boundary of
Dθ be the image of a point on the boundary of Dζ . In other words, it is necessary that B be
onto. This is proved in §3.
3. Proof B Is Onto
Denote the boundary sets of Dθ and Dζ by ∂θ and ∂ζ respectively. Theorem 2 of BarndorffNielsen and Schou (1973) showed that B maps Dζ onto Dθ and that this mapping is oneto-one. But B is no longer one-to-one on the boundary ∂ζ . It is non-trivial to show that
B maps ∂ζ onto ∂θ . After careful investigation and discussion with an expert in point set
topology we were not able to find any suitable theorem which is applicable to this situation
and so for completeness we have included a proof from first principles.
Theorem 1. ∂θ = B(∂ζ )
Proof. First we show that ∂θ ⊂ B(∂ζ ). Let D̄θ and D̄ζ denote the closures of Dθ and Dζ
respectively. Since B is a polynomial and hence continuous on D̄ζ , B(D̄ζ ) ⊂ B(Dζ ) = D̄θ .
Meanwhile, B(D̄ζ ) = Dθ ∪ B(∂ζ ) is closed since D̄ζ is compact and therefore B(D̄ζ ) ⊃ D̄θ .
Hence B(D̄ζ ) = D̄θ , so Dθ ∪ B(∂ζ ) = Dθ ∪ ∂θ . Since B is a homeomorphism between Dθ and
Dζ , it follows that Dθ is open since Dζ is open. Dθ ∩ ∂θ = ∅. Hence ∂θ ⊂ B(∂ζ ).
Next, we show ∂θ ⊃ B(∂ζ ). Let ϑ ∈ ∂ζ . There exists a sequence {ϑn } ∈ Dζ such
that ϑn → ϑ. Hence B(ϑn ) = ϑn ∈ Dθ → B(ϑ) = ϑ by the continuity of B on D̄ζ .
If ϑ ∈ Dθ , B−1 (ϑn ) = ϑn → B−1 (ϑ) = ϑ ∈ Dζ by the continuity of B−1 on Dθ , which
causes a contradiction with ϑ ∈ ∂ζ . Therefore, B(ϑ) ∈
/ Dθ . It follows that B(ϑ) ∈ ∂θ since
Dθ ∪ B(∂ζ ) = Dθ ∪ ∂θ . Hence ∂θ ⊃ B(∂ζ ).
4. Test for Estimate on the Noninvertible Boundary
Monahan (1984) showed that inside the invertible region, B−1 , may be obtained using

4

the recursive formula,
2
θi,k−1 = (θi,k − θk,k θk−i,k )/(1 − θk,k
),

i = 1, . . . , k − 1; k = q, . . . , 1,

(2)

where θi = θi,q , ζi = θi,i , i = 1, . . . , q. More generally, for the closed invertible region, define
B− : D̄θ −→ D̄ζ using the recursion,
2
θi,k−1 = (θi,k − θk,k θk−i,k )/(1 − θk,k
),

if |θj,j | < 1, for all j ≥ k,

= 0 otherwise,

(3)

where i = 1, . . . , k − 1; k = q, . . . , 1. In Dθ , B− is the same as B−1 and, using mathematical
induction, it may be shown that all points on the boundary of Dθ are mapped into boundary
points in Dζ . As an illustration,
B(ζ1 , 1, ζ3, ζ4 ) = (−ζ3 (ζ4 + 1) , 1 − ζ4 , ζ3 (ζ4 + 1) , ζ4)
and
B− (−ζ3 (ζ4 + 1) , 1 − ζ4 , ζ3 (ζ4 + 1) , ζ4 ) = (0, 1, ζ3, ζ4 ) .
Let θ̂ denote estimates which belong to the closed invertible region and let ζ̂ = B− (θ̂),
where θ̂ ∈ D̄θ . Then |ζ̂i | ≤ 1 and θ̂ is on the boundary if and only if ζ̂i = ±1 for at least one
i ∈ {1, . . . , q}. In practice, we need to take into account the finite precision and rounding
error in our computations so we may declare an estimate is on the boundary if |1 − |ζ̂i || < ǫ
for any i ∈ {1, . . . , q}. In the simulation example below, we took ǫ = 10−6 . If a different
software environment were used it might be advisable to take ǫ somewhat larger. The use
of this test is illustrated in §5.
5. Simulation Experiment
A simulation experiment was conducted to see how the probability of an estimate on the
boundary depends on the model order q and series length n. An MA (1) with parameter θ1 =
−0.9, −0.6, . . . , 0.9, was simulated 100 times and for each simulation the exact maximum
likelihood estimates were determined using the innovation algorithm and the NMinimize
5

function in Mathematica (Wolfram, 2005) for each of the MA(q) models with q = 1, 2, 3, 4.
An estimate was counted as on the boundary if any one of ζ̂1 , . . . , ζ̂q was within 10−6 of ±1.
The results are summarized in Table 1. We see that the probability of an estimate on the
boundary increases with the model order q, decreases with sample size n and increases with
the distance of the true parameter to the unit boundary. Note that the test for an estimate
on the boundary requires Theorem 1. For the model MA(1), the results agree well with the
previous studies mentioned above.
In another simulation experiment, ARMA (1, q), q = 1, 2, 3, 4 models were fit to a series
of length 25 generated by an ARMA (1, 1) model with parameters φ = −0.5, 0, 0.5 and
θ1 = −0.9, −0.6, . . . , 0.9. It was found that the probability of a root on the moving average
boundary was quite small but appeared to be nonzero in all cases.
6. Concluding Remarks
Theorem 1 is also useful in the maximum likelihood estimation of ARMA (p, q) models,
φ(B)zt = θ(B)at where φ(B) = 1 − φ1 B − . . . φp B p and θ(B) = 1 − θ1 B − . . . θq B q . For
a causal-stationary process, φ(B) = 0 has all roots outside the unit circle and for model
identifiability, in the econometric sense (Harvey, 1990, §3.6), we require that θ(B) = 0 has
all roots on or inside the unit circle. The results §2 and §4 may be extended to the case.
Complete details of the maximum likelihood algorithm for ARMA (p, q) models, as well as
of the simulations in §5, are given in Mathematica notebooks available from the authors.

6

Acknowledgements
The authors would like to thank Dr. André Boivin for his suggestions and comments on
Theorem 1.
References
Barndorff-Nielsen, O. and Schou G. (1973), On the Parametrization of Autoregressive Models
by Partial Autocorrelations, Journal of Multivariate Analysis 3, 408–419.
Box, G.E.P., Jenkins, G.M. Reinsel, G.C., (1994), Time Series Analysis: Forecasting and
Control (Prentice-Hall, Englewood Cliffs, 3rd ed.).
Brockwell, P.J. and Davis, R.A. (1991), Time Series: Theory and Methods (Springer-Verlag,
New York, 2nd ed.).
Cryer, J. D. and Ledolter, J. (1981), Small-sample properties of the maximum likelihood
estimator in the first-order moving average model, Biometrika 68, 691–694.
Hamilton, J.D. (1994), Time Series Analysis (Princeton University Press, Princeton).
Harvey, A. (1990), The Econometric Analysis of Time Series (MIT Press, Cambridge, 2nd
ed.).
Kang, K.M. (1975). A Comparison of Estimators for Moving Average Proesses. Unpublished
manuscript, Australian Bureau of Statistics.
Marriott, J. M. and Smith, A. F. M. (1992), Reparametrization Aspects of Numerical
Bayesian Methodology for Autoregressive Moving-average Models, Journal of Time Series
Analysis 13, 327–343.
McLeod, A.I. and Quenneville, B. (2000), Mean likelihood estimators, Statistics and Computing 11, 57–65.
Monahan, J.F. (1984), A note on enforcing stationarity in autoregressive-moving average
models, Biometrika 71, 403–404.
Wolfram, S. (2005), The Mathematica Book 5 (Wolfram Media, Champaign).

7

Table 1: Proportion of times an estimate was on the noninvertible boundary in 100 simulations of a series of length n from an MA (1) with parameter θ1 when it is fit with an
MA (q),

q = 1, 2, 3, 4.

n

θ

MA (1)

MA (2)

MA (3)

MA (4)

25 −0.9

0.53

0.51

0.51

0.46

25 −0.6

0.08

0.15

0.23

0.53

25 −0.3

0.00

0.03

0.17

0.27

25

0.0

0.01

0.05

0.24

0.35

25

0.3

0.02

0.06

0.23

0.27

25

0.6

0.14

0.21

0.37

0.39

25

0.9

0.49

0.51

0.49

0.57

50 −0.9

0.36

0.36

0.37

0.32

50 −0.6

0.01

0.03

0.06

0.11

50 −0.3

0.00

0.03

0.04

0.08

50

0.0

0.00

0.00

0.02

0.04

50

0.3

0.01

0.02

0.02

0.04

50

0.6

0.01

0.03

0.03

0.07

50

0.9

0.26

0.33

0.28

0.36

8

