Xycoon logo
Time Series Analysis
Home    Site Map    Site Search    Free Online Software    
horizontal divider
vertical whitespace

Time Series Analysis - ARIMA models - Model Estimation

[Home] [Up] [Identification] [ARIMA Checking] [ARIMA Forecasting] [ARIMA Extensions] [ARIMA Estimation]

V.I.2 Univariate Stochastic ARIMA Model Estimation

The estimation of ARMA parameters in practice is not straightforward. Though many computer algorithms and computer programs exist for ARMA estimation, care should be taken with respect to some important aspects. Since it is not our intention to go to far into the details of estimation algorithms, only some important pitfalls and problems are considered. These problems are especially interesting (important) for those who seek to apply the methodology in practice.

First, consider the formulation of the likelihood function. An ARIMA model contains three different kinds of parameters:


the p AR-parameters;


the q MA-parameters;


and the variance of the error term.

This amount to a total of p + q + 1 parameters to be estimated (see also remark). These parameters are always estimated on using the stationary time series (c.q. a time series which is stationary with respect to itís variance and mean): sometimes it is necessary to introduce an additional parameter m (a constant term) to be estimated such that the total amount of parameters to be estimated is p + q + 2.

Time Series Analysis - ARIMA models - Model Estimation


likelihood function holds true.

The so-called log-likelihood function is


where SSR is the Sum of Squared Residuals

Box and Jenkins (1976) showed that the exact log-likelihood of an ARIMA model can be written as


Note the following definitions




Hence, using (V.I.2-4), it follows that


The backforecasting procedure is identical to the forecasting algorithm, except that the stationary time series is used in reversed order. Q' is identified by some criteria which ensure that the backforecasts have converged. This means that the backforecast should not be significantly different from the (zero) mean of e(t).

If backforecasting is not used (c.q. if the conditional likelihood criterion is used) the parameter estimates may deviate severely from the true parameters. Especially if the model contains MA parameters, exact likelihood estimation is necessary (see also the "truncation remainder").

Note that the variance-covariance matrix of the estimated parameters is given by the inverse of the information matrix


which is derived from the likelihood function. Hence, not only the parameter estimation, but also the variances of the parameters are dependent on the use of the backforecasting procedure.


The large sample variance can be shown to be


Above that, Box and Jenkins prove that


are uncorrelated (for large samples). In fact, almost all estimation properties in time series analysis are large sample properties. The MLE estimates for ARIMA parameters are consistent, normally distributed, and asymptotically efficient.

It is possible that an estimation algorithm yields parameter values for an MA-process which lie outside the invertibility region. In such a case it is possible to find a SSR which is smaller than the true minimum.

The estimation process can be adequately performed by Marquardt's algorithm for nonlinear least squares as described in many references (BOX and JENKINS 1976), (MELARD 1984), (GARDNER, HARVEY, and PHILLIPS), etc...


Remark that it is assumed that there is no seasonality involved in the ARIMA process. In the case of seasonality, there would be p + P + q + Q + 1 parameters (P and Q being the seasonal counterparts of p and q respectively). Furthermore it is assumed that no constant term is required in the ARIMA process.

vertical whitespace

ARIMA Checking
ARIMA Forecasting
ARIMA Extensions
ARIMA Estimation
horizontal divider
horizontal divider

© 2000-2022 All rights reserved. All Photographs (jpg files) are the property of Corel Corporation, Microsoft and their licensors. We acquired a non-transferable license to use these pictures in this website.
The free use of the scientific content in this website is granted for non commercial use only. In any case, the source (url) should always be clearly displayed. Under no circumstances are you allowed to reproduce, copy or redistribute the design, layout, or any content of this website (for commercial use) including any materials contained herein without the express written permission.

Information provided on this web site is provided "AS IS" without warranty of any kind, either express or implied, including, without limitation, warranties of merchantability, fitness for a particular purpose, and noninfringement. We use reasonable efforts to include accurate and timely information and periodically updates the information without notice. However, we make no warranties or representations as to the accuracy or completeness of such information, and it assumes no liability or responsibility for errors or omissions in the content of this web site. Your use of this web site is AT YOUR OWN RISK. Under no circumstances and under no legal theory shall we be liable to you or any other person for any direct, indirect, special, incidental, exemplary, or consequential damages arising from your access to, or use of, this web site.

Contributions and Scientific Research: Prof. Dr. E. Borghers, Prof. Dr. P. Wessa
Please, cite this website when used in publications: Xycoon (or Authors), Statistics - Econometrics - Forecasting (Title), Office for Research Development and Education (Publisher), http://www.xycoon.com/ (URL), (access or printout date).

Comments, Feedback, Bugs, Errors | Privacy Policy