TIME SERIES ANALYSIS II
VÍTĚZSLAV VESELÝ
Department of Statistics and Operations Research
University of Malta *
Email: vvese01@um.edu.mt
CONTENTS
•   General notation and abbreviations
•   Discrete LTI-systems: impulse and frequency response, causality and stability
•   Mean square convergence of series Yt = YlTL-ao ipjXt-j
•   Best linear prediction (Yule-Walker system of equations)
•   Partial autocorrelation function
•   ARMA models for stationary time series, AR and MA models as their more simple case
•   Causality and invertibility for ARMA models
•   Identification, parameter estimation and verification for ARMA models
•   Asymptotic properties of some estimates
•   ARIMA and SARIMA models for time series with station-arity defects in the mean
Date: September 25, 2008.
1Visiting lecturer, 2-nd semester from 4-th February to 12-th July 2002
1. General notation and abbreviations
s := v ot v =: s ... denoting expression v by symbol s. iff stands for if and only if.
Sets and mappings:
•  N, Z, R, C ... natural numbers, integers, real and complex numbers, respectively.
•  Zjv := {0,1,... , TV - 1} ... residuals modulo TV e N.
•  R+ ... the set of all non-negative real numbers.
•  exp X ... class of all subsets of the set X.
•  card M .. . cardinality of a set M.
•   (-)+ : R —> R+ ... mapping defined as (x)+ = max(0, x).
•   {a,tí), [a,b], (a,b], [a,b) .. .intervals on real line. 3(a,b)     =     {x\ min(a,6) < x < max(a,6)} J[a,6]      =     {a; | min(a,6) < x < max(a,6)}.
•   f(A)  :=  {y e Y\y = f(x),x e A C X} ...range (image) of set A under mapping / : X —> Y.
•   /_1(ß) := {x e X I f (x) e B} C X ... inverse image of set 73 C Y under mapping / : X —> Y.
•  I a ■ ■ ■ indicator function of set A <Z X:
J1    for re e A 1 0    otherwise
•  A„ | ... increasing or non-decreasing sequence of numbers or sets.
•  A„ I ... decreasing or non-increasing sequence of numbers or sets.
•   X^ľ=i Ai '■= Uľ=i A-i ... union of a family of sets which are pairwise disjoint.
•  Ac := X — A ... complement of set A C X in X where X is a priori known from the context.
•  A := liminfn^oo An := LC=i C(jLn Ai ■ ■ ■ inferior limit of a sequence of sets.
2
. A  :=  limsup^A,   :=  Dľ= iU£„4/  • limit of a sequence of sets.
• A = limn^oo A„ iff A = A, clearly
An j A implies lim«-^ An = \J™=1 ^« and An i A implies limn^oo An = n„°=i A„. Vectors and matrices:
. superior
x := [xi,... ,xn\'   ...vector of numbers (by default column vector if not stated otherwise).
•   x + h := [xi + h,... , xn + h]   , h e C.
•   xt := [xtl,... ,xtk]T e Ck where t = [h,... ,tk]T G Nfc, U e {l,...,n}   fori = l,...,fc.
•   x(i) := [xi,. .. ,Xi-i,Xi+i,. .. ,xn]    for any 1 < í < n.
•   f (x) := f(xi,... , x„), dx := dx\ ... dx„.
•   0, 0nXi ... vector of n zero entries.
'■— [aij] — [A(i,j)] ■ ■ ■ matrix of size m x n.
•  1Z(A) := {í/ I y = Ax} ... range space of matrix oper-
11(A) := {y I ator A. N{A) := {x operator A.
AT := lau] ■
Ax = 0} ... null space (kernel) of matrix
A* := [a
I, In  : =
det A ..
0;  0mxn
matrix transpose. ji] ... matrix adjoint.
Inxn = [Sij] ■ ■ ■ identity matrix of order n. determinant of a square matrix A. . .. zero matrix of size mxn. xx     0      ...       0 0     x2     ...       0
diag(a;)   : =
. diagonal ma-
. i-th row of matrix A using
0       0      ...      Xn
trix.
•   A(i,\) := [an,... ,air MATLAB style.
•  A(:,j):=[ Q>\j , • • • , Q>mj
]    ... j-th column of matrix A
using MATLAB style.
3
•   A := [n;... ; rm] = [si,... , s„] ... forming matrix A row-by-row or columnwise using MATLAB style.
•   A > 0 (or A > 0) ... positively (semi)deflnite (non-negatively definite) matrix.
•   (x,y) := "}2™=1Xiyi = y*x ...scalar (inner) product of vectors x and y.
•   ||a;|| := \Z^2"=1\xi\2 = \J(x, x) ... Euclidean norm of vector x.
Random variables and random vectors:
•  X ... random variable.
•  X := [Xi,... ,X„] ...(real) random vector, indexing conventions listed above for number vectors are adopted accordingly.
•  /it := /itx := EX ... expectation of random variable X.
•  fj, := /ix := EX := [EXi,... , EXn]T ... expectation of random vector X.
•  a2 := a\ := varX := E|X - EX|2 = E|X|2 - |EX|2 > 0 . .. variance of random variable X.
•  axY := cov(X, Y) := E(X - EX)(Y - EY) = EX Y -(EX)(EY) ... covariance of random variables X and Y.
•   Ex := varX := [cov^X,-)] = E(X-EX)(X-EX)T = EXXT-(EX)(EX)T .. . variance matrix of random vector X.
•   Exy := cov(X, Y) := [cov(Xi; Yj)] = E(X - EX)(Y -EY)T = EXYT - (EX)(EY)T ... covariance matrix of X and Y.
It holds:
•  varX = cov(X,X).
•  cov(Y,X) = cov(X,Y).
•  COY(Y,rXr>Y,s Ys) = ErEs00^.^) and hence in particular:
4
•  var(X + Y) = varX + cov(X, Y) + cov( Y, X) + varY = varX + 2cov(X, Y) + varY.
•  cov(X, X) = varX.
•  cov(Y, X) = cov(X, Y)T implies:
•  varX = (varX)T ... variance matrix X is symmetrical.
•   Given number vectors a and c, and matrices B and D of compatible sizes then
cov(a+BX, c+DY) = cov(BX, DY) = B cov(X, Y) DT i).X = Y
•  var(a + BX) = cov(a + BX, a + BX) = cov(BX, BX) = Bvar(X)BT
I). a = 0, B = bT
•  0 < var(bTX) = bTvarXb implies:
•  varX > 0 ...variance  matrix is  non-negatively
positive and consequently it has non-negative eigen
i values Xi and its square root matrix E^ having eigen
i values X? may be constructed such that:
•  cov(£rXr,£sYs)  =  £r£scov(Xr,Ys)  and hence in particular:
•  var(X + Y) = varX + cov(X, Y) + cov(Y, X) + varY = varX + 2cov(X, Y) + varY.
5
References
[1]   Jiří Anděl. Statistická analýza časových rad. SNTL, Praha, 1976.
[2]   Peter J.  Brockwell and Richard A. Davis.  Introduction to   Time Series
and Forecasting. Springer-Verlag, New York, 2-nd edition, 2002. [3]   Peter J. Brockwell and Richard A. Davis. Time Series:  Theory and Methods. Springer-Verlag, New York, 2-nd edition, 1991 (corrected 2-nd printing
1993). [4]   Peter J. Brockwell and Richard A. Davis. ITS M for Windows. A   User's
Guide to  Time Series Modelling and Forecasting. Springer-Verlag, New
York, 1994. 2 diskettes included. [5]   James D. Hamilton.   Time Series Analysis.  Princeton University Press,
Princeton, NJ 08540, 1994. [6]   Lennart   Ljung.   SYSTEM  IDENTIFICATION:   Theory  for   the   User.
Prentice Hall, Inc., Englewood Cliffs, NJ, 1987.
Doc. RNDr. Vítězslav Veselý, CSc., Department of Applied Mathematics and Computer Science, Masaryk University of Brno, Lipová 41a, 602 00 Brno, Czech republic.
TEL.: +420-549498330, FAX:  +420-549491720
E-mail address: vesely@math.muni.cz,   URL:   http://www.math.muni .cz/~vesely
6