Data Analysis

RANDOM SEQUENCES IN MATRIX NOTATION

Let us write the jointly distributed random variables in form of a column matrix, (random vector)

X = [\begin{matrix} x_{1} \\ x_{2} \\ ⋮ \\ x_{n} \end{matrix}],

X^{T} = [x_{1}, x_{2}, \dots, x_{n}],

E [X] = [\begin{matrix} E {x_{1}} \\ E {x_{2}} \\ ⋮ \\ E {x_{n}} \end{matrix}] .

(4.1)

Its transpose is a row matrix. The expectation is a column matrix with elements equal to the expectations of the random numbers $x_{k}, 1 \leq k \leq n$ .

The square $(n \times n)$ matrix with elements $cov {x_{i}, x_{i}}$ is called the covariance matrix of the random vector $X$ and denoted by $C_{X}$

C_{X} = E \{(X - E \{X\}) {(X - E \{X\})}^{T}\},

(4.2)

its elements are

X = [\begin{matrix} var {x_{1}} & cov {x_{1}, x_{2}} & \dots & cov {x_{1}, x_{n}} \\ cov {x_{2}, x_{1}} & var {x_{2}} & \dots & cov {x_{2}, x_{n}} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ cov {x_{n}, x_{1}} & cov {x_{n}, x_{2}} & \dots & var {x_{n}} \end{matrix}] .

(4.3)

The covariance matrix is symmetric. For a symmetric square matrix the eigenvalues are real and the eigenvectors are orthogonal. A matrix $A$ is said to be positive definite if $X^{T} A X$ for all vectors $X \neq 0$ . It is easy to see that the matrix $C_{X}$ has to be positive definite. If the values of the elements of the covariance matrix are estimated from observation it is necessary to transform the matrix to a symmetric form and change the terms so that all the eigenvalues are positive.

In matrix notation the density function of a n jointly normally distributed random variables $X^{T} = [x_{1}, x_{2}, \dots, x_{n}]$ is

p_{x_{1}, \dots, x_{m}} (x_{1} \dots x_{m}) = \sqrt{\frac{{(2 π)}^{n}}{| C_{X} |}} \exp [- \frac{1}{2} {(X - E {X})}^{T} C_{X}^{- 1} (X - E {X})]

(4.4)

where $| C_{X} |$ denotes the determinant and $C_{X}^{- 1}$ the inverse of the covariance matrix.

Let us now look at some simple examples of Gaussian random sequences

Example 4.1. Let us consider the simple case the expectation is a zero column matrix and the covariance matrix is proportional to the unit matrix $I$ , $C_{X} = σ^{2} I$ . Thus it is a diagonal matrix. When substituted into the expression for jointly normally distributed random variables it follows that the elements of the sequence are mutually independent. It follows that $X = σ U$ or in elements $x_{k} = σ u_{k}, 1 \leq k \leq n$ where $U$ is a Gaussian white noise sequence all elements have the same variance $σ^{2}$ .

Example 4.2. Let us consider the random sequence defined by the following difference equation and initial value

The covariance matrix has the following form

\begin{matrix} X (0) = 0, \\ X (s) - X (s - 1) = σ u (s), 1 \leq s \leq n \end{matrix}

It means the sequence has independent increments with equal $σ^{2}$ variances.

The covariance matrix has the following form

C_{X} = σ^{2} [\begin{matrix} 1 & 1 & 1 & 1 & \dots & 1 \\ 1 & 2 & 2 & 2 & \dots & 2 \\ 1 & 2 & 3 & 3 & \dots & 3 \\ 1 & 2 & 3 & 4 & \dots & 4 \\ ⋮ & ⋮ & ⋮ & ⋮ & ⋱ & ⋮ \\ 1 & 2 & 3 & 4 & \dots & n \end{matrix}] .

It is easy to verify that the elements of the covariance matrix are given by the following expression

The covariance matrix has the following form

c (i, j) = σ^{2} \min (i, j) .

Let us generalize the results of Example 4.2 to the case of not unit intervals but of length $Δ t$ . This results in a change in the notation of the coefficient $σ^{2} \to σ^{2} Δ t$ . This leads to the following form of the difference equation:

[X (s) - X (s - 1)] = σ \sqrt{Δ t} u (0),

(4.5)

and the expression for the element of the covariance matrix becomes

The covariance matrix has the following form

c (i, j) = σ^{2} \min (i Δ t, j Δ t) = σ^{2} \min (t_{i}, t_{j}) .

(4.6)

This random difference equation may be used to study the case when $Δ t \to 0$ . The result is that the sequence tends to a continuous function with no derivative in any point.

Example 4.3. Let us consider the random sequence defined by the following difference equation and initial value

\begin{matrix} A_{0} (0) = σ_{0} u (0), \\ A_{0} (s) = γ A_{0} (s - 1) + β σ_{0} u (s), 1 \leq s \leq n - 1 \end{matrix}

where $u (s)$ is an element of a white noise sequence and the condition that the elements $A_{0} (0)$ and $A_{0} (1)$ have equal variances has to be satisfied. The condition yields the relation

σ_{0}^{2} = γ^{2} σ_{0}^{2} + β^{2} σ_{0}^{2} \Rightarrow β = \sqrt{{1 - γ}^{2}}

It is easy to verify that the covariance matrix has the following form

The covariance matrix has the following form

C_{X} = σ^{2} [\begin{matrix} 1 & γ & γ^{2} & γ^{3} & \dots & γ^{n - 1} \\ γ & 1 & γ & γ^{2} & \dots & γ^{n - 2} \\ γ^{2} & γ & 1 & γ & \dots & γ^{n - 3} \\ γ^{2} & γ^{2} & γ & 1 & \dots & γ^{n - 4} \\ ⋮ & ⋮ & ⋮ & ⋮ & ⋱ & ⋮ \\ γ^{n - 1} & γ^{n - 2} & γ^{n - 3} & γ^{n - 4} & \dots & 1 \end{matrix}] .

It is easy to verify that the general expressions for the elements of the covariance matrix are

c_{i j} = γ^{| i - j |} σ_{0}^{2}

The covariance matrix has same values on all diagonals. The values depend upon the distances of the points $| i - j |$ .

Let us generalize the results of the Example 4.3 by introduction of the following change of notation in parameters in the expressions for the elements of the covariance matrix $γ = e^{- η Δ t}$ . It follows

c_{i j} = σ_{0}^{2} {[e^{- η Δ t}]}^{| i - j |} .

(4.7)

In the new notations the random difference equation becomes

\begin{matrix} A_{0} (0) = σ_{0} u (0), \\ A_{0} (s) = e^{- η Δ t} A_{0} (s - 1) + \sqrt{1 - e^{- 2 η Δ t}} σ_{0} u (s), 1 \leq s \leq n - 1 . \end{matrix}

(4.8)

This form is suitable to study the influence of the value of $Δ t$ on the behaviour of the solution. The final result is: when $Δ t \to 0$ the difference equation tends to an Itô random differential equation with a solution that is a continuous function with no derivative at any point.

Numerical examples

Example 1
The script file pwsema04 gives examples of simple random sequences: 1) White Gaussian Sequence, 2) Brownian Motion Sequence, 3) not differentiable stationary process.

Download
Scilab: pwsema04.sci
Octave/Matlab: pwsema04.m