STOCHASTIC PROCESSES, MEAN SQUARE CALCULUS

Let $\left\{x_t,t\in T\right\}$ be a stochastic process. For any finite set $\left[t_1,\dots ,t_n\right]$, $t_i\in T$ the joint distribution function of the random variables $x\left(t_1\right),\dots ,x\left(t_n\right)$ is called a finite dimensional distribution of the process. The stochastic process can be characterized by specifying the joint density function $p\left(x_{t_1},\dots ,x_{t_n}\right)$.

Classification of Stochastic Processes

Example 5.1. At times 0, 1, 2, … we toss a fair coin. For each time we define a random variable

We assume that the start is at $x_0=0$ and the random variable is defined as

It is clear that $x_t$, $n=0,1,2,\dots$ is a discrete parameter chain.

The most important are the first order and the second order densities $p\left(x_t\right)$ and $p\left(x_t,x_{\tau }\right)$,  $t,\tau \in T$

The mean value function (expected value) is defined as

The (auto) correlation function is defined as

The (auto) covariance function is defined as

STATIONARY PROCESSES

A process is strictly stationary if it has the same probability laws

for all finite sets $t_i\in T$ and for any $\tau \in T$.

The process is strictly stationary of order $k$ if it holds for $n\le k$. For $k=1$ the first order density is independent of time

Thus the expected value $m\left(t\right)$ is constant.

The second order density can depend only on $t_2-t_1$, thus the correlation and covariance functions are

The stochastic process $x_t,t\in T$ is said to be weakly stationary (or stationary in the wide sense, or covariance stationary) if it has finite second moments, the mean value function is constant and the correlation function is a function of distance $\tau$.

The stochastic process $x_t,t\in T$ is said to have strictly stationary increments if the process $\left\{x(t+s)-x\left(t\right),t\in T\right\}$ is strictly stationary for every $s\in T$ .

Convergence of Random Sequences

There are a number of ways in which a sequence may converge as $n\to \infty$.

The sequence $\left\{x_t\right\}$ is said to converge to $x$ with probability 1 if

for almost all realizations $\omega$. (Does not hold perhaps for event $A$ with probability $\mathrm{Pr}\left(A\right)=0$.

The sequence $\left\{x_t\right\}$ is said to converge to $x$ in probability if, for every $\epsilon >0$

The sequence $\left\{x_t\right\}$ is said to converge to $x$ in mean square if

We write $\mathrm{l.i.m.}{x}_n=x$.

The Cauchy criterion

is a necessary and sufficient condition for mean square covergence.

Mean Square Calculus

The random function $x_t$ is said to be continuous in mean square at $t\in T$ if

The correlation function of the random function is $R\left(t_1,t_2\right)$. Let us introduce a random function $y=x(t+h)-x\left(t\right)$ and calculate the cross correlation function

and then the correlation function

Finally upon substitution

It follows that for $t_1=t_2=t$

and thus the random function is mean square continuous if the correlation function is continuous at $t_1=t_2=t$.

Theorem. The random function $X_t$ is mean square continuous at $t\in T$ if, and only if $R\left(t_1,t_2\right)$ is continuous at if, and only if $\left(t,t\right)$.

Let us consider a derivative of a stochastic process.

The problem is simple when we know how to calculate the realizations of the stationary process $X\left(t\right)$ with a mean value $m\left(t\right)$ and a covariance function $R\left(t_1,t_2\right)$. If we can calculate the derivatives of all realizations, then the problem of the derivative is the problem of derivatives of a family of deterministic functions.

In view of the Cauchy criterion a sufficient condition for mean square differentiability of $X\left(t\right)$ is the convergence to zero with $h$, $s$ of $E\left\{|\frac{X\left(t+h\right)-X\left(t\right)}{h}-\frac{X\left(t+s\right)-X\left(t\right)}{s}|\right\}$.

The final result is the theorem.

Theorem. The random function $X_t$ is mean square differentiable at $t\in T$ if, and only if ${\partial }^{2}R\left(t_1,t_2\right)/\partial t_1\partial t_2$ exists at $\left(t,t\right)$.

For stationary processes it is easier to prove similar theorems.

A stationary process $X_t$ is mean square continuous at $t\in T$ if, and only if $R\left(\tau \right)$ is continuous at $\tau =0$.

A stationary process $X_t$ is mean square differentiable at $t\in T$ if, and only if $dR\left(\tau \right)/d\tau$ and $d^{2}R\left(\tau \right)/{\left(d\tau \right)}^2$ exist at $\tau =0$.

For example let us consider the stationary random process with the following covariance function

The covariance function at $\tau =0$, thus the random process is continuous. The covariance function is not differentiable for $\tau =0$. (The right side and left side derivatives are different.). The second derivative does not exist, the random function is not differentiable.

Let us consider a Riemann integral of a stochastic process.

We introduce a partition of the interval $\left[a,b\right]$, $a,b\in T$, $a\le b$, $a=t_0<t_1<\dots <t_n=b$. We denote $\rho ={\max }_i\left(t_{i+1}-t_i\right)$, and $t_i\le t_i^{,}<t_{i+1}$.

The random function is mean square Riemann integrable if the following limit, which then defines the integral, exists

Theorem. The random function $X_t$ is mean square Riemann integrable over $\left[a,b\right]$ if, and only if, $R\left(t_1,t_2\right)$ is Riemann integrable over $\left[a,b\right]\times \left[a,b\right]$ (The Riemann integral $\underset{a}{\overset{b}{\int }}\underset{a}{\overset{b}{\int }}R\left(t_1,t_2\right)d{t}_{1}d{t}_2$ exist).

If a random function $X_t$ is mean square integrable over $\left[a,b\right]$ for every $t\in \left[a,b\right]$, then

is a random function of t defined on $\left[a,b\right]$.

Theorem. The random function $X_t$ is mean square integrable over $\left[a,b\right]$ for every $t\in \left[a,b\right]$. Then $Y_t=\underset{a}{\overset{t}{\int }}{X}_{\tau }d\tau$ is mean square continuous on $\left[a,b\right]$. If $X_t$ is ms continuous on $\left[a,b\right]$, then $Y_t$ is ms differentiable on $\left(a,b\right)$ and ${\dot{Y}}_t=X_t$.

The Fundamental Theorem on ms Calculus.

Let ${\dot{X}}_t$ be ms Riemann integrable on $\left[a,b\right]$. Then

with probability one.

The Brownian Motion Process

A random function $\left\{X_t,t\in T\right\}$ has independent increments if, for all finite sets $\left\{t_i:t_i<t_{i+1}\right\}\in T$ the random variables (vectors) $X_{t_2}-X_{t_1},X_{t_3}-X_{t_2},\dots ,X_{t_n}-X_{t_{n-1}}$ are independent.

The $\left\{X_t,t\in T\right\}$ process has stationary independent increments if, in addition $X_{t+h}-X_{\tau +h}$ has the same distribution as $X_t-X_{\tau }$ for every $t>\tau \in T$ and every $h$.

A random function $B\left(t\right)$, $t\ge 0$ is a Brownian motion (Wiener or Wiener-Lévy) process if

It follows from (ii) that $X\left(t\right)-X\left(\tau \right)$ is also normally distributed for every $t,\tau \ge 0$. It remains to specify the distribution of increments $X\left(t\right)-X\left(\tau \right)$ for all $t>\tau \ge 0$. The mean, $E\left\{X\left(t\right)-X\left(\tau \right)\right\}$ in view of (iii) is zero. From the definition it follows:

The function does not decrease when $t$ increases. Thus the equation

has a solution

It is the only solution.

Finally the mean value of the increment is zero $m\left(t\right)=0$ and its variance is $\mathrm{var}\left\{B\left(t\right)-B\left(\tau \right)\right\}={\sigma }^2\left(t-\tau \right)$.

Let us calculate the correlation (the covariance is the same) function. For $t>\tau$

Therefore in general for the Brownian motion process the correlation function is

The correlation function is continuous for every $\left(t,\tau \right)$. Thus the process is mean square continuous on $\left[0,\infty \right]$. The second derivative ${\partial }^{2}R\left(t,\tau \right)/\partial t\partial \tau$ exists at no $\left(t,t\right)$, thus the process is ms differentiable nowhere. It is Riemann integrable on every $t$ interval.

To compute the values we have to chose the time increment $\Delta t$ and and to write the covariance function in terms of the increment

This leads to the following form of the difference equation:

where $u\left(s\right)$ is a term of the Gaussian white noise sequence with mean value zero and unit variance. The expressions (5.18) and (5.19) reduce the problem in continuum to the solution of a difference equation as described in chapter 4 (compare with the relations (4.6) and (4.5)).

It should be noted that $\left[X\left(s\right)-X\left(s-1\right)\right]/\sqrt{\Delta t}=\sigma u\left(s\right)$ and this expression is an invariant with respect to the choice of the increment $\Delta t$, and not the relation for the derivative in finite differences.