A **stochastic process** is a series of developments over time that follow probabilistic laws. - Unlike other statistical models, a stochastic process has its observations ordered by time, and are in general not possible to draw two samples at the same moment in time. Modeling an underlying stochastic process is a common method of time-series analysis. - The set of all "possible" time series that could have been realized, given the stochastic process, is called an **ensemble**. ### Describing Stochastic Processes Stochastic processes $(X_{t})$ can often be described as a function of past values and an unobservable error process $\epsilon_{t}$: $X_{T}=f(X_{1},\dots,X_{T-1},\epsilon_{T})$ Useful metrics of the process include the mean $\mu_{t}=\mathbb{E}[X_{t}]$, the variance $\sigma^{2}_{t}=\mathrm{Var}(X_{t})$, and the **autocovariance** $\gamma(t_{1},t_{2})=\mathbb{E}[(X_{t_{1}}-\mu_{t_{1}})(X_{t_{2}}-\mu_{t_{2}})]$which is analogous to the [[Autocorrelations|autocorrelations]] of a particular time series, which can be an [[Estimating Stochastic Processes#Estimating Autocorrelations|estimate of the theoretical acv.f.]] of the underlying stochastic process. ### Stationarity A stochastic process $(X_{t})$ is **stationary** if the joint distribution of $X(t_{1}),\dots,X(t_{k})$ is independent of a shift $\tau$, for any $t_{1},\dots,t_{k},\tau$. - For $k=1$, this requires a constant mean $\mu_{t}=\mu$ and variance $\sigma^{2}_{t}=\sigma^{2}$. - For $k=2$, this means that the joint distribution of $X(t_{1}),X(t_{1}+\tau)$ only depends on the **lag** $\tau$, and in particular so is $\gamma(t_{1},t_{1}+\tau)=\gamma(\tau)$, the **autocovariance coefficient at lag** $\tau$ **(acv.f.)**. - This coefficient can be standardized to be the **autocorrelation function (ac.f.)** $\rho(\tau):= \frac{\gamma(\tau)}{\gamma(0)}$ Approximately stationary distributions commonly arise when the process has a "limit" called an **equilibrium distribution**, where the distribution change very little when $t$ is large. A **second-order stationary** or **weakly stationary** distribution has a constant mean $\mu$ and its acv.f. $\rho(\tau)$ only depends on the lag $\tau$. - These are necessary but non-sufficient for a truly stationary time series. - However, for normal processes where joint distributions are Gaussians, this characterization is sufficient for strict stationarity.