> [!note] Setup > Fix a probability triple $(\Omega, \mathcal{F}, \mathbb{P})$, and $m\mathcal{F}$, the set of measurable functions $\Omega \to \mathbb{R}$. > $m \mathcal{F}^{+}$ is the subset of non-negative measurable functions. > A random variable $X$ is just any function in $m \mathcal{F}$. > [!definition|*] Expectations > The **expectation** of $X$ is $\mathbb{E}[X]:= \int _{\Omega}X ~ d\mathbb{P}, $and the **probability** of the event $A \in \mathcal{F}$ is $\mathbb{P}[A]:=\mathbb{E}[\mathbf{1}_{_{A}}]$. ### Convergence Theorems Since expectations are just integrations, the convergence theorems also hold for them: **Monotone Convergence Theorem**: if $(X_{n}) \in m \mathcal{F}^{+}, (X_{n}) \nearrow X ~\mathrm{a.e.}$, then $\mathbb{E}[X_{n}] \to X$. **Dominated Convergence Theorem**: if $(X_{n}) \in m\mathcal{F}^{+}$ and $|X_{n}| \le Y$ for some $Y \in m \mathcal{F}$, then $(X_{n}) \to X$ implies $\mathbb{E}[|X_{n}-X|] \to 0$. **Fatou's Lemma**: if $(X_{n}) \subseteq m \mathcal{F}$, then $\mathbb{E}[\liminf_{n} X_{n}] \le \liminf_{n} \mathbb{E}[X_{n}]$. ### Inequality Theorems > [!theorem|*] Markov's Theorem > If $X \in m \mathcal{F}^{+}$ and $g:\mathbb{R} \to [0,\infty]$ is measurable and non-decreasing, then $\mathbb{E}[g(X)] \ge \mathbb{E}[g(X)\mathbf{1}_{X \ge c}] \ge g(c)\cdot\mathbb{P}[X \ge c].$ In particular with $g:X\mapsto X$ and $g: X \mapsto \exp(\theta X)$, $\begin{align*} \mathbb{E}[X] &\ge c \cdot\mathbb{P}[X \ge c]\\[0.4em] \mathbb{P}[X \ge c] &\le \exp(-\theta c)\cdot\mathbb{E}[\exp(\theta X)] , \end{align*}$where choosing the optimal $\theta$ can give decent bounds. > [!theorem|*] Jensen's Inequality > If $f: I \to \mathbb{R}$ is convex on an interval $I$, and $X: \Omega \to I$ is an integrable RV, then $f(\mathbb{E}[X]) \le \mathbb{E}[f(X)].$ > > > [!proof]- > > First note that at any point $m \in I$, there is a tangent line that lies below the graph of $f$. > > If for contradiction $\exists x < m < y$ such that$\frac{f(m)-f(x)}{m-x}\ge \frac{f(m)-f(y)}{m-y},$![[Convexity.png|w50|center]] > > then $x,y,m$ form a triangle, and the chord $x,y$ goes under the graph of $f$. > > > > Hence setting $\mu=m$, we can always have $s: \sup_{x < \mu} \frac{f(\mu)-f(x)}{\mu-x} \le s \le \inf_{y > \mu} \frac{f(\mu)-f(y)}{\mu-y}.$ > > Then with slope $s$, $f(X)-f(\mu) \ge s(X - \mu).$ Taking the expectation gives $\mathbb{E}[f(X)] - f(\mathbb{E}[X]) \ge 0.$ ### $\mathcal{L}^{p}$ Spaces **Monotonicity of $\mathcal{L}^{p}$ norms**: if $1 \le p < r <\infty$, $X \in \mathcal{L}^{r}$, then $X \in \mathcal{L}^{p}$, and $\| X \|_{p} \le \| X \|_{r}$. For any $p \in [1,\infty)$, $\mathcal{L}^{p}$ is complete. ### Expectations Under Independence For independent random variables $X, Y$, their (measure-theoretic) expectations follow the classic identities: - $\mathbb{E}[XY]=\mathbb{E}[X] \cdot \mathbb{E}[Y]$. > [!proof] > By considering $X=X^{+} -X^{-}$ (same for $Y$), WLOG assume $X,Y$ are non-negative. > > Let $\mathcal{A} := \sigma(X)$, $\mathcal{B} := \sigma(Y)$. Then approximate $X, Y$ with simple functions $\begin{align*} f_{i}(\omega)&:= \sum_{p}a_{p}\mathbf{1}_{A_{i,p}},\\ g_{i}(\omega) &:= \sum_{p}b_{p} \mathbf{1}_{B_{i, p}} \end{align*}$for families of sets $\{ A_{i,p} \} \subset \mathcal{A}$, $\{ B_{i,p} \} \subset \mathcal{B}$, and $f_{i} \nearrow X,g_{i}\nearrow Y$. Then the identity $\mathbb{E}[f_{i}g_{i}]=\mathbb{E}[f_{i}]\cdot \mathbb{E}[g_{i}]$holds for all $i$ by switching the (finite) summation and expectation, then noting that $\mathbb{E}[\mathbf{1}({{A_{i,p_{1}}}})\cdot \mathbf{1}(B_{i, p_{2}})]=\mathbb{P}[A_{i, p_{1}} \text{ and }B_{i, p_{2}}]= \mathbb{P}[A_{i, p_{1}}] \cdot \mathbb{P}[B_{i, p_{2}}]$as $\mathcal{A}, \mathcal{B}$ are independent. Lastly, let $i \to \infty$ and apply the MCT.