Random variables, distributions, CDFs and joint densities

Random variables and the CDF

A random variable $X$ gives each outcome of an experiment a number.

• A discrete random variable takes countably many values and has a PMF $p(x) = P(X = x)$.
• A continuous random variable has a PDF $f(x) \ge 0$ with $P(a \le X \le b) = \int_a^b f(x)\,dx$ and $\int_{-\infty}^\infty f = 1$.

The cumulative distribution function $F(x) = P(X \le x)$ works for both kinds.

• $F$ is non-decreasing and right-continuous, with $F(-\infty) = 0$ and $F(\infty) = 1$.
• $P(a < X \le b) = F(b) - F(a)$.
• For a continuous $X$: $f(x) = F'(x)$, and $P(X = a) = 0$. So $<$ and $\le$ give the same probabilities.
• For a discrete $X$: $F$ is a step function, and each jump equals $P(X = x)$.

Example 1. $F(x) = 1 - e^{-2x}$ for $x \ge 0$.

• The PDF is $f(x) = 2e^{-2x}$: an exponential distribution with rate 2.
• $P(1 < X \le 2) = e^{-2} - e^{-4} \approx 0.1353 - 0.0183 = 0.117$.
• The median $m$ solves $1 - e^{-2m} = \frac12$, so $m = \frac{\ln 2}{2} \approx 0.347$. That is below the mean of 0.5, because the distribution is skewed right.

Example 2: a step CDF. $X$ takes the values 1, 2 and 4 with probabilities 0.2, 0.5 and 0.3. Then:

• $F$ jumps to 0.2 at 1, to 0.7 at 2, and to 1 at 4;
• $F(3) = 0.7$;
• $P(X > 1.5) = 1 - 0.2 = 0.8$.

Expectation and variance rules

• Definitions: $E[X] = \sum x\,p(x)$ or $\int x f(x)\,dx$.
• LOTUS: $E[g(X)] = \sum g(x)p(x)$ or $\int g(x) f(x)\,dx$. There is no need to find the distribution of $g(X)$.
• Linearity: $E[aX + bY + c] = aE[X] + bE[Y] + c$. This always holds, even for dependent variables.
• Variance: $\operatorname{Var}(X) = E[X^2] - (E[X])^2$ and $\operatorname{Var}(aX + b) = a^2\operatorname{Var}(X)$.
• $\operatorname{Var}(X + Y) = \operatorname{Var}X + \operatorname{Var}Y + 2\operatorname{Cov}(X, Y)$. The covariance term vanishes for independent variables.
• For independent $X$ and $Y$: $E[XY] = E[X]E[Y]$.

Example 3. For $X$ in Example 2:

• $E[X] = 0.2 + 1.0 + 1.2 = 2.4$;
• $E[X^2] = 0.2 + 2.0 + 4.8 = 7$;
• $\operatorname{Var}(X) = 7 - 5.76 = 1.24$;
• $\operatorname{Var}(3X - 5) = 9 \times 1.24 = 11.16$.

The distributions to know

Distribution	PMF / PDF	Mean	Variance
Bernoulli($p$)	$P(1) = p$, $P(0) = 1 - p$	$p$	$p(1 - p)$
Binomial($n, p$)	$\binom{n}{k}p^k(1 - p)^{n-k}$	$np$	$np(1 - p)$
Geometric($p$): trials up to the first success	$(1 - p)^{k-1}p$, for $k \ge 1$	$1/p$	$(1 - p)/p^2$
Poisson($\lambda$)	$e^{-\lambda}\lambda^k/k!$	$\lambda$	$\lambda$
Discrete uniform on $\{1, \dots, n\}$	$1/n$	$\frac{n + 1}{2}$	$\frac{n^2 - 1}{12}$
Uniform($a, b$)	$\frac{1}{b - a}$	$\frac{a + b}{2}$	$\frac{(b - a)^2}{12}$
Exponential($\lambda$)	$\lambda e^{-\lambda x}$	$1/\lambda$	$1/\lambda^2$
Normal($\mu, \sigma^2$)	$\frac{1}{\sigma\sqrt{2\pi}}e^{-(x - \mu)^2/2\sigma^2}$	$\mu$	$\sigma^2$
$\chi^2_k$	sum of $k$ squared standard normals	$k$	$2k$
$t_\nu$	$Z/\sqrt{\chi^2_\nu/\nu}$	0 (for $\nu > 1$)	$\frac{\nu}{\nu - 2}$ (for $\nu > 2$)

Discrete examples

Example 4: binomial. With $n = 5$ and $p = 0.3$: $P(X = 2) = \binom52(0.3)^2(0.7)^3 = 10 \times 0.09 \times 0.343 = 0.3087$.

Example 5: Poisson. Calls arrive at $\lambda = 3$ per minute.

• $P(0) = e^{-3} \approx 0.0498$.
• $P(X \le 1) = e^{-3}(1 + 3) = 4e^{-3} \approx 0.199$.
• $P(X \ge 2) = 1 - 0.199 = 0.801$.