Probability and Statistics

Probability models uncertainty; statistics summarizes observed data. First identify the experiment, sample space, and assumptions such as independence or equal likelihood. For a distribution, match the story to its parameters before applying a formula. For data summaries, distinguish a typical value from spread and check whether the observations are weighted or equally represented.

Random variables and expectation

Idea. A random variable assigns a numerical value to each outcome. A discrete variable has a probability mass function (PMF); a continuous variable has a density whose integral over an interval gives probability. For finite expectation, $E[X]=\sum_x xP(X=x)$ or $\int xf(x)dx$; variance is $E[(X-E[X])^2]$.

Solved examples.

1. Question. A fair coin is tossed once. Let X=1 for heads and 0 for tails. Find its distribution and mean. Solution. (1) P(X=0)=P(X=1)=1/2. (2) $E[X]=0(1/2)+1(1/2)=1/2$. Answer. Bernoulli(1/2); mean 1/2. Trap. The indicator of an event has mean equal to the event's probability.
2. Question. X takes 0, 2, 5 with probabilities 0.2, 0.5, 0.3. Find E[X]. Solution. (1) Check probabilities sum to 1. (2) Weighted sum is $0(0.2)+2(0.5)+5(0.3)=2.5$. Answer. 2.5. Trap. Do not take the unweighted average of support values.
3. Question. For the same variable, find variance. Solution. (1) Mean is 2.5. (2) $E[X^2]=0+4(0.5)+25(0.3)=9.5$. (3) $Var(X)=E[X^2]-E[X]^2=9.5-6.25=3.25$. Answer. 3.25. Shortcut. The computational variance identity applies when second moment is finite.
4. Question. A density is $f(x)=2x$ for $0\le x\le1$ and 0 otherwise. Find P(X>1/2). Solution. (1) Integrate density: $\int_{1/2}^1 2x\,dx$. (2) Antiderivative is $x^2$. (3) $1-1/4=3/4$. Answer. 0.75. Trap. For a continuous variable, a point has probability zero; use area over an interval.
5. Question. If Y=3X+2, express mean and variance in terms of those of X. Solution. (1) Linearity gives $E[Y]=3E[X]+2$. (2) Adding a constant does not change variance. (3) Scaling by 3 multiplies variance by $3^2$. Answer. $E[Y]=3E[X]+2$ and $Var(Y)=9Var(X)$. Trap. Variance scales quadratically, not linearly.

Binomial and Poisson distributions

Idea. A binomial count models n independent Bernoulli trials with the same success probability p: $P(X=k)=\binom nkp^k(1-p)^{n-k}$ for integer $0\le k\le n$. A Poisson count with rate $\lambda>0$ has $P(X=k)=e^{-\lambda}\lambda^k/k!$ for nonnegative integer k; its mean and variance are both λ.

Solved examples.

1. Question. A fair coin is tossed 4 times. Probability of exactly 2 heads? Solution. (1) Trials are independent, n=4,p=1/2. (2) Apply binomial formula: $\binom42(1/2)^2(1/2)^2=6/16$. Answer. 3/8. Trap. “Exactly” uses one value k; “at most” requires summing several.
2. Question. A component fails with probability 0.1 independently. In 3 components, find probability of no failures. Solution. (1) Success may mean “no failure,” with probability 0.9. (2) All three must be nonfailures: $0.9^3$. Answer. 0.729. Trap. The binomial count can be defined for either outcome; state what counts as success.
3. Question. For $X\sim Binomial(10,0.2)$, find mean and variance. Solution. (1) Mean $np=10(0.2)=2$. (2) Variance $np(1-p)=10(0.2)(0.8)=1.6$. Answer. Mean 2, variance 1.6. Condition. This formula requires independent trials with common p.
4. Question. A server receives events at Poisson rate 2 per minute. Find probability of zero events in one minute. Solution. (1) Here λ=2. (2) At k=0, probability is $e^{-2}2^0/0!$. Answer. $e^{-2}$, about 0.1353. Trap. k! equals 1 at k=0.
5. Question. A Poisson variable has mean 3. Find P(X=2). Solution. (1) λ=3 and k=2. (2) $P=e^{-3}3^2/2!=4.5e^{-3}$. (3) Numerically this is about 0.2240. Answer. $4.5e^{-3}$. Trap. Keep the exact expression if rounding is not requested.

Uniform, normal, and exponential distributions

Idea. A continuous uniform variable on [a,b] has constant density $1/(b-a)$ and mean $(a+b)/2$. A normal variable is described by mean μ and standard deviation σ; standardization is $Z=(X-\mu)/\sigma$. An exponential waiting time with rate λ has density $\lambda e^{-\lambda x}$ for x≥0 and the memoryless property.

Solved examples.

1. Question. X is uniform on [2,8]. Find P(3≤X≤5). Solution. (1) Total interval length is 6. (2) Target length is 2. (3) Probability is length ratio $2/6=1/3$. Answer. 1/3. Condition. Endpoints have zero probability, so inclusive endpoints do not change it.
2. Question. If X is uniform on [4,10], find mean and variance. Solution. (1) Mean is $(4+10)/2=7$. (2) Variance is $(b-a)^2/12=36/12=3$. Answer. 7 and 3. Trap. Uniform variance uses interval width squared divided by 12.
3. Question. X is normal with mean 50 and standard deviation 5. Find the z-score for x=60. Solution. (1) $z=(60-50)/5$. (2) z=2. Answer. 2. Interpretation. The value is two standard deviations above the mean; a tail probability needs a normal table or CDF.
4. Question. For a standard normal Z, express P(-1≤Z≤1) using the CDF Φ. Solution. (1) CDF difference is $\Phi(1)-\Phi(-1)$. (2) By symmetry $\Phi(-1)=1-\Phi(1)$. Answer. $2\Phi(1)-1$, approximately 0.6827. Trap. The 68% rule is an approximation, not an exact identity.
5. Question. Waiting time X is exponential with rate λ=0.5 per minute. Find P(X>4). Solution. (1) Survival function is $e^{-\lambda t}$. (2) Substitute λ=0.5,t=4. Answer. $e^{-2}$, approximately 0.1353. Trap. Rate units are inverse time; λt must be dimensionless.

Mean, median, mode, and standard deviation

Idea. The mean uses every value and is sensitive to outliers; the median is the middle after sorting; the mode is a most frequent value and may not be unique. Standard deviation is the square root of variance. For a population divide squared deviations by N; for the usual unbiased sample variance divide by n−1.