MATH 2421: Lecture 24

Date: 2024-11-27 11:59:13

Reviewed:

Topic / Chapter:

summary

❓Questions

Notes

Limit Theorem

Laws of large numbers
- weak law of large numbers: for iid r.v. $X_{1}, X_{2}, \dots$ w/ common mean $μ$
  - then, for any $ε > 0$ $n \to \infty lim P (\frac{X _{1} + \dots + X _{n}}{n} - μ \geq ε) ⟶ 0$
- proof: under additional assumption
  - assumption: r.v. w/ finite variance $σ^{2}$ , it's given that $E [\frac{X _{1} + \dots + X _{n}}{n}] = μ$ $var (\frac{X _{1} + \dots + X _{n}}{n}) = \frac{var ( X _{1} + \dots + X _{n} )}{n ^{2}} = \frac{n σ ^{2}}{n ^{2}} = \frac{σ ^{2}}{n}$
  - from Chebyshev's inequality: $P (\frac{X _{1} + \dots + X _{n}}{n} - μ \geq ε) \leq \frac{var ([ X _{1} + \dots + X _{n} ] / n )}{ε ^{2}} = \frac{σ ^{2}}{n ε ^{2}} ⟶ 0$
Applications
- casino games
  - in long run / repetition: casino's profit = expected value of gambling
  - e.g. $35 in$ 1/38 $c han ce, w i t h$ 1 cost per trial
- insurance
- polling: estimating opinions of a large population
- stock market: average return converge to the historical average
Central limit theorem
- 👨‍🏫 one of the most remarkable result
  - stating: sum of large no. of indep. r.v.: follows approximately normal dist.
    - not only a simple method for approximation of sum
    - but also explaining remarkable fact for empirical freq.
      - of natural population exhibiting bell-shape curves
- central limit theorem: w/ iid r.v. $X_{1}, X_{2}, \dots$ w/ $μ, σ^{2}$
  - distribution of: $\frac{X _{1} + \dots + X _{n} - n μ}{σ n}$
  - tends to standard normal as $n \to \infty$ : $n \to \infty lim P (\frac{X _{1} + \dots + X _{n} - n μ}{σ n} \leq x) = \frac{1}{2 π} \int_{- \infty}^{x} e^{- t^{2} /2} d t$
    - LHS: CDF, RHS: $\frac{1}{2 π} ϕ (x)$
  - thus: $P (a < \frac{X _{1} + \dots + X _{n} - n μ}{σ n} \leq b) \approx \frac{1}{2 π} \int_{a}^{b} e^{- t^{2} /2} d t$
  - examples
    - human heigh, IS, test scores, blood pressure, etc.
  - remarks: while the theorem itself is also important
    - discoveries made on the way to the theorem were also significant
  - $B in (n, p)$ : can be approximated by normal distribution as:
    - it's sum of iid Bernoulli r.v.
Examples
- w/ $X_{1}, i = 1, 2, \dots, 10$ independent r.v.
  - each w/ uniform distribution over $(0, 1)$
  - calculate: approximation to $P (X_{1} + \dots + X_{1} 0 > 6)$
  - $μ = E (X_{i}) = \frac{1}{2}$
  - $var (X_{i}) = \frac{1}{12} = σ^{2}$ $P (X_{1} + \dots + X_{1} 0 > 6) = P (\frac{X _{1} + \dots + X _{1} 0 - 5}{10/12} > \frac{6 - 5}{10/12}) \approx P (z > 1.095) = 0.1379$
- no. of students enrolling in a psychology class: Poisson r.v. w/ $μ = 100$
  - prof. in charge: will teach the course in 2 sessions, if 120 or more enrolls
  - what's the probability of teaching 2 sessions?
  - $X \sim P o i sso n (100)$
  - $P (X \geq 120) = \sum_{k = 120}^{\infty} e^{- 100} \frac{10 0 ^{k}}{k !}$
    - exact answer, but we might better approximate
  - $X = X_{1} + X_{2} + \dots + X_{100}$ w/ $X_{i}$ iid and $X_{i} \sim P o i sso n (1)$ $P (X \geq 120) = P (X_{1} + \dots + X_{100} \geq 120) = P (X_{1} + \dots + X_{100} \geq 119.5) = P (\frac{X _{1} + \dots + X _{100} - 100}{100} \geq \frac{119.5 - 100}{100}) \approx P (z \geq 1.96) cont. correct.$
- normal approximation: w/ $X_{1}, X_{2}, \dots, X_{20}$ of independent uniformly distributed r.v. over $(0, 1)$
  - estimate / upper bound: $P (k = 1 \sum 20 X_{k} > 15)$
  - use Markov's inequality to obtain an upper bound $P (k = 1 \sum 20 X_{k} > 15) \leq \frac{E ( \sum _{k = 1}^{20} X _{k} )}{15} = \frac{\sum _{k = 1}^{20} E ( X _{k} )}{15} = \frac{20 \times \frac{1}{2}}{15} = \frac{2}{3}$
  - use Chebyshev's inequality to obtain an upper bound $Y P (k = 1 \sum 20 X_{k} > 15) = k = 1 \sum 20 X_{k} = P (Y > 15) = P (Y - 10 > 5) \leq P (∣ Y - 10∣ > 5) \leq \frac{var ( Y )}{5 ^{2}} = \frac{var ( \sum _{k = 1}^{20} X _{k} )}{5 ^{2}} = \frac{\sum _{k = 1}^{20} var ( X _{k} )}{5 ^{2}} = \frac{20 \times \frac{1}{12}}{25} = \frac{1}{15}$
  - use Central Limit Theorem to approximate $P (k = 1 \sum 20 X_{k} > 15) = P (\frac{\sum _{k = 1}^{20} X _{k} - 10}{20/12} > P (\frac{15 - 10}{20/12})) \approx P (z > 15) = 0.00005$
    - although Markov and Chebyshev's upper bound is not very tight, it's widely used
Strong law of large numbers
- best-known result in probability theory
- strong law of large numbers
  - w/ $X_{1}, X_{2} \dots$ a seq. of iid r.v.
    - each w/ finite mean $μ = E (X_{i})$
  - then, w/ probability 1: $n \to \infty lim \frac{X _{1} + X _{2} + \dots + X _{n}}{n} \to μ$
  - alternatively: $P ({n \to \infty lim \frac{X _{1} + X _{2} + \dots + X _{n}}{n} \to μ}) = 1$
Example
- suppose: seq. of independent trials of some experiment
  - let $E$ : fixed event of experiment
    - let $P (E)$ : chance of $E$ occurring on any particular trial
  - let: $X_{i} {10 if E occurs on the i th trial otherwise$
  - w/ Strong law of large numbers, w/ $P = 1$ $\frac{X _{1} + X _{2} + \dots + X _{n}}{n} \to E (X) = P (E)$
  - $\frac{X _{1} + X _{2} + \dots + X _{n}}{n}$ : corresponds to proportion of time $E$ occurs in first $n$ trials
  - thus: interpret the result as:
  - w/ $P = 1$ , limiting proportion of times that $E$ occurs: $P (E)$

Final exam remarks

Remarks
- Dec. 9th, 12:30 - 2:30 (2 hr)
  - TST Sports Ctr Arena (Seafront)
- covers: Ch. 5-8 (from continuous r.v.)
  - yet, Ch. 1-4 might be implicitly included
    - e.g. famous discrete distributions from Ch. 4
      - Be, Bin, Geom, Poisson
- conditional variance: only concept is required
- join distribution of $> 2$ r.v. / joint MFG is not covered in final
  - 👨‍🏫 Beta, Gamma, Cauchy, etc. won't be major component
- exam format
  - each question: 20 pt,
  - w/ possible subproblems
  - 6 questions
- 1 A4 size 2 sided cheat sheet allowed
- normal table: provided
  - same as one in past papers
- calculator: anything without internet access
Some policies
- seating plan to be released
- bring your SID, calculator, cheat sheet, etc.

Ch. 7 Revision

Example
- mean of hypergeometric:
  - $n$ balls chosen randomly from an urn w/ $N$ balls, where $m$ are white
  - find: expected number of white balls selected
  - find: variance of white balls selected $var (X) = var (X_{1} + \dots + X_{m}) = i = 1 \sum m var (X_{i}) + i \neq = j \sum cov (X_{i}, X_{j})$
  - $X_{i} \sim B er (n / N)$
  - $var (X) = p (1 - p) = \frac{n}{N} (1 - \frac{n}{N}) = \frac{n ( N - n )}{N ^{2}}$
  - for $i \neq = j$ $cov (X_{i}, X_{j}) E (X_{i} X_{k}) cov (X_{i}, X_{j}) = E (X_{i} X_{k}) - E (X_{i}) \cdot E (X_{j}) = E (X_{i} X_{k}) - \frac{n ^{2}}{N ^{2}} = P (X_{i} = 1, X_{j} = 1) = \frac{( n - 1 N - 1 )}{( n N )} = \frac{n !}{( n - 2 )!} \frac{( N - 2 )!}{N !} = \frac{n ( n - 1 )}{N ( N - 1 )} = \frac{n ( n - 1 )}{N ( N - 1 )} - \frac{n ^{2}}{N ^{2}}$ $var (X) = m \cdot \frac{n ( N - n )}{N ^{2}} + m (m - 1) [\frac{n ( n - 1 )}{N ( N - 1 )} - \frac{n ^{2}}{N ^{2}}] = \frac{mn ( N - n ) ( N - m )}{N ^{2} ( N - 1 )}$

MATH 2421: Probability