Introduction

information

These are my notes on MATH 2421: Probability

Course information

Course code: MATH 2421
Course title: Probability
Semester: 24/25 Fall
Credit: 4
Grade: A-F
TMI
- Prerequisite: MATH 1014 OR MATH 1020 OR MATH 1024
- Corequisite: MATH 2011 OR MATH 2023
- Exclusion: IEDA 2520, MATH 2431, ELEC 2600, ELEC 2600H, ISOM 3540

Description

The course covers the basic principles of probability theory. Topics include combinatorial analysis used in computing probabilities, the axioms of probability, conditional probability and independence of events, discrete and continuous random variables; joint, marginal, and conditional densities, moment generating function; binomial, Poisson, gamma, exponential, Gamma, Beta, Cauchy, univariate, and bivariate normal distributions; laws of large numbers; central limit theorem

My section

Section: L1 / T1B
Time:
- Lecture: MoWe 12:00PM - 01:20PM
- Tutorial: Tu 06:00PM - 06:50PM
Venue:
- Lecture: LG3009, Lift 10-12
- Tutorial: Rm 4580, Lift 27-28
Instructor: WANG, Ke
- Room: Rm 3434
- Email: [email protected]
Teaching Assistants:
- CHEONG, Kha Man ([email protected])

Grading scheme

Assessment Task	Percentage
5 HW	15%
Mid-Term	35%
Final Exam	50%

Required texts

"A First Course in Probability" (8th or other Edition) by Sheldon Ross (Publisher: Prentice Hall)

Optional resources

N/A

MATH 2421: Lecture 1

Date: 2024-09-02 03:12:22

Reviewed:

Topic / Chapter: Introduction & Principle of Counting

summary

❓Questions

can you elaborate more on "combinatorial proof"? What is the difference between algebraic ad combinatorial?

Notes

Probability by Combinatorial Analysis: Introduction

What is, and why probability?

probability: how likely something is to happen
- why should we learn it?
  1. randomness becomes prevalent in real life
    - e.g. revenue of Apple store at NPM next month: random
  2. probability: fundamental tool for modelling, analyzing randomness
    - e.g. what is the most likely revenue of apple store at APM next month?
  3. probability: fundamental tool for statistics, ML, etc.
Intuitions on probability
- what is the chance of selecting a red ball, among 1 green, 4 blue, and 4 red balls?
  - $\frac{4}{9} = \frac{no. of reds}{no. of balls}$
- what if we select 2 balls simultaneously?
  - what's the change that both balls are red, from 2 blue, 2 red, and 1 green balls?
  - 👨‍🎓: $\frac{2}{5} \cdot \frac{1}{4} = \frac{1}{10}$
    - 👨‍🏫: correct!
Combinations
- case study: a communication system is of $n$ identical antennas in a linear order
  - the system runs as long as no 2 conseq. antennas are defective
  - and exactly $m$ out of $n$ antennas are defective
  - what is the probability that the system does not fail?
- for a special case: m=2; n=4
  - all possible config. are:
    
    0 1 1 0
    
    0 1 0 1
    
    1 0 1 0
    
    0 0 1 1
    
    1 0 0 1
    
    1 1 0 0
  - in this case: $\frac{3}{5} = \frac{1}{2} = 50%$
- for general $n, m$ , the probability is
  - $= \frac{no. of well-working config.}{no. of all possible config.}$
- 👨‍🎓 programming style: can we use DP / recursion or Markov chain..?
- combinatorial analysis : mathematical theory of counting

Principles of Counting

Experiments
- experiment
  - used to denote a process whose outcome is random
  - examples
    - randomly toss a coin, roll a die, etc.
- basic principle of counting
  - With two experiments performed, one resulting in any of $m$ possible outcomes , while another in any of $n$ possible outcomes , then together there are $mn$ possible outcomes of the two experiments
  - can be proved by enumerating all possible outcomes as pairs
- generalized basic principle of counting
  - With $r$ experiments performed, and $\forall i \in [1, r]$ experiment $Exp_{i}$ resulting in any of $n_{i}$ possible outcomes, then together there are $\prod_{i = 1}^{r} n_{i}$ possible outcomes of the $r$ experiments

Permutations

Permutations
- permutation: how many different (distinct) ordered arrangement of items are possible
- w/ $n$ distinct objects, the total number of different permutations are:
  - $n (n - 1) (n - 2) \dots 3 \cdot 2 \cdot 1 = n!$
  - where $0! = 1$ by convention
  - proof: experiment being equivalent to arrange $n$ objects into $n$ positions
- for $n$ objects which $\forall i \in [1, r]$ $n_{i}$ are alike, there are $\frac{n !}{\prod _{i = 1}^{r} n _{i} !}$
  - proof: first compute permutation of $n$ objects
  - then. for a set of same objects, consider themselves as a (sub) permutation too
  - thus, divide it by no. of permutations that $n_{i}$ objects can have
- example: how many ways to rearrange the word "Mississippi"?
  - $\frac{11 !}{1 ! 4 ! 2 ! 4 !}$ (for M, s, p, and i, respectively)
Seating in circle
- now, only the relative position matters, not linear
- to count no. of arrangements, we can first compute all "linear" permutations
  - and divide it by no. of seats (e.g. where index 0 can be located)
  - thus, there are $(n - 1)!$ possible arrangements!
- if it is a necklace: $\frac{( n - 1 )!}{2}$ as necklace (unlike Chinese tables) can be flipped!

Combinations

Introduction
- how many ways to choose $r$ items from $n$ , when the order does not matter?
  - total permutation: $\frac{n !}{( n - r )!}$
  - no. of orderings: $r!$
  - no. of combination: $\frac{n !}{( n - r )! r !}$
- theorem: w/ $n$ distinct objects, no. of possible groups of choosing $r$ items are given by: $\frac{n !}{( n - r )! r !}$
  - also denoted by $_{n} C_{r}$ or $(r n)$
    - latter one: aka binomial coefficient
  - read " $n$ choose $r$ "
- properties
  - for $r = 0, 1, \dots n$
    - $(r n) = (n - r n)$
  - $(0 n) = (n n) = 1$
  - if $r < 0∣∣ r > n$ : $(r n) = 0$
Useful combinatorial identities
- theorem: for $1 \leq r \leq n$ :
  - $(r n) = (r - 1 n - 1) + (r n - 1)$
  algebraic proof
  
  $(r - 1 n - 1) = \frac{( n - 1 )!}{( n - r )! ( r - 1 )!} = \frac{( n - 1 )!}{( n - r - 1 )! ( r - 1 )!} \frac{1}{n - r} - - - (r n - 1) = \frac{( n - 1 )!}{( n - r - 1 )! ( r )!} = \frac{( n - 1 )!}{( n - r - 1 )! ( r - 1 )!} \frac{1}{r} - - - RHS = \frac{( n - 1 )!}{( n - r - 1 )! ( r - 1 )!} (\frac{1}{n - r} + \frac{1}{r}) = \frac{( n - 1 )!}{( n - r - 1 )! ( r - 1 )!} \frac{r + n - r}{( n - r ) r} = \frac{( n - 1 )!}{( n - r - 1 )! ( r - 1 )!} \frac{n}{( n - r ) r} = \frac{( n )!}{( n - r )! ( r )!} = (r n) = L H S$
  combinatorial proof
  - or: cheat bijections
  - $(r n)$ : no. of ways of choosing $r$ no. from $1, 2, \dots n$
  - count in a different ways:
    - case 1: 1 is chosen $(r - 1 n - 1)$
    - case 2: 1 is not chosen $(r n - 1)$
- theorem:
Examples
- example 1: committee
  - a committee of 3: from a group of 20 ppl
  1. how many possible committees?
    - $_{n} C_{r} =_{20} C_{3} = \frac{20 !}{17 ! 3 !} = \frac{20 * 19 * 18}{6} = 20 * 19 * 3 = 1140$
  2. Peter and Paul refuse to serve in the same committee; how many possibilities considering that?
    - $1140 - 18 = 1122$
    - 18: no. of committee where two guys serve together
    - alternatively: you can choose
      - when neither are in the committee + one of P & P are in committee
      - $(3 18) + (1 2) (3 19)$
- example 2: antenna defection
  - total $n$ antenna; $m$ defective & $n - m$ functional; all functional / defective antenna are indistinguishable among them. how many linear orderings w/ no two consec. defectives?
  - procedure
    1. align functional ones in linear order
    2. and, there are $n - m + 1$ spots where defectives could be located
    3. thus: $(m n - m + 1)$

MATH 2421: Lecture 2

Date: 2024-09-04 01:57:45

Reviewed:

Topic / Chapter: Combinations

summary

❓Questions

Notes

Combinations (cont.)

Useful combinatorial identities (cont.)
- binomial theorem: let $n \in Z^{+}$ , then
  
  $(x + y)^{n} = k = 0 \sum n (k n) x^{k} y^{n - k}$
  - $(k n)$ : often referred to as the binomial coefficient
  - combinatorial proof $(x + y)^{n} = x^{3} + 3 x^{2} y + 3 x y^{2} + y^{3} x^{2} + x y + y x + y^{2} (x + y) (x + y) (x + y) \dots (x + 1)$
    - $k$ brackets contribute $x$ , $n - k$ brackets for $y$
Problems
- how many subsets are there of a set consisting of $n$ elements?
  - intuition: there are $(k n)$ ways of choosing $k$ elements from $n$
  - alternatively: you can assign following binary values to each elements
    - $b_{i} {1, if element i is included 0, if element i is not included$
    - and there are exactly $2^{n}$ combinations!
  $\sum_{k = 0}^{n} (k n) = 2^{n}$
- show that following equation holds true
  - $\sum_{k = 0}^{n} (- 1)^{k} (k n) = 0$
  - proof (algebraic) $0 = (1 - 1)^{n} = \sum_{k = 0}^{n} (- 1)^{k} (k n)$
- $(0 n) + (2 n) + (4 n) \dots = (1 n) + (3 n) + (5 n) \dots$
  - can be shown from the previous proof
  - 👨‍🏫 can you show combinatorial proof?

Multinomial Coefficients

Multinomial coefficients
- a set of $n$ distinct items: to be divided into $r$ different groups of respective size $(n_{1}, n_{2}, \dots, n_{r})$ where $\sum n_{i} = n$
  - how many divisions are possible? $= (n _{1} n) (n _{2} n - n _{1}) (n _{3} n - n _{1} - n _{2}) \dots (n _{r} n - n _{1} \dots - n _{r - 1}) = \frac{n !}{n _{1} ! ( n - n _{1} )!} \frac{( n - n _{1} )!}{( n - n _{1} - n _{2} )! n _{2} !} \dots \frac{n _{r} !}{0 ! n _{r} !} = \frac{n !}{n _{1} ! n _{2} ! \dots n _{r} !}$
alternatively:
- it's the same as arranging $n$ objects linearly ( $n!$ )
- and putting the first $n_{1}$ objects to group 1, next $n_{2}$ objects to group 2, etc.
  - as the order doesn't matter within the set, you can divide it by $n_{1}!, n_{2}!,$ and so forth
let $(n _{1} , n _{2} , \dots , n _{r} n) := \frac{n !}{n _{1} ! n _{2} ! \dots n _{r} !}$
- and call it multinomial coefficients
- i.e. no. of ways in dividing $n$ elements into $r$ subgroups of each size $n_{i}$ , total in $n$
Theorem
- multinomial theorem $(x_{1} + x_{2} + \dots + x_{r})^{n} = (n_{1}, \dots, n_{r}) : n_{1} + \dots + n_{r} = r \sum (n _{1} , n _{2} , \dots , n _{r} n) x_{1}^{n_{1}} x_{2}^{n_{2}} \dots x_{r}^{n_{r}}$
  - exercise: expand $(x_{1} + x_{2} + x_{3})^{5} = ?$
  - exercise 2: expand $(x_{1} + x_{2} + x_{3} + x_{4} + x_{5})^{3} = ?$
Problems
- police department: of 10 officers
  - each group: size of 5,3,2
  - how many divisions exist?
  - $\frac{10}{5 , 3 , 2} = \frac{10 !}{5 ! 3 ! 2 !} = 7 * 8 * 9 * 10/2 = 2520$
- ten children: divided into team A, B of 5 groups. how many divisions?
  - $(5 , 5 10) = \frac{10 !}{5 ! 5 !} = 252$
- 10 children at playground divide themselves into teo teams of 5 each. how many divisions?
  - i.e. order of team A / B doesn't matter
  - $252/2 = 126$

Number of Integer Solutions

Theorem
- theorem: there are $(r - 1 n - 1)$ distinct positive integer-valued vectors that satisfies the equation
  - $x_{1} + x_{2} + \dots + x_{r} = n$
  - where $x > 0$ for $i = 1, \dots r$
  - proof: stars and bars method
    - put undistinguishable $n$ stars into $r$ bins (labeled from $1.. = r$ )
      - s.t. $n$ bin is empty
    - let $x_{i}$ be the number of stars in the $i$ th bin ( $1.. = r$ )
    - it's the same as placing $r - 1$ bars on $n - 1$ slots (between stars)
- theorem there are $(r - 1 n + r - 1)$ distinct non-negative integer-valued vectors that satisfies the equation
  - proof
    - change "non-negative" into "positive" $x_{1} + x_{2} + \dots + x_{r} (y_{1} - 1) + (y_{2} - 1) + \dots + (y_{r} - 1) y_{1} + y_{2} + \dots + y_{r} - r y_{1} + y_{2} + \dots + y_{r} = n = n = n = n + r$
    - and $y_{i}$ is a positive integer
    - thus $(r - 1 n + r - 1)$
  - or, stars and bars method: now consider bars as objects too
    - it's basically aligning $n$ stars and $r - 1$ bars
Problems
- investor: w/ 20K dollars to invest among 4 possible investment, in unit of thousand dollars.
  - if the total of 20k is to be invested: how man strategies (only holding) are possible?
    - 4 slots, with 20 stars
    - $(3 20 + 4 - 1) = \frac{23 !}{20 ! 3 !} = \frac{21 * 22 * 23}{6} = 1771$
  - what if not all the money need be invested?
    - same as creating another investment of "nothing"
    - $(4 20 + 5 - 1) = \frac{24 !}{20 ! 4 !} = \frac{21 * 22 * 23 * 24}{24} = 10626$
- there are $n$ antennas with $m$ defective among them
  - how many orders there exist in which no two defectives are consecutive?
  - $n - m + 1$ slots, $m$ bars
  - $(m n - m + 1)$
- 👨‍🏫 other approach
  - .

Axioms of Probability: Introduction

Recap
- Ch. 1: used intuitive definitions to calculate probability
- translated "calculate probability" into counting no. of total outcomes & no. of interesting outcomes
- however: it can't deal w/ experiments where no. of total outcome & interesting outcomes are $\infty$
- we shall rigorously formalize probability
- terminologies
  - random experiments outcomes
  - event sample space
  - rigorous definition of probability

Sample Space and Events

Terminology
- experiment: activity or procedure that produces distinct & well-defined possibilities
  - i.e. outcomes
  - basic object of probability
- sample space: set of ALL possible outcomes of an experiment
  - often denoted by $S$
  - e.g. head / tail on coin tossing
- event: any SUBSET $E$ of sample space
  - if random experiment produces an outcome in event $E$
    - we say "event $E$ occurs"
- size of sample space / event: can be finite or infinite

MATH 2421: Lecture 3

Date: 2024-09-09 12:46:15

Reviewed:

Topic / Chapter: Event Operations and Probability

summary

❓Questions

why do we need to use infinity in defining probability when we can use the concept of exhaustive events?
- 👨‍🏫 They (use of $\infty$ or $n$ ) are equivalent and it's matter of personal preference.
- We have used infinity version to follow the textbook

Notes

Operations on Events

Four basic operations
- result of operation: also a collection of outcome = event
- union: $A \cup B$
  - either event $A$ or event $B$ occurs
- intersection: $A \cap B$
  - both event $A$ and event B occur
  - short: $A B$
- complement: $A^{c}$
  - even A does NOT occur
- symmetric difference: $A Δ B = (A \cup B$ \ $A B)$
  - EITHER $A$ or $B$ occurs, but not both
Additional terms
- inclusion of events
  - $\forall e \in E, e \in F ⟹ E \subset F$ or $F \supset E$
  - i.e. occurrence of $E$ implies $F$
- disjoint
  - when two events share no common outcomes
  - $A \cap B = \emptyset$
- mutually exclusive
  - when events $A_{1}, A_{2}, \dots, A_{k}$ are pairwise disjoint
  - $\forall i, j \in [1, k] \land i \neq = j, A_{i} \cap A_{j} = \emptyset$
- exhaustive
  - when union of some events $A_{1}, A_{2}, \dots, A_{k}$ is the sample space
  - $⋃_{i = 1}^{k} A_{i} = S$
- partition
  - when events $A_{1}, A_{2}, \dots, A_{k}$ are both exclusive and exhaustive
  - $⋃_{i = 1}^{k} A_{i} = S$ and $\forall i, j \leq N \land i \neq = j, A_{i} \cap A_{j} = \emptyset$
Fundamental laws
- commutative law
  - $A \cap B = B \cap A$
  - $A \cup B = B \cup A$
- associative law
  - $A \cap (B \cap C) = (A \cap B) \cap C$
  - $A \cup (B \cup C) = (A \cup B) \cup C$
- distributive law
  - $A \cap (B \cup C) = (A \cap B) \cup (A \cap C)$
  - $A \cup (B \cap C) = (A \cup B) \cap (A \cup C)$
- DeMorgan's law
  - $(⋃_{i = 1}^{k} A_{i})^{c} = ⋂_{i = 1}^{k} A_{i}^{c}$
  - $(⋂_{i = 1}^{k} A_{i})^{c} = ⋃_{i = 1}^{k} A_{i}^{c}$
Lemma
- for any events $A, B$ : $A = (A \cap B) \cup (A \cap B^{c})$
  - proof: as above is $= A \cap (B \cup B^{c}) = A$
    - following distributive law

Axioms of Probability

Probability definition
- probability: function that assigns numbers to events
  - numbers: characterize how likely this event occurs
- primitive definition
  - assuming an experiment w/ sample space $S$ can be repeated
  - for each even $E \in S$ , $n (E)$ : number of times $E$ occured in the first $n$ repetitions
  - probability is defined by:
  - $P (E) = lim_{n \to \infty} \frac{n ( E )}{n}$
  - l.e. limiting proportion of time that $E$ occurs
- problems
  - how do we know the limit of $\frac{n ( E )}{n}$ exists or not, for a seq. of repetitions?
  - even if limits exist for all sequences, how do we know that the limits are the same = i.e. converge?
- Kolmogorov Axiom (modern probability definition)
  - by Andrey Kolmogorov in 1933
  - probability, denoted by $P$ is a function on the collection of events satisfying
  1. for any event $A$ : $0 \leq P (A) \leq 1$
  2. let $S$ be the sample space, then: $P (S) = 1$
  3. for any sequence of mutually exclusive events $A_{1}, A_{2}, \dots$ $P (i = 1 ⋃ \infty A_{i}) = i = 1 \sum \infty P (A_{i})$
    - also written as, for all $N = 1, 2, \dots$ $P (i = 1 ⋃ N A_{i}) = i = 1 \sum N P (A_{i})$
- event $A ⟶ P (\cdot) [0, 1]$
Example
- on a fair six-faced dice, when $A = {1, 3}$ , $B = {2, 4}$
  - by Kolmogorov's axiom, what are $P (A)$ and $P (B)$ ?
  - $\forall i \in [1, 6], P (i) = 1/6$
  - as each faces are mutually exclusive (cannot have two or more faces at once)
  - $P (A) = P (1 \cup 3) = P (1) + P (3) = \frac{1}{3}$
  - $P (A) = P (2 \cup 4) = P (2) + P (4) = \frac{1}{3}$

Properties of Probability

Theorem
- $P (\emptyset) = 0$
  - proof: take $A_{1} = S$ , $A_{2} = A_{3} = \dots = \emptyset$
  - $A_{1}, A_{2}, \dots$ are mutually exclusive
  - by axiom 3: $P (S) 0 P (\emptyset) = P (i = 1 ⋃ \infty A_{i}) = P (A_{1}) + P (A_{2}) + \dots = P (S) + P (A_{2}) + \dots = P (A_{2}) + \dots = 0$
- for any finite sequence of mutually exclusive events $A_{1}, A_{2}, \dots, A_{n}$
  - $P (⋃_{i = 1}^{n} A_{i}) = \sum_{i = 1}^{b} P (A_{i})$
  - proof: let $\forall m > n, A_{m} = \emptyset$
  - $A_{1}, A_{2}, \dots$ : still mutually exclusive
  - apply axiom 3 to find: $P (i = 1 ⋃ \infty A_{i}) = i = 1 \sum \infty P (A_{i}) = i = 1 \sum n P (A_{i}) // as \forall m \geq n, P (m) = P (\emptyset) = 0$
- let $A$ be an event, then $P (A^{c}) = 1 - P (A)$
  - proof: $S = A \cup A^{c}$
  - $A \cap A^{c} = \emptyset$
  - $P (S) = P (A \cup A^{c}) = P (A) + P (A^{c})$
  - $⟹ P (A^{c}) = P (S) - P (A) = 1 - P (A)$
- if $E \in F$ , then
  - $P (E) \leq P (F)$
  - proof: $F = FE \cup F E^{c} = E \in F E \cup F E^{c}$
  - since $E \cap F E^{c} = \emptyset$
  - by Axiom 3, $P (F) = P (E \cup F E^{c}) = P (E) + \geq 0 P (F E^{c})$
  - $⟹ P (F) \geq P (E)$
- let $A, B$ be any two events, then
  - $P (A \cup B) = P (A) + P (B) - P (A B)$
  - proof by diagram
    - $P (A \cup B) = {I, II, III}$
    - $P (A) = P (I, II)$
    - $P (B) = P (II, III)$
    - $P (A \cap B) = P (II)$
    - $P (A \cup B) = P (A) + P (B) - P (A \cap B)$
  - rigorous way
    - $A \cup B = A \cup (B A^{c})$
    - $A, B A^{c}$ : disjoint
    - using axiom 3: $P (A \cup B) = P (A \cup (B A^{c})) = P (A) + P (B A^{c})$
    - since $B = B A \cup B A^{c}$ , $B A, B A^{c}$ : disjoint
    - using axiom 3: $P (B) = P (B A) + P (B A^{c}) = P (A B) + P (B A^{c})$
      - $⟹ P (A B^{c}) = P (B) - P (A B)$
    - $∴ (A \cup B) = P (A) + P (B) - P (A B)$
- Inclusion-Exclusion principle
  - let $A_{1}, A_{2}, \dots, A_{n}$ be any events, then: $P (A_{1} \cup A_{2} \cup \dots \cup A_{n}) = i = 1 \sum n P (A_{i}) - i \leq i_{1} < i_{2} \leq n \sum P (A_{i_{1}} A_{i_{2}}) + \dots + (- 1)^{r + 1} i \leq i_{1} < \dots < i_{r} \leq n \sum P (A_{i_{1}} \dots A_{i_{r}}) + \dots + (- 1)^{n + 1} P (A_{1} \dots A_{n})$
    - where $\sum_{i \leq i_{1} < i_{2} \leq n} P (A_{i_{1}} A_{i_{2}})$ means sum of every possible combination of $P (A_{i_{1}} A_{i_{2}})$
    - 👨‍🏫 won't be used for $n \geq 4$
Example
- let $A, B$ two events s.t. $P (B) = \frac{5}{8}$ f
  - and $P (A \cap B) = \frac{1}{2}$ . find $P (B \cap A^{c})$
  - $B = B A \cup B A^{c}$
  - $P (B) = P (B A) + P (B A^{c})$
  - $P (B A^{c}) = P (B) - P (B A) = \frac{5}{8} - \frac{1}{2} = \frac{1}{8}$
- J is taking two books for vacation.
  - $A$ : reading first book; $P (A) = 0.5$
  - $B$ : reading second book; $P (B) = 0.4$
  - $P (A B) = 0.3$
  - what is $P (A^{c} B^{c}) = P ((A \cup B)^{c})$
    - $P ((A \cup B)^{c}) = 1 - P (A \cup B)$
    - $P (A \cup B) = P (A) + P (B) - P (A B) = 0.5 + 0.4 - 0.3 = 0.6$
    - $P ((A \cup B)^{c}) = 1 - 0.6 = 0.4$
- formula for $P (A \cup B \cup C)$ $P (A \cup B) P (A \cup B \cup C) = P (A) + P (B) - P (A B) = P (A \cup B) + P (C) - P ((A \cup B) \cap C) = P (A) + P (B) - P (A B) + P (C) - P ((A \cup B) \cap C) = P (A) + P (B) - P (A B) + P (C) - P (A C \cup BC) = P (A) + P (B) - P (A B) + P (C) - (P (A C) + P (BC) - P (A C \cap BC)) = P (A) + P (B) - P (A B) + P (C) - P (A C) - P (BC) + P (A C \cap BC) = P (A) + P (B) - P (A B) + P (C) - P (A C) - P (BC) + P (A BC) = P (A) + P (B) + P (C) - P (A B) - P (A C) - P (BC) + P (A BC)$
  - or, consult diagram for easy understanding

MATH 2421: Tutorial 1

Date: 2024-09-10 17:17:12

Reviewed:

Topic / Chapter: Exercises

summary

❓Questions

Notes

Introduction to Tutorial

Instructor: TA CHEONG, Kha Man
Feel free to attend different section's tutorial!

Exercises

Problem 1 (permutation)
- Five people, designed as A, B, C, D E, are arranged in linear order.
1. How many ways to arrange these five people?
  - $5! = 120$ ways
2. How many ways to arrange these five people, if they are arranged in a circle?
  - as you can rotate the same combination into five different order:
  - $5! /5 = 24$ ways
Problem 2 (permutation)
- How many different ways can 3 red, 4 yellow and 2 blue bulbs be arranged in a string of Christmas tree lights with 9 sockets? $\frac{9 !}{4 ! 3 ! 2 !} = \frac{9 \cdot 8 \cdot 7 \cdot 6 \cdot 5}{3 ! 2 !} = \frac{9 \cdot 8 \cdot 7 \cdot 5}{2 !} = 9 \cdot 4 \cdot 7 \cdot 5 = 9 \cdot 140 = 1260$
- 1260 ways?
Problem 3 (combinations)
- A committee of 5 people is to be chosen from a group of 6 men and 4 women. How many committees are possible if
1. there are no restrictions?
  - $(5 10) = \frac{20 !}{( 10 - 5 )! 5 !} = \frac{10 \cdot 9 \cdot 8 \cdot 7 \cdot 6}{5 !} = \frac{9 \cdot 8 \cdot 7 \cdot 6}{3 \cdot 4} = 9 \cdot 4 \cdot 7 = 252$
  - 252 ways
2. one particular person must be chosen on the committee?
  - $(4 9) = \frac{9 !}{( 9 - 4 )! 4 !} = 126$
3. there are to be 3 men and 2 women?
  - $(3 6) \cdot (2 4) = 20 \cdot 6 = 120$
  - 120 ways
4. there is to be a majority of women?
  - either: 3 woman or 4
  - 3 women: $(2 6) \cdot (3 4) = 15 \cdot 4 = 60$
  - 4 women: $(1 6) \cdot (4 4) = 6 \cdot 1 = 6$
  - 60+6=66 ways
Problem 4 (combinations)
- If 4 Maths books are selected from 6 different Maths books, and 3 English books are selected from 5 different English books, how many ways can the seven books be arranged on a shelf (linear order):
1. if there are no restrictions?
  - $(4 6) \cdot (3 5) \cdot 7! = 756000$
2. if 4 Maths books remain together?
  - $(4 6) \cdot (3 5) \cdot 4! 4! = 18000$
  - one $4!$ for arranging 4 "groups" (3 individual English books, and 1 group of math books)
  - and another for order within the math books
3. if Maths and English books alternate?
  - $(4 6) \cdot (3 5) \cdot 4! 3! = 21600$
4. if a Math book is at the beginning of the shelf?
  - $(4 6) \cdot (3 5) \cdot 4 \cdot 6! = 432000$

MATH 2421: Lecture 4

Date: 2024-09-11 12:02:33

Reviewed:

Topic / Chapter:

summary

❓Questions

Notes

Sample Spaces Having Equally Likely Outcomes

Assuming Equally-Likely
- for many experience: natural to assume all outcomes in finite sample space: equally likely to occur
  - e.g. fair coin / die
- notation
  - $1 = P (S) = P (⋃_{i = 1}^{N} {s_{i}}) = \sum_{i = 1}^{n} P ({s_{i}}) = N c$
  - $c = 1/ N$ ; or $P ({s_{i}}) = \frac{1}{∣ S ∣}$
- then, follows from Axiom 3 that, for $E \subset S$
  - $P (E) = \frac{no. of outcomes in E}{no. of outcomes in S}$
Examples
- trump cards
  - assuming all 52 cards in a deck is equally likely to occur
  - and $A$ : drawing diamond; $B$ : drawing Jack
  - what is $P (A), P (B), P (A \cup B)$
  - $P (A) : \frac{13}{52} = \frac{1}{4}$
  - $P (B) : \frac{4}{52} = \frac{1}{13}$
  - $P (A \cup B) : \frac{1}{4} + \frac{1}{13} - \frac{1}{52} = \frac{16}{52} = \frac{4}{13}$
- drawing random balls
  - if 3 balls are randomly drawn from set of 6 white and 5 black balls
  - what's probability that one of the drawn balls is white
    - and the other two black?
  - 3 ordering possible
    - $W BB : 6 \times 5 \times 4 = 120$
    - $B W B : 5 \times 6 \times 4 = 120$
    - $BB W : 5 \times 4 \times 6 = 120$
  - total ways: $(3 11) = 990$
  - $\frac{120 * 3}{990} = \frac{36}{99} = \frac{4}{11}$
  - alternatively
    - no. of choosing 1 white and 2 black:
    - $= (1 6) (2 5)$
    - $P (A) = \frac{( 1 6 ) ( 2 5 )}{( 3 11 )}$
- birthday problem 2
  - how large must the group be, so that there is a probability io greater than 0.5, that someone will have the same birthday as your do?
  - exclude Feb 29 for calculation; assume unifom distribution
  - $P (A) = 1 - P (A^{c}) = 1 - P (no one among n people w/ same BD as you)$
  - $= 1 - (\frac{364}{365})^{n}$
  - $n$ s.t. $P (A) > 0.5 ⟹ 1 - (\frac{364}{365})^{n} > 0.5$
  - $n ln \frac{364}{365} < ln 0.5$ // switch side
  - $n ln \frac{364}{365} < ln \frac{1}{2} = - ln 2$
  - $ln \frac{364}{365} < ln 364 - ln 365$
  - $n < - \frac{l n 2}{l n 364 - l n 365}$
- birthday problem 1
  - what is the probability that in a group of $n$ people, at least $2$ of the will have the same birthday?
  - $P (B) = 1 - P (B^{c}) = 1 - P (all having diff. birthday)$
  - $= 1 - \frac{365 \times 364 \times \dots \times ( 365 - n + 1 ) n terms}{365 ^{n}}$
  - $= 1 - \frac{365 !}{( 365 - n )! 365 ^{n}}$
  - $= 1 - \frac{\prod _{i = 0}^{n - 1} ( 365 - i )}{365 ^{n}}$
  - or: $= \prod_{i = 0}^{n - 1} (\frac{365 - i}{365}) = \prod_{i = 0}^{n - 1} (1 - \frac{i}{365})$
  - and as $1 + x \approx e^{x}$ when $∣ x ∣$ is small
    - $\approx \prod_{i = 0}^{n - 1} (e^{- \frac{i}{365}})$
    - $= e^{- \frac{1}{365} \frac{( n - 1 ) n}{2}}$
    - $= e^{- \frac{( n - 1 ) n}{365 * 2}}$
    - as $ln 0.5 \approx - 0.6931$
    - $- 0.6931 = - \frac{( n - 1 ) n}{365 * 2}$
    - $\dots n \approx 25$
  - $P (B) \approx 1 - e^{- \frac{( n - 1 ) n}{365 * 2}} = P_{n}$

Conditional Probability and Independence: Introduction

Why conditional probability
- We can leverage the information we have!

Conditional Probability

Definition
- $P (B ∣ A) = \frac{∣ A \cap B ∣}{∣ A ∣}$
  - $= \frac{\frac{∣ A \cap B ∣}{∣ S ∣}}{\frac{∣ A ∣}{∣ S ∣}} = \frac{P ( A \cap B )}{P ( A )}$
  - ⭐conditional probability: $P (B ∣ A) = \frac{P ( A \cap B )}{P ( A )}$
    - read: $P (B)$ given that $A$ has occurred
  - if $A \subseteq B$ , then $P (B ∣ A) = 1$
- if $P (F) > 0$ , then
  - $P (B ∣ A) = \frac{P ( A B )}{P ( A )}$ (simple notation change)
- ⭐ by Kolmogorov's axioms, is $P (\cdot ∣ F)$ a valid probability?
  - check
    - $0 \leq P (E ∣ F) \leq 1$
      - true as $P (EF) < P (F)$
    - $P (S ∣ F) = 1$
      - true as $P (SF) = P (F)$ (as $F \subseteq S$ )
    - $P (⋃_{i = 1}^{\infty} A_{i} ∣ F) = \sum_{i = 1}^{\infty} P (A_{i} ∣ F)$ $P (i = 1 ⋃ \infty A_{i} ∣ F) = \frac{P (( ⋃ _{i = 1}^{\infty} A _{i} ) F )}{P ( F )} = \frac{P ( ⋃ _{i = 1}^{\infty} A _{i} F )}{P ( F )} = \frac{\sum _{i = 1}^{\infty} P ( A _{i} F )}{P ( F )} = i = 1 \sum \infty \frac{P ( A _{i} F )}{P ( F )} = i = 1 \sum \infty P (A_{i} ∣ F)$
    - holds as $A_{i} F$ are mutually exclusive to one another
  - thus, $P (\cdot ∣ F)$ is a valid probability
    - and therefore properties of probability holds for $P (\cdot ∣ F)$ , such as:
    - $P (E^{c} ∣ F) = 1 - P (E ∣ F)$
    - $P (A \cup B ∣ F) = P (A ∣ F) + P (B ∣ F) - P (A B ∣ F)$
Examples
- if we know that event $A = {1, 3, 5}$ has occurred
  - then $P (B = {1, 2, 3}) = \frac{2}{3}$ , not $\frac{1}{2}$ !
- suppose 2 fair dice are are rolled
  - and we observe that the first die is a $3$
  - given that, what is the probability that the sum of 2 dice equals 8?
    - i.e. second die = $5$ , so $\frac{1}{6}$
  - or: $P (B ∣ A) = \frac{P ( A \cap B )}{P ( A )} = \frac{\frac{1}{36}}{\frac{1}{6}}$
    - $= \frac{1}{6}$
- student: taking 1-hr exam
  - suppose $P (A)$ : student will finish exam in less than $x$ hours: $\frac{x}{2}$ ( $\forall0 \leq x \leq 1$ )
  - given that student: still working after $0.75$ hrs ( $L_{0.75}^{c}$ )
  - what's the conditional probability that the full hour will be used? ( $L_{1}^{c}$ )
  - $P (L_{1}^{c} ∣ L_{0.75}^{c}) = \frac{P ( L _{0.75}^{c} \cap L _{1}^{c} )}{P ( L _{0.75}^{c} )} = \frac{P ( L _{1}^{c} )}{P ( L _{0.75}^{c} )} = \frac{1 - 1.0/2}{1 - 0.75/2} = \frac{1/2}{5/8} = \frac{4}{5}$
    - 80%

MATH 2421: Lecture 5

Date: 2024-09-16 12:03:51

Reviewed:

Topic / Chapter:

summary

❓Questions

Notes

Conditional Probability (cont.)

Theorems
- multiplication rule
  - if $P (F) > 0$ , then
  - $P (EF) = P (E ∣ F) P (F)$
- general multiplication rule
  - for events $A_{1}, A_{2}, \dots, A_{n}$
  - $P (A_{1} A_{2} \dots A_{n}) = P (A_{1}) P (A_{2} ∣ A_{1}) P (A_{3} ∣ A_{1} A_{2}) \dots P (A_{n} ∣ A_{1} A_{2} \dots A_{n - 1})$
  - 👨‍🎓 more like, nested use of multiplication rule
  - explanation $R H S = P (A_{1}) \frac{P ( A _{1} A _{2} )}{P ( A _{1} )} \frac{P ( A _{1} A _{2} A _{3} )}{P ( A _{1} A _{2} )} \dots \frac{P ( A _{1} \dots A _{n} )}{P ( A _{1} \dots A _{n - 1} )}$
    - and cancel, cancel
    - or, as mentioned above, nested use $P (A_{1} A_{2} \dots A_{n}) = P (A_{n} ∣ A_{1} A_{2} \dots A_{n - 1}) P (A_{1} A_{2} \dots A_{n - 1}) = P (A_{n} ∣ A_{1} A_{2} \dots A_{n - 1}) P (A_{n - 1} ∣ A_{1} A_{2} \dots A_{n - 2}) P (A_{1} A_{2} \dots A_{n - 2}) = \dots$
Examples
- bin: with 25 light bulbs
  - A: 5: good & functional for more than 30 days
  - B: 10: partially defective; fail in the second day of use
  - C: rest: totally defective
  - if: a random bulb initially lights up ( $A \cup B$ )
    - what is the probability of it working after 1 week? ( $A$ ) $P (A ∣ (A \cup B)) = \frac{P ( A ) \cap P ( A \cup B )}{P ( A \cup B )} = \frac{P ( A )}{P ( A \cup B )} = \frac{5/25}{15/25} = \frac{1}{3}$
- Celine: $1/2$ chance of getting A in French, and $2/3$ chance for chemistry
  - is her decision: based on fair coin
  - what is the chance that she gets an A in chemistry?
  - $A :=$ Celine getting A
  - $C :=$ Celine taking chemistry
  - $P (A C) = P (A ∣ C) P (C) = \frac{2}{3} \frac{1}{2} = \frac{1}{3}$
- suppose: an urn w/ 8 red balls & 4 white balls
  - we draw 2 balls, one at a time, without replacement
  - assuming fair chance, what is the probability that both balls drawn are red?
  - w/ conditional probability
    - $R_{1} :=$ first ball drawn: red
    - $R_{2} :=$ second ball drawn: red
    - $P (R_{1} R_{2}) = P (R_{2} ∣ R_{1}) P (R_{1}) = \frac{7}{11} \frac{8}{12} = \frac{14}{33}$
  - w/ combinatorial analysis
    - i.e. counting no. out interested outcomes / all possible outcomes
    - no. of choosing 2 red balls / no. of choosing arbitrary balls
    - $= \frac{( 2 8 )}{( 2 12 )} = \frac{8 * 7/2}{12 * 11/2} = \frac{14}{33}$
- three cards: selected successively at random, without replacement
  - 52 playing cards
  - calculate probability of receiving in order: K,Q, and J
  - conditional probability
    - $P (J Q K) = P (K) P (J Q ∣ K) = P (K) P (Q ∣ K) P (J ∣ Q K) = \frac{4}{52} \frac{4}{51} \frac{4}{50}$
  - combinatorial analysis
    - no. of outcome w/ K-Q-J / no. of total outcomes
    - $= \frac{4 * 4 * 4}{52 * 51 * 50}$
  - box of fuses w/ 20 fuses
    - 5 are defective
    - 3 fuses: selected randomly and removed from all
    - what's the probability that all 3 are defective?
    - conditional probability
      - $P (R_{1} R_{2} R_{3})$ $P (R_{1}) P (R_{2} ∣ R_{1}) P (R_{3} ∣ R_{1} R_{2}) = \frac{5}{20} \frac{4}{19} \frac{3}{18} = \frac{1}{19 * 6} = \frac{1}{114}$
    - combinatorial analysis
      - $(3 5) / (3 20) = \frac{5 * 4 * 3/3 !}{20 * 19 * 18/3 !} = \frac{1}{114}$
  - If six cards are selected at random (without replacement) from a standard deck of 52 cards, what is the probability there will be no pairs?
    - conditional probability
      - $= \frac{52}{52} \frac{48}{51} \frac{44}{50} \frac{40}{49} \frac{36}{48} \frac{32}{47}$
      - each turn:
    - combinatorial analysis
      - $52 \cdot 48 \cdot 44 \cdot 40 \cdot 36 \cdot 32/ \frac{52 !}{( 52 - 6 )!}$

Total Probability

Introduction
- how can we compute probability under different conditions / cases?
- e.g. probability of getting 3 on a fair die
  - case 1: 4-faced fie is chosen: $1/4$
  - case 2: 6-faced fie is chosen: $1/6$
  - assuming both cases happen with equal chance
  - $P (X = 3) = \frac{1}{4} /2 + \frac{1}{6} /2 = \frac{5}{24}$
Theorem
- let $A, B$ be any two event, then
  - $P (B) = P (B ∣ A) P (A) + P (B ∣ A^{c}) P (A^{c})$
  - 👨‍🏫 or: prob. of $B$ occurs = weighted average of the conditional probabilities of $B$
    - conditioning on either $A$ occurs or $A$ does not occur
  - proof: w/ Venn diagram
    - $B = B A \cup B A^{c}$ , $B A \cap B A^{c} = \emptyset$ $P (B) = P (B A) + P (B A^{c}) = P (B ∣ A) P (A) + P (B ∣ A^{c}) P (A^{c})$
- $A_{1}, A_{2}, \dots A_{n}$ partitions the sample space $S$ if:
  - they are mutually exclusive: $A_{i} \cap A_{j} = \emptyset, \forall i \neq = j$
  - they are exhaustive: $⋃_{i = 1}^{n} A_{i} = S$
- law of total probability
  - if $A_{1}, A_{2}, \dots A_{n}$ partitions the sample space and $P (A_{i}) > 0$
  - let B any event, then
  - $P (B) = P (B ∣ A_{1}) P (A_{1}) + \dots + P (B ∣ A_{n}) P (A_{n})$
    - proof:
      - because $⋃_{i = 1}^{n} A_{i} = S$
      - $B = B \cap S = B \cap (⋃_{i = 1}^{n} A_{i}) = ⋃_{i = 1}^{n} A \cap A_{i}$
      - because $A_{i} \cap A_{j} = \emptyset, \forall i \neq = j$
      - $⋃_{i = 1}^{n} B \cap A_{i} = \sum P (B A_{i}) = \sum P (B ∣ A_{i}) P (A_{i})$
Examples
- MCQ exam
  - a student: either knows the answer at probability $p$ , or guesses at probability $1 - p$
  - there are $m$ alternatives
  - what is the probability that he answered it correctly?
    - $K$ : student knows; $P (K) = p$
    - $G$ : student guesses: $P (G) = 1 - p$
    - $c$ : student answers correctly $P (C) = P (C ∣ K) P (K) + P (C ∣ K^{c}) P (K^{c}) = 1 \cdot p + \frac{1}{m} (1 - p) = p + \frac{1 - p}{m}$
  - what is the probability that the student knew the answer, given that he answered it correctly?
    - $p (k ∣ c) = \frac{P ( k c )}{P ( c )} = \frac{P ( k )}{P ( c )} = \frac{p}{p + \frac{1 - p}{m}}$
    - 👨‍🏫 aka: Bayes formula
- Insurance company: categorizes people into 2 groups: accident-prone and others
  - accident-prone person: will have accident in a fixed 1 yr time with probability of 0.4
    - 0.2 for others
  - if 30% of population is accident prone, what's the change that new policyholder will have an accident within an year?
    - $A$ : person being accident prone
    - $X$ : person having accident in 1 year $P (X) = P (X ∣ A) P (A) + P (X ∣ A^{c}) P (A^{c}) = 0.4 \cdot 0.3 + 0.2 \cdot 0.7 = 0.26$
- party support
  - 40% of people support party $A$ , 30% $B$ , 20% $C$ , 10% $D$
  - $Q$ : certain policy
    - 50% of $A$ supporter supports it
    - 40% of $B$ supporter supports it
    - 30% of $C$ supporter supports it
    - 100% of $D$ supporter supports it
  - what is the probability that a random citizen supports the policy?
    - $P (Q) = P (Q ∣ A) P (A) + P (Q ∣ B) P (B) + P (Q ∣ C) P (C) + P (Q ∣ D) P (D)$
    - $= 0.4 * 0.5 + 0.3 * 0.4 + 0.2 * 0.3 + 0.1 * 1 = 0.48$

Bayes' Theorem

Introduction
- if $A_{1}, A_{2}, \dots A_{n}$ partitions the sample space $S$
  - and $P (A_{i}) > 0$
  - then, for any $1 \leq i \leq n$ $P (A_{i} ∣ B) = \frac{P ( B ∣ A _{i} ) P ( A _{i} )}{P ( B ∣ A _{1} ) P ( A _{1} ) + \dots + P ( B ∣ A _{n} ) P ( A _{n} )}$
  - proof:
    - $P (A_{i} ∣ B) = \frac{P ( B A _{i} )}{P ( B )} = \frac{P ( B ∣ A _{i} ) P ( A _{i} )}{P ( B )}$
    - and expand the denominator

MATH 2421: Tutorial 2

Date: 2024-09-21 22:25:32

Reviewed:

Topic / Chapter:

summary

❓Questions

Notes

Tutorial 2

Properties of Probability
- 60% of students: wear neither a ring / necklace
- 20% wears ring; 30% wear a necklace
- what is the probability of student wearing
- a ring / necklace?
  - $P (R^{c} N^{c}) = 0.6$
  - $P (R) = 0.2; P (N) = 0.3$
  - $P (R \cup N) = 1 - P (R^{c} N^{c}) = 0.4$
- a ring & necklace?
  - $P (RN) = P (R) + P (N) - P (R \cup N) = 0.1$
Sample Spaces Having Equally Likely Outcomes
- Five people, designated as A, B, C, D, E, are arranged in linear order. Assuming that each possible order is equally likely, what is the probability that
- there is exactly one person between A and B?
  - $\frac{2 \cdot 3 ! \cdot 3}{5 !} = \frac{3}{10}$
- there are exactly two people between A and B?
  - $\frac{2 \cdot 3 ! \cdot 2}{5 !} = \frac{2}{10} = \frac{1}{5}$
- there are three people between A and B?
  - $\frac{2 \cdot 3 ! \cdot 1}{5 !} = \frac{1}{10}$
Conditional Probability
- from years of production: $2/1000$ of robots are defective.
- if robot is not defective, it tests negative for 97% of time
- if robot is defective, it tests positive for 99.5% of time
- probability that a randomly selected robot: negative result?
  - $T$ : event of being tested positive
  - $D$ : event of actually being defective
  - $P (T^{c}) = P (T^{c} ∣ D) P (D) + P (T^{c} ∣ D^{c}) P (D^{c}) = 0.005 \cdot 0.002 + 0.97 \cdot 0.998 = 96.807%$
- probability that a randomly selected robot is defective, given positive result?
  - $P (D ∣ T) = \frac{P ( D \cap T )}{P ( T )} = \frac{0.002 \cdot 0.995}{0.03193} = \frac{0.00199}{0.03193} = 6.23%$
General Multiplication Rule
- deck of 52: divided randomly into 4 distinct piles of 13 cards each
- probability that each pile: has exactly 1 ace?
  - $\frac{( 1 4 ) ( 12 48 )}{( 13 52 )} \frac{( 1 3 ) ( 12 36 )}{( 13 39 )} \frac{( 1 2 ) ( 12 24 )}{( 13 26 )} \frac{( 1 1 ) ( 12 12 )}{( 13 13 )}$
Inclusion-Exclusion Principle
- Suppose that there are $N$ distinct types of coupons and each time one obtains a coupon it is, independent of prior selections, equally likely to be any one of the $N$ types. Denote $D_{n}$ as the number of distinct types of coupons that are contained in the first $n$ selections. Find $P (D_{n} = k)$ .
- probability that distinct tickets $\leq k$ : $\frac{( k N ) k ^{n}}{N ^{n}}$
  - no. of ways to choose $k$ types * no. of ways to choosing $n$ tickets within $k$ / total ways of choosing $n$ tickets within $N$
  - must be multiplied by $P (A)$ in order to be correct.
- let $A$ : no empty category exists within $k$
- $A^{c}$ : $\exists$ empty slot
- $B_{i}$ : $i$ -th slot is empty
- $P (A^{c}) = P (⋃_{i = 1}^{k} B_{i})$ $P (i = 1 ⋃ k B_{i}) = t hm 1 \leq i \leq k \sum P (B_{i}) - 1 \leq i < j \leq k \sum P (B_{i} B_{j}) + \dots + (- 1)^{k + 1} P (B_{1} B_{2} \dots B_{k}) = (1 k) (\frac{k - 1}{k})^{n} - (2 k) (\frac{k - 2}{k})^{n} \dots + (- 1)^{k + 1} (k k) (\frac{k - k}{k})^{n} = r = 1 \sum k (- 1)^{r - 1} (r k) (\frac{k - r}{k})^{n}$
- final answer: $(k N) \frac{k ^{n}}{N ^{n}} \times r = 1 \sum k (- 1)^{r - 1} (r k) (\frac{k - r}{k})^{n} = (k N) r = 1 \sum k (- 1)^{r - 1} (r k) (\frac{k - r}{N})^{n}$

MATH 2421: Lecture 6

Date: 2024-09-23 11:41:56

Reviewed:

Topic / Chapter: Bayes theorem & independence

summary

❓Questions

Notes

Bayes' Theorem

Bayes' theorem
- given $A_{1}, A_{2}, \dots, A_{n}$ partitions $S$
  - and $P (A_{i}) > 0$ for all
  - and $P (B)$ any event, then: $P (A_{i} ∣ B) = \frac{P ( B ∣ A _{i} ) P ( A _{i} )}{P ( B ∣ A _{1} ) P ( A _{1} ) + \dots + P ( B ∣ A _{b} ) P ( A _{n} )}$
- proof
  - $P (A_{i} ∣ B) = \frac{P ( B A _{i} )}{P ( b )} = \frac{P ( B ∣ A _{i} ) P ( A _{i} )}{P ( B )}$
  - then: law of total probability
- terminology
  - $P (A_{i})$ prior probability
  - $B$ : evidence
  - $P (A_{i} ∣ B)$ posterior probability
Examples
- I saw a mule!
  - but it is surely mistaken, as there are only horse and donkey in the village.
  - there are 30% horses and 70% donkeys
    - prior belief: there is 30% chance to encounter a donkey
  - if: horse is mistaken as a mule w/ 0.4
    - and donkey is mistaken w/ 0.8
  - then $P (H ∣ saw mule) = \frac{0.4 \cdot 0.3}{0.4 \cdot 0.3 + 0.8 \cdot 0.7} = 0.176$
  - then $P (D ∣ saw mule) = \frac{0.8 \cdot 0.7}{0.4 \cdot 0.3 + 0.8 \cdot 0.7} = 0.823$
- Covid tests
  - PCR test vs. RAT
  - different cases:
    - $TP$ : sick people correctly identified as sick
    - $FP$ : healthy people incorrectly identified as sick
    - $TN$ : healthy people correctly identified as healthy
    - $FN$ : sick people incorrectly identified as healthy
  - terminology
    - sensitivity: $\frac{TP}{P} = \frac{TP}{TP + FN}$
      - i.e. probability of a positive result given patient: is sick
    - specificity: $\frac{TN}{N} = \frac{TN}{TN + FP}$
      - i.e. probability of a negative result given patient: is healthy
    - $1 - specificity$ : prob. of $FP$
    - prevalence:
  - prevalence of covid: 1%
    - test: w/ sensitivity 95%
      - specificity: 99%
    - test result: positive
    - probability that you have covid?
    - $COV := {have the covid}$
    - $+ := {test result: positive}$ $P (CO V ∣ +) = \frac{P ( + ∣ CO V ) P ( COV )}{P ( + )} = \frac{P ( + ∣ CO V ) P ( COV )}{P ( + ∣ COV ) P ( COV ) + P ( + ∣ COV ^{c} ) P ( COV ^{c} )} = \frac{0.95 \cdot 0.01}{0.95 \cdot 0.01 + ( 1 - 0.99 ) \cdot 0.99} = \frac{0.95 \cdot 0.01}{0.95 \cdot 0.01 + 0.01 \cdot 0.99} \approx 48.97%$
- class exercise
  - if $P (COV) =$ 5% or 8%, how does $P (COV ∣ +)$ change? $P (CO V ∣ +) = \frac{P ( + ∣ CO V ) P ( COV )}{P ( + ∣ COV ) P ( COV ) + P ( + ∣ COV ^{c} ) P ( COV ^{c} )} = \frac{0.95 \cdot 0.05}{0.95 \cdot 0.05 + ( 1 - 0.99 ) \cdot 0.95} = \frac{0.95 \cdot 0.05}{0.95 \cdot 0.05 + 0.01 \cdot 0.95} = 83.33%$ $P (CO V ∣ +) = \frac{P ( + ∣ CO V ) P ( COV )}{P ( + ∣ COV ) P ( COV ) + P ( + ∣ COV ^{c} ) P ( COV ^{c} )} = \frac{0.95 \cdot 0.08}{0.95 \cdot 0.08 + ( 1 - 0.99 ) \cdot 0.92} = \frac{0.95 \cdot 0.08}{0.95 \cdot 0.08 + 0.01 \cdot 0.92} = 89.2%$
  - how about $P (COV ∣ -)$ ? $P (CO V ∣ -) = \frac{P ( - ∣ CO V ) P ( COV )}{P ( - ∣ COV ) P ( COV ) + P ( - ∣ COV ^{c} ) P ( COV ^{c} )} = \frac{0.05 \cdot 0.01}{0.05 \cdot 0.01 + 0.99 \cdot 0.99} \approx 0.05%$
- class exercise 2
  - $+ + := {test result: positive twice in a row}$ $P (CO V ∣ + +) = \frac{P ( + + ∣ CO V ) P ( COV )}{P ( + + ∣ COV ) P ( COV ) + P ( + + ∣ COV ^{c} ) P ( COV ^{c} )} = \frac{0.9 5 ^{2} \cdot 0.01}{0.9 5 ^{2} \cdot 0.01 + ( 1 - 0.99 ) ^{2} \cdot 0.99} = \frac{0.9 5 ^{2} \cdot 0.01}{0.9 5 ^{2} \cdot 0.01 + 0.0 1 ^{2} \cdot 0.99} \approx 98.91$
    - 👨‍🏫 sensitivity & specificity matter much, esp. it's multiplied twice
- blood type
  - 95% effective in detecting a disease when it is present
  - false positive: 1% for healthy person
  - prevalence: 0.5%
  - $P (DIS ∣ +) = ?$ $P (DIS ∣ +) = \frac{P ( + ∣ DIS ) P ( DIS )}{P ( + ∣ DIS ) P ( DIS ) + P ( + ∣ DIS ^{c} ) P ( DIS ^{c} )} = \frac{0.95 \cdot 0.005}{0.95 \cdot 0.005 + 0.01 \cdot 0.995}$
- plane: missing
  - equally likely: in 3 possible regions
    - $R_{i}$ : plane is in region $i$ = $1/3$
  - $1 - β_{i}$ : probability that plane: found upon search in $i$ -th region
    - when plane is indeed there
  - $E$ : search in region 1: unsuccessful
    - = plane not found in region 1 upon search
  - $P (R_{i} ∣ E)$ for $i = 1, 2, 3$
    - $P (R_{i} ∣ E) = \frac{P ( E ∣ R _{i} ) P ( R _{i} )}{P ( E )}$
      - for each $i$
    - $1 - β_{1} = P (E^{c} ∣ R_{1}) ⟹ P (E ∣ R_{1}) = 1 - P (E^{c} ∣ R_{1}) = 1 - (1 - β_{1}) = β$
    - $P (E ∣ R_{2}) = P (E ∣ R_{3}) = 1$
    - $P (R_{i} ∣ E) = \frac{P ( E ∣ R _{i} ) P ( R _{i} )}{P ( E )} = \frac{P ( E ∣ R _{i} ) \frac{1}{3}}{\frac{1}{3} β _{1} + \frac{1}{3} \cdot 1 + \frac{1}{3} \cdot 1} = \frac{P ( E ∣ R _{i} )}{β _{1} + 2}$
    - $P (R_{1} ∣ E) = \frac{P ( E ∣ R _{1} )}{β _{1} + 2} = \frac{β}{β _{1} + 2} < \frac{1}{3}$
      - 👨‍🏫 less chance of unsuccessful search, if the plane is actually there!
    - $P (R_{2} ∣ E) = \frac{P ( E ∣ R _{2} )}{β _{1} + 2} = \frac{1}{β _{1} + 2} > \frac{1}{3}$
    - $P (R_{3} ∣ E) = \frac{P ( E ∣ R _{3} )}{β _{1} + 2} = \frac{1}{β _{1} + 2} \frac{1}{3}$
- assembly plant w/ 3 machine types
  - each is of proportion: 30% / 45% / 25%
  - and defective rate: 2% / 3% / 2%
  - given a chosen machine is defective, what is the probability of it being type 1? $P (I ∣ D) = \frac{P ( D ∣ I ) P ( I )}{P ( D ∣ I ) P ( I ) + P ( D ∣ II ) P ( II ) + P ( D ∣ III ) P ( III )} = \frac{0.02 \cdot 0.3}{0.02 \cdot 0.3 + 0.03 \cdot 0.45 + 0.02 \cdot 0.25} = \frac{0.006}{0.0245} = 24.49%$

Independence

Independence
- two events $A, B$ : independent if
  - ⭐ $P (A B) = P (A) P (B)$
  - and dependent if $P (A B) \neq = P (A) P (B)$
- motivation: if $P (B) > 0$ , then
  - $P (A ∣ B) = \frac{P ( A B )}{P ( B )} = \frac{P ( A ) P ( B )}{P ( B )} = P (A)$
  - i.e. $A$ : independent of $B$ if
    - knowledge that $B$ has occurred: does NOT change the probability that $A$ occurs
- if $A, B$ are independent, then so are:
  - $A, B^{c}$ $P (A) P (A B^{c}) = P (A B \cup A B^{c}) = P (A B) + P (A B^{c}) = P (A) - P (A B) = P (A) - P (A) P (B) = P (A) [1 - P (B)] = P (A) P (B^{c})$
  - $A^{c}, B$
  - $A^{c}, B^{c}$
  - 👨‍🏫 and similarly for others
Examples
- card: selected at random from ordinary deck of $52$ cards
  - if $E$ : event that selected card = Ace
  - $F$ : is is spade
  - then $E, F$ : independent
  - $P (EF) = 1/52; P (E) = 4/52; P (F) = 13/52; P (EF) = P (E) P (F)$
    - thus they are independent
  - 👨‍🏫 they are not disjoint (i.e. $E \cap F \neq = \emptyset$ ), but they are independent
- two coins: flipped
  - all 4 outcomes: equally likely
  - $E$ : first coin lands head
  - $F$ : first coin lands tail
  - $P (EF) = 1/4; P (E) = 2/2; P (F) = 2/4; P (EF) = P (E) P (F)$
- another two fair coins
  - $E$ : first coin lands head
  - $F$ : only one coin lands head
  - $P (EF) = 1/4; P (E) = 2/4; P (F) = 2/4; P (EF) = P (E) P (F)$
- 2 fair dice
  - $E_{1}$ : sum of dice = 6
  - $F$ : first die = 4
  - are $E_{1}, F$ independent?
  - $P (E_{1} F) = 1/36; P (E_{1}) P (F) = \frac{5}{36} \frac{1}{6}; P (E_{1} F) \neq = P (E_{1}) P (F)$
- 2 fair dice
  - $E_{1}$ : sum of dice = 7
  - $F$ : first die = 4
  - are $E_{1}, F$ independent?
  - $P (E_{1} F) = 1/36; P (E_{1}) P (F) = \frac{6}{36} \frac{1}{6}; P (E_{1} F) = P (E_{1}) P (F)$

MATH 2421: Tutorial 3

Date: 2024-09-24 18:06:07

Reviewed:

Topic / Chapter:

summary

❓Questions

Notes

Tutorial

Tutorial
- $n$ evens are independent if: for every sub-collection $A_{i_{1}}, \dots, A_{i_{r}}$
  - and meaningful sub-collection: must have $\geq 2$ elements
  - i.e. $2^{n} - n - 1$
    - all subset - no. of individual events - empty set

Exercises

Independent events
- each independently component: function at $p_{i}$ prob.
- that is the prob. that any of $n$ component works?
- as, for independent events, $P (A B) = P (A) P (B)$
- Solution: $P (⋃) = \sum p_{i} - \sum_{i < j} p_{i} p_{j} + \dots + (- 1)^{n + 1} \prod_{1 < i < n} p_{i}$
- 👨‍🏫 or, more simply:
  - let $s$ : system functions
    - ans $s_{i}$ : component $i$ functions
  - $P (s) = 1 - P (s^{c}) = 1 - ⋂_{1 < i < n} s_{i}^{c} = 1 - \prod_{1 < i < n} (1 - p_{i})$
Bayes' theorem
- 3 types of flashlight
  - $P (T > 100∣ I) = 0.7$
    - $P (I) = 0.2$
  - $P (T > 100∣ II) = 0.4$
    - $P (II) = 0.3$
  - $P (T > 100∣ III) = 0.3$
    - $P (III) = 0.5$
- a) $P (T > 100)$ ?
  - sum of products: $P (T > 100∣ I) P (I) + P (T > 100∣ II) P (II) + P (T > 100∣ III) P (III) = 0.14 + 0.12 + 0.15 = 0.41)$
- b) $P (j ∣ T > 100)$ for $j = I, II, III$ ?
  - $= \frac{P ( T > 100∣ j ) P ( j )}{P ( T > 100 )}$
  - $I : 0.14/0.41 = 0.3415$
  - $II : 0.12/0.41 = 0.2927$
  - $III : 0.15/0.41 = 0.3659$
Month Hall problem
- 1 car and 2 goats
- you can choose a door, $1$
- after your choice, the host removes one door with a goat behind, $2$
- then, does changing the door make you more likely to get the car?
- let $P (D_{i} = C)$ probability that car is behind it
- let $P (O_{i})$ probability that door $i$ was opened
  - $P (O_{2} ∣ D_{1} = C) = \frac{1}{2}$
  - $P (O_{2} ∣ D_{2} = C) = 0$
  - $P (O_{2} ∣ D_{3} = C) = 1$ // no other box to open $P (D_{1} = C ∣ O_{2}) = \frac{\frac{1}{2} \frac{1}{3}}{\frac{1}{2} \frac{1}{3} + 0 \cdot \frac{1}{3} + 1 \cdot \frac{1}{3}} = 1/3$ $P (D_{2} = C ∣ O_{2}) = \frac{0 \cdot \frac{1}{3}}{\frac{1}{2} \frac{1}{3} + 0 \cdot \frac{1}{3} + 1 \cdot \frac{1}{3}} = 0$ $P (D_{3} = C ∣ O_{2}) = \frac{1 \cdot \frac{1}{3}}{\frac{1}{2} \frac{1}{3} + 0 \cdot \frac{1}{3} + 1 \cdot \frac{1}{3}} = 2/3$
- thus: better to change
Random Monty Hall problem
- What if the host opens a door that we did not pick, but which might contain car?
- 1/2 $P (D_{1} = C ∣ D_{3} = G) = \frac{P ( D _{3} = G ∣ D _{1} = C ) \cdot P ( D _{1} = C )}{P ( D _{3} = G ∣ D _{1} = C ) P ( D _{1} = C ) + P ( D _{3} = G ∣ D _{1} = G ) P ( D _{1} = G )} = \frac{1 \cdot 1/3}{1 \cdot 1/3 + 1/2 \cdot 2/3} = \frac{1}{2}$

MATH 2421: Lecture 7

Date: 2024-09-25 12:07:25

Reviewed:

Topic / Chapter:

summary

❓Questions

❓ can we say $A$ is independent of itself?
- 👨‍🏫 only if $A \in {\emptyset, S}$

Notes

Independence (cont.)

Independence (cont.)
- three events: independent iff:
  - $P (A BC) = P (A) P (B) P (C)$
  - $P (A B) = P (A) P (B)$
  - $P (BC) = P (B) P (C)$
  - $P (A C) = P (A) P (C)$
  - last three: pairwise independent
- if $A, B, C$ independent => $A$ independent of any objects formed of $B, C$
  - $P (A (B \cup C)) = P (A) P (B \cup C)$ as: $P (A (B \cup C)) = P (A B \cup A C) = P (A B) + P (A C) - P (A B \cap A C) = P (A) P (B) + P (A) P (C) - P (A BC) = P (A) P (B) + P (A) P (C) - P (A) P (BC) = P (A) (P (B) + P (C) - P (BC)) = P (A) (P (B \cup C))$
  - $P (A (BC)) = P (A) P (BC)$ as: $P (A (BC)) = P (A BC) = P (A) P (B) P (C) = P (A) P (BC)$
- events $A_{1}, A_{2}, \dots, A_{n}$ are independent, if:
  - $\forall$ sub-collection of $A_{i_{1}}, \dots, A_{i_{r}}$
    - $P (A_{i_{1}} \dots A_{i_{r}}) = P (A_{i_{1}}) \dots P (A_{i_{r}})$
      - for $r \in [1, n]$
  - for $n = 4$ , verify:
    - $r = 4$ ; $(4 4) = 1$ condition
      - $P (A_{1} A_{2} A_{3} A_{4}) = P (A_{1}) P (A_{2}) P (A_{3}) P (A_{4})$
    - $r = 3$ ; $(3 4) = 4$ conditions
      - $P (A_{1} A_{2} A_{3}) = P (A_{1}) P (A_{2}) P (A_{3})$
      - and similarly so one
    - $r = 2$ ; $(2 4) = 6$ conditions
      - // pairwise independent
Examples
- two fair dice: thrown
  - $A$ : sum of dice = 7
    - $P (A) = 1/6$
  - $B$ : first dice = 4
    - $P (B) = 1/6$
  - $C$ : second dice = 3
    - $P (C) = 1/6$
  - $P (A B) = P (A) P (B)$
  - $P (A C) = P (A) P (C)$
  - yet $P (A BC) = 1/36 \neq = 1/216 = 1/6 \cdot 1/36 = P (A) P (BC)$
    - $A$ dependent on $BC$
- loaded coin w/ $P (H) = p$
  - loaded coin: tossed $n$ times
  - probability of getting at least 1 head in these tosses?
    - $P (⋃ A_{i}) = P ()$
    - $P (⋃ A_{i}) = P ((⋂ A_{i}^{c})^{c}) = 1 - P (⋂ A_{i}^{c}) = independence 1 - (1 - p)^{n}$
  - probability of getting exactly $k$ head in these tosses?
    - if first $k$ tosses: give heads
      - remaining $(n - k)$ tosses: all tail
    - event: $A_{1} A_{2} \dots A_{k} A_{k}^{c} \dots A_{n}^{c}$
      - $P (A_{1} A_{2} \dots A_{k} A_{k}^{c} \dots A_{n}^{c}) = p^{k} (1 - p)^{n - k}$
      - and there are $(k n)$ ways to choose it
    - thus final solution: $(k n) p^{k} (1 - p)^{n - k}$
- system: of $n$ independent components
  - is parallel if it functions when at least one component functions
  - if component $i$ functions in $p_{i}$
  - then what is the probability that the system functions? $P (B_{i}) = p_{1}; P (B) = 1 - P (B^{c}) = 1 - P ((⋃ B_{i})^{c}) = 1 - P (⋂ B_{i}^{c}) = 1 - P (1 \leq i < n \prod (1 - p_{i}))$

Random Variables: Def. of Random Variables

Introduction
- Need for random variable
  - often: we are interested in some function / expression of outcome
  - rather than outcome itself
  - e.g. roll a pair of dice; what is the distribution of two faces' sum?
    - i.e. not interested in face of each die
    - function: $X : S \to R$
    - $(i, j) \mapsto i + j$
    - range: $[2, 12]$
- 20 questions in a MCQ. Each question w/ 5 alternatives
  - student: answers all 20 questions randomly
  - independently choose one alternative in each questions
  - interested in: no. of correct answers
Random variables
- random variable: mapping from sample space to real numbers
  - rv: random variables
  - denoted by capitals, $X, Y, Z$
- discrete random variable
  - rv w/ finite || countable range
  - e.g. number of defective items
- continuous random variable
  - range: an interval over the real line
  - weight of an item, time taken, etc.
Examples
- urn: w/ 20 chips w/ number $[1, 20]$
  - three chips: chosen at random
  - $X$ : largest among the three chips
  - then, what's the range of $X$ ?
    - $[3, 20]$ (discrete)
- toss 3 coins. $Y$ : no. of heads appearing
  - $Y \in [0, 1, 2, 3]$ (discrete)
  - $P (Y = 0) = \frac{1}{8}$
  - $P (Y = 1) = \frac{3}{8}$
  - $P (Y = 2) = \frac{3}{8}$
  - $P (Y = 3) = \frac{1}{8}$
  - ${Y = 0}, {Y = 1}, {Y = 2}, {Y = 3}$
    - partition of $S$
- from above urn, compute the probability of $X \geq 17$
  - $P (X = 3) = 1/ (3 20)$
  - $P (X = 6) = 1 (2 5) / (3 20)$
    - choose $6$ , and 2 other that is smaller that $6$
    - i.e. within $[1, 5]$
  - $P (X = k) = (2 k - 1) / (3 20)$ $P (X \geq 17) = P (X = 17) + P (X = 18) + P (X = 19) + P (X = 20) = ((2 16) + (2 17) + (2 18) + (2 19)) / (3 20) = 0.5087$
- three balls: randomly chosen from urn w/ 3W, 3R, 5B balls
  - we win $1 p er Wba ll c h ose n; l ose$ 1 per R ball chosen.
  - $X$ : total winnings of the experiment
  - compute respective probability for all domain of $X$
    - $P (X = 0) = P (bbb, w r b) = \frac{( 3 5 ) + ( 1 3 ) ( 1 3 ) ( 1 5 )}{( 3 11 )} = \frac{55}{165}$
    - $P (X = 1) = P (w bb, ww r) = \frac{( 1 3 ) ( 2 5 ) + ( 2 3 ) ( 1 3 )}{( 3 11 )} = \frac{39}{165}$
    - $P (X = 2) = P (ww b) = \frac{( 2 3 ) ( 1 5 )}{( 3 11 )} = \frac{15}{165}$
    - $P (X = 3) = P (www) = \frac{( 3 3 )}{( 3 11 )} = \frac{1}{165}$
  - and $P (X = k) = P (X = - k)$
  - probability of winning money
    - $P (X > 0) = P (X = 1) + P (X = 2) + P (X = 3) = \frac{39}{165} + \frac{15}{165} + \frac{1}{165} = 1/3$

Discrete Random Variables

Discrete random variables
- rv: discrete if range of $X$ : either finite / countably infinite
  - using: capital $X, Y, Z$ for variable
  - and $x, y, z$ for value
- suppose rv $X$ is discrete, taking value of $x_{i}$
  - probability mass function of $X$ , $p_{X}$ is $p_{X} = {P (X = x) 0 if x = x_{1}, x_{2}, \dots otherwise$
    - for PMF: range must be well-determined
    - PMF: fully characterizes a rv
    - can be notated as $p$
- properties of PMF
  - $p_{X} (x_{i}) \geq 0$ for $i, 1, 2, \dots$
  - $p_{X} (x) = 0$ for other values of $x$
    - domain of pMF itself: entire real line
  - ⭐ since $X$ must take on one of the values of $x_{i}$
    - $\sum_{i = 1}^{\infty} p_{X} (x_{i}) = 1$
Examples
- rv $X$ : only takes $0, 1, 2, \dots$ value if $p_{X}$ is of the form
  - $p (k) = c \frac{λ ^{k}}{k !}, k = 0, 1, \dots$
  - where $λ > 0$ : fixed positive value; $c$ : suitably chosen constant
    - $c$ : also known as a normalizing constant
  - a) what is this suitable constant
    - as $\sum_{i = 1}^{\infty} p_{X} (x_{i}) = 1$
    - $c \sum_{k = 0}^{\infty} \frac{λ ^{k}}{k !} = c e^{λ} = 1$
      - Taylor series
      - $c = e^{- λ}$
      - 👨‍🏫 sometimes, $c$ is almost impossible to calculate
  - b) compute $P (X = 0)$
    - $P (X = 0) = p (0) = e^{- λ} \frac{λ ^{0}}{0 !} = e^{- λ}$
  - c) compute $P (X > 2)$
    - $P (X > 2) = 1 - P (X \leq 2) = 1 - p (0) - p (1) - p (2)$
    - $P (X > 2) = 1 - e^{- λ} - e^{- λ} λ - e^{- λ} λ^{2} /2$

MATH 2421: Lecture 8

Date: 2024-09-30 11:59:52

Reviewed:

Topic / Chapter:

summary

❓Questions

Notes

Discrete Random Variables

Cumulative distribution function
- cumulative distribution function: or d.f. of $X$ :
  - $F_{X} : R \to R$
  - $F_{X} (x) = P (X \leq x)$ for $x \in R$
- like pmf: cdf fully characterizes random variable
- $F_{X} \in [0, 1]$
- for discrete $X$ , $F_{X}$ : a step function
  - taking jump of $p (x_{i})$ upon reaching it
  - for $x_{1} < x_{2} < \dots < x_{n}$
  - then $F$ constant in $[x_{i}, x_{i + 1})$
  - to know cdf, knowing pdf of following range is enough
    - $(- \infty, x_{1}), [x_{1}, x_{2}), [x_{2}, x_{3}), \dots [x_{n - 1}, x_{n}), [x_{n}, \infty)$
- ⭐ more properties of CDF (in general)
  - $F_{X}$ : monolithic / non-decreasing
  - $0 \leq F_{X} \leq 1$
  - always starts at / near 0 for small $x$
    - and end at or near $1$ for large $x$
  - discrete r.v.'s cdf: a step function
  - $F (x)$ : right-continuous
    - i.e. can approach the limit only from the right
Examples
- if $X$ w/ pmf by
  - $p (1) = \frac{1}{4}; p (2) = \frac{1}{2}; p (3) = \frac{1}{8};; p (4) = \frac{1}{8}$
  - then cdf:
    - $(- \infty, 1); 0$
    - $[1, 2) = \frac{1}{4}$
      - jump of $p_{X} (1)$
    - $[2, 3) = \frac{3}{4};$
      - jump of $p_{X} (2)$
    - $[3, 4) = \frac{7}{8};$
      - jump of $p_{X} (3)$
    - $[4, \infty) = 1;$
      - jump of $p_{X} (4)$
- no. of mortgage approved per week: given below
  
  approved 0 1 2 3 4 5 6
  
  probability 0.1 0.1 0.2 0.3 0.15 0.1 0.05
  - prob: fewer than 4 mortgage approved
    - $p (0) + p (1) + p (2) + p (3) = 0.7$
  - prob: more than 2, but no more that 5 approved
    - $p (3) + p (4) + p (5) = 0.55$
    - or: $P (X \leq 5) - P (X \leq 2) = F (5) - F (2)$
  - cdf chart: trivial
- given cdf, show pdf $F (X) = ⎩ ⎨ ⎧ 0, 0.1, 0.2, 0.4, 0.7, 0.85, 0.95, 1, if x < 0; if 0 \leq x < 1; if 1 \leq x < 2; if 2 \leq x < 3; if 3 \leq x < 4; if 4 \leq x < 5; if 5 \leq x < 6; if 6 \leq x;$
  - simply by $p (x) = F (x) - F (x - 1)$
  - and $p (x_{1}) = F (x_{1})$
  $x$ 0 1 2 3 4 5 6
  
  $p_{X} (x)$ 0.1 0.1 0.2 0.3 0.15 0.1 0.05

Expected Values

Expected value
- expectation or expected value of discrete random var $X$ :
  - $E (X) = \sum_{x} x p_{X} (x)$
  - aka $E [X], μ_{X}$
  - $X$ : a random var
  - but $E (x)$ : a deterministic number
- i.e. a weighted average over all possible value
  - and some of weight: 1
- usually: we use $W, X, Y, Z$ for r.v. (as they are functions)
  - and $w, x, y, z$ for the values
- interpretations of expectation
  1. weighted average of possible values of $X$
    - weight: $P (X = x)$
  2. measure of central location of r.v. $X$
  3. expectation of average val of r.v. over large no. of experiments
    - further connected w/ ch. 8: law of large numbers
Example
- suppose: $X$ only takes only 0 / 1
  - and $P (X = 0) = 1 - p$ and $P (X = 1) = p$
  - random variable: called Bernoulli r.v. of parameter $p$
  - denoted by $X \sim B e (p)$
  - $E (X) = p$
- expected value of a fair 6-face die:
  - i.e. $\sum_{i = 1}^{6} \frac{i}{6} = \frac{7}{2} = 3.5$
- newly wed couple: continue to have children until they have 1 of each sex
  - if: boy-girl chance are independent & equally likely
  - then how many children should this couple expect?
  - assume: get sex 1 for first try
  - and let $E (X^{'})$ : no. of tries until getting sex $2$
  - $E (X) = 1 + E (X^{'})$
  - $E (X^{'}) = \frac{1}{2} \cdot 1 + \frac{1}{2} (1 + E (X^{'})) = 1 + \frac{1}{2} E (X^{'})$
  - $\frac{1}{2} E (X^{'}) = 1$
  - $E (X^{'}) = 2$
  - $E (X) = 1 + E (X^{'}) = 3$
  - prof's way
    1. $p (i)$ : prob. that first $i - 1$ children are girls, and then boy
    2. compute $E (X)$ $E (X) = i = 2 \sum \infty i P_{X} (i) = i = 2 \sum \infty i (\frac{1}{2})^{i - 1} = i = 1 \sum \infty i (\frac{1}{2})^{i - 1} - 1$
    - and, from calculus:
      - $1 + x + x^{2} + x^{3} + \dots = \sum_{i = 1}^{\infty} x^{i} = \frac{1}{1 - x}$ if $∣ x ∣ < 1$
      - $0 + 1 + 2 x + 3 x^{2} + \dots = \sum_{i = 1}^{\infty} i x^{i} = \frac{1}{( 1 - x ) ^{2}}$
        
        derivation
      - as $x = 1/2$ , $E (X) = \frac{1}{( 1 - x ) ^{2}} - 1 = 4 - 1 = 3$

Expectation of a Function of a Random Variable

Expectation of value
- if $P (X = - 1) = 0.2, P (X = 0) = 0.5, P (X = 1) = 0.3$
  - $Y = X^{2}$
  - then $P (Y = 1) = P (X = - 1) + P (X = 1) = 0.5$
- given $X$ and function $g (X)$ , how can we compute $E [g (X)]$ ?
  - standard: compute pmf of $g (X)$ , and compute $E [g (X)]$ by definition
    - $Y \in {0, 1}$
    - $P (Y = 0) = 0.5$
    - $P (Y = 1) = 0.5$
    - $E (Y) = 0.5$
  - alternatively: you can break it down $E (Y) = E (X^{2}) = 0^{2} \times 0.5 + 1^{2} \times 0.5 = 0^{2} \times 0.5 + 1^{2} \times (0.2 + 0.3) = 0^{2} \times p_{X} (0) + (- 1)^{2} \times p_{X} (- 1) + 1^{2} \times p_{X} (1) = 0 \times p_{X} (0) + 1 \times p_{X} (- 1) + 1 \times p_{X} (1)$
  - in general: $E (g (x)) = \sum g (x) p_{X} (x)$
  - 👨‍🏫 no need to compute pmf for $g (x)$ separately
- theorem: r.v. $X$ w/ value $x_{i}$ for $i \geq 1$ for $p_{X} (x_{i})$
  - then for any real value function $g$
  $E [g (X)] = i \sum g (x_{i}) p_{X} (x_{i}) = x \sum g (x_{i}) p_{X} (x)$
- proof:
  - idea: group together all terms in $\sum_{i} g (x_{i}) p (x_{i})$ w/ same value for $g (x_{i})$ , then represent it as sum per each distinct value of $y_{j} = g (x_{i})$
  $i \sum g (x_{i}) p (x_{i}) = j \sum i : g (x_{i}) = y_{j} \sum g (x_{i}) p (x_{i}) = j \sum i : g (x_{i}) = y_{j} \sum y_{j} p (x_{i}) = j \sum y_{j} i : g (x_{i}) = y_{j} \sum p (x_{i}) = j \sum y_{j} P (g (X) = y_{j}) = E [g (X)]$
- $E (X^{2}) = \sum_{x} x^{2} p_{X} (x)$
  - is called a second moment of $X$
  - $E (X^{k})$ for $k \geq 1$ : called k-th moment of $X$
- let $μ = E (X), g (x) = (x - μ)^{k}$ then
  - $E (X - μ)^{k} = E [(X - μ)^{k}]$ (notation)
  - is called $k$ -th central moment
- remarks
  - $E (X)$ : aka first moment / mean of $X$
  - first central moment: $0$
  - second central moment: $E [(X - μ)^{2}]$
    - aka variance of $X$
- let $a, b$ : constants, then
  - $E [a X + b] = a E (X) + b$
  - 👨‍🏫 expectation: a linear operator
- proof:
  - $g (x) = a x + b$ $E [a X + b] = x \sum [a x + b] p (x) = x \sum [a x p (x) + b p (x)] = x \sum a x p (x) + x \sum b p (x) = a x \sum x p (x) + b x \sum p (x) = a E (X) + b$

MATH 2421: Lecture 9

Date: 2024-10-02 12:08:29

Reviewed: 2024-10-07 16:01:23

Topic / Chapter:

summary

❓Questions

Notes

Expectation of a Function of a Random Variable (cont.)

Expectation of value
- for non-negative integer-valued r.v. $X$
  - $E (X) = \sum_{k = 1}^{\infty} P (X \geq k) = \sum_{k = 0}^{\infty} P (X > k)$
  - tail sum formula fo expectation
- proof $k = 1 \sum \infty P (X \geq k) = P (X = 1) + P (X = 2) + P (X = 3) + P (X = 4) + \dots + P (X = 2) + P (X = 3) + P (X = 4) + \dots + P (X = 3) + P (X = 4) + \dots + P (X = 4) + \dots = P (X = 1) + 2 P (X = 2) + 3 P (X = 3) + 4 P (X = 4) + \dots = E (X)$

Variance and Standard Deviation

Variance & standard deviation
- for random variable $X$ w/ mean $μ$ , variance is given by:
  - $var (X) = E (X - μ)^{2}$
  - i.e. measure of scattering / spread of the values of $X$
- standard deviation of $σ_{X}$ or $SD (X)$ :
  - $σ) X = var (X)$
- $var (X) = E (X^{2}) - [E (X)^{2}]$
- proof:
  - w/ $X$ discrete r.v., $E (X) = μ$ $var (X) = E (x - μ)^{2} = x \sum (x - μ)^{2} p (x) = x \sum (x^{2} - 2 xμ + μ^{2}) p (x) = x \sum x^{2} p (x) - 2 xμ p (x) + μ^{2} p (x) = x \sum x^{2} p (x) - x \sum 2 xμ p (x) + x \sum μ^{2} p (x) = x \sum x^{2} p (x) - 2 μ x \sum x p (x) + μ^{2} x \sum p (x) = x \sum x^{2} p (x) - 2 μ E (x) + μ^{2} = x \sum x^{2} p (x) - 2 E (x)^{2} + E (x)^{2} = x \sum x^{2} p (x) - E (x)^{2} = E (x^{2}) - E (x)^{2}$
- remarks
  - $var (X) \geq 0$ as it's squared
  - $var (X) = 0$ iff $X$ is a degenerate r.v.
    - i.e. taking only one value: mean
    - special case of some inequality
  - following tha formula: $E (X^{2}) \geq [E (X)]^{2} \geq 0$
- $var (a X + b) = a^{2} var (X)$
- proof $var (a X + b) = E (a x + b - E (a x + b))^{2} = E (a x + b - a E (x) - b)^{2} = E (a x - a E (x))^{2} = E (a (x - E (x)))^{2} = a^{2} E (x - E (x))^{2} = a^{2} var (X)$
- $SD (a X + b) = ∣ a ∣ SD (X)$
  - similarly, just sqrt at each side
Problems
- calculate $var (X)$ is $X$ : outcome when a fair die is rolled
- consider the prob. distribution on stock $A, B$
  - find expected return:
  X Y
  
  prob A return B return
  
  0.2 1% 10%
  
  0.3 2% 6%
  
  0.3 3% 2%
  
  0.2 4% -2%
  - $E (A) = 0.2 \cdot 0.01 + 0.3 \cdot 0.02 + 0.3 \cdot 0.03 + 0.2 \cdot 0.04 = 2.5%$
  - $E (B) = 0.2 \cdot 0.10 + 0.3 \cdot 0.06 + 0.3 \cdot 0.02 - 0.2 \cdot 0.02 = 4%$
  - find variance & SD of return
    - $var (X) = E (X^{2}) - (μ_{x})^{2}$

Discrete Random Variables arising from Repeated Trials

Modeling repeated trials
- repeating
  1. each trial results in an event, or not
  - occurrence of event: success
  - non-occurrence of event: failure
  - $p := P (success)$
  - $q := 1 - p = P (failure)$
  1. each trial w/ success probability $p$ , failure with $q = 1 - p$
  2. repeating trials independently
  - such trial: Bernoulli (p) trials
- Bernoulli random variable
  - performing experiment only once: $X = {10 success failure$
    - $P (X = 1) = p$ , $P (X = 0) = 1 - p$
    - $E (X) = p$ ; $var (X) = p (1 - p)$
      - as $E (X^{2}) - (E (X))^{2} = p - p^{2}$
  - denote by $B e (p)$
  - aka $X \sim B e (p)$
  - aka $X \sim B er (p)$
  - aka $X \sim B er n o u ll i (p)$
  - distribution of Bernoulli r.v.: Bernoulli distribution
- Binomial r.v.
  - experiment $n$ times and define
    - $X =$ no. of success in $n$ $B e (p)$ trials
    - $P (X = k) = (k n) p^{k} q^{n - k}$
  - denoted by $B in (n, p)$
    - binomial r.v., Binomial distribution
    - $X \sim B in (n, p)$
  - is it a valid pmf?
    - i.e. show $\sum_{k = 0}^{n} p (X = k) = 1$ $k = 0 \sum n p (X = k) = k = 0 \sum n (k n) p^{k} q^{n - k} = (p + q)^{n} = (1)^{n} = 1$
- if $X \sim B in (n, p)$ then $E (X) = n p$ , $var (X) = n p (1 - p)$
- proof on $E (X)$ $E (X) = k = 0 \sum n k P (X = k) = k = 0 \sum n k (k n) p^{k} q^{n - k} = k = 1 \sum n k \frac{n !}{( n - k )! k !} p^{k} q^{n - k} = k = 1 \sum n \frac{n !}{( n - k )! ( k - 1 )!} p^{k} q^{n - k} = k = 1 \sum n \frac{n ( n - 1 )! p ^{k} q ^{n - k}}{(( n - 1 ) - ( k - 1 ))! ( k - 1 )!} = k = 1 \sum n (k - 1 n - 1) n p^{k} q^{n - k} = n p k = 1 \sum n (k - 1 n - 1) p^{k - 1} q^{n - k} = n p k = 1 \sum n (k - 1 n - 1) p^{k - 1} q^{(n - 1) - (k - 1)} = l = k - 1 n p l = 0 \sum n - 1 (l n - 1) p^{l} q^{(n - 1) - l} = n p (p + q)^{n - 1} = n p$
- proof on $var (X)$ $x^{2} E (x^{2}) var (X) E [x (x - 1)] E (x^{2}) = x (x - 1) + x = E [x (x - 1)] + E (x) = E (X^{2}) - [E (X)]^{2} = E [x (x - 1)] + E (x) - [E (X)]^{2} = k = 0 \sum n k (k - 1) P (x = k) = k = 0 \sum n k (k - 1) (k n) p^{k} q^{n - k} = k = 2 \sum n k (k - 1) (k n) p^{k} q^{n - k} = k = 2 \sum n k (k - 1) \frac{n !}{k ! ( n - k )!} p^{k} q^{n - k} = k = 2 \sum n \frac{n \cdot ( n - 1 ) \cdot ( n - 2 )!}{( k - 2 )! (( n - 2 ) - ( k - 2 ))!} p^{k} q^{n - k} = k = 2 \sum n n \cdot (n - 1) p^{k} q^{n - k} (k - 2 n - 2) = n \cdot (n - 1) p^{2} k = 2 \sum n p^{k - 2} q^{(n - 2) - (k - 2)} (k - 2 n - 2) = l = k - 2 n \cdot (n - 1) p^{2} l = 0 \sum n - 2 p^{l} q^{(n - 2) - l} (l n - 2) = n \cdot (n - 1) p^{2} (p + q)^{n - 2} = n \cdot (n - 1) p^{2} = E [x (x - 1)] + E (x) - [E (X)]^{2} = n \cdot (n - 1) p^{2} + n p - (n p)^{2} = (n^{2} - n) p^{2} + n p - (n p)^{2} = (n^{2} - n) p^{2} + n p - n^{2} p^{2} = n^{2} p^{2} - n p^{2} + n p - n^{2} p^{2} = - n p^{2} + n p = n p (1 - p)$
- geometric random variable
  - let $X$ : no. of $B e (p)$ trials to obtain the first success
  - $P (X = k) = p q^{k - 1}$
  - and $E (X) = \frac{1}{p}$
  - and $var (X) = \frac{1 - p}{p ^{2}}$
  - denoted by $G eo m (p)$
- calculus tools
  - $\sum_{i = 0}^{\infty} x^{i} = \frac{1}{1 - x}$ for $∣ x ∣ < 1$
  - $\sum_{i = 1}^{\infty} i x^{i - 1} = \frac{1}{( 1 - x ) ^{2}}$ for $∣ x ∣ < 1$
  - $\sum_{i = 2}^{\infty} i (i - 1) x^{i - 2} = \frac{2}{( 1 - x ) ^{3}}$ for $∣ x ∣ < 1$
- proof on $E (X) = \frac{1}{p}$ $E (X) = k = 1 \sum \infty k P (X = k) = k = 1 \sum \infty k p q^{k - 1} = p k = 1 \sum \infty k q^{k - 1} = \frac{p}{( 1 - q ) ^{2}} = \frac{p}{p ^{2}} = \frac{1}{p} x = q$
- 👨‍🏫 show the variance one yourself! $var (X) E [x (x - 1)] var (X) = E [X^{2}] - [E (X)]^{2} = E [x (x - 1)] + E (x) - [E (X)]^{2} = E [x (x - 1)] + \frac{1}{p} - \frac{1}{p ^{2}} = E [x (x - 1)] + \frac{p - 1}{p ^{2}} = k = 1 \sum \infty k (k - 1) P (X = k) = k = 1 \sum \infty k (k - 1) p q^{k - 1} = pq \cdot k = 1 \sum \infty k (k - 1) q^{k - 2} = pq \cdot k = 2 \sum \infty k (k - 1) q^{k - 2} = pq \cdot \frac{2}{( 1 - q ) ^{3}} = pq \cdot \frac{2}{p ^{3}} = 2 (1 - p) \cdot \frac{1}{p ^{2}} = 2 (1 - p) \cdot \frac{1}{p ^{2}} + \frac{p - 1}{p ^{2}} = (2 (1 - p) + p - 1) \cdot \frac{1}{p ^{2}} = (2 - 2 p + p - 1) \cdot \frac{1}{p ^{2}} = (1 - p) \cdot \frac{1}{p ^{2}} = \frac{1 - p}{p ^{2}}$
- another version of geometric distribution
  - $X^{'}$ : no. of failures in $B e (p)$ trials
    - in order to obtain the first success
  - and $X = X^{'} + 1$
  - $P (X^{'} = k) = p q^{k}$ for $k = 0, 1, \dots$
  - $E (X^{'}) = \frac{1 - p}{p}$
  - $Var (X^{'}) = \frac{1 - p}{p ^{2}}$
  - conventionally: geometric seq. means $X$ (at least in this course)
- negative Binomial r.v.
  - $X$ : no. of $B e (p)$ to get $r$ success
  - $X = r, r + 1, \dots$
  - $P (X = k) = (r - 1 k - 1) p^{r} q^{k - r}$
    - $k \geq r$
    - 👨‍🎓 better: $(r - 1 k - 1) p^{r - 1} q^{(k - 1) - (r - 1)} p$
  - $E (X) = \frac{r}{p}$
  - $var (X) = \frac{r ( 1 - p )}{p ^{2}}$
  - denoted by: $NB (r, p)$
  - $G eo m (p) = NB (1, p)$
  - 👨‍🏫 it's called negative binomial, because it is extended from negative binomial theorem
    - $(x + y)^{- m}$
    - or: $(r - 1 k - 1) = (- 1)^{r - 1} (r - 1 - ( k - r + 1 ))$

MATH 2421: Lecture 10

Date: 2024-10-07 12:02:15

Reviewed:

Topic / Chapter:

summary

❓Questions

Notes

Discrete Random Variables arising from Repeated Trials (cont.)

Practice
- gambler: makes seq. of 1 dollar bets on black
  - success: winning $1, f ai l u re : l os in g$ 1
  - wins if ball stops on one of $18.38$ positions
    - lose otherwise
- in small town: out of 12 accidents, at least 4 happened on Friday 13th
  - does it mean that Friday 13th seems to be auspicious?
  - suppose: probability accident to occur on Friday 13th: same as other days
    - $= 1/30$
    - $X \sim Bin (12, 1/30)$
  - then, probability of four or more accidents on Friday the 13th:
    - $1 - \sum_{i = 0}^{3} (i 12) (\frac{1}{30})^{i} (\frac{29}{30})^{12 - i} \approx 0.000493$
    - such rare probability; good reason to believe so (hypothetically)
- geometric distribution: plays important role in theory of queues
  - e.g. line of customers
  - assume: each small time unit, either 0 or 1 new customers arrive
  - probability that a customer arrive: $p$ , not arriving: $q := 1 - p$
  - time $T$ until next arrival: has a geometric distribution
  - what is the probability no customer arrives in next $k$ time units?
    - i.e. $P (T > k)$
  - $T \sim Geom (p)$ $P (T > K) = j = k + 1 \sum \infty P (T = j) = j = k + 1 \sum \infty q^{j - 1} p = \frac{q ^{k} p}{1 - q} = q^{k}$
  - or: no success in sequence of $k$ consecutive time units
    - thus $q^{k}$
- 10 students: randomly pick one number from $[0, 9]$
  - let $X$ : r.v. on number of students who picked no. 8
  - find the probability: more than 1 student pick no. 8
  - $X =$ no. of success in 10 $Be (1/10)$ trials
    - $X \sim Bin (10, 1/10)$ $P (X > 1) = 1 - P (X \leq 1) = 1 - P (X = 0) - P (X = 1) = 1 - (0 10) (\frac{1}{10})^{0} (\frac{9}{10})^{10} - (1 10) (\frac{1}{10})^{1} (\frac{9}{10})^{9} = 0.2639$
- communication system w/ $n$ components
  - each: function w/ probability $p$ , independently
  - total system: operate effectively if at least one-half of components function
  - for what values of $p$ : 5-component system works better than 3-component?
    - $X_{n} \sim Bin (n, p)$ : number of components working in $n$ -component system $P (5-component works) = P (X_{5} \geq 3) = P (X_{5} = 3) + P (X_{5} = 4) + P (X_{5} = 5) = (3 5) p^{3} (1 - p)^{2} + (4 5) p^{4} (1 - p) + (5 5) p^{5}$ $P (3-component works) = P (X_{3} \geq 2) = P (X_{3} = 2) + P (X_{3} = 3) = (2 3) p^{2} (1 - p) + (3 3) p^{3}$
- given $P (X_{5} \geq 3) > P (X_{3} \geq 2)$
  - $(3 5) p^{3} (1 - p)^{2} + (4 5) p^{4} (1 - p) + (5 5) p^{5} > (2 3) p^{2} (1 - p) + (3 3) p^{3}$
  - $⟹ 3 (p - 1)^{2} (2 p - 1) > 0 ⟹ \frac{1}{2} < p < 1$

Poisson Random Variable

Poisson Random Variable
- ⭐ r.v. $X$ : Poisson distribution w/ parameter $λ$
  - if for $X = 0, 1, 2$ $P (X = k) = \frac{e ^{- λ} λ ^{k}}{k !}$
    - for $k = 0, 1, 2, \dots$
- defines: probability mass function
  - $\sum_{k = 0}^{\infty} P (X = k) = e^{- λ} \sum_{k = 0}^{\infty} \frac{λ ^{k}}{k !} = e^{- λ} e^{λ} = 1$
- notation: $X \sim Poisson (λ)$
- theorem: if $X \sim Poisson (λ)$ , $E (X = λ) = λ; var (X) = λ$ $E (X) = k = 0 \sum \infty k P (x = k) = k = 1 \sum \infty k \frac{e ^{- λ} λ ^{k}}{k !} = e^{- λ} k = 1 \sum \infty \frac{λ ^{k}}{( k - 1 )!} = e^{- λ} j = 0 \sum \infty \frac{λ ^{j + 1}}{j !} = λ e^{- λ} j = 0 \sum \infty \frac{λ ^{j}}{j !} = λ e^{- λ} \cdot e^{λ} = λ$ $var (X) E (x (x - 1)) var (X) = E (x^{2}) - (E (x)^{2}) = E (x (x - 1)) + E (x) - (E (x))^{2} = k = 0 \sum \infty k (k - 1) P (X = k) = k = 0 \sum \infty k (k - 1) e^{- λ} \frac{λ ^{k}}{k !} = k = 2 \sum \infty k (k - 1) e^{- λ} \frac{λ ^{k}}{k !} = k = 2 \sum \infty e^{- λ} \frac{λ ^{k}}{( k - 2 )!} = j = 0 \sum \infty e^{- λ} \frac{λ ^{j + 2}}{j !} = e^{- λ} λ^{2} j = 0 \sum \infty \frac{λ ^{j}}{j !} = e^{- λ} λ^{2} e^{λ} = λ^{2} = λ^{2} + E (x) - (E (x))^{2} = λ^{2} + λ - λ^{2} = λ$
- Poisson random variable: tremendous range of applications
  - for determining probability of counts over time
- example
  - no. of traffic accidents occurring on a highway in a day
  - crashes of a computer network per week
  - no. of people joining a line in an hours
  - no. of customers per day
  - no. of goals in a hockey game
  - no. of typos per page of an essay
- Poisson random variable: can be used for approximation for binomial r.v. w/ parameter $(n, p)$
  - if $n$ is larger $p$ is small enough
    - i.e. $n p$ : moderate size
  - suppose $X$ binomial r.v. w/ $(n, p)$ and $λ = n p$
  - then $P (X = k) n \to \infty lim (1 + \frac{x}{n})^{n} (1 - \frac{λ}{n})^{n} (1 - \frac{λ}{n})^{k} P (X = k) = \frac{n !}{( n - k )! k !} p^{k} (1 - p)^{n - k} = \frac{n !}{( n - k )! k !} (\frac{λ}{n})^{k} (1 - \frac{λ}{n})^{n - k} = \frac{n !}{( n - k )! n ^{k}} (\frac{λ ^{k}}{k !}) \frac{( 1 - λ / n ) ^{n}}{( 1 - λ / n ) ^{k}} = e^{x} \approx e^{- λ} \frac{n !}{( n - k )! n ^{k}} \approx 1 \approx 1 \approx e^{- λ} \frac{λ ^{k}}{k !}$
  - $Bin(n,p) =$ no. of successes in $n$ $Bernoulli (p)$ trials
    - law of rare events: if $n$ is large and $p$ is small
    - then $Bin (n, p) \approx Poisson (n p)$
    - ⭐ usually if $n > 20$ and $n p < 15$
  - remarks
    - w/ $n$ independent trials w/ success probability $p$
    - when $n$ is large and small $p$ for moderate $n p$
      - no. of success occurring: Poisson random $λ = n p$
    - examples
      - no. of misprints on a page
      - no. of people in community living to 100 yearsno. of wrong telephone no. that are dialed in a day
      - no. of people entering a store on a given day
      - with large $n$ , above and many others become approximately Poisson
- on $t$ units on time and rate $λ$ occurrences per unit time, then
  - $Y_{t} \sim Poisson (λ t)$
  - $Y_{t}$ : r.v. of count of occurrences of even over period of time $t$
  - proof: non trivial
Problems
- which of following r.v. has infinite range?
  - $Be (p), Poisson (λ), Bin (n, p)$
  - answer: Poisson
    - Be: ${0, 1}$
    - Bin: $[0, n]$
    - Poisson: $[0, \infty)$
- suppose: no. of typographical errors on a page: w/ Poisson distribution
  - parameter: $λ = 1/2$
  - calculate prob. there exists at least one error on a page
  - $P (X \geq 1) = 1 - P (X = 0) = 1 - e^{- 1/2} \frac{( 1/2 ) ^{0}}{0 !} = 1 - e^{- 1/2} \approx 39.3%$
- suppose: probability of an item produced by a machine w/ defective rate $0.1$
  - find: probability that sample of 10 samples: contain at most 1 defective item
  - $X$ : no. of defective items among 10 samples
  - $X \sim Bin (10, 0.1)$
  - $P (X \leq 1) = P (X = 0) + P (X = 1) = (0 10) (0.1)^{0} (0.9)^{10} + (1 10) (0.1)^{1} (0.9)^{9} = 73.61%$
  - w/ Poisson approximation: $λ = n p = 1$
    - $X \sim Poisson (1)$
    - $P (X \leq 1) = P (X = 0) + P (X = 1) = e^{- 1} \frac{1 ^{0}}{0 !} + e^{- 1} \frac{1 ^{1}}{1 !} = 2 e^{- 1} \approx 73.58%$
      - 👨‍🏫 pretty close!
- suppose: during a particular minute of day
  - $n = 2, 000, 000$ people: serviced in a particular telephone service area
  - independently decide whether to place an emergency call
  - w/ probability $p = .000005$
  - and let $T$ : actual random no. of 911 callers in that minute
  - find: $P (T = 10)$
  - $P (T = 10) = (10 2 , 000 , 000) (0.000005)^{10} (0.999995)^{1, 999, 990}$
    - 👨‍🏫 don't try to run it on calculator!
  - approximating: $λ = 2, 000, 000 \cdot .000005 = 10$
    - $P (T = 10) \approx e^{- 10} \frac{1 0 ^{10}}{10 !} = 0.1251$
- during lab experiment: average no. of radioactive parties passing counter per 1 ms: 4
  - given no. of particles passing follows a Poisson distribution
  - what is the probability that 6 particles enter the counter in a given ms?
  - $X$ : no. of passing through counter in 1 ms
  - $X \sim Poisson (λ)$
  - $E (X) = 4 = λ$
    - 👨‍🎓 and, for Poisson, $E (X) = var (X) = λ$
  - $P (X = 6) = e^{- 4} \frac{4 ^{6}}{6 !} \approx 0.1042$
- studying earthquakes in California
  - w/ reading over 6.7 on Richter scale
  - on average: 1.5 earthquakes w/ such condition per year
  - $λ = 1.5$ : rate of the occurrence of earthquakes
  - let $X$ : rv of number of earthquakes above 6.7 in upcoming year, then
  - $X \sim Poisson (1.5)$
  - probability that there will be 5 earthquakes w/ reading over 6.7
    - in upcoming year:
      - $P (X = 5) = \frac{( 1.5 ) ^{5} e ^{- 1.5}}{5 !} = 0.0141$
    - in next 4 years:
      - on $t$ units on time and rate $λ$ occurrences per unit time, then
      - $Y_{t} \sim Poisson (λ t)$
      - $Y_{t}$ : r.v. of count of occurrences of even over period of time $t$
      - $P (Y_{4} = 5) = \frac{( λ t ) ^{5} e ^{- λ t}}{5 !} = \frac{( 1.5 \times 4 ) ^{5} e ^{- 6}}{5 !} = 0.1601$
    - 👨‍🎓 6 in next 4 years:
      - $P (Y_{4} = 6) = \frac{( 1.5 \times 4 ) ^{6} e ^{- 1.5 \times 4}}{6 !} = 0.1606$
- average no. of homes solve by agency: 2 homes / day (Poisson)
  - what is the probability that exactly 10 homes will be sold by agency in the next 30 days?
  - $λ = 2, λ t = 60$
  - $Y_{10} = e^{- 60} \frac{( 60 ) ^{10}}{10 !} \approx 1.459 \times 1 0^{- 15}$

Hypergeometric Random Variable

Hypergeometric Random Variable
- suppose: w/ set of $N$ balls, w/ $m$ red balls and $N - m$ blue balls
  - and choose: $n$ of such balls without replacement
  - $X$ : no. of red balls in sample $P (X = x) = \frac{( x m ) ( n - x N - m )}{( n N )}$
    - for $x = 0, 1, \dots, min (m, n)$
- random variable w/ PMF given as above: hypergeometric r.v.
  - denoted by: $H (n, N, m)$ $E (X) = \frac{nm}{N}, var (X) = \frac{nm}{N} [\frac{( n - 1 ) ( m - 1 )}{N - 1} + 1 - \frac{nm}{N}]$

MATH 2421: Lecture 11

Date: 2024-10-09 12:02:10

Reviewed:

Topic / Chapter:

summary

❓Questions

Notes

Hypergeometric Random Variable (cont.)

Practice
- purchaser of electrical component: buys in lots of size 10
  - policy: inspect 3 from a lot
  - and all must be non-defective
  - w/ 30% of lots having 4 defective components
  - and 70% w/ 1 defective component
  - what proportion of lots: does purchaser reject?
  - $X \sim H (n, N, m)$
    - $N = 10$
    - $n = 3$
  - $m = 1∣∣4$
    - $P (m = 1) = 0.7$
    - $P (m = 4) = 0.3$ $P (acceptance) P (rejection) = P (X = 0) = P (X = 0∣ m = 4) P (m = 4) + P (X = 0∣ m = 1) P (m = 1) = \frac{( 0 4 ) ( n - 0 10 - 4 )}{( 3 10 )} \cdot 0.3 + \frac{( 0 1 ) ( n - 0 10 - 1 )}{( 3 10 )} \cdot 0.7 = 0.54 = 1 - 0.54 = 0.46$

Expected Value of Sum of Random Variables

Expected value of sum of random variables
- 👨‍🏫 more discussion in Ch. 7; important!
- for a r.v. $X$ , let $X (s)$ : val. of $X$ when $s \in S$ being output
- if $X, Y$ : both r.v. $⟹$ their sum is also r.v.
- $Z = X + Y$ : also r.v.
  - $Z (s) = X (s) + Y (s)$
  - r.v.: map from sample space to real line
- theorem: let $p (x) = P ({s})$ : probability that $s$ : output of experiment
  - then $E [X] = s \in S \sum X (s) p (s)$
- proof
  - suppose, for distinct values of $X$ are $x_{i} \geq 1$
  - for each $i$ : $S_{i}$ : event $X = x_{i}$
    - i.e. $S_{i} = {s : X (s) = x_{i}}$ $E [X] = i \sum x_{i} P {X = x_{i}} = i \sum x_{i} P (S_{i}) = i \sum x_{i} s \in S_{i} \sum p (s) = i \sum s \in S_{i} \sum x_{i} p (s) = i \sum s \in S_{i} \sum X (s) p (s) = s \in S \sum X (s) p (s)$
- ⭐ linearity of expectation
  - for r.v. $X_{1}, X_{2}, \dots, X_{n}$ (no independence req.) $E [i = 1 \sum n X_{i}] = i = 1 \sum n E [X_{i}]$
- proof $Z E (Z) := i = 1 \sum n X_{i} = s \in S \sum Z (s) p (s) = s \in S \sum (i = 1 \sum n X_{i}) p (s) = s \in S \sum (i = 1 \sum n X_{i} p (s)) = i = 1 \sum n (s \in S \sum X_{i} p (s)) = i = 1 \sum n E [X_{i}]$
Practice
- suppose: experiment consists of flipping a coin 5 times
  - result: only heads & tails
  - $X$ : no. of heads in first 3 flips
  - $Y$ : no. of heads in last 2 flips
  - then $Z = X + Y$ - for any combination
- suppose: two independent flip of a coin
  - head w/ probability of $p$ maid
  - $X$ : no. of heads obtained
  - $P (X = 0) = P (t, t) = (1 - p)^{2}$
  - $P (X = 1) = P (h, t) + P (t, h) = 2 p (1 - p)$
  - $P (X = 2) = P (h, h) = p^{2}$
  - thus $E [X] = 1 \cdot 2 p (1 - p) + 2 \cdot p^{2} = 2 p$
  - as well as $E [X] = X (h, h) p^{2} + X (h, t) p (1 - p) + X (t, h) (1 - p) p + X (t, t) (1 - p)^{2} = 2 p^{2} + p (1 - p) + (1 - p) p = 2 p$
- find: expected total no. of success from $n$ trials
  - when trial $i$ : success w/ probability $p_{i}$
  - let $x_{i} = {10 i -th: success otherwise$ $E (X) = E (i = 1 \sum n x_{i}) = i = 1 \sum n E (x_{i}) = i = 1 \sum n p_{i}$
  - special case: $\forall p_{i} = p$ and trials independent:
    - then $X \sim Bin(n,p)$
    - $E (X) = \sum_{i = 1}^{n} p = n p$
- $n$ identical & independent trials
  - each trial: yields success w/ same $p$
    - $X_{i} = 1$ if success
  - proof
    - show: $\sum_{i = 1}^{n} X_{i} \sim Bin (n, p)$
    - by def. $X = \sum_{i = 1}^{n} X_{i}$ : total no. of success in $n$ independent $Bernoulli (p)$
    - thus $\sum_{i = 1}^{n} X_{i} \sim Bin (n, p)$
    - ⭐⭐ w/ same $p$ , then always
      - $X = \sum_{i = 1}^{n} X_{i}$
        
        where $X \sim Be (p)$
        
        and $x_{i}$ all independent
- for $X \sim Bin (n, p), var (X) = ?$ $E (X) var (X) E (X^{2}) E (X_{i}^{2}) i \neq = j, x_{i} x_{j} E (x_{i} x_{j}) ⟹ E (X^{2}) var (X) = n p = E (X^{2}) - (E (X))^{2} = E [(i = 1 \sum n x_{i})^{2}] = E [((i = 1 \sum n x_{i}) \cdot (j = 1 \sum n x_{j}))^{2}] = E (i = 1 \sum n x_{i}^{2} + i = 1 \sum n j \neq = 1 \sum n x_{i} \cdot x_{j}) = E [i = 1 \sum n x_{i}^{2}] + E i = 1 \sum n j \neq = 1 \sum n x_{i} \cdot x_{j} = i = 1 \sum n E (x_{i}^{2}) + i = 1 \sum n j \neq = 1 \sum n E (x_{i} \cdot x_{j}) = E (X_{i}) = p = {10 if x_{i} = 1, x_{j} = 1 = 1 \times P (X_{i} = 1, X_{j} = 1) + 0 \times P (otherwise) = P (X_{i} = 1) P (X_{j} = 1) = p^{2} = i = 1 \sum n p + i = 1 \sum n j \neq = 1 \sum n p^{2} = n p + n (n - 1) p^{2} = E (X^{2}) - (E (X))^{2} = n p + n (n - 1) p^{2} - (n p)^{2} = 0 // as X_{i} \in {0, 1}$

Continuous Random Variables: Introduction

Continuous r.v.
- r.v. w/ range being an interval over real line
  - weight / time / length ...
  - considered separate as:
    - 👨‍🏫 you can't get exactly 30 cm long ruler, etc.
- for a continuous r.v. $X$
  - $\forall t \in R, P (X = t) = 0$
  - and thus $P (a < X < b) = P (a \leq X \leq b)$
    - and $P (a < X < b) = P (a < X < b) + P (X = a) + P (X = b) = P (a \leq X \leq b)$
- for cont. r.v. $X$ , w/ property s.t.
  - $f_{X}$ : non-negative function, $\forall x \in R$ $P (X \in B) = \int_{B} f_{X} (x) d x$
  - then: $f_{X}$ : probability density function (pdf)
- for all $B = [a, b]$ $P (a \leq X \leq b) = \int_{a}^{b} f_{X} (x) d x = F_{X} (b) - F_{X} (a)$
- $P (X = a)$ as: $P (X = a) = P (a \leq X \leq a) = \int_{a}^{a} f (x) d x = 0$
- distribution function (cdf): defined by $F_{X} (x) = P (X \leq x) = \int_{- \infty}^{x} f_{X} (t) for x \in R$
  - 👨‍🏫 same as discrete!
- w/ fundamental theorem of calculus:
  - $F_{X}^{'} (x) = f_{X} (x), x \in R$
  - density: derivative of cumulative distribution function $P (x - \frac{ε}{2} \leq X \leq x + \frac{ε}{2}) = \int_{x - ε /2}^{x + ε /2} f_{X} (x) d x \approx ε f (x)$
  - (think of it as a area of small slice of rectangle)
  - $f (x)$ : measure of how likely that r.v. will be near $x$
    - $\neq =$ probabilities
    - $f (x) \in [0, \infty)$

MATH 2421: Lecture 12

Date: 2024-10-14 11:50:44

Reviewed:

Topic / Chapter:

summary

❓Questions

Notes

Continuous Random Variables: Introduction (cont.)

Continuous r.v. (cont.)
- remarks
  - for cont. r.v. $P (X = x) = 0, \forall x \in (- \infty, \infty)$
  - $F_{X}$ : continuous
- simple facts
  - $F_{X} (- \infty) = P (X \leq - \infty) = 0$
  - $F_{X} (+ \infty) = P (X \leq + \infty) = 1$
- determining constant in PDF $1 = P (- \infty < X < \infty) = \int + - \infty^{\infty} f_{X} (s) d x$
Problems
- for $X$ a cont. r.v. w/ following PDF: $f_{X} (x) = {c (4 x - 2 x^{2}) 0 0 < x < 2 otherwise$
  - compute value of $c$ $\int_{- \infty}^{\infty} f_{X} (x) d x c = \int_{0}^{2} f_{X} (x) d x = \int_{0}^{2} c (4 x - 2 x^{2}) d x = c (2 x^{2} - \frac{2}{3} x^{3}) ∣_{0}^{2} = c (8 - \frac{16}{3}) = - \frac{8}{3} c = 1 = - \frac{3}{8}$
  - compute value of $P (X > 1)$ $- \frac{3}{8} (2 x^{2} - \frac{2}{3} x^{3}) ∣_{1}^{2} = - \frac{3}{8} ((8 - \frac{16}{3}) - \frac{4}{3}) = \frac{3}{8} \cdot \frac{4}{3} = \frac{1}{2}$
- electrical appliance: function for a random amount of time $T$ $f r (t) = {c e^{- λ t} 0 0 < t < \infty otherwise$
  - exponential distribution
  - CDF: $F_{T} (t) = {1 - e^{- λ t} 0 t \geq 0 otherwise$
  - value of $c$ ? $\int_{0}^{\infty} c e^{- λ t} d t c = - \frac{c e ^{- λ t}}{λ} ∣_{0}^{\infty} = \frac{c}{λ} = 1 = λ$
  - for any $s, t > 0$ , $P (T > s + t ∣ T > s = P (T > t))$ $P (T > t) P (T > s) P (T > s + t) P (T > s + t ∣ T > s) = - \frac{λ e ^{- λ t}}{λ} ∣_{t}^{\infty} = \frac{λ e ^{- λ t}}{λ} = e^{- λ t} = e^{- λ s} = e^{- λ (s + t)} = \frac{e ^{- λ (s + t)}}{e ^{- λ s}} = e^{- λ t} = P (T > t)$
  - property s.t.
    - $P (T > s + t ∣ T > s) = P (T > t)$
      - i.e. memoryless property
- lifetime in hour of a radio tube: given by $f (x) = {0 100/ x^{2} x \leq 100 x > 100$
  - what is the probability that exactly 2 of 5 such tubes: have to be replaced within first 150 hours? $P (T < 150) = \int_{- in f t y}^{150} f (x) d x = \int_{100}^{150} 100/ x^{2} d x = - 100/ x ∣_{100}^{150} = 1 - \frac{2}{3} = \frac{1}{3}$
  - desired property $P (X = 2) = (2 5) P (T < 150)^{2} P (T \geq 150)^{3} = \frac{10 \cdot 2 ^{2}}{3 ^{5}}$

Expectation and Variance of Continuous Random Variables

Expected value and SD
$E (X) var (X) S D (X) = \int_{\infty}^{\infty} x f_{X} (x) d x = \int_{\infty}^{\infty} (x - E (X)) f_{X} (x) d x = E [(X - μ X)^{2}] = var (X)$
- theorem
  - for any real value function $g$
  - ⭐⭐ $E [g (X)] = \int_{- \infty}^{\infty} g (x) f_{X} (s) d x$
  - then same linearity, variance, or variance of linear function
- theorem: tail sum formula
  - for $X$ a nonnegative continuous random variable $E (X) = \int_{0}^{\infty} P (X > x) d x = \int_{0}^{\infty} P (X \geq x) d x$
- proof $\int_{0}^{\infty} P (X > x) d x \int_{0}^{\infty} P (X > x) = \int_{0}^{\infty} \int_{x}^{\infty} f (y) d y d x = \int_{0}^{\infty} (\int_{0}^{y} d x) f (y) d y = \int_{0}^{\infty} y f (y) d y = E [X] = \int_{0}^{\infty} x f (x) d x // order change$
Problems
- find the mean and variance of the random variable $X$ $f (x) = {1/ (b - a) 0 (a \leq x \leq b) otherwise$
  - $X$ : uniform distribution over $[a, b]$ $E (X) = \int_{- \infty}^{\infty} x f (x) d x = \int_{a}^{b} \frac{x}{b - a} d x = \frac{x ^{2}}{2 ( b - a )} ∣_{a}^{b} = \frac{b ^{2} - a ^{2}}{2 ( b - a )} = \frac{a + b}{2}$ $var (X) E (X^{2}) var (X) = E (X^{2}) - (E (X))^{2} = \int_{- \infty}^{\infty} x^{2} f (x) d x = \int_{a}^{b} \frac{x ^{2}}{b - a} d x = \frac{x ^{3}}{3 ( b - a )} ∣_{a}^{b} = \frac{b ^{3} - a ^{3}}{2 ( b - a )} = \frac{a ^{2} + b ^{2} + ab}{3} = \frac{a ^{2} + b ^{2} + ab}{3} - (\frac{a + b}{2})^{2} = \frac{4 a ^{2} + 4 b ^{2} + 4 ab - 3 a ^{2} - 6 ab - 3 b ^{2}}{12} = \frac{a ^{2} - 2 ab + b ^{2}}{12} = \frac{( a - b ) ^{2}}{12}$
- find $E (e^{X})$ w/ $X$ of following pdf $f (x) = {2 x 0 0 \leq x \leq 1 otherwise$ $E (X) = \int_{- \infty}^{\infty} e^{x} f (x) d x = \int_{0}^{1} 2 x e^{x} d x = I BP 2 x e^{x} ∣_{0}^{1} - \int_{0}^{1} e^{x} d x = 2 e - e + 1 = e + 1$
- stick of length 1: broken at random
  - determine expected length of the piece containing point of length $p$ from one end ( $A$ )
  - when $0 \leq p \leq 1$ is fixed
  - let $u$ : break point, then length denoted by $x (u)$ :
    - if $u < p$ then $1 - u$
    - else: $u$
    - $u \sim uniform ([0, 1])$
  - finding: $E [x (u)]$ $E [x (u)] = \int_{- \infty}^{\infty} x (u) f (u) d u = \int_{0}^{1} x (u) f (u) d u = \int_{0}^{p} (1 - u) f (u) d u + \int_{p}^{1} u f (u) d u = (u - \frac{u ^{2}}{2}) ∣_{0}^{p} + (\frac{u ^{2}}{2}) ∣_{p}^{1} = (p - \frac{p ^{2}}{2}) + (\frac{1}{2} - \frac{p ^{2}}{2}) = \frac{1}{2} + p - p^{2}$
- upon meeting: being $s$ minutes early costs you $cs$
  - while being late $s$ minutes cost $k s$
  - suppose: traveling time is a cont. r.v. given by $f$
    - if $x \geq 0$ , given by $f$
    - if $x < 0$ : 0
  - determine: what time to depart in order to minimize cost
  - suppose: you leave $t$ minutes before the appointment
    - step 1: find expected cost $g (t)$
    - step 2: minimize $g (t)$
  - let $C_{t} (X)$ be the cost $C_{t} (X) = {k (x - t) c (t - x) x \geq t = arrive late x < t = arrive early$
  - expected value: $E [C_{t} (x)] = \int_{- \infty}^{\infty} C_{t} (x) f (x) d x = \int_{0}^{\infty} C_{t} (x) f (x) d x = \int_{0}^{t} c (t - x) f (x) d x + \int_{t}^{\infty} k (x - t) f (x) d x = g (t)$
  - minimizing $g (t)$ (finding optimal value, etc.) $g (t) g^{'} (t) = c t \int_{0}^{t} f (x) d x - \int_{0}^{t} x f (x) d x + k \int_{t}^{\infty} x f (x) d x - k t \int_{t}^{\infty} f (x) d x = c t \int_{0}^{t} f (x) d x - \int_{0}^{t} x f (x) d x - k \int_{\infty}^{t} x f (x) d x + k t \int_{\infty}^{t} f (x) d x = c \int_{0}^{t} f (x) d x + c t f (t) - c t f (t) - k t f (t) + k \int_{\infty}^{t} f (x) d x + k t f (t) = c \int_{0}^{t} f (x) d x + k \int_{\infty}^{t} f (x) d x = c \int_{0}^{t} f (x) d x - k \int_{t}^{\infty} f (x) d x$
  - recall that: $F (t) = \int_{- \infty}^{t} f (x) d x = \int_{0}^{t} f (x) d x$
    - and $\int_{t}^{\infty} f (x) d x = \int_{0}^{\infty} f (x) d x - \int_{0}^{t} f (x) d x$
    - $P (X > t) = 1 - P (X \leq t) = 1 - F (t)$
    - $g^{'} (t) = c F (t) - k (1 - F (t)) = (c + k) F (t) - k$
    - $g^{'} (t) = 0 ⟹ (c + k) F (t) = k = 0 ⟹ F (t) = \frac{k}{c + k}$
    - thus $t = F^{- 1} (\frac{k}{c + k})$
  - $g^{''} (t) = [(c + k) F (t) = k]^{'} = (c + k) F^{'} (t) = (c + k) f (t) \geq 0$
    - if $g^{''} (t^{*}) > 0$ $g (t)$ : w/ absolute minimal at $t^{*}$

MATH 2421: Lecture 13

Date: 2024-10-21 11:56:40

Reviewed:

Topic / Chapter:

summary

❓Questions

Notes

Uniform Distribution

Uniform distribution
- $X$ : uniformly distributed aver the interval $(a, b)$ if pdf is given by: $f_{X} (x) = {\frac{1}{b - a} 0 a < x < b otherwise$
  - denoted by: $X \sim U (a, b)$
  - every neighbor: equally likely
- CDF: $F_{X} (x) = - \infty in f x f_{X} (y) d y = ⎩ ⎨ ⎧ 0 \frac{x - a}{b - a} 1 x < a a \leq x < b otherwise$
- similarly shows that: $E (X) = \frac{a + b}{2}$ , $Va r (X) = \frac{( b - a ) ^{2}}{12}$
Problems
- buses: arrive at stop on 15-min intervals from 7:15 am
  - passenger: arrives at stop uniformly at random time between $7 : 00, 7 : 30$ .
  - find probability: the man arriving in less that 5 mins after arrival
    - i.e. $7 : 10 - 15, 7 : 25 - 30$
    - $P (10 < X < 15 or 25 < X < 30)$
    - $10/30 = 1/3$
  - find probability: the man arriving more than 10 minutes late
    - 👨‍🏫 exercise!

Normal Distribution

Normal distribution
- $X$ : normally distributed w/ parameters $μ, σ^{2}$ , pmf given by: $f_{X} (x) = \frac{1}{2 π σ} e^{- \frac{1}{2} (\frac{x - μ}{σ})^{2}}$
- notation: $X \sim N (μ, σ^{2})$
  - $E (X) = μ, Va r (X) = σ^{2}$
- density function: bell-shaped, always positive. symmetric on and peaks at $μ$
Standard normal r.v.
- $Z \sim N (0, 1)$ , pdf: $ϕ (x)$ , cdf: $Φ$
  - $ϕ (x) = \frac{1}{2 π} e^{- x^{2} /2}$
  - $Φ (x) = \frac{1}{2 π} \int_{- \infty}^{x} e^{- y^{2} /2} d y$
- theorem: for $X \sim N (μ, σ^{2})$ , $Z \sim N (0, 1)$ $Z P (a < X \leq b) = \frac{X - μ}{σ} \sim N (0, 1) = P (\frac{a - μ}{σ} < Z \leq \frac{b - μ}{σ}) = Φ (\frac{b - μ}{σ}) - Φ (\frac{a - μ}{σ})$
- note: don't try to evaluate cdf of general normal distribution by hand
  - simple impossible :p
- some intuitions
  - bell curve's left half: same size as right
  - total area / region: 1
  - for $X \sim N (μ, σ^{2})$ , $P (X \leq μ) = P (X \geq μ) = \frac{1}{2}$
- properties of the standard normal
  1. $P (Z \geq 0) = P (Z \leq 0) = 0.5$ (aligned by center)
  2. $- Z \sim N (0, 1)$ (symmetric)
  3. $P (Z \leq x) = 1 - P (Z > x)$ for $x \in (- \infty, \infty)$
  4. $P (Z \leq - x) = P (Z \geq x)$ for $x \in (- \infty, \infty)$
  5. ⭐ if $Y \sim N (μ, σ^{2})$ . then $Z = \frac{Y - μ}{σ} \sim N (0, 1)$
  6. ⭐ if $Z \sim N (0, 1)$ . then $Y = a Z + b \sim N (b, 1^{2})$ for $a, b \in R$
    - 👨‍🏫 more generally: linear transformation of a normal r.v. is still a normal r.v.
  - $\forall x, Φ (- x) = 1 - Φ (x)$
- $q$ -th quantile of a r.v. $X$ : $z_{q}$ s.t. $P (X \leq z_{q}) = q$
  - for $Z \sim N (0, 1)$ , $Φ (z_{q}) = q$
Problems
- when $X \sim N (65, 5^{2})$ , compute $P (47.5 < X \leq 80)$
  - $= P (X \leq 80) - P (X \leq 47.5) = P (Z \leq 15/5) - P (Z \leq - 17.5/5) = Φ (3) - (1 - Φ (3.5)) \approx 1$
- width of a slot of duralumin in forging: normally distributed with $(μ, σ) = (0.9000, 0.0030)$
  - specification limits: $0.9000 \pm 0.0050$
  - what percentage of forgings will be defective?
    - $P (0.9000 - 0.0050 > X) + P (0.9000 + 0.0050 < X) = P (- 5/3 > Z) + P (5/3 < Z) = 2Φ (- 5/3)$
    - $= 2 (1 - Φ (5/3)) = 9.5%$
  - what is the maximum allowable value of $σ$ to permit no more than 1 in 100 defectives, when $μ = 0.9000$ ?
    - $2Φ (- 0.0050/ σ) \leq 0.01$ , $z_{0.005} \approx - 2.576$ , $σ = - 0.0050/ (- 2.576) = 0.00194$

MATH 2421: Tutorial 6

Date: 2024-10-22 18:08:37

Reviewed:

Topic / Chapter:

summary

❓Questions

Notes

Tutorial 6

Tutorial
- exponential distribution
  - if $X \sim exp (λ)$ , then
    - $f (x) = λ e^{- λ x}$ $x \geq 0$ , $E (X) = 1/ λ$ , $Va r (X) = 1/ λ^{2}$
    - memoryless property: $P_{X} (X > s + t ∣ X > s) = P_{X} (X > t)$
  - 👨‍🏫 $X$ : waiting time for first arrival
  - shares the parameter with Poisson distribution
  - as expected time to wait: $1/ λ$
Exercises
1. continuous r.v.
  - for some constant $c$ , pdf of $X$ is: $f (x) = {c x^{n} 0 0 < x < 1 otherwise$
  - find $c$ , $P (X > x), 0 < x < 1$
  - when $n = 4$ , find $E (X)$ and $Va r (X)$
  - $\int_{0}^{1} c x^{n} d x$
2. normal dist.
  - total sales: follow normal distribution
  - $μ = 2, 500, 000$ , $σ = 300, 000$
  - probability: firm's sale fall within 150,000 of the mean?
    - $2 \cdot (Φ (0.5) - 0.5) = 2 \cdot (0.6915 - 0.5) = 0.383$
  - determine: sales level w/ 9% of being exceeded next year
    - $z_{0.91} = 1.34$
    - $2, 500, 000 + 1.34 \cdot 300, 000 = 2902000$
3. normal dist.
  - weights of kindergarten students: $\sim N (19.5, 3. 2^{2})$
  - find probability: exactly $4/11$ randomly selected students weigh less than $20$
  - $P (X \leq 20) = P (Z \leq 0.5/3.2) = Φ (0.156) = 0.5636$
  - and $Y \sim B in (11, 0.5623)$
    - $P (Y = 4) = (4 11) (0.5623)^{4} (1 - 0.5624)^{7} = 0.1014$
4. exponential dist.
  - $X$ : exponential r.v. w/ $μ = 1/ λ$
  - show: $E (X^{k}) = \frac{k !}{λ ^{k}}$ $E (X^{k}) = \int_{- \infty}^{\infty} X^{k} \cdot λ e^{- λ x} d x$
5. Chi-squared dist.
  - if $X \sim N (0, 1)$ , find pdf of $X^{2}$
  - $p (x) = 2 \cdot \frac{1}{2 π} e^{- 0.5 x}$
    - pdf of $X \sim N (0, 1)$ times two, as two $x$ exists with same probability
    - $χ^{'} = [0, \infty)$

MATH 2421: Lecture 15

Date: 2024-10-28 11:58:17

Reviewed:

Topic / Chapter:

summary

❓Questions

Notes

Function of a Continuous Random Variable (cont.)

Theorem
- theorem: w/ $X$ being cont. r.v. having pdf $f_{X}$
  - suppose: $g (x)$ strictly monotonic, differentiable function of $x$
    - 👨‍🏫 i.e. inverse function exists
    - then r.v. $Y = g (X)$ has pdf given by $f_{Y} (y) = {f_{X} (g^{- 1} (y)) ∣ \frac{d}{d y} g^{- 1} (y) ∣ 0 if y = g (x) for some x if y \neq = g (x) for all x$
    - $g^{- 1} (y)$ : value of $x$ s.t. $g (x) = y$
- proof
  - assume: $g (x)$ increasing
    - and same for $g^{- 1}$
  - $F_{Y} (y) = P (g (X) \leq y) = P (X \leq g^{- 1} (y)) = F_{X} (g^{- 1} (y))$
    - and, from chain rule:
      - ⭐ $f_{Y} (y) = f_{X} (g^{- 1} (y)) \frac{d}{d y} g^{- 1} (y)$
      - and, as $g^{- 1} (x)$ non-decreasing, its derivative is non-negative
    - when $y \neq = g (x)$ , $F_{Y} (y)$ either $0$ or $1$ : leading to $f_{Y} (y) = 0$
- Remarks: if $X$ cont. r.v. w/ strictly increasing cdf $F_{X} (x)$
  1. $F \sim U (0, 1)$
  2. $X \sim F_{X}^{- 1} (u), u \sim U (0, 1)$
    - thus, we can perform inverse transformation method
    - generating cont. r.v. w/ distribution $F_{X}$ from $u \sim U (0, 1)$
    - $Y = F_{X}^{- 1} (U)$ , then $F_{Y} (y) = P (Y \leq y) = P (F_{X}^{- 1} (U) \leq y) = P (U \leq F_{X} (y)) = F_{X} (y)$
Example
- let $X \sim N (0, 1)$ , find cdf, pdf. of $Y = X^{2}$
  - if $y < 0$ : $F_{Y} (y) = 0$
  - if $y \geq 0$ $F_{Y} (y) = P (Y \leq y) = P (X^{2} \leq y) f_{Y} (y) = F_{Y}^{'} (y) = P (- y \leq X \leq y) = \int_{- y}^{y} \frac{1}{2 π} e^{- \frac{x ^{2}}{2}} d x = 2 \int_{0}^{y} \frac{1}{2 π} e^{- \frac{x ^{2}}{2}} d x = (2 \int_{0}^{y} \frac{1}{2 π} e^{- \frac{x ^{2}}{2}} d x)^{'} = \frac{1}{2 π} e^{- \frac{y}{2}} (y)^{'} = \frac{1}{2 π} e^{- \frac{y}{2}} y^{- 1/2}$
  - actually $Y = X^{2} \sim Γ (0.5, 0.5)$
    - aka Chi-squared distribution
- same $X$ , $Y = e^{X}$ , aka lognormal r.v.
  - define pdf $f_{Y}$
  - $Y = e^{X} > 0$
  - for $y \leq 0$ , $F_{Y} (y) = 0, f_{Y} (y) = 0$
  - else: $F_{Y} (y) f_{Y} (y) = P (Y \leq y) = P (e^{X} \leq y) = P (X \leq ln y) = \int_{- \infty}^{l n y} \frac{1}{2 π} e^{- \frac{x ^{2}}{2}} d x = F_{Y} (y)^{'} = (\int_{- \infty}^{l n y} \frac{1}{2 π} e^{- \frac{x ^{2}}{2}} d x)^{'} = \frac{1}{2 π} e^{- \frac{( l n y ) ^{2}}{2}} (ln y)^{'} = \frac{1}{y 2 π} e^{- \frac{( l n y ) ^{2}}{2}}$
  - also, as $g (Y)$ strictly monotonic
  - $g^{- 1} (y) = ln y$ for $y > 0$
    - also $(g^{- 1})^{'} = 1/ y$
  - using formula, we obtain: $f_{Y} = f_{X} (g^{- 1} (y)) ∣ \frac{d}{d y} g^{- 1} (y) ∣ = f_{X} (g^{- 1} (y)) ∣ \frac{d}{d y} g^{- 1} (y) ∣ = \frac{1}{2 π} e^{- \frac{( l n y ) ^{2}}{2}} \cdot ∣ \frac{1}{y} ∣ = \frac{1}{y 2 π} e^{- \frac{( l n y ) ^{2}}{2}}$
- $X$ : cont. r.v. w/ pdf $f_{X}$
  - $Y = X^{n}$ and $n$ is odd
  - find: $f_{Y}$
  - 👨‍🏫 exercise $f_{Y} (y) = f_{X} (n y) \cdot \frac{n y}{n y}$
- let $Y$ be cont. r.v. w/ cdf $F$
  - assume $F$ : strictly increasing function
  - define the r.b. by $X = F (Y)$
  - possible values of $X \in [0, 1]$
  - for $x \in [0, 1]$ $F_{X} (x) = P (X \leq x) = P (F (Y) \leq x) = P (F^{- 1} (F (Y)) \leq F^{- 1} (x)) = P (Y \leq F^{- 1} (x)) = F (F^{- 1} (x)) = x$
  - PDF of $x$ : $f_{X} (x) = (F_{X} (x))^{'} = x^{'} = 1$ for $x \in [0, 1]$
  - thus, $X \sim U (0, 1)$
- generating an exponential r.v.
  - $F (x) = 1 - e^{- λ x}$ for $x \geq 0$
  - then $F^{- 1} (u)$ : value $x$ s.t.
    - $u = F (x) = 1 - e^{- λ x}$
    - $x = - \frac{1}{λ} lo g (1 - u)$
  - $X = - \frac{1}{λ} lo g (1 - U) \sim E x p (λ)$
  - and $1 - U$ : also follows $\sim U (0, 1)$
    - thus $X = - \frac{1}{λ} lo g (U) \sim E x p (λ)$

Joint Distribution Functions

Introduction
- often: we are interested in multiple r.v. at the same time
  - e.g. a student's age, gender, major, year of study
  - on particular day, number of vehicle accidents, deaths, and major injuries
- generalizing distribution function: for multiple r.v.s
  - joint distribution function
Joint distribution function
- for r.v. $X, Y$ in same $S$ , define joint distribution function of $X, Y$ aas $F_{X, Y} (x, y) = P (X \leq x, Y \leq y) x, y \in R$
  - generalization of $F_{X} (x) = P (X \leq x)$
  - often abbreviated: joint d.f.
- remarks ${X \leq x; Y \leq y} = {s \in S : X (s) \leq x \cap Y (s) \leq y} = {s \in S : X (s) \leq x} \cap {s \in S : Y (s) \leq y} = {Y \leq x} \cap {Y \leq y}$
- distribution of $X$ : obtained from joint d.f. by: $F_{X} (x) = y \to \infty lim F_{X, Y} (x, y) = y \to \infty lim P (X \leq x, Y \leq y) = P (X \leq x)$
  - then $F_{X}$ : marginal distribution function of $X$ (= cdf)
- properties
  - 👨‍🏫 all derived from:
    - $X \leq \infty = S$
    - $X \leq - \infty = \emptyset$
  - $F_{X, Y} (- \infty, y) = 0$ for any $y \in R$
  - $F_{X, Y} (\infty, y) = F_{Y} (y)$ for any $y \in R$
  - $F_{X, Y} (x, - \infty) = 0$ for any $x \in R$
  - $F_{X, Y} (x, \infty) = F_{X} (x)$ for any $x \in R$
  - $F_{X, Y} (\infty, \infty) = 1$
  - $F_{X, Y} (- \infty, - \infty) = 0$
  - $F_{X, Y} (\infty, - \infty) = F_{X, Y} (- \infty, \infty) = 0$
- using joint d.f. to compute two r.v.
  - $P (X > a, Y > b) = 1 - F_{X} (a) - F_{Y} (b) + F_{X, Y} (a, b)$
    - think about it in geometry! $P (a_{1} < X \leq a_{2}, b_{1} < Y < b_{2}) = F_{X, Y} (a_{2}, b_{2}) - F_{X, Y} (a_{1}, b_{2}) - F_{X, Y} (a_{2}, b_{1}) + F_{X, Y} (a_{1}, b_{1})$
  - inclusion matters here!
- proof
  - $A = {X \leq a}, B = {Y \leq b}$
  - then ${X > a, Y > b} = A^{c} B^{c} = (A \cup B)^{c}$
  - thus: $P (X > a, Y > b) = P ((A \cup B)^{c}) = 1 - P (A \cup B) = 1 - P (A) - P (B) + P (A B) = 1 - F_{X} (a) - F_{Y} (b) + F_{X, Y} (a, b)$
Joint probability mass function
- joint pmf of $X, Y$ : $p_{X, Y} (x, y) = P (X = x, Y = y)$
- marginal pmf
  - $p_{X} (x) = P (X = x) = \sum_{y \in R} p_{X, Y} (x, y)$
  - $p_{Y} (y) = P (Y = y) = \sum_{x \in R} p_{X, Y} (x, y)$
- conditions to check for validity
  - $p_{X, Y} (x, y) \geq 0, \forall x, y \in R$
  - $\sum_{x} \sum_{y} p_{X, Y} (x, y) = 1$
- useful calculations
  1. $P (a_{1} < X \leq a_{2}, b_{1} < Y \leq b_{2}) = \sum_{a_{1} < x \leq a_{2}} \sum_{b_{1} < y \leq b_{2}} p_{X, Y} (x, y)$
  2. $F_{X, Y} (a, b) = P (X \leq a, Y \leq b) = \sum_{x \leq a} \sum_{y \leq b} p_{X, Y} (x, y)$
  3. $P (X > a, Y > b) = \sum_{x > a} \sum_{y > b} p_{X, Y} (x, y)$
Example
- suppose: $3$ balls randomly selected from urn
  - w/ $3, 4, 5$ red, white, and blue balls respectively
  - $R, W$ : rn. of white balls chosen
  - pmf of $X, Y$ is:
  r\w 0 1 2 3 $P (R = r)$
  
  0 10/220 40/220 30/220 4/220 84/220
  
  1 30/220 60/220 18/220 0 108/220
  
  2 15/220 12/220 0 0 27/220
  
  3 1/220 0 0 0 1/220
  
  $P (W = w)$ 56/220 112/220 48/220 4/220
  - last row & column: marginal pmf
- suppose: 15% of family in community w/ no children
  - 20% w/ 1
  - 35% w/ 2
  - 30% w/ 3
  - each child: equally likely to be a boy or girl
  - $B$ : no. of boys, $G$ : no. of girls
  $i$ \ $j$ 0 1 2 3 $P (B = i)$
  
  0 0.1500 0.1000 0.0875 0.0375 0.3750
  
  1 0.1000 0.1750 0.1125 0 0.3875
  
  2 0.0875 0.1125 0 0 0.2000
  
  3 0.0375 0 0 0 0.0375
  
  $P (G = j)$ 0.3750 0.3875 0.2000 0.0375

Joint Continuous Random Variables

Joint and marginal pdf
- $X, Y$ : jointly continuous r.v. if there exists joint pdf for every set $C \subset R^{2}$ $P ((X, Y) \in C) = \int \int_{(x, y) \in C} f_{X, Y} (x, y) d x d y$
- marginal pdf: given by $f_{X} (x) = \int_{- \infty}^{\infty} f_{X, Y} (x, y) d y$
- conditions to check: same as discrete
  - $f_{X, Y} (x, y) \geq 0, \forall x, y \in R$
  - $\int_{- \infty}^{\infty} \int_{- \infty}^{\infty} f_{X, Y} (x, y) d x d y = 1$
- some useful calculations
  1. for $A, B \subset R$ , take $C = A \times B$ $P (X \in A, Y \in B) = \int_{A} \int_{B} f_{X, Y} (x, y) d y d x$
  2. for $a_{1}, a_{2}, b_{1}, b_{2} \in R$ where $a_{1} < a_{2}, b_{1} < b_{2}$ $P (a_{1} < X \leq a_{2}, b_{1} < Y \leq b_{2}) = \int_{a_{1}}^{a_{2}} \int_{b_{1}}^{b_{2}} f_{X, Y} (x, y) d y d x$
  3. ⭐ for $a, b \in R$ $F_{X, Y} (a, b) = P (X \leq a, Y \leq b) = \int_{- \infty}^{a} \int_{- \infty}^{b} f_{X, Y} (x, y) d y d x$
  - thus: $f_{X, Y} (x, y) = \frac{\partial ^{2}}{\partial x \partial y} F_{X, Y} (x, y)$
Examples
- joint pdf: given by $f_{X, Y} (x, y) = {2 e^{- x} e^{- 2 y} 0 - < x < \infty, 0 < y < \infty otherwise$
  1. $P (X > 1, Y < 1)$ $P (X > 1, Y < 1) = \int_{1}^{\infty} \int_{0}^{1} 2 e^{- x} e^{- 2 y} d y d x = \int_{1}^{\infty} (- e^{- x} e^{- 2 y}) ∣_{0}^{1} d x = \int_{1}^{\infty} - e^{- x} (e^{- 2} - 1) d x = e^{- x} (e^{- 2} - 1) ∣_{1}^{\infty} = - e^{- 1} (e^{- 2} - 1) = e^{- 1} - e^{- 3}$
  2. $P (X < Y)$ $P (X < Y) = \int_{0}^{\infty} \int_{x}^{\infty} 2 e^{- x} e^{- 2 y} d y d x = \int_{0}^{\infty} - e^{- x} e^{- 2 y} ∣_{x}^{\infty} d x = \int_{0}^{\infty} e^{- 3 x} d x = - \frac{1}{3} e^{- 3 x} ∣_{0}^{\infty} = \frac{1}{3}$
  3. $P (X \leq x)$ $P (X \leq a) P (X \leq x) = \int_{0}^{a} \int_{0}^{\infty} 2 e^{- x} e^{- 2 y} d y d x = \int_{0}^{a} - e^{- x} e^{- 2 y} ∣_{0}^{\infty} d x = \int_{0}^{a} e^{- x} d x = - e^{- x} ∣_{0}^{a} = 1 - e^{- a} = 1 - e^{- x}$
  4. $p_{X} (x)$ $p_{X} (x) = \int_{0}^{\infty} 2 e^{- x} e^{- 2 y} d y = - e^{- x} e^{- 2 y} ∣_{0}^{\infty} = e^{- x}$ = $X \sim E x p (1)$
  5. $F_{Y} (y)$ $F_{Y} (a) F_{Y} (y) = \int_{0}^{\infty} \int_{0}^{a} 2 e^{- x} e^{- 2 y} d y d x = \int_{0}^{\infty} - e^{- x} e^{- 2 y} ∣_{0}^{a} d x = \int_{0}^{\infty} - e^{- x} (e^{- 2 a} - 1) d x = e^{- x} (e^{- 2 a} - 1) ∣_{0}^{\infty} = - (e^{- 2 a} - 1) ∣_{0}^{\infty} = 1 - e^{- 2 a} = 1 - e^{- 2 y}$

MATH 2421: Lecture 16

Date: 2024-10-30 11:52:18

Reviewed:

Topic / Chapter:

summary

❓Questions

❓ let joint pdf of $X, Y$ be circular: e.g. $X^{2} + Y^{2} = k$
- and let joint pdf $X, Y, Z$ be cylinder: extending in Z axis from the circle
- then, can we say $(X, Y) ⊥ ⊥ Z$ ?
  - 👨‍🏫 yes!
❓ what's so complex about continuous-discrete joint distribution?
- 👨‍🏫 its cdf is not well established

Notes

Joint Distribution Functions (cont.)

Examples
- joint density of $X, Y$ : given by $f (x, y) = {e^{- (x + y)} 0 0 < x < \infty, 0 < y < \infty otherwise$
  - find: density of $X / Y$
  - $F (a) = P (A \leq a) = P (X / Y \leq a) = P (X \leq aY)$ $\int_{0}^{\infty} \int_{0}^{a y} e^{- (x + y)} d x d y = \int_{0}^{\infty} - e^{- (x + y)} ∣_{0}^{a y} d y = \int_{0}^{\infty} - e^{- (a + 1) y} + e^{- y} d y = \frac{1}{a + 1} e^{- (a + 1) y} - e^{- y} ∣_{0}^{\infty} = \frac{1}{a + 1} - 1$
  - and $f (a) = F^{'} (a) = - \frac{1}{( a + 1 ) ^{2}}$

Independent Random Variables

Independent r.v.
- two r.v. $X, Y$ : independent if: $P (X \in A, Y \in B) = P (X \in A) P (Y \in B)$
  - for any $A, B \subset R$
  - i.e. value of $X$ not influencing probability of $Y$ , vice versa
  - else: they are dependent
- theorem: following statements are equivalent for jointly discrete / cont. r.v.
  1. r.v. $X, Y$ are independent
  2. $\forall x, y \in R$ , we have: $p_{X, Y} (x, y) = p_{X} (x) p_{Y} (y)$
    - joint pmf = product of marginal pmf
  3. $\forall x, y \in R$ , we have: $F_{X, Y} (x, y) = F_{X} (x) F_{Y} (y)$
    - joint cdf = product of marginal cdf
- r.v. $X, Y$ independent: iff $\exists g, h : R \to R$ s.t. for all $x, y \in R$
  - $f_{X, Y} (x, y) = g (x) h (y)$
  - ( $g, h$ ) not necessarily being $p df$
  - 👨‍🏫 joint pdf: factorizable
    - same for cdf
- proof:
  - for continuous case, independence $⟹$ factorizable
    - for its pdf
  - now: $f_{X, Y} = h (x) g (y)$ $1 = \int_{- \infty}^{\infty} \int_{- \infty}^{\infty} f_{X, Y} (x, y) d x d y = \int_{- \infty}^{\infty} h (x) d x \int_{- \infty}^{\infty} g (y) d y = C_{1} C_{2}$
    - where $C_{1} = \int_{- \infty}^{\infty} h (x) d y, C_{2} = \int_{- \infty}^{\infty} g (y) d y$ :
      - $f_{X} (x) = \int_{- \infty}^{\infty} f_{X} (x, y) d y = C_{2} h (x)$
      - $f_{Y} (y) = \int_{- \infty}^{\infty} f_{Y} (x, y) d x = C_{1} g (y)$
    - as $C_{1} C_{2} = 1$ , $f_{X, Y} (x, y) = f_{X} (x) f_{Y} (y)$
Examples
- given 3 balls randomly selected from urn
  - containing 3 red, 4 white, 5 blue
  - let $R, W$ : no. of red / white chosen
  - are $R, W$ : independent?
  - $P (R = 3, W = 3) = 0 \neq = P (R = 3) P (W = 3)$
- w/ $n + m$ independent trials, having common success probability $p$
  - $X$ : no. of success in first $n$ trials
  - $Y$ : no. of success in last $m$ trials
  - $X$ doesn't influence $Y$ , and vice versa $P (X = x, Y = y) = P (X = x) P (Y = y) = (x n) p^{x} (1 - p)^{n - x} (y m) p^{y} (1 - p)^{m - y}$
  - $X, Y$ : independent
  - similarly $P (Z = z) = (z m + n) p^{z} (1 - p)^{m + n - z}$ $P (X = x, Z = z) = P (X = x, X + Y = z) = P (X = x, Y = z - x) = (x n) p^{x} (1 - p)^{n - x} (z - x m) p^{z - x} (1 - p)^{m - z + x}$
  - and $P (X = x, Z = z) \neq = P (X = x) P (Z, z)$
- man an woman: decided to meet at a location
  - each person: independently arrive at u.a.r. between 12 noon ro 1 pm
  - find probability: first to arrive waiting longer than 10 minutes
  - $X$ : time past 12 noon, in minutes, man arrives
  - $Y$ : time past 12 noon, in minutes, woman arrives
  - $f_{X} (x) = \frac{1}{60}$ if $0 \leq x \leq 60$
  - $f_{Y} (y) = \frac{1}{60}$ if $0 \leq y \leq 60$
  - $f (x, y) = \frac{1}{60}^{2}$ if $0 \leq x, y \leq 60$
  - finding: $P (X \geq Y + 10) + P (X + 10 \leq Y) = 2 P (X \geq Y + 10)$ $P (X \geq Y + 10) = \int_{0}^{50} \int_{y + 10}^{60} (\frac{1}{60})^{2} d x d y = \int_{0}^{50} (\frac{1}{60})^{2} x ∣_{y + 10}^{60} d y = \int_{0}^{50} \frac{1}{60} - \frac{y + 10}{6 0 ^{2}} d y = \int_{0}^{50} \frac{1}{60} - \frac{y}{6 0 ^{2}} - \frac{10}{6 0 ^{2}} d y = \frac{y}{60} - \frac{y ^{2} /2}{6 0 ^{2}} - \frac{10 y}{6 0 ^{2}} ∣_{0}^{50} = \frac{3000 - 1250 - 500}{3600} = \frac{1250}{3600} = \frac{25}{72}$
  - thus $\frac{25}{36}$ ..?
- Buffon's needle problem: table ruled w/ equidistant parallel lines a $D$ apart
  - needle: length $L \leq D$
    - randomly thrown on the table
  - what is the probability that needle: intersect one of the lines?
- if $f (x, y) = 24 x y$ , $0 < x < 1, 0 < 1, 0 < x + y < 1$
  - are r.v. independent?
  - solution 1
    - find marginal pdf, and compare $f_{X} (x) f_{Y} (y) f (x, y) = \int_{0}^{1 - x} 24 x y d y = 12 x y^{2} ∣_{0}^{1 - x} = 12 x (1 - x)^{2} = 12 y (1 - y)^{2} \neq = f_{X} (x) f_{Y} (y)$
  - solution 2
    - let $I (x, y) = {10 if 0 < x < 1, 0 < y < 1, 0 < x + y < 1 otherwise$
  - $f (x, y) = 24 x y I (x, y)$ for all $x, y \in R$
    - and $f (x, y)$ : not factorizable
      - and thus: $X, Y$ are not independent
    - i.e. factorizable: we can define $I (x, y) = I (x) I (y)$ for all $x, y \in R$
  - basically: $I$ is factorizable if its region is shown as square, cube, etc.
    - which edges are all parallel to the axis
    - otherwise: impossible
- $N$ traffic accidents occur per day w/ $N \sim P o i sso n (λ)$
  - each accident: major & minor
    - and it is a major accident w/ $p \in (0, 1)$
  - let $X, Y$ : denote no. of major & minor accidents, respectively
  - $X + Y = N$ , if $N = n$ , then $X \sim B in (n, p)$
  1. find joint pmf of $X, Y$
    - for $i, j \geq 0$ $P (X = i, Y = j) = P (X = i, X + Y = i + j) = P (X = i, N = i + j) = P (X = i ∣ N = i + j) P (N = i + j) = (i i + j) p^{i} (1 - p)^{j} P (N = i + j) = (i i + j) p^{i} (1 - p)^{j} e^{- λ} \frac{λ ^{i + j}}{( i + j )!} = \frac{( i + j )!}{i ! j !} p^{i} (1 - p)^{j} e^{- λ} \frac{λ ^{i + j}}{( i + j )!} = p^{i} (1 - p)^{j} e^{- λ} \frac{λ ^{i + j}}{i ! j !} = e^{- λ} \frac{( λ p ) ^{i}}{i !} \frac{( λ ( 1 - p ) ) ^{j}}{j !} as X \sim B in (n, p)$
  2. are $X, Y$ independent?
    - yes, as it's factorizable by:
      - $g (i) = e^{- λ} \frac{( λ p ) ^{i}}{i !}$ within range
      - $h (i) = \frac{( λ ( 1 - p ) ) ^{j}}{j !}$ within range
  3. can you identify: distribution of $X, Y$ ?
    - 👨‍🏫 exercise!
    - $X \sim P o i sso n (λ p)$
    - $Y \sim P o i sso n (λ)$

MATH 2421: Lecture 17

Date: 2024-11-04 11:44:02

Reviewed:

Topic / Chapter:

summary

❓Questions

❓ does $f_{X, Y, Z} (x, y, z) = f_{X} (x) f_{Y} (y) f_{Z} (z)$ implies pairwise independence, etc.?
- 👨‍🏫 Yes. As the condition is $\forall x, y, z \in R$ , it's fine

Notes

Independent Random Variables (cont.)

More than 2 r.v.
- joint pdf of multiple r.v. (e.g. $X, Y, Z$ ) can be described as: $P ((X, Y, Z) \in C) = \int\int \int_{(x, y, z) \in C} f_{X, Y, Z} (x, y, z) d x d y d z$
- multiple cont. r.v. (e.g. $X, Y, Z$ ) are independent if, $\forall x, y, z \in R$ : $f_{X, Y, Z} (x, y, z) = f_{X} (x) f_{Y} (y) f_{Z} (z)$
Examples
- suppose $X, Y$ : independent standard normal dist.
  - find: joint Pdf of $X, Y$
  - from independence: $p_{X, Y} (x, y) = p_{X} (x) \cdot p_{Y} (y)$
  - $= \frac{1}{2 π σ ^{2}} e^{- \frac{x + y}{2}}$
- let $X, Y, Z$ be independent & uniformly distributed over $(0, 1)$
  - compute $P (X \geq Y Z)$
  - $P (X \geq k) = 1 - k$
  - $P (Y Z = k) = P (Y = y \land Z = k / y) = P (Y = y) P (Z = k / y)$ $P (X \geq Y Z) = \int\int \int_{x \geq yz} f_{X, Y, Z} (x, y, z) d x d y d z = \int_{0}^{1} \int_{0}^{1} \int_{yz}^{1} 1 d x d y d z = \int_{0}^{1} \int_{0}^{1} 1 - yz d y d z = \int_{0}^{1} y - y^{2} z /2 ∣_{0}^{1} d z = \int_{0}^{1} 1 - z /2 ∣_{0}^{1} d z = z - z^{2} /4 ∣_{0}^{1} = 3/4$

Sum of Independent random Variables

X, Y being continuous & independent
- theorem: on independence, for $x, y \in R$
  - $f_{X, Y} (x, y) = f_{X} (x) f_{Y} (y)$
  - following: $F_{X, Y} (x) f_{X, Y} (x) = \int_{- \infty}^{\infty} F_{X} (x - t) f_{Y} (y) d t = \int_{- \infty}^{\infty} F_{Y} (x - t) f_{X} (t) d = \int_{- \infty}^{\infty} f_{X} (x - t) f_{Y} (y) d t = \int_{- \infty}^{\infty} f_{Y} (t) f_{X} (x - t y) d t$
    - ⭐👨‍🏫 important
  - $f_{X + Y}$ : "convolution of $f_{X}, f_{Y}$ "
- proof
- some of 2 important Gamma d.v.
  - $X \sim Γ (α, λ) \sim X \sim Γ (α, λ)$
  - then $X + Y \sim Γ (a + b + λ)$
- proof
- ⭐👨‍🏫 sum of independent normal r.v.
  - for $X_{i}$ where $X$
  - use linearity
- d
X, Y being discrete & independent
- pmf of $X + Y$ , each being discrete r.v.: $p_{X + Y} (t) = P (X + Y = t) = x \sum P (X = x, Y = t - x) = p_{X, Y} (x, t - x)$
- similarly, for continuous r.v.: $p_{X + Y} (t) = P (X + Y = t) = \int_{x} P (X = x, Y = t - x) d x = p_{X, Y} (x, t - x)$
Exercises
- $X, Y : in d e p . w / c m o m o n core d i s t . a rc j o m o a -$ X+t\leq (0,2) $<! - - - f u r t h er w or k - - > - -$ 2-x $- re ma r k : i f$ d_{X+Y}(x,) w/ triangular dist."
  - $X + Y$ : dimctop $d$ smooth triangular
- basketball team: plays a 44-game season
  - 26 games: against class A; 18: against class B
  - chance of against class A: $0.4$
  - chance of against class B: $0.7$
  - result of different games: independent
  - $X_{A}, X_{B}$ : no. of wins against class $A, B$
  - $X_{A} \sim B in (26, 0.4)$
  - $X_{B} \sim B in (18, 0.7)$
  - compute $P (X_{A} + X_{B} \geq 25)$ or $P (25 \leq X_{A} + X_{B} \leq 44)$ (👨‍🏫 either works)
    - 👨‍🏫 use normal approximate
    - $B in (n, p) \approx N (n p, n pq)$
    - $X_{A} \approx N (10.4, 6.24)$
    - $X_{B} \approx N (12.6, 3.78)$
    - sum of independent normal: normal dist.
    - $X_{A} + X_{B} \approx N (13, 10.02)$ $P (X_{A} + X_{B} \geq 25) = P (X_{A} + X_{B} \geq 24.5) \approx P (\frac{X _{A} + X _{B} - 24}{10.02} \geq \frac{24.5 - 23}{10.02}) = P (Z \geq \frac{1.5}{10.02}) = 1 - P (Z \leq \frac{1.5}{10.02}) = 1 - Φ (\frac{1.5}{10.02}) \approx 0.3178$
  - compute $P (X_{A} > X_{B})$
    - $P (X_{A} > X_{B}) = P (X_{A} \geq X_{B} + 1) = P (X_{A} - X_{B} \geq 1)$
    - $- X_{B} \approx N (- 12.6, 3.78)$
    - then $X_{A} - X_{B} \approx N (10.4 - 12.6, 6.24 + 3.78) = N (- 2.2, 10.02)$ $P (X_{A} - X_{B} \geq 1) = P (X_{A} - X_{B} \geq 0.5) \approx P (\frac{X _{A} - X _{B} + 2.2}{10.02} \geq \frac{0.5 + 2.2}{10.02}) = P (Z \geq \frac{2.7}{10.02}) = 1 - P (Z \leq \frac{2.7}{10.02}) = 1 - Φ (\frac{2.7}{10.02}) \approx 0.1968$
- sum of 2 indep. binomial r.v.
  - $X \sim B in (n, p); Y \sim B in (m, p)$ , $X, Y$ indep.
  - find pmf of $X + Y$ $P (X + Y = k) = i = 0 \sum k$
- s

Lecture 18

MATH 2421: Lecture 19

Date: 2024-11-11 11:54:21

Reviewed:

Topic / Chapter:

summary

❓Questions

Notes

Joint PDF of Functions of RV

Generalization to multiple variables
- 👨‍🏫 for exams: you are expected to do max. $2$
- Jacobian determinant expands, to $n \times n$
Example
- let $X_{1}, X_{2}$ : jointly continuous r.v. w/ pdf $f_{X_{1}, X_{2}}$
  - let $Y_{1} = X_{1} + X_{2}, Y_{2} = X_{1} - X_{2}$
  - find joint density function of $Y_{1}, Y_{2}$ in terms of $f_{X_{1}, X_{2}}$
  - d $f_{Y_{1}, Y_{2}} (y_{1}, y_{2}) = f_{X_{1}, X_{2}} (\frac{1}{2} (y_{1} + y_{2}), \frac{1}{2} (y_{1} - y_{2})) \frac{\partial ( y _{1} , y _{2} )}{\partial ( x _{1} , x _{2} )}^{- 1} = \frac{1}{2} f_{X_{1}, X_{2}} (\frac{1}{2} (y_{1} + y_{2}), \frac{1}{2} (y_{1} - y_{2}))$
- suppose $X, Y$ : independent standard normal variable
  - show: $X + Y, X - Y$ are independent normal variables
  - let $X_{1} = X; X_{2} = Y$
  - and $Y_{1} = X + Y; Y_{2} = X - Y$
  - as they are independent: $f_{X_{1}, X_{2}} (x_{1}, x_{2}) = f_{X_{1}} (x_{1}) f_{X_{2}} (x_{2}) = \frac{1}{2 π} e^{- \frac{x _{1}^{2} + x _{2}^{2}}{2}}$
  - by previous example: joint pdf of $Y_{1}, Y_{2}$ is $f_{Y_{1}, Y_{2}} (y_{1}, y_{2}) = \frac{1}{2} f_{X_{1}, X_{2}} (\frac{1}{2} (y_{1} + y_{2}), \frac{1}{2} (y_{1} - y_{2})) = \frac{1}{2} \frac{1}{2 π} e^{- \frac{1}{2} (\frac{1}{4} (y_{1} + y_{2})^{2} + \frac{1}{4} (y_{1} - y_{2})^{2})} = \frac{1}{4 π} e^{- \frac{1}{2} (\frac{1}{2} y_{1}^{2} + \frac{1}{2} y_{2}^{2})} = \frac{1}{4 π} e^{- \frac{1}{4} (y_{1}^{2} y_{2}^{2})} = \frac{1}{4 π} e^{- \frac{1}{4} y_{1}^{2}} \frac{1}{4 π} e^{- \frac{1}{4} y_{2}^{2}}$
  - as con
  - $Y_{1} \sim N (0, 2)$
- let $X, Y$ be independent standard normal
  - then $R = X^{2} + Y^{2}, Θ = arctan (\frac{Y}{X})$
  - $f_{X, Y} (x, y) = \frac{1}{2 π} e^{- \frac{x ^{2} + y ^{2}}{2}}$ ${r = x^{2} + y^{2} Θ = arctan (\frac{y}{x}) {x = r cos Θ y = r sin Θ$
  - pdf of $R, Θ$
  - domain: $r > 0 \cap Θ \in [0, 2 π)$ $\frac{\partial ( x , y )}{\partial ( r , Θ )} = \frac{\partial x}{\partial r} \frac{\partial y}{\partial r} \frac{\partial x}{\partial Θ} \frac{\partial y}{\partial Θ} = cos Θ sin Θ - r sin Θ r cos Θ = r cos^{2} - (- r sin^{2}) = r \geq 0$
  - thus, pdf of $R, Θ$ : $f_{R, Θ} (r, Θ) = f_{X, Y} (x, y) = \frac{1}{2 π} e^{- \frac{r ^{2}}{2}} r$
  - for joint pdf $f_{R, Θ} (r, θ)$ is factorization, $R, Θ$ are independent
    - $Θ$ : uniform distribution
      - 👨‍🏫 rotation ignorance property of normal vectors
  - $R$ : Rayleigh distribution
- if $X, Y$ : independent Gamma r.v. w/ parameters $(α, γ), (β, γ)$
  - compute joint density of $U = X + Y; V = X / (X + Y)$ ${u = x + y v = \frac{x}{x + y} ⟹ {x = uv y = u - uv = u (1 - v)$
- Finally, joint pdf of $X, Y$ : $f_{X, Y (x . y)} = f_{X} (x) f_{Y} (y) = A ^{α + β}$ $\frac{λ e ^{- λ u} ( λ u ) ^{α + β + 1}}{Γ ( α + β )}$
- try harder later

Jointly Distributed R.V w/ n>2

Jointly distributed r.v.
- $F_{X, Y, Z} (x, y, z) := P (X \leq x, Y \leq y, Z \leq z)$
- marginal distribution, namely:
  - $F_{X, Y} (x, y) := lim_{z \to \infty} F_{X, Y, Z} (x, y, z)$
- similar for density function

Expectation of Sum of Random Variables

Expectation of sum of random variables
- theorem: $a \leq X \leq b ⟹ a \leq E (X) \leq b$
- proof:
- if $X, Y$ : jointly discrete w/ joint pmf $p_{X, Y}$
  - $E [g (X, Y)] = \sum_{y} \sum_{x} g (x, y) p_{X, Y} (x, y)$
- if $X, Y$ : jointly continuous w/ joint pdf $p_{X, Y}$
  - $E [g (X, Y)] = \int_{y} \int_{x} g (x, y) p_{X, Y} (x, y) d x d y$
- remarks
  1. if $g (x, y) \geq 0$ whenever $p_{X, Y} > 0$ , then $E [g (X, Y)] \geq 0$
  2. $E [g (X, Y) + h (X, Y)] = E [g (X, Y)] + E [h (X, Y)]$
  3. $E [g (X) + h (Y)] = E [g (X)] + E [h (Y)]$
  4. Monotone property, if $X \leq Y$ , then $E (X) \leq E (Y)$
- important special case
  - mean of sum: sum of means
  - $E (X + Y) = E (X) + E (Y)$
    - leads to: linearity of expectation regardless of independency
  - to compute the sum: marginal pdf / pmf is enough
- d
Example
- accident: at point $X$ w/ uniformed distributed on a road of length $L$
  - ambulance: at location $Y$ , uniformly distributed on the same road
  - find: expected distance between the ambulance and point of the accident
  - $X \sim U (0, L); Y \sim Y (0, L)$
  - joint pdf: multiplication! $f_{X, Y} (x, y) = {\frac{1}{L ^{2}} 0 if x \in (0, L), y \in (0, L) otherwise$ $E [∣ X - Y ∣] = \int\int ∣ x - y ∣ f_{X, Y} (x, y) d x d y = \int_{0}^{L} \int_{0}^{L} ∣ x - y ∣ \frac{1}{L ^{2}} d x d y = 2 \int_{0}^{L} \int_{y}^{L} (x - y) \frac{1}{L ^{2}} d x d y = 2 \int_{0}^{L} (\frac{x ^{2}}{2 L ^{2}} - \frac{x y}{L ^{2}}) ∣_{y}^{L} d y = 2 \int_{0}^{L} \frac{1}{2} - \frac{y}{L} + \frac{y ^{2}}{2 L ^{2}} d y = 2 (\frac{y}{2} - \frac{y ^{2}}{2 L} + \frac{y ^{3}}{6 L ^{2}}) ∣_{0}^{L} = 2 (\frac{L}{2} - \frac{L}{2} + \frac{L}{6}) = \frac{L}{3}$
- sample mean: let $X_{1}, \dots, X_{n}$ : independent & identically distributed r.v.
  - w/ distribution function $F$ and expected value $μ$
  - such sequence of r.v.: constitute a sample from distribution $F$
  - sample mean $\overset{ˉ}{X}$ : defined as $\overset{ˉ}{X} = \frac{1}{n} k = 1 \sum n X_{k}$
  - $E (\overset{ˉ}{X}) = E [\frac{1}{n} \sum_{k = 1}^{n} X_{k}] = \frac{1}{n} \sum_{k = 1}^{n} E [X_{k}] = \frac{1}{n} \sum_{k = 1}^{n} μ = μ$
- Boole's inequality (skipped)
- mean of hypergeometric
  - $n$ balls: selected from $N$ balls of which $m$ are white
    - find expected no. of white balls selected
  - $X$ : no. of white balls selected
  - use indicator random variable
    - $E (X) = \sum_{k} E (X_{k}) = \sum_{k} P (x_{k} = 1)$
  - $E (X_{i}) = \frac{m}{N}$
  - $E (X) = n \cdot \frac{m}{N}$

Math 2421: Lecture 20

Date: 2024-11-13 12:01:10

Reviewed:

Topic / Chapter:

summary

❓Questions

Notes

Expectation of Sum of Random Variables

Example
- $n$ people: throw their hats to center of a room
  - hats: mixed up, find expected no. of people getting back their own hats
  - $E (\sum) = \sum (E) = \sum (P) = n \times \frac{1}{n}$
- suppose: $N$ different types of coupons exists
  - and each time: obtaining a coupon is equally likely to be any of $N$ types
  - find: expected no. of different types with $n$ run
    - denote answer by $X$
      - $X = \sum X_{i}$ for $i \in [1, N]$
    - $X_{i}$ : 1 if $i$ -th type is contained in the $n$ company
    - $E (X) = \sum E (X_{i})$
      - $E (X) = P (X_{i} = i) = 1 - P (none of tickets being type i)$
      - $= 1 - \frac{( N - 1 ) ^{n}}{N ^{n}}$
      - $E (X) = N \times (1 - \frac{( N - 1 ) ^{n}}{N ^{n}})$
  - find expected no. of coupons needed before completing at lease one set of each type
    - let $Y$ : no. of coupons needed to get a complete set
    - $Y = Y_{0} + Y_{1} + \dots Y_{N - 1}$
    - $Y_{0}$ : no. of coupons you need in order to get a first new type
      - $= 1$
    - $Y_{1}$ : no. of coupons you need in order to get a second a new type (after $Y_{0}$ )
      - $E (Y_{1}) = 1 \times (\frac{N - 1}{N}) + \frac{1}{N} (1 + E (Y_{1}))$
      - $E (Y_{1}) = \frac{N}{N - 1}$
    - $Y_{i}$ : no. of coupons to get a new type after collecting $i - 1$ types
      - $E (Y_{i}) = 1 \times (\frac{N - i}{N}) + \frac{i}{N} (1 + E (Y_{i}))$
      - $E (Y_{i}) = \frac{N}{N - i}$
      - $Y_{i} \sim G eo m (p_{i})$
      - where success probability $p_{i} = \frac{N - i}{N}$
    - thus: $Y = \sum_{i = 0}^{N - 1} \frac{N}{N - i} = N (\sum_{i = 1}^{N} \frac{1}{i}) \approx N lo g N$

Covariance, Variance of Sums, and Correlations

Covariance
- covariance of jointly distributed r.v. $X, Y$ :
  - $cov (X, Y) = E [(X - μ_{X}) (Y - μ_{Y})]$
  - measures: how $X, Y$ vary together
- common formula
  - $cov (X, Y) = cov (Y, X)$
  - $cov (X, X) = var (X)$
  - $cov (X, a) = E [(X - μ_{X}) (a - a)] = 0$
  - if, for larger value of $X$ : corresponds to larger values of $Y$
    - then $cov (X, Y) > 0$
    - else: $cov (X, Y) < 0$
- remark: if $cov (X, Y) \neq = 0$ , $X, Y$ : correlated
  - else: $X, Y$ : uncorrelated
- correlation: does not imply causation
- theorem: $cov (X, Y) = E (X Y) - E (X) E (Y)$
  - similar to $var (X) = E (X^{2}) - (E (X))^{2}$
- proof $cov (X, Y) = E [(X - μ_{X}) (Y - μ_{Y})] = E [(X - μ_{X}) (Y - μ_{Y})]$
- theorem: if $X, Y$ independent, then for any $g, h : R \to R$ , we have
  - $E [g (X) h (Y)] = E [g (X)] E [h (Y)]$
  - $E [X Y] = E X E [Y]$
- theorem: if $X, Y$ independent: then $cov (X, Y) = 0$
  - 👨‍🏫 however: $cov (X, Y)$ doesn't mean that $X, Y$ are independent
- proof: set some $X$ s.t. $E (X) = 0$ , and let $Y$ dependent on $X$
  - and let $X Y = 0$ always
  - then $cov (X, Y) = E [X Y] - E [X] [E [Y] = 0]$
- theorem: $var (k = 1 \sum n X_{k}) = k = 1 \sum n var (X_{k}) + 2 1 \leq i < j \leq n \sum cov (X_{i}, X_{j})$
- proof:
- w/ independent $X_{i}$ $var (k = 1 \sum n X_{k}) = k = 1 \sum n var (X_{k})$
  - under independence: $var (\sum) = \sum var$
- proof:
Properties of covariance
1. $var (X) = cov (X, X)$
2. $cov (X, Y) = cov (Y, X)$
3. $cov (\sum_{i = 1}^{n} a_{i} X_{i}, \sum_{i = 1}^{m} n_{j} Y_{i}) = \sum_{i = 1}^{n} \sum_{j = 1}^{m} a_{i} b_{j} cov (X_{i}, Y_{j})$
- 👨‍🏫 no independence assumed
- d
Covariance
Example
- sample variance
  - $X_{i} - \overset{ˉ}{X}$ is called deviation
  - sample variance: $S^{2} = \sum_{i = 1}^{n} \frac{( X _{i} - X ˉ ) ^{2}}{n - 1}$
  - find $var (\overset{ˉ}{X})$
    - $var (\overset{ˉ}{X}) = var (\frac{1}{n} \sum_{i = 1}^{n} X_{i}) = \frac{1}{n ^{2}} var (\sum_{i = 1}^{n} X_{i})$
    - w/ independence: $= \frac{1}{n ^{2}} \sum_{i = 1}^{n} var (X_{i}) = \frac{1}{n ^{2}} \cdot n σ^{2} = \frac{σ ^{2}}{n}$
  - find $E [S^{2}]$ $E (S^{2}) E (X_{i} - \overset{ˉ}{X})^{2} cov (X_{i}, \frac{1}{n} j = 1 \sum n X_{j}) cov (X_{i}, X_{j}) cov (X_{i}, \overset{ˉ}{X}) E (X_{i} - \overset{ˉ}{X})^{2} E (S^{2}) = E (\frac{1}{n - 1} i = 1 \sum n (X_{i} - \overset{ˉ}{X})^{2}) = \frac{1}{n - 1} i = 1 \sum n E [(X_{i} - \overset{ˉ}{X})^{2}] = E [(X_{i} - μ) - (\overset{ˉ}{X} - μ)]^{2} = E [(X_{i} - μ)^{2} + (\overset{ˉ}{X} - μ)^{2} - 2 (X_{i} - μ) (\overset{ˉ}{X} - μ)] = E [(X_{i} - μ)^{2}] + E [(\overset{ˉ}{X} - μ)^{2}] - 2 E [(X_{i} - μ) (\overset{ˉ}{X} - μ)] = var (X_{i}) + var (\overset{ˉ}{X}) - 2 cov (X_{i}, \overset{ˉ}{X}) = σ^{2} + \frac{1}{n} σ^{2} - 2 cov (X_{i}, \overset{ˉ}{X}) = \frac{1}{n} j = 1 \sum n cov (X_{i}, X_{j}) = 0 = \frac{1}{n} cov (X_{i}, X_{i}) = \frac{1}{n} var (X_{i}) = \frac{σ ^{2}}{n} = σ^{2} - \frac{1}{n} σ^{2} = \frac{n - 1}{n} σ^{2} = σ^{2} independence if i \neq = j$
  - thus, sample variance w/ $n - 1$ is a good estimator

MATH 2421: Lecture 21

Date: 2024-11-18 11:59:35

Reviewed:

Topic / Chapter:

summary

❓Questions

Notes

Covariance, Variance of sums, and Correleations (cont.)

Covariance
- correlation (coefficient) of r.v. $X, Y$ : defined by $ρ (X, Y)$ defined by: $ρ (X, Y) = \frac{cov ( X , Y )}{var ( X ) var ( Y )} = \frac{cov ( X , Y )}{σ _{X} σ _{Y}}$
- thus: $- 1 \leq ρ (X, Y) \leq 1$
- proof $0 \leq var (\frac{X}{σ _{X}} + \frac{Y}{σ _{Y}^{2}}) = \frac{var ( X )}{σ _{X}^{2}} + \frac{var ( Y )}{σ _{Y}^{2}} + \frac{n cov ( X , Y )}{σ _{X} σ _{Y}} =$
- remarks
  1. d
  2. $ρX, Y = 1$ iff $Y = a X + b$ w/ $σ_{Y} σ_{X} = 0$
    - tjere $a = σ_{X} / σ_{X}$
  3. s
  - if $Y = a X + b$ w/ $a = σ_{Y} / σ_{X}$
  1. d
  2. independence of $X, Y$ $ρ (X, Y)$ implies
    - but the opposite: doesn't get support
- d
Example
- variance of binomial
  - $X \sim B in (n, p)$ : given by $X = I_{1} + I_{2} + \dots + I_{n}$
  - $I_{k}$ : whether $k$ -th trial is success / not
  - $var (X) = var (\sum_{k = 1}^{n}) = \sum_{k = 1}^{n} var (I_{k}) = \sum_{k = 1}^{n} p (1 - p) = n p (1 - p)$
- example
  - $X = \pm 1$ , w/ fair probability
  - $Y = 1$ if $X = 1$ , $Y = - 1$ if $Y = - 1$
  - $E (X) = E (Y) = 0$ , and as $E (X Y) = 1$ , $cov (X, Y) = 1$
  - $ρ (X, Y_{1}) = ρ (X, Y_{2}) = 1$

Conditional expectation

Conditional expectation
1. w/ $X, Y$ : jointly distributed discrete r.v. $E [X [Y = u]] := x \sum x p_{X ∣ Y} (x, y)$
  - w/ $p_{Y} (y) > 0$
  - $p_{X ∣ Y} (c, y) = \frac{p _{X, Y} ( x , y )}{p _{Y} ( y )} = \frac{joint}{marginal}$
    - a valid PMF!
2. w/ $X, Y$ : jointly distributed continuous r.v. $E [X [Y = u]] := \int_{- 1\infty}^{\infty} x f_{X ∣ Y} (x, y)$
  - w/ $p f_{Y} (y) > 0$
  - $p_{X ∣ Y} (c, y) = \frac{p _{X, Y} ( x , y )}{p _{Y} ( y )} = \frac{joint}{marginal}$
  - w/ $p_{Y}$ a valid PDF!
- important formula
  1. s $E [g (X) ∣ Y = y] = {\sum_{x} g (x) p_{X ∣ Y} (x ∣ y) \int_{- \infty}^{\infty} g (x) f_{X ∣ Y} (x ∣ y) d x discrete continuous$
    - thus: conditional expectation of sum = sum of conditional expectation $E [k = 1 \sum n X_{k} ∣ Y = y] = k = 1 \sum n E [X_{k} ∣ Y = y]$
- furthermore, consider $E [X ∣ Y]$ as function of $Y$ , $g (Y)$
  - e.g. $E [X ∣ Y = y] = y /2 - 5$ , then $E [X ∣ Y] = Y /2 - 5$
- theorem: $E [X] = E [g (Y) E (X ∣ Y)] = {\sum_{y} E (X ∣ Y = y) P (Y = y) \int_{- \infty}^{\infty} E (X ∣ Y = y) f_{Y} (y) d y discrete Y continuous Y$
  - i.e. nesting property of expectation
  - $E_{Y} [E_{X} [X ∣ Y]]$
  - and $g_{Y} = E_{X} [X ∣ Y]$
- d
Example
- $f (x, y) = {\frac{e ^{- x / y} e ^{- y}}{y} 0 0 < x < \infty, 0 < y < \infty otherwise$
- $f_{X ∣ Y} (x ∣ y) = \frac{f ( x , y )}{f _{Y} ( y )}$
- for $y > 0$ : $f_{Y} (y) = \int_{- \infty}^{\infty} f (x, y) d x = \int_{0}^{\infty} e^{- x / y} \cdot \frac{1}{y} e^{- y} d x = e^{- y} \int_{0}^{\infty} e^{- x / y} \cdot \frac{1}{y} d x = e^{- y}$
- $f_{X ∣ Y} (x ∣ y) = {d$
- d
f

MATH 2421: Lecture 22

Date: 2024-11-20 11:27:40

Reviewed:

Topic / Chapter:

summary

❓Questions

Notes

Conditional Expectation (cont.)

Conditional expectation
- we may also use this approach to compute probabilities
  - s.t. $X = I_{A}$ w/ $A$ being an event
    - then $E (I_{A}) = P (A)$
    - then $E (I_{A} ∣ Y = y) = P (A ∣ Y = y)$
  - we obtain: $P (A) = E (I_{A}) = E [E (I_{A} = y)]$ $= {\sum_{y} E (I_{A} ∣ Y = y) P (Y = 1) \int_{y} E (I_{A} ∣ Y = y) f_{y} (y) d y Y being discrete Y being continuous$
  - we obtain: $P (A) = E (I_{A}) = E [E (I_{A} = y)]$
    - ${\sum_{y} P (A ∣ Y = y) P (Y = 1) \int_{y} P (A ∣ Y = y) f_{y} (y) d y Y being discrete Y being continuous$
  - e.
- d
Example
- miner: trapped in a mine w/ 3 doors
  - first door: to a safety place after 3 hours
  - second door: back to same place after 5 hours
  - third door: back to same place after 7 hours
  - choosing each door: equally likely
    - and previous choice is forgotten
  - find: expected length of time, until the miner reach the safety
  - let $X$ : length of time until reaching the safety
    - $Y$ : door number he chooses $E (X ∣ Y = 1) E (X ∣ Y = 2) E (X ∣ Y = 3) P (Y = 1) E (X) = E_{Y} (E (X ∣ Y)) \frac{1}{3} E (X) E (X) = 3 = 5 + E (X) = 7 + E (X) = P (Y = 2) = P (Y = 3) = \frac{1}{3} = E (X ∣ Y = 1) P (Y = 1) + E (X ∣ Y = 2) P (Y = 2) + E (X ∣ Y = 3) P (Y = 3) = \frac{1}{3} \cdot 3 + \frac{1}{3} (5 + E (X)) + \frac{1}{3} (7 + E (X)) = 5 = 15$
- expectation of a random sum
  - suppose: $X_{1}, X_{2}, \dots$ are iid w/ common mean $μ$
  - let $N$ non-negative, integer valued r.v. independent of $X_{i}$
  - our interest: mean of $T = \sum_{k = 1}^{n} X_{k}$
    - where $T$ : taken to be zero when $N = 0$
  - solution $E (Y) = E (k = 1 \sum N X_{k}) = E_{N} [E (∣ N)] = n = 0 \sum \infty E (\sum X_{k} ∣ N = n) P (N = n) = n = 0 \sum \infty E (\sum n X_{k} ∣ N = n) P (N = n) = n = 0 \sum \infty E (\sum n X_{k}) P (N = n) = n = 0 \sum \infty k = 1 \sum n E (X_{k}) P (N = n) = n = 0 \sum \infty n μ P (N = n) = μ n = 0 \sum \infty n P (N = n) = μ E (N)$
- suppose: no. of people getting department store on a day: r.v. w/ $μ = 50$
  - amounts of money spent by individual customers: indep. r.v. w/ common mean $8$
  - suppose amount of money spent by a customer: independent of total no. customers entering
    - what is: expected amount of money spent on a given day?
- $X, Y$ : independent w/ pdf $f_{X}, f_{Y}$ . Compute $P (X < Y)$
  - conditioning on the value of $Y$ $P (X < Y) = \int_{- \infty}^{\infty} P (X < Y ∣ Y = y) f_{Y} (y) d y = \int_{- \infty}^{\infty} P (X < y ∣ Y = 1) f_{Y} (y) d y = \int_{- \infty}^{\infty} P (X < Y) f_{Y} (y) d y$
- let $U$ be u.r.v. and define conditional density is $X$ $p$
- $\frac{x ^{k} ( 1 - k ) ^{n - k}}{B ( k + 1 , n - k + 1 )}$

Conditional Variance

Conditional variance
- pre-req $var (X) = E [(X - E (X))^{2}] = E (X^{2}) - [E (X)]^{2}$ $var (X ∣ Y = y) var (X ∣ Y) = E [(X - E (X ∣ Y = y))^{2} ∣ Y = y] = E (X^{2} ∣ Y = y) - [E (X ∣ Y = y)^{2}] = h (y) = h (y) function of Y$
- conditional variance of $X$ given $Y = y$ $var (X ∣ Y) \equiv E [(X - E [X ∣ Y])^{2} ∣ Y]$
- notably: $var (X) = E [var (X ∣ Y)] + var (E [X ∣ Y])$
- proof
- 👨‍🏫 not often used, and its application is not expected
Example
- 👨‍🎓 (very complicated computation)

Moment Generating Function

Moment generating function
- moment generating function of r.v. $X$ , $M_{X}$ is defined as: $M_{X} (t) = E [e^{tX}] = {\sum_{X} e^{t x} p_{X} (x) \int e^{t x} f_{X} (x) d x if X discrete if X continuous$
- called as MFG as:
  - it generates moments of r.v. for $n \geq 0$
  - $E (X^{n}) = M_{X}^{(n)} (0)$
    - where $M_{X}^{(n)} (0) := \frac{d ^{n}}{d t ^{n}} M_{X} (t) ∣_{t = 0}$
- proof: by Taylor series expansion $E [e^{tX}] = E [k = 0 \sum \infty \frac{( tX ) ^{k}}{k !}] = k = 0 \sum \infty \frac{E ( tX ) ^{k}}{k !} = k = 0 \sum \infty \frac{E ( X ) ^{k}}{k !} t^{k}$
  - and $M_{X} (t) = \sum_{k = 0}^{\infty} \frac{M _{X}^{k} ( 0 )}{k !} t^{k}$
    - equating coefficient of $t^{n}$ , we get $M_{X}^{(n)} (0) = E (X^{n})$
  - 👨‍🏫 not all r.v. has a mgf
    - e.g. for cauchy distribution, $E (X^{n})$ does not exist for $n \geq 1$ , and $X$ has no mgf
      - 👨‍🏫 for advanced topics, people use complex numbers and find ways
- theorem: multiplicative property
  - if $X, Y$ independent:
  - $M_{X + Y} (t) = M_{X} (t) M_{Y} (t)$
- proof $E [e^{t (X + Y)}] = E [e^{tX} e^{t Y}] = E [e^{tX}] [e^{t Y}] = M_{X} (t) M_{Y} (t)$
- theorem: uniqueness property
  - let $X, Y$ be r.v. w/ their MGF $M_{X}, M_{Y}$
  - suppose there exists an $h > 0$ s.t.
    - $M_{X} (t) = M_{Y} (t), \forall t \in (- h, h)$
  - then $X, Y$ have the same distribution (i.e. $F_{X} = F_{Y}$ or $f_{X} = f_{Y}$ )

MATH 2421: Lecture 23

Date: 2024-11-25 12:00:07

Reviewed:

Topic / Chapter:

summary

❓Questions

Notes

Moment Generating Function (cont.)

Moment generating function
Examples
- when $X \sim B e (p), M (t) = 1 - p + p e^{t}$ $M (t) = E (e^{tX}) = e^{t \cdot 0} P (X = 0) + e^{t \cdot 1} P (X = 1) = 1 - p + p e^{t}$
- when $X \sim B in (n, p), M (t) = (1 - p + p e^{t})^{n}$
  - $X = \sum_{i = 1}^{n} X_{i}$ , $X_{i}$ being independent and $X_{i} \sim B e (p)$ $M (t) = E (e^{tX}) = E (e^{t (X_{1} + \dots + X_{n})}) = E (e^{t X_{1}} e^{t X_{2}} \dots e^{t X_{n}}) = E (e^{t X_{1}}) \dots E (e^{t X_{n}}) = [M_{X_{i}} (t)]^{n} = (1 - p + p e^{t})^{n}$
- $X \sim G eo m (p), M (t) = \frac{p e ^{t}}{1 - ( 1 - p ) e ^{t}}$
  - 👨‍🏫 exercise!
- $X \sim P o i sso n (λ), M (t) = exp (λ (e^{t} - 1))$ $M (t) = E (e^{t x}) = k = 0 \sum \infty e^{t k} P (X = k) = k = 0 \sum \infty e^{t k} \frac{e ^{- λ} λ ^{k}}{j !} = e^{- λ} k = 0 \sum \infty \frac{( e ^{t} λ ) ^{k}}{j !} = e^{- λ} e^{e^{t} λ} = e^{(e^{t} - 1) λ}$
- when $X \sim B in (n, p), M (t) = (1 - p + p e^{t})^{n}$
  - suppose $n \geq 4$
  - derive the 4th moment by computing 4th derivative
  - 👨‍🏫 tedious!
- when $X \sim U (α, β), M (t) = \frac{e ^{βt} - e ^{α} t}{( β - α ) t}$ $M (t) = E (e^{tX}) = \int_{- \infty}^{\infty} e^{t x} f (x) d x = \int_{β}^{α} e^{t x} \frac{1}{β - α} d x = \frac{1}{β - α} \int_{α}^{β} e^{t x} d x = \frac{1}{β - α} \frac{1}{t} e^{t x} ∣_{α}^{β} = \frac{e ^{βt} - e ^{α t}}{( β - α ) t}$
  - when $X \sim E x p (λ), M (t) = \frac{λ}{λ - t}$ for $t < λ$ $E [e^{tX}] = \int_{0}^{\infty} e^{t x} λ e^{- λ x} d x = λ \int_{0}^{\infty} e^{- (λ - t) x} d x = \frac{λ}{λ - t}$
    - $t < λ$ as otherwise, $E^{e^{tX}} < 0$ - somewhat impossible:
      - as $λ$ = frequency can't be negative!
  - when $X \sim N (μ, σ^{2}), M (t) = e x p (μ t + σ^{2} t^{2} /2)$
    - $X = σ Z + μ$ , where $Z \sim N (0, 1)$
    - compute $M_{Z} (t)$ first $M_{Z} (t) Z^{2} /2 - tZ \int_{- \infty}^{\infty} \frac{1}{2 π} e^{- (Z^{2} /2 - tZ)} d Z M_{X} (t) = E (e^{tZ}) = \int_{- \infty}^{\infty} e^{tZ} \frac{1}{2 π} e^{- (Z^{2} /2)} d Z = \int_{- \infty}^{\infty} \frac{1}{2 π} e^{- (Z^{2} /2 - tZ)} d Z = \frac{1}{2} (Z^{2} - 2 tZ + t^{2} - t^{2}) = \frac{1}{2} (Z - t)^{2} - \frac{1}{2} (t^{2}) = \int_{- \infty}^{\infty} \frac{1}{2 π} e^{- \frac{1}{2} (Z - t)^{2}} e^{\frac{t ^{2}}{2}} d Z = e^{\frac{t ^{2}}{2}} \int_{- \infty}^{\infty} \frac{1}{2 π} e^{- \frac{1}{2} (Z - t)^{2}} d Z = e^{\frac{t ^{2}}{2}} = E (e^{t (σ Z + μ)}) = E (e^{t σ Z + t μ}) = E_{Z} (e^{t σ Z} e^{t μ}) = e^{t μ} E_{Z} (e^{t σ Z}) = e^{t μ} M_{Z} (t σ) = e^{t μ} e^{\frac{t ^{2} σ ^{2}}{2}} = e^{t μ + \frac{t ^{2} σ ^{2}}{2}} simplify pdf of N (t, 1)$
  - suppose $M_{X} (t) = e^{3 (e^{(t - 1)})}$ , find $P (X = 0)$
    - given: Poisson distribution
    - $P (X = 0) = e^{- 3}$
  - sum of independent Binomial r.v. w/ the same success probability: binomial
    - $X \sum B in (n, p)$ , $Y \sim B in (m, p)$ , $X, Y$ being independent $E [e^{t (X + Y)}] = E [e^{tX}] E [e^{t Y}] = [1 - p + p e^{t}]^{n} [1 - p + p e^{t}]^{m} = [1 - p + p e^{t}]^{r + m}$
    - thus $X + Y \sim B in (n + m, p)$
  - similar for Poisson r.v.
  - sum of independent normal r.v. is normal
    - $X \sim N (μ_{1}, σ_{1}^{2})$
    - $Y \sim N (μ_{2}, σ_{2}^{2})$ $E [e^{t (X + Y)}] = E [e^{tX}] E [e^{t Y}] = exp (μ_{1} t + σ_{1}^{2} t^{2} /2) exp (μ_{2} t + σ_{2}^{2} t^{2} /2) = exp ((μ_{1} + μ_{2}) t + (σ_{1}^{2} + σ_{2}^{2}) t^{2} /2)$
  - find all moments of exponential dist. of parameter $λ > 0$
    - $X \sim E x p (λ)$ $E [e^{tX}] = \frac{λ}{λ - t} = \frac{1}{1 - t / λ} = k = 0 \sum \infty \frac{t ^{k}}{λ ^{k}} = k = 0 \sum \infty \frac{k ! / λ ^{k}}{k !} t^{k}$
    - thus: $E [X^{k}] = \frac{k !}{λ ^{k}}$
  - chi-squared r.v. w/ $n$ degrees of freedom: $χ_{n}^{2}$
    - given as: $\sum_{i = 1}^{n} Z_{1}^{2}$
      - $Z_{i}$ : independent std. normal r.v.
    - then $M (t)$ : $M (t) = E (e^{t (Z_{1}^{2} + Z_{2}^{2} + \dots Z_{n}^{2})}) = E (e^{t Z_{1}^{2} + t Z_{2}^{2} + \dots t Z_{n}^{2}}) = E (e^{t Z_{1}^{2}}) E (e^{t Z_{2}^{2}}) \dots E (e^{t Z_{n}^{2}}) = E (e^{t Z_{1}^{2}})^{n}$ $E (e^{t Z^{2}}) \frac{1}{σ ^{2}} σ^{2} σ E (e^{t Z^{2}}) M (t) = \int_{- \infty}^{\infty} e^{t Z^{2}} \frac{1}{2 π} e^{- Z^{2} /2} d Z = \int_{- \infty}^{\infty} \frac{1}{2 π} e^{- Z^{2} /2 (1 - 2 t)} d Z = \int_{- \infty}^{\infty} \frac{1}{2 π} e^{- \frac{Z ^{2}}{2 σ ^{2}}} d Z = σ \int_{- \infty}^{\infty} \frac{1}{2 π σ} e^{- \frac{Z ^{2}}{2 σ ^{2}}} d Z = σ = 1 - 2 t = \frac{1}{1 - 2 t} = \frac{1}{1 - 2 t} = \frac{1}{1 - 2 t} = (\frac{1}{1 - 2 t})^{n /2}$

Joint Moment Generating Functions

Joint MGFs
- for multiple variables:
  - $M (t_{1}, \dots, t_{n}) = E [e^{t_{1} X_{1} + \dots + t_{n} X_{n}}]$
- thus: for joint distribution of $X, Y$
  - exists joint cdf, pmf/pdf, and joint mgf
    - joint mgf: $M (t, s) = E (e^{t x + s Y})$
- individual mgf: can be obtained from joint mgf (like marginal)
  - $M_{X_{i}} (t) = E [e^{t X_{i}}] = M (0, \dots, 0, t, 0, \dots, 0)$
- 👨‍🏫 not required for the final exam
Multiplication rule
- $M (t_{1}, \dots, t_{n}) = M_{X_{1}} (t_{1}) \dots M_{X_{n}} (t_{n})$
  - iff each $X_{i}$ independent
- proof: too advanced for this course
  - but some generalization of the original multiplication rule $M (t_{1}, \dots, t_{n}) = E [e^{t_{1} X_{1} + \dots + t_{n} X_{n}}] = E [e^{t_{1} X_{1}} \dots e^{t_{n} X_{n}}] = E [e^{t_{1} X_{1}}] \dots E [e^{t_{n} X_{n}}] = M_{X_{1}} (t_{1}) \dots M_{X_{n}} (t_{n})$
Example
- let $X, Y$ be independent normal r.v. w/ $μ, σ^{2}$
  - as we know that $X + Y, X_{Y}$ are independent, we get: $E [e^{t (X + Y) + s (X - Y)}] = E [e^{(t + s) X + (t - s) Y}] = E [e^{(t + s) X}] E [e^{(t - s) Y}] = e^{μ (t + s) + σ^{2} (t + s)^{2} /2} e^{μ (t - s) + σ^{2} (t - s)^{2} /2} = e^{2 μ t + σ^{2} t^{2}} e^{σ^{2} s^{2}}$
    - if we considered the original preceding as a sum of normal r.v. with $2 μ, 2 σ^{2}$
      - and indep. normal r.v. w/ $0, 2 σ^{2}$
      - joint MGF: uniquely determines the joint distribution
      - thus: $X + Y$ and $X - Y$ are independent normal r.v.

Limit Theorem

Introduction
- 2 important concepts: laws of large numbers & central limit theorem
- LLN: concerned w/ stating conditions under the average of a seq. of r.v. converges to expected average
- CLT: concerned w/ determining conditions: where sum of large number of r.v. has probability distribution approximately normal
Chebyshev's inequality and weak law of large numbers
- inequalities below: for estimating tail probabilities w/ minimal information on r.v.
  - e.g. its mean / variance
- Markov's inequality: for non-negative r.v. $X$ and for $a > 0$ : $P (X \geq a) \leq \frac{E ( X )}{a}$
- proof: (~= first moment) $E (X) = \int_{- \infty}^{\infty} x f_{X} (x) d x = \int_{0}^{\infty} x f_{X} (x) d x \geq \int_{a}^{\infty} x f_{X} (x) d x \geq \int_{a}^{\infty} a f_{X} (x) d x = a P (X \geq a)$
- Chebyshev's inequality: for r.v. $X$ w/ finite mean $μ$ , variance $σ^{2}$ , then for $a > 0$ : $P (∣ X - μ ∣ \geq a) \leq \frac{σ ^{2}}{a ^{2}}$
- proof: (~= first moment) $P (∣ X - μ ∣ \geq a) = P (∣ X - μ ∣^{2} \geq a^{2}) \leq \frac{E ( ∣ X - μ ∣ ^{2} )}{a ^{2}} = \frac{var ( X )}{a ^{2}}$
- Markov's & Chebyshev's: deriving bounds on probabilities with very few information
  - mean / mean & variance
  - if the actual distribution is not known: it's a decent measure!
Example
- no. of items produced in factory: a r.v. w/ mean 50
  - what is: probability that production exceed 75?
    - $50/75 = \frac{2}{3}$
  - if variance of production is 25, what is the probability of production being within $[40, 60]$ ?
    - $25/1 0^{2} = \frac{2}{4}$
    - $1 - \frac{1}{4} = \frac{3}{4}$

Tutorial 7

Tutorial 8

Tutorial 9

Tutorial 10

MATH 2421: Tutorial 11

Date: 2024-11-26 18:03:39

Reviewed:

Topic / Chapter:

summary

❓Questions

Notes

Tutorial

Tutorial
- conditional expectation & variance
  - conditional expectation $E (X ∣ Y = y) = x \sum x p_{X ∣ Y} (x ∣ y)$ $E (k \sum X_{k} ∣ Y = y) = k \sum E (X_{k} ∣ Y = y)$
  - law of total expectation $E [E (X ∣ Y)] = E [X]$
    - 👨‍🏫 idea: separating randomness
      - $g (y) := E (X ∣ Y = y)$
      - then $E_{Y} [g (y)] = E [X]$
  - conditional variance $Var (X ∣ Y) = E ((X - E (X ∣ Y))^{2} ∣ Y)$ $Var (X) = E (Var (X ∣ Y)) + Var (E (X ∣ Y))$
- moment generating function
  - $M_{X} (t = E (e^{tX}))$
  - $E (X^{n}) = M_{X}^{(n)} (0) = \frac{d ^{n}}{d t ^{n}} M_{X} (t) ∣_{t = 0}$
  - if $X, Y$ independent: $M_{X + Y} (t) = M_{X} (t) M_{Y} (t)$
  - uniqueness: MGF determines distribution uniquely
    - i.e. if two r.v. have same MGF: then same distribution
- law of large number (LLN)
  - if $X_{1}, X_{2}, \dots, X_{n}$ : i.i.d. r.v. w/ $μ < \infty$ $\overset{ˉ}{X}_{n} ⟶ P / a . s . μ$
    - i.e. convergence in probability
    - $lim_{n \to \infty} P (∣ \overset{ˉ}{X}_{n} - μ ∣ > ε) \to 0$
    - a.s.: almost sure
    - convergence in probability
      - 👨‍🏫 probability of not converging = 0
    - 👨‍🏫 for stronger result, a stronger assumption is needed
  - can be computed without knowing the distribution of individual r.v.
    - $μ$ must exist, though
- central limit theorem (CLT)
  - if $X_{1}, X_{2}, \dots, X_{n}$ : i.i.d. r.v. w/ $μ, σ^{2} < \infty$ $\frac{( \sum X _{i} ) - n μ}{σ n} = \frac{X ˉ _{n} - μ}{σ / n} ⟶ d N (0, 1)$
  - i.e. convergence in distribution
  - strength of convergence: almost sure > in probability > in distribution
    - one on the left: implies one on the right
Examples
1. skipped
2. solution
  1. let $N = min {n \geq 1; X_{n} > X_{0}}$
    - show: $X_{n}$ w/ distribution $F + (1 - F) lo g (1 - F)$
    - $X_{i} < X_{0}$ for all $i \in [1, n - 1]$
    - idea: law of total probability, solving one by one
    - $P (X_{N} \leq x) = \sum P (X_{N} \leq x ∣ N = n) P (N = n)$ $P (N = n) = P (X_{0} < X_{n} \cap X_{1}, \dots, X_{n - 1} < X_{0}) = \frac{1}{n + 1} \cdot \frac{1}{n} = \frac{1}{n} - \frac{1}{n + 1}$
    - explanation: set $X_{n} = max, X_{0} = max {{X_{i}} </span> {X_{n}}}$
    - $P (X_{N} \leq x ∣ N = n) = [F (n)]^{n + 1}$ $P (X_{N} \leq x) [F (X) - 1] \sum \frac{[ F ( X ) ] ^{n}}{n} + F (n) = \sum [F (n)]^{n + 1} (\frac{1}{n} - \frac{1}{n + 1}) = F + (1 - F) lo g (1 - F)$
    - integrate both sides to show: one converges to $- lo g$
  2. consider: a very simple case for intuition
    - $n = 2$ : $P (X_{2} > X_{0} \leq X_{1}) = P (X_{2} > X_{1}) - P (X_{1} \geq X_{0} \geq X_{2})$ $P (M > m) = P (X_{m} > X_{0} \leq \dots) = \frac{1}{( n + 1 )!} = P (X_{2} > \dots) - P (\dots)$
3. $X \sim$ Poisson w/ $Λ \sim E x p (μ)$
  - show: $X \sim G eo m$
  - $L amb d a \sim E x p (μ)$
  - Poisson's MGF: $M_{X} (t) = E (e^{Λ (e^{t} - 1)}) = E (e^{t Λ}) = \frac{μ}{μ - ( e ^{t} - 1 )} t = e^{t} - 1 Λ \approx Exp (μ)$
  - $M_{Λ} (t) = \frac{μ}{μ - t}$

MATH 2421: Lecture 24

Date: 2024-11-27 11:59:13

Reviewed:

Topic / Chapter:

summary

❓Questions

Notes

Limit Theorem

Laws of large numbers
- weak law of large numbers: for iid r.v. $X_{1}, X_{2}, \dots$ w/ common mean $μ$
  - then, for any $ε > 0$ $n \to \infty lim P (\frac{X _{1} + \dots + X _{n}}{n} - μ \geq ε) ⟶ 0$
- proof: under additional assumption
  - assumption: r.v. w/ finite variance $σ^{2}$ , it's given that $E [\frac{X _{1} + \dots + X _{n}}{n}] = μ$ $var (\frac{X _{1} + \dots + X _{n}}{n}) = \frac{var ( X _{1} + \dots + X _{n} )}{n ^{2}} = \frac{n σ ^{2}}{n ^{2}} = \frac{σ ^{2}}{n}$
  - from Chebyshev's inequality: $P (\frac{X _{1} + \dots + X _{n}}{n} - μ \geq ε) \leq \frac{var ([ X _{1} + \dots + X _{n} ] / n )}{ε ^{2}} = \frac{σ ^{2}}{n ε ^{2}} ⟶ 0$
Applications
- casino games
  - in long run / repetition: casino's profit = expected value of gambling
  - e.g. $35 in$ 1/38 $c han ce, w i t h$ 1 cost per trial
- insurance
- polling: estimating opinions of a large population
- stock market: average return converge to the historical average
Central limit theorem
- 👨‍🏫 one of the most remarkable result
  - stating: sum of large no. of indep. r.v.: follows approximately normal dist.
    - not only a simple method for approximation of sum
    - but also explaining remarkable fact for empirical freq.
      - of natural population exhibiting bell-shape curves
- central limit theorem: w/ iid r.v. $X_{1}, X_{2}, \dots$ w/ $μ, σ^{2}$
  - distribution of: $\frac{X _{1} + \dots + X _{n} - n μ}{σ n}$
  - tends to standard normal as $n \to \infty$ : $n \to \infty lim P (\frac{X _{1} + \dots + X _{n} - n μ}{σ n} \leq x) = \frac{1}{2 π} \int_{- \infty}^{x} e^{- t^{2} /2} d t$
    - LHS: CDF, RHS: $\frac{1}{2 π} ϕ (x)$
  - thus: $P (a < \frac{X _{1} + \dots + X _{n} - n μ}{σ n} \leq b) \approx \frac{1}{2 π} \int_{a}^{b} e^{- t^{2} /2} d t$
  - examples
    - human heigh, IS, test scores, blood pressure, etc.
  - remarks: while the theorem itself is also important
    - discoveries made on the way to the theorem were also significant
  - $B in (n, p)$ : can be approximated by normal distribution as:
    - it's sum of iid Bernoulli r.v.
Examples
- w/ $X_{1}, i = 1, 2, \dots, 10$ independent r.v.
  - each w/ uniform distribution over $(0, 1)$
  - calculate: approximation to $P (X_{1} + \dots + X_{1} 0 > 6)$
  - $μ = E (X_{i}) = \frac{1}{2}$
  - $var (X_{i}) = \frac{1}{12} = σ^{2}$ $P (X_{1} + \dots + X_{1} 0 > 6) = P (\frac{X _{1} + \dots + X _{1} 0 - 5}{10/12} > \frac{6 - 5}{10/12}) \approx P (z > 1.095) = 0.1379$
- no. of students enrolling in a psychology class: Poisson r.v. w/ $μ = 100$
  - prof. in charge: will teach the course in 2 sessions, if 120 or more enrolls
  - what's the probability of teaching 2 sessions?
  - $X \sim P o i sso n (100)$
  - $P (X \geq 120) = \sum_{k = 120}^{\infty} e^{- 100} \frac{10 0 ^{k}}{k !}$
    - exact answer, but we might better approximate
  - $X = X_{1} + X_{2} + \dots + X_{100}$ w/ $X_{i}$ iid and $X_{i} \sim P o i sso n (1)$ $P (X \geq 120) = P (X_{1} + \dots + X_{100} \geq 120) = P (X_{1} + \dots + X_{100} \geq 119.5) = P (\frac{X _{1} + \dots + X _{100} - 100}{100} \geq \frac{119.5 - 100}{100}) \approx P (z \geq 1.96) cont. correct.$
- normal approximation: w/ $X_{1}, X_{2}, \dots, X_{20}$ of independent uniformly distributed r.v. over $(0, 1)$
  - estimate / upper bound: $P (k = 1 \sum 20 X_{k} > 15)$
  - use Markov's inequality to obtain an upper bound $P (k = 1 \sum 20 X_{k} > 15) \leq \frac{E ( \sum _{k = 1}^{20} X _{k} )}{15} = \frac{\sum _{k = 1}^{20} E ( X _{k} )}{15} = \frac{20 \times \frac{1}{2}}{15} = \frac{2}{3}$
  - use Chebyshev's inequality to obtain an upper bound $Y P (k = 1 \sum 20 X_{k} > 15) = k = 1 \sum 20 X_{k} = P (Y > 15) = P (Y - 10 > 5) \leq P (∣ Y - 10∣ > 5) \leq \frac{var ( Y )}{5 ^{2}} = \frac{var ( \sum _{k = 1}^{20} X _{k} )}{5 ^{2}} = \frac{\sum _{k = 1}^{20} var ( X _{k} )}{5 ^{2}} = \frac{20 \times \frac{1}{12}}{25} = \frac{1}{15}$
  - use Central Limit Theorem to approximate $P (k = 1 \sum 20 X_{k} > 15) = P (\frac{\sum _{k = 1}^{20} X _{k} - 10}{20/12} > P (\frac{15 - 10}{20/12})) \approx P (z > 15) = 0.00005$
    - although Markov and Chebyshev's upper bound is not very tight, it's widely used
Strong law of large numbers
- best-known result in probability theory
- strong law of large numbers
  - w/ $X_{1}, X_{2} \dots$ a seq. of iid r.v.
    - each w/ finite mean $μ = E (X_{i})$
  - then, w/ probability 1: $n \to \infty lim \frac{X _{1} + X _{2} + \dots + X _{n}}{n} \to μ$
  - alternatively: $P ({n \to \infty lim \frac{X _{1} + X _{2} + \dots + X _{n}}{n} \to μ}) = 1$
Example
- suppose: seq. of independent trials of some experiment
  - let $E$ : fixed event of experiment
    - let $P (E)$ : chance of $E$ occurring on any particular trial
  - let: $X_{i} {10 if E occurs on the i th trial otherwise$
  - w/ Strong law of large numbers, w/ $P = 1$ $\frac{X _{1} + X _{2} + \dots + X _{n}}{n} \to E (X) = P (E)$
  - $\frac{X _{1} + X _{2} + \dots + X _{n}}{n}$ : corresponds to proportion of time $E$ occurs in first $n$ trials
  - thus: interpret the result as:
  - w/ $P = 1$ , limiting proportion of times that $E$ occurs: $P (E)$

Final exam remarks

Remarks
- Dec. 9th, 12:30 - 2:30 (2 hr)
  - TST Sports Ctr Arena (Seafront)
- covers: Ch. 5-8 (from continuous r.v.)
  - yet, Ch. 1-4 might be implicitly included
    - e.g. famous discrete distributions from Ch. 4
      - Be, Bin, Geom, Poisson
- conditional variance: only concept is required
- join distribution of $> 2$ r.v. / joint MFG is not covered in final
  - 👨‍🏫 Beta, Gamma, Cauchy, etc. won't be major component
- exam format
  - each question: 20 pt,
  - w/ possible subproblems
  - 6 questions
- 1 A4 size 2 sided cheat sheet allowed
- normal table: provided
  - same as one in past papers
- calculator: anything without internet access
Some policies
- seating plan to be released
- bring your SID, calculator, cheat sheet, etc.

Ch. 7 Revision

Example
- mean of hypergeometric:
  - $n$ balls chosen randomly from an urn w/ $N$ balls, where $m$ are white
  - find: expected number of white balls selected
  - find: variance of white balls selected $var (X) = var (X_{1} + \dots + X_{m}) = i = 1 \sum m var (X_{i}) + i \neq = j \sum cov (X_{i}, X_{j})$
  - $X_{i} \sim B er (n / N)$
  - $var (X) = p (1 - p) = \frac{n}{N} (1 - \frac{n}{N}) = \frac{n ( N - n )}{N ^{2}}$
  - for $i \neq = j$ $cov (X_{i}, X_{j}) E (X_{i} X_{k}) cov (X_{i}, X_{j}) = E (X_{i} X_{k}) - E (X_{i}) \cdot E (X_{j}) = E (X_{i} X_{k}) - \frac{n ^{2}}{N ^{2}} = P (X_{i} = 1, X_{j} = 1) = \frac{( n - 1 N - 1 )}{( n N )} = \frac{n !}{( n - 2 )!} \frac{( N - 2 )!}{N !} = \frac{n ( n - 1 )}{N ( N - 1 )} = \frac{n ( n - 1 )}{N ( N - 1 )} - \frac{n ^{2}}{N ^{2}}$ $var (X) = m \cdot \frac{n ( N - n )}{N ^{2}} + m (m - 1) [\frac{n ( n - 1 )}{N ( N - 1 )} - \frac{n ^{2}}{N ^{2}}] = \frac{mn ( N - n ) ( N - m )}{N ^{2} ( N - 1 )}$