• 👨‍🏫 only if $A\in \{\varnothing ,S\}$

Independence (cont.)

• three events: independent iff:
  • $P(ABC)=P(A)P(B)P(C)$
  • $P(AB)=P(A)P(B)$
  • $P(BC)=P(B)P(C)$
  • $P(AC)=P(A)P(C)$
  • last three: pairwise independent
• if $A,B,C$ independent => $A$ independent of any objects formed of $B,C$
  • $P(A(B\cup C))=P(A)P(B\cup C)$ as: $$\begin{array}{rl}P(A(B\cup C))& =P(AB\cup AC)\\ & =P(AB)+P(AC)-P(AB\cap AC)\\ & =P(A)P(B)+P(A)P(C)-P(ABC)\\ & =P(A)P(B)+P(A)P(C)-P(A)P(BC)\\ & =P(A)(P(B)+P(C)-P(BC))\\ & =P(A)(P(B\cup C))\end{array}$$
  • $P(A(BC))=P(A)P(BC)$ as: $$\begin{array}{rl}P(A(BC))& =P(ABC)\\ & =P(A)P(B)P(C)\\ & =P(A)P(BC)\end{array}$$
• events $A_1,A_2,\dots ,A_n$ are independent, if:
  • $\forall$ sub-collection of $A_{i_1},\dots ,A_{i_r}$
    • $P(A_{i_1}\dots A_{i_r})=P(A_{i_1})\dots P(A_{i_r})$
      • for $r\in [1,n]$
  • for $n=4$, verify:
    • $r=4$; $\left(\genfrac{}{}{0ex}{}{4}{4}\right)=1$ condition
      • $P(A_1{A}_2{A}_3{A}_4)=P(A_1)P(A_2)P(A_3)P(A_4)$
    • $r=3$; $\left(\genfrac{}{}{0ex}{}{4}{3}\right)=4$ conditions
      • $P(A_1{A}_2{A}_3)=P(A_1)P(A_2)P(A_3)$
      • and similarly so one
    • $r=2$; $\left(\genfrac{}{}{0ex}{}{4}{2}\right)=6$ conditions
      • // pairwise independent

Examples

• two fair dice: thrown
  • $A$: sum of dice = 7
    • $P(A)=1/6$
  • $B$: first dice = 4
    • $P(B)=1/6$
  • $C$: second dice = 3
    • $P(C)=1/6$
  • $P(AB)=P(A)P(B)$
  • $P(AC)=P(A)P(C)$
  • yet $P(ABC)=1/36\ne 1/216=1/6\cdot 1/36=P(A)P(BC)$
    • $A$ dependent on $BC$
• loaded coin w/ $P(H)=p$
  • loaded coin: tossed $n$ times
  • probability of getting at least 1 head in these tosses?
    • $P(\bigcup A_i)=P()$
    • $$\begin{gathered} P(\bigcup A_i)=P((\bigcap A_i^c{)}^c)=1-P(\bigcap A_i^c) \\ \stackrel{\text{independence}}{=}1-(1-p{)}^n \end{gathered}$$
  • probability of getting exactly $k$ head in these tosses?
    • if first $k$ tosses: give heads
      • remaining $(n-k)$ tosses: all tail
    • event: $A_1{A}_2\cdots A_k{A}_k^c\cdots A_n^c$
      • $P(A_1{A}_2\cdots A_k{A}_k^c\cdots A_n^c)=p^k(1-p{)}^{n-k}$
      • and there are $\left(\genfrac{}{}{0ex}{}{n}{k}\right)$ ways to choose it
    • thus final solution: $\left(\genfrac{}{}{0ex}{}{n}{k}\right)p^k(1-p{)}^{n-k}$
• system: of $n$ independent components
  • is parallel if it functions when at least one component functions
  • if component $i$ functions in $p_i$
  • then what is the probability that the system functions? $$\begin{array}{r}P(B_i)=p_1;P(B)=1-P(B^c)=1-P((\bigcup B_i{)}^c)\\ =1-P(\bigcap B_i^c)=1-P(\prod _{1\le i<n}(1-p_i))\end{array}$$

Introduction

• Need for random variable
  • often: we are interested in some function / expression of outcome
  • rather than outcome itself
  • e.g. roll a pair of dice; what is the distribution of two faces' sum?
    • i.e. not interested in face of each die
    • function: $X:S\to \mathbb{R}$
    • $(i,j)\mapsto i+j$
    • range: $[2,12]$
• 20 questions in a MCQ. Each question w/ 5 alternatives
  • student: answers all 20 questions randomly
  • independently choose one alternative in each questions
  • interested in: no. of correct answers

Random variables

• random variable: mapping from sample space to real numbers
  • rv: random variables
  • denoted by capitals, $X,Y,Z$
• discrete random variable
  • rv w/ finite || countable range
  • e.g. number of defective items
• continuous random variable
  • range: an interval over the real line
  • weight of an item, time taken, etc.

Examples

• urn: w/ 20 chips w/ number $[1,20]$
  • three chips: chosen at random
  • $X$: largest among the three chips
  • then, what's the range of $X$?
    • $[3,20]$ (discrete)
• toss 3 coins. $Y$: no. of heads appearing
  • $Y\in [0,1,2,3]$ (discrete)
  • $P(Y=0)=\frac{1}{8}$
  • $P(Y=1)=\frac{3}{8}$
  • $P(Y=2)=\frac{3}{8}$
  • $P(Y=3)=\frac{1}{8}$
  • $\{Y=0\},\{Y=1\},\{Y=2\},\{Y=3\}$
    • partition of $S$
• from above urn, compute the probability of $X\ge 17$
  • $P(X=3)=1/\left(\genfrac{}{}{0ex}{}{20}{3}\right)$
  • $P(X=6)=1\left(\genfrac{}{}{0ex}{}{5}{2}\right)/\left(\genfrac{}{}{0ex}{}{20}{3}\right)$
    • choose $6$, and 2 other that is smaller that $6$
    • i.e. within $[1,5]$
  • $P(X=k)=\left(\genfrac{}{}{0ex}{}{k-1}{2}\right)/\left(\genfrac{}{}{0ex}{}{20}{3}\right)$ $$\begin{array}{rl}P(X\ge 17)& =P(X=17)+P(X=18)+P(X=19)+P(X=20)\\ & =((\genfrac{}{}{0ex}{}{16}{2})+(\genfrac{}{}{0ex}{}{17}{2})+(\genfrac{}{}{0ex}{}{18}{2})+(\genfrac{}{}{0ex}{}{19}{2}))/\left(\genfrac{}{}{0ex}{}{20}{3}\right)\\ & =0.5087\end{array}$$
• three balls: randomly chosen from urn w/ 3W, 3R, 5B balls
  • we win $1perWballchosen;lose$1 per R ball chosen.
  • $X$: total winnings of the experiment
  • compute respective probability for all domain of $X$
    • $P(X=0)=P(bbb,wrb)=\frac{\left(\genfrac{}{}{0ex}{}{5}{3}\right)+\left(\genfrac{}{}{0ex}{}{3}{1}\right)\left(\genfrac{}{}{0ex}{}{3}{1}\right)\left(\genfrac{}{}{0ex}{}{5}{1}\right)}{\left(\genfrac{}{}{0ex}{}{11}{3}\right)}=\frac{55}{165}$
    • $P(X=1)=P(wbb,wwr)=\frac{\left(\genfrac{}{}{0ex}{}{3}{1}\right)\left(\genfrac{}{}{0ex}{}{5}{2}\right)+\left(\genfrac{}{}{0ex}{}{3}{2}\right)\left(\genfrac{}{}{0ex}{}{3}{1}\right)}{\left(\genfrac{}{}{0ex}{}{11}{3}\right)}=\frac{39}{165}$
    • $P(X=2)=P(wwb)=\frac{\left(\genfrac{}{}{0ex}{}{3}{2}\right)\left(\genfrac{}{}{0ex}{}{5}{1}\right)}{\left(\genfrac{}{}{0ex}{}{11}{3}\right)}=\frac{15}{165}$
    • $P(X=3)=P(www)=\frac{\left(\genfrac{}{}{0ex}{}{3}{3}\right)}{\left(\genfrac{}{}{0ex}{}{11}{3}\right)}=\frac{1}{165}$
  • and $P(X=k)=P(X=-k)$
  • probability of winning money
    • $P(X>0)=P(X=1)+P(X=2)+P(X=3)=\frac{39}{165}+\frac{15}{165}+\frac{1}{165}=1/3$

Discrete random variables

• rv: discrete if range of $X$: either finite / countably infinite
  • using: capital $X,Y,Z$ for variable
  • and $x,y,z$ for value
• suppose rv $X$ is discrete, taking value of $x_i$
  • probability mass function of $X$, $p_X$ is $$p_X=\{\begin{array}{llc}P(X=x)& & \text{if }x=x_1,x_2,\dots \\ 0& & \text{otherwise}\end{array}$$
    • for PMF: range must be well-determined
    • PMF: fully characterizes a rv
    • can be notated as $p$
• properties of PMF
  • $p_X(x_i)\ge 0$ for $i,1,2,\dots$
  • $p_X(x)=0$ for other values of $x$
    • domain of pMF itself: entire real line
  • ⭐ since $X$ must take on one of the values of $x_i$
    • ${\sum }_{i=1}^{\infty }{p}_X(x_i)=1$

Examples

• rv $X$: only takes $0,1,2,\dots$ value if $p_X$ is of the form
  • $p(k)=c\frac{{\lambda }^k}{k!},k=0,1,\dots$
  • where $\lambda >0$: fixed positive value; $c$: suitably chosen constant
    • $c$: also known as a normalizing constant
  • a) what is this suitable constant
    • as ${\sum }_{i=1}^{\infty }{p}_X(x_i)=1$
    • $c{\sum }_{k=0}^{\infty }\frac{{\lambda }^k}{k!}=c{e}^{\lambda }=1$
      • Taylor series
      • $c=e^{-\lambda }$
      • 👨‍🏫 sometimes, $c$ is almost impossible to calculate
  • b) compute $P(X=0)$
    • $P(X=0)=p(0)=e^{-\lambda }\frac{{\lambda }^0}{0!}=e^{-\lambda }$
  • c) compute $P(X>2)$
    • $P(X>2)=1-P(X\le 2)=1-p(0)-p(1)-p(2)$
    • $P(X>2)=1-e^{-\lambda }-e^{-\lambda }\lambda -e^{-\lambda }{\lambda }^2/2$