• 👨‍🏫 They (use of $\infty$ or $n$) are equivalent and it's matter of personal preference.
• We have used infinity version to follow the textbook

Four basic operations

• result of operation: also a collection of outcome = event
• union: $A\cup B$
  • either event $A$ or event $B$ occurs
• intersection: $A\cap B$
  • both event $A$ and event B occur
  • short: $AB$
• complement: $A^c$
  • even A does NOT occur
• symmetric difference: $A\Delta B=(A\cup B$\ $AB)$
  • EITHER $A$ or $B$ occurs, but not both

Additional terms

• inclusion of events
  • $\forall e\in E,e\in F⟹E\subset F$ or $F\supset E$
  • i.e. occurrence of $E$ implies $F$
• disjoint
  • when two events share no common outcomes
  • $A\cap B=\varnothing$
• mutually exclusive
  • when events $A_1,A_2,\dots ,A_k$ are pairwise disjoint
  • $\forall i,j\in [1,k]\wedge i\ne j,A_i\cap A_j=\varnothing$
• exhaustive
  • when union of some events $A_1,A_2,\dots ,A_k$ is the sample space
  • ${\bigcup }_{i=1}^k{A}_i=S$
• partition
  • when events $A_1,A_2,\dots ,A_k$ are both exclusive and exhaustive
  • ${\bigcup }_{i=1}^k{A}_i=S$ and $\forall i,j\le N\wedge i\ne j,A_i\cap A_j=\varnothing$

Fundamental laws

• commutative law
  • $A\cap B=B\cap A$
  • $A\cup B=B\cup A$
• associative law
  • $A\cap (B\cap C)=(A\cap B)\cap C$
  • $A\cup (B\cup C)=(A\cup B)\cup C$
• distributive law
  • $A\cap (B\cup C)=(A\cap B)\cup (A\cap C)$
  • $A\cup (B\cap C)=(A\cup B)\cap (A\cup C)$
• DeMorgan's law
  • $({\bigcup }_{i=1}^k{A}_i{)}^c={\bigcap }_{i=1}^k{A}_i^c$
  • $({\bigcap }_{i=1}^k{A}_i{)}^c={\bigcup }_{i=1}^k{A}_i^c$

Lemma

• for any events $A,B$: $A=(A\cap B)\cup (A\cap B^c)$
  • proof: as above is $=A\cap (B\cup B^c)=A$
    • following distributive law

Probability definition

• probability: function that assigns numbers to events
  • numbers: characterize how likely this event occurs
• primitive definition
  • assuming an experiment w/ sample space $S$ can be repeated
  • for each even $E\in S$, $n(E)$: number of times $E$ occured in the first $n$ repetitions
  • probability is defined by:
  • $P(E)={\lim }_{n\to \infty }\frac{n(E)}{n}$
  • l.e. limiting proportion of time that $E$ occurs
• problems
  • how do we know the limit of $\frac{n(E)}{n}$ exists or not, for a seq. of repetitions?
  • even if limits exist for all sequences, how do we know that the limits are the same = i.e. converge?
• Kolmogorov Axiom (modern probability definition)
  • by Andrey Kolmogorov in 1933
  • probability, denoted by $P$ is a function on the collection of events satisfying
  1. for any event $A$: $$0\le P(A)\le 1$$
  2. let $S$ be the sample space, then: $$P(S)=1$$
  3. for any sequence of mutually exclusive events $A_1,A_2,\dots$ $$P(\bigcup _{i=1}^{\infty }{A}_i)=\sum _{i=1}^{\infty }P(A_i)$$
     • also written as, for all $N=1,2,\dots$ $$P(\bigcup _{i=1}^N{A}_i)=\sum _{i=1}^{N}P(A_i)$$
• event $A\stackrel{P(\cdot )}{⟶}[0,1]$

Example

• on a fair six-faced dice, when $A=\{1,3\}$, $B=\{2,4\}$
  • by Kolmogorov's axiom, what are $P(A)$ and $P(B)$?
  • $\forall i\in [1,6],P(i)=1/6$
  • as each faces are mutually exclusive (cannot have two or more faces at once)
  • $P(A)=P(1\cup 3)=P(1)+P(3)=\frac{1}{3}$
  • $P(A)=P(2\cup 4)=P(2)+P(4)=\frac{1}{3}$

Theorem

• $P(\varnothing )=0$
  • proof: take $A_1=S$, $A_2=A_3=\cdots =\varnothing$
  • $A_1,A_2,\dots$ are mutually exclusive
  • by axiom 3: $$\begin{array}{rl}P(S)& =P(\bigcup _{i=1}^{\infty }{A}_i)=P(A_1)+P(A_2)+\dots \\ & =P(S)+P(A_2)+\dots \\ 0& =P(A_2)+\dots \\ P(\varnothing )& =0\end{array}$$
• for any finite sequence of mutually exclusive events $A_1,A_2,\dots ,A_n$
  • $P({\bigcup }_{i=1}^n{A}_i)={\sum }_{i=1}^{b}P(A_i)$
  • proof: let $\forall m>n,A_m=\varnothing$
  • $A_1,A_2,\dots$: still mutually exclusive
  • apply axiom 3 to find: $$\begin{array}{rl}P(\bigcup _{i=1}^{\infty }{A}_i)& =\sum _{i=1}^{\infty }P(A_i)\\ & =\sum _{i=1}^{n}P(A_i)\text{ // as }\forall m\ge n,P(m)=P(\varnothing )=0\end{array}$$
• let $A$ be an event, then $P(A^c)=1-P(A)$
  • proof: $S=A\cup A^c$
  • $A\cap A^c=\varnothing$
  • $P(S)=P(A\cup A^c)=P(A)+P(A^c)$
  • $⟹P(A^c)=P(S)-P(A)=1-P(A)$
• if $E\in F$, then
  • $P(E)\le P(F)$
  • proof: $F=FE\cup F{E}^c\stackrel{E\in F}{=}E\cup F{E}^c$
  • since $E\cap F{E}^c=\varnothing$
  • by Axiom 3, $P(F)=P(E\cup F{E}^c)=P(E)+\underset{\ge 0}{\underbrace{P(F{E}^c)}}$
  • $⟹P(F)\ge P(E)$
• let $A,B$ be any two events, then
  • $P(A\cup B)=P(A)+P(B)-P(AB)$
  • proof by diagram
    • $P(A\cup B)=\{I,II,III\}$
    • $P(A)=P(I,II)$
    • $P(B)=P(II,III)$
    • $P(A\cap B)=P(II)$
    • $P(A\cup B)=P(A)+P(B)-P(A\cap B)$
  • rigorous way
    • $A\cup B=A\cup (B{A}^c)$
    • $A,B{A}^c$: disjoint
    • using axiom 3: $P(A\cup B)=P(A\cup (B{A}^c))=P(A)+P(B{A}^c)$
    • since $B=BA\cup B{A}^c$, $BA,B{A}^c$: disjoint
    • using axiom 3: $P(B)=P(BA)+P(B{A}^c)=P(AB)+P(B{A}^c)$
      • $⟹P(A{B}^c)=P(B)-P(AB)$
    • $\therefore (A\cup B)=P(A)+P(B)-P(AB)$
• Inclusion-Exclusion principle
  • let $A_1,A_2,\dots ,A_n$ be any events, then: $$\begin{array}{rl}P(A_1\cup A_2\cup \cdots \cup A_n)& =\sum _{i=1}^{n}P(A_i)-\sum _{i\le i_1<i_2\le n}P(A_{i_1}{A}_{i_2})+\dots \\ & +(-1{)}^{r+1}\sum _{i\le i_1<\cdots <i_r\le n}P(A_{i_1}\cdots A_{i_r})\\ & +\cdots +(-1{)}^{n+1}P(A_1\cdots A_n)\end{array}$$
    • where ${\sum }_{i\le i_1<i_2\le n}P(A_{i_1}{A}_{i_2})$ means sum of every possible combination of $P(A_{i_1}{A}_{i_2})$
    • 👨‍🏫 won't be used for $n\ge 4$

Example

• let $A,B$ two events s.t. $P(B)=\frac{5}{8}$f
  • and $P(A\cap B)=\frac{1}{2}$. find $P(B\cap A^c)$
  • $B=BA\cup B{A}^c$
  • $P(B)=P(BA)+P(B{A}^c)$
  • $P(B{A}^c)=P(B)-P(BA)=\frac{5}{8}-\frac{1}{2}=\frac{1}{8}$
• J is taking two books for vacation.
  • $A$: reading first book; $P(A)=0.5$
  • $B$: reading second book; $P(B)=0.4$
  • $P(AB)=0.3$
  • what is $P(A^c{B}^c)=P((A\cup B{)}^c)$
    • $P((A\cup B{)}^c)=1-P(A\cup B)$
    • $P(A\cup B)=P(A)+P(B)-P(AB)=0.5+0.4-0.3=0.6$
    • $P((A\cup B{)}^c)=1-0.6=0.4$
• formula for $P(A\cup B\cup C)$ $$\begin{array}{rl}P(A\cup B)& =P(A)+P(B)-P(AB)\\ P(A\cup B\cup C)& =P(A\cup B)+P(C)-P((A\cup B)\cap C)\\ & =P(A)+P(B)-P(AB)+P(C)-P((A\cup B)\cap C)\\ & =P(A)+P(B)-P(AB)+P(C)-P(AC\cup BC)\\ & =P(A)+P(B)-P(AB)+P(C)-(P(AC)+P(BC)-P(AC\cap BC))\\ & =P(A)+P(B)-P(AB)+P(C)-P(AC)-P(BC)+P(AC\cap BC)\\ & =P(A)+P(B)-P(AB)+P(C)-P(AC)-P(BC)+P(ABC)\\ & =P(A)+P(B)+P(C)-P(AB)-P(AC)-P(BC)+P(ABC)\end{array}$$
  • or, consult diagram for easy understanding