MATH 2421: Lecture 6

Date: 2024-09-23 11:41:56

Reviewed:

Topic / Chapter: Bayes theorem & independence

summary

❓Questions

Notes

Bayes' Theorem

Bayes' theorem
- given $A_{1}, A_{2}, \dots, A_{n}$ partitions $S$
  - and $P (A_{i}) > 0$ for all
  - and $P (B)$ any event, then: $P (A_{i} ∣ B) = \frac{P ( B ∣ A _{i} ) P ( A _{i} )}{P ( B ∣ A _{1} ) P ( A _{1} ) + \dots + P ( B ∣ A _{b} ) P ( A _{n} )}$
- proof
  - $P (A_{i} ∣ B) = \frac{P ( B A _{i} )}{P ( b )} = \frac{P ( B ∣ A _{i} ) P ( A _{i} )}{P ( B )}$
  - then: law of total probability
- terminology
  - $P (A_{i})$ prior probability
  - $B$ : evidence
  - $P (A_{i} ∣ B)$ posterior probability
Examples
- I saw a mule!
  - but it is surely mistaken, as there are only horse and donkey in the village.
  - there are 30% horses and 70% donkeys
    - prior belief: there is 30% chance to encounter a donkey
  - if: horse is mistaken as a mule w/ 0.4
    - and donkey is mistaken w/ 0.8
  - then $P (H ∣ saw mule) = \frac{0.4 \cdot 0.3}{0.4 \cdot 0.3 + 0.8 \cdot 0.7} = 0.176$
  - then $P (D ∣ saw mule) = \frac{0.8 \cdot 0.7}{0.4 \cdot 0.3 + 0.8 \cdot 0.7} = 0.823$
- Covid tests
  - PCR test vs. RAT
  - different cases:
    - $TP$ : sick people correctly identified as sick
    - $FP$ : healthy people incorrectly identified as sick
    - $TN$ : healthy people correctly identified as healthy
    - $FN$ : sick people incorrectly identified as healthy
  - terminology
    - sensitivity: $\frac{TP}{P} = \frac{TP}{TP + FN}$
      - i.e. probability of a positive result given patient: is sick
    - specificity: $\frac{TN}{N} = \frac{TN}{TN + FP}$
      - i.e. probability of a negative result given patient: is healthy
    - $1 - specificity$ : prob. of $FP$
    - prevalence:
  - prevalence of covid: 1%
    - test: w/ sensitivity 95%
      - specificity: 99%
    - test result: positive
    - probability that you have covid?
    - $COV := {have the covid}$
    - $+ := {test result: positive}$ $P (CO V ∣ +) = \frac{P ( + ∣ CO V ) P ( COV )}{P ( + )} = \frac{P ( + ∣ CO V ) P ( COV )}{P ( + ∣ COV ) P ( COV ) + P ( + ∣ COV ^{c} ) P ( COV ^{c} )} = \frac{0.95 \cdot 0.01}{0.95 \cdot 0.01 + ( 1 - 0.99 ) \cdot 0.99} = \frac{0.95 \cdot 0.01}{0.95 \cdot 0.01 + 0.01 \cdot 0.99} \approx 48.97%$
- class exercise
  - if $P (COV) =$ 5% or 8%, how does $P (COV ∣ +)$ change? $P (CO V ∣ +) = \frac{P ( + ∣ CO V ) P ( COV )}{P ( + ∣ COV ) P ( COV ) + P ( + ∣ COV ^{c} ) P ( COV ^{c} )} = \frac{0.95 \cdot 0.05}{0.95 \cdot 0.05 + ( 1 - 0.99 ) \cdot 0.95} = \frac{0.95 \cdot 0.05}{0.95 \cdot 0.05 + 0.01 \cdot 0.95} = 83.33%$ $P (CO V ∣ +) = \frac{P ( + ∣ CO V ) P ( COV )}{P ( + ∣ COV ) P ( COV ) + P ( + ∣ COV ^{c} ) P ( COV ^{c} )} = \frac{0.95 \cdot 0.08}{0.95 \cdot 0.08 + ( 1 - 0.99 ) \cdot 0.92} = \frac{0.95 \cdot 0.08}{0.95 \cdot 0.08 + 0.01 \cdot 0.92} = 89.2%$
  - how about $P (COV ∣ -)$ ? $P (CO V ∣ -) = \frac{P ( - ∣ CO V ) P ( COV )}{P ( - ∣ COV ) P ( COV ) + P ( - ∣ COV ^{c} ) P ( COV ^{c} )} = \frac{0.05 \cdot 0.01}{0.05 \cdot 0.01 + 0.99 \cdot 0.99} \approx 0.05%$
- class exercise 2
  - $+ + := {test result: positive twice in a row}$ $P (CO V ∣ + +) = \frac{P ( + + ∣ CO V ) P ( COV )}{P ( + + ∣ COV ) P ( COV ) + P ( + + ∣ COV ^{c} ) P ( COV ^{c} )} = \frac{0.9 5 ^{2} \cdot 0.01}{0.9 5 ^{2} \cdot 0.01 + ( 1 - 0.99 ) ^{2} \cdot 0.99} = \frac{0.9 5 ^{2} \cdot 0.01}{0.9 5 ^{2} \cdot 0.01 + 0.0 1 ^{2} \cdot 0.99} \approx 98.91$
    - 👨‍🏫 sensitivity & specificity matter much, esp. it's multiplied twice
- blood type
  - 95% effective in detecting a disease when it is present
  - false positive: 1% for healthy person
  - prevalence: 0.5%
  - $P (DIS ∣ +) = ?$ $P (DIS ∣ +) = \frac{P ( + ∣ DIS ) P ( DIS )}{P ( + ∣ DIS ) P ( DIS ) + P ( + ∣ DIS ^{c} ) P ( DIS ^{c} )} = \frac{0.95 \cdot 0.005}{0.95 \cdot 0.005 + 0.01 \cdot 0.995}$
- plane: missing
  - equally likely: in 3 possible regions
    - $R_{i}$ : plane is in region $i$ = $1/3$
  - $1 - β_{i}$ : probability that plane: found upon search in $i$ -th region
    - when plane is indeed there
  - $E$ : search in region 1: unsuccessful
    - = plane not found in region 1 upon search
  - $P (R_{i} ∣ E)$ for $i = 1, 2, 3$
    - $P (R_{i} ∣ E) = \frac{P ( E ∣ R _{i} ) P ( R _{i} )}{P ( E )}$
      - for each $i$
    - $1 - β_{1} = P (E^{c} ∣ R_{1}) ⟹ P (E ∣ R_{1}) = 1 - P (E^{c} ∣ R_{1}) = 1 - (1 - β_{1}) = β$
    - $P (E ∣ R_{2}) = P (E ∣ R_{3}) = 1$
    - $P (R_{i} ∣ E) = \frac{P ( E ∣ R _{i} ) P ( R _{i} )}{P ( E )} = \frac{P ( E ∣ R _{i} ) \frac{1}{3}}{\frac{1}{3} β _{1} + \frac{1}{3} \cdot 1 + \frac{1}{3} \cdot 1} = \frac{P ( E ∣ R _{i} )}{β _{1} + 2}$
    - $P (R_{1} ∣ E) = \frac{P ( E ∣ R _{1} )}{β _{1} + 2} = \frac{β}{β _{1} + 2} < \frac{1}{3}$
      - 👨‍🏫 less chance of unsuccessful search, if the plane is actually there!
    - $P (R_{2} ∣ E) = \frac{P ( E ∣ R _{2} )}{β _{1} + 2} = \frac{1}{β _{1} + 2} > \frac{1}{3}$
    - $P (R_{3} ∣ E) = \frac{P ( E ∣ R _{3} )}{β _{1} + 2} = \frac{1}{β _{1} + 2} \frac{1}{3}$
- assembly plant w/ 3 machine types
  - each is of proportion: 30% / 45% / 25%
  - and defective rate: 2% / 3% / 2%
  - given a chosen machine is defective, what is the probability of it being type 1? $P (I ∣ D) = \frac{P ( D ∣ I ) P ( I )}{P ( D ∣ I ) P ( I ) + P ( D ∣ II ) P ( II ) + P ( D ∣ III ) P ( III )} = \frac{0.02 \cdot 0.3}{0.02 \cdot 0.3 + 0.03 \cdot 0.45 + 0.02 \cdot 0.25} = \frac{0.006}{0.0245} = 24.49%$

Independence

Independence
- two events $A, B$ : independent if
  - ⭐ $P (A B) = P (A) P (B)$
  - and dependent if $P (A B) \neq = P (A) P (B)$
- motivation: if $P (B) > 0$ , then
  - $P (A ∣ B) = \frac{P ( A B )}{P ( B )} = \frac{P ( A ) P ( B )}{P ( B )} = P (A)$
  - i.e. $A$ : independent of $B$ if
    - knowledge that $B$ has occurred: does NOT change the probability that $A$ occurs
- if $A, B$ are independent, then so are:
  - $A, B^{c}$ $P (A) P (A B^{c}) = P (A B \cup A B^{c}) = P (A B) + P (A B^{c}) = P (A) - P (A B) = P (A) - P (A) P (B) = P (A) [1 - P (B)] = P (A) P (B^{c})$
  - $A^{c}, B$
  - $A^{c}, B^{c}$
  - 👨‍🏫 and similarly for others
Examples
- card: selected at random from ordinary deck of $52$ cards
  - if $E$ : event that selected card = Ace
  - $F$ : is is spade
  - then $E, F$ : independent
  - $P (EF) = 1/52; P (E) = 4/52; P (F) = 13/52; P (EF) = P (E) P (F)$
    - thus they are independent
  - 👨‍🏫 they are not disjoint (i.e. $E \cap F \neq = \emptyset$ ), but they are independent
- two coins: flipped
  - all 4 outcomes: equally likely
  - $E$ : first coin lands head
  - $F$ : first coin lands tail
  - $P (EF) = 1/4; P (E) = 2/2; P (F) = 2/4; P (EF) = P (E) P (F)$
- another two fair coins
  - $E$ : first coin lands head
  - $F$ : only one coin lands head
  - $P (EF) = 1/4; P (E) = 2/4; P (F) = 2/4; P (EF) = P (E) P (F)$
- 2 fair dice
  - $E_{1}$ : sum of dice = 6
  - $F$ : first die = 4
  - are $E_{1}, F$ independent?
  - $P (E_{1} F) = 1/36; P (E_{1}) P (F) = \frac{5}{36} \frac{1}{6}; P (E_{1} F) \neq = P (E_{1}) P (F)$
- 2 fair dice
  - $E_{1}$ : sum of dice = 7
  - $F$ : first die = 4
  - are $E_{1}, F$ independent?
  - $P (E_{1} F) = 1/36; P (E_{1}) P (F) = \frac{6}{36} \frac{1}{6}; P (E_{1} F) = P (E_{1}) P (F)$

MATH 2421: Probability