Bài giảng Probability & Statistics - Lecture 4: Basic probability - Bùi Dương Hải

Lecture 4. BASIC PROBABILITY  Probability  Outcome – Event  Complement Event, Intersection Event, Union Event  Mutually Exclusive, Independent, Collectively Exhausive, Partitions  Bernoulli Formula  Total Probability, Bayes’ Theorem  [1] Chapter 4, pp. 169 - 212 PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 1 “Problem of Points”  2 players A and B, contributed 50 Franc each.  The game is winner – loser only (no draw), A and B are equally-likely to win in each match.  Game rule: play 9 matches, the one wins more is final winner and takes all of 100F  But the game had stopped after 7 matches, and scores at that time of A and B are 4 and 3, respectively.  How could they distribute the money? PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 2 4.1. Probability  Probability is quantitative measure of uncertainty, the chance that an uncertain event will occur.  Probability: Subjective and Objective  Probability of event A is denoted by P(A)  0 ≤ P(event) ≤ 1  P(always) = 1  P(impossible) = 0  () > (): A is more possible to occur than B PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 3 Experiment – Outcome - Event Experiment Outcome Event Flip a coin Head, tail ‘Head’, ‘tail’ Toss a die 1,2,3,4,5,6 dot(s) ‘greater than 3’ Do an exam Score = 0, 1, 2,, 10 ‘pass’; ‘excellent’ Invest in a project Profit: (+), (–), zero ‘Non negative’ ‘profitable’ Apply for a job Pass, fail Do a job Salary = PROBABILITY & STATISTICS– Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 4 Classical Probability  Sample space: all basic outcomes, denoted by Ω  Assumes all basic outcomes in the sample space are equally-likely to occur  Number of basic outcomes:  Number of basic outcomes for event A:  Probability of event A: = PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 5 Classical Probability Example 4.1. Flipping a “fair” coin once, the probability that the Head is up = ? Example 4.2. Flipping a fair coin two times. What is the probability of (a) “There are 2 Heads” (b) “There are 1 Head, 1 Tail” (c) “There are 2 Tails” PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 6 Classical Probability Example 4.3: Flip a coin three times, what is the probability of (a) There are 2 Heads only? (b) There are Heads only? (c) There are 2 Heads, given the first is Head? (d) There are 2 Heads, given the first is Tail? Example 4.4 There is a box contains 6 white balls and 4 black balls. Random pick up 2 balls, calculate the probability of event that both balls are white, in 3 cases: (a) Pick up one, replace, then the next (b) Pick up one by one, without replacement (c) Pick up 2 balls simultaneously PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 7 Example Male Female Sum Freq. 160 240 400 PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 8 Male Female Sum Prob. 0.4 0.6 1 Freq. Male Female Sum Single 60 80 140 Married 100 160 260 Sum 160 240 400 Prob. Male Female Sum Single 0.15 0.2 0.35 Married 0.25 0.4 0.65 Sum 0.4 0.6 1  Frequency Table Probability Table  Cross-Frequency Table Joint Probability Table 4.2. Probability vs Proportion  Number of experiments:  Frequency of event A:  Proportion of A:  When n is large enough: () ≈ / Ex. In 100,000 new-borns, there were 51,000 boys, then the probability of “new-born is boy” is about 0.51 Ex. In 4,000 students, there are 600 fail in subject A, then probability of “Fail in subject A” is about 0.15 PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 9 4.3. Complement Event  Complement of A : all outcomes that not belong to A  Denoted by Ā Ex.  A = “two flipped coins are Heads”  A = ?  B = “both picked balls are White”  B = ?  C = “all of students passed”  C = ? PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 10 A Ω Ā Law of Complement  Law: () = – () Ex. Flipping coin twice, what is the Probability of “Have at least one Tail” ?  Complement of “Have at least one tail” is “No any Tail”, or “Two Heads”  Probability = 1 – P(Two Heads) = 1 – ¼ = ¾  PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 11 4.4. Intersection Event  Intersection of A and B: all outcomes that belong to both A and B,  Denoted by ∩  ∩ ̅ = Ø Ex. Pick 2 balls from the box of Blacks and Whites, A is “The first is white” , B is “The second is white” AÇB is “Both balls are white” PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 12 A B Ω AÇB Mutually Exclusive  A and B are Mutually exclusive : have no common  i.e., A Ç B = Ø  A and Ā ? Ex. Pick up 2 balls, A = “two Whites”, B = “two Blacks”, C = “At least one White”  A and B are mutually exclusive; B and C are mutually exclusive, but A and C are NOT mutually exclusive, PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 13 A B Ω Conditional Probability  Conditional probability : probability of one event, given that another event has occurred.  The conditional probability of B given that A has occurred, (B given A) denoted by P(B | A) Ex. Flip a coin 3 times P(2 Heads) = 3/8 = 0.375 P(2 Heads | The first is Head) = 2/4 = 0.5 P(2 Heads | The first is Tail) = 1/4 = 0.25 P(2 Heads | There are at least one Head) = PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 14 Conditional Probability Ex. Pick up one ball from box of 6 Whites and 4 Blacks, then one more. Let A = “the first ball is White”, B = “The second ball is White”  Without replacement of the first ball: P(B | A) = 5/9  Replacement of the first ball: P(B | A) = 6/10 Ex. Pick up 3 balls one-by-one, without replacement  P(The third is White | Two firsts are Whites) =  P(The third is White | Two firsts are 1 White 1 Black) =  P(The third is White | Two firsts are Blacks) = PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 15 Independent Events  A and B are independent : A does not affect B, and B does not affect A  Û ( | ) = () and (|) = ()  A and B are not independent  dependent Ex. From box of 6 Whites, 4 Blacks, pick up one by one Let A = “the 1st is White”, B = “The 2nd is White”.  Replace the first  A and B are independent (|) = (|Ā) = 6/10 ; (|) = 6/10  Without replacement  A and B are dependent (|) = 5/9 ; (|Ā) = 6/9 PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 16 Law of Intersection  ∩ = × = × (|)  A and B are independent Û ∩ = × ()  Conditional Probability = ∩ PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 17 Law of Intersection Ex. Pick one ball from box of 6 whites and 4 blacks, then one more. Let A = “the first ball is White”, B = “The second ball is White”.  The Probability of “Two White” = (Ç) (Ç) = () × (|)  Without replacement Ç = × =  Replacement Ç = × = PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 18 Example Example 4.5. The chance that a student passes the subjects A and B are 0.6 and 0.8, respectively. However, if passed subject A, the chance for him to pass subject B is 0.9 (a) What is probability that he passes both subjects? (b) Whether “Pass subject A” and “Pass subject B” are independent? (c) What is probability of passing subject A, given B has been passed? PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 19 In General  Intersection of n events A1 and A2 and and An ∩ ∩ ⋯ ∩  A1, A2,,An are totally independent Û ( ∩ ∩ ⋯ ∩ ) = () × ⋯ × () Ex. Pick 5 balls from box of 6 Whites and 4 Backs, with replacement, the Probability that all of them are white is 6 10 × 6 10 × 6 10 × 6 10 × 6 10 = 0.6 PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 20 4.5. Union Events  Union of A and B: all outcomes belong to either A or B  Denoted by ∪  ∪ includes: ∩ ; ∩ ; ̅ ∩ Ex. Pick 2 balls from box of 6Ws, 4Bs A = “the first is W”, B = “the second is W”,  then AB is “at least one W” PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 21 A B Ω Law of Union ∪ = + − ( ∩ )  A and B are mutually exclusive events Û ∩ = Ø Û ( ∩ ) = 0 Û ( ∪ ) = () + () PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 22 A B Ω A B Ω Example Example 4.6. The chance that a candidate passes the subjects A and B are 0.6 and 0.8, respectively. However, if passed subject A, the chance for him to pass subject B is 0.9. What is probability that (s)he: (a) pass both subjects? (b) pass at least one subject? (c) fail in both subjects? (d) fail in at least one subject? PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 23 4.6. Joint Probability Ex. The chance that a candidate passes the subjects A and B are 0.6 and 0.8, respectively. The chance of passing both is 0.54. Building the join probability table PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 24 Join Probability Table Subject B Pass B Fail B Sum Sub. A Pass A 0.54 0.06 0.6 Fail A 0.26 0.14 0.4 Sum 0.8 0.2 1 Joint Probability Example 4.7. Base on the Join Probability table, build the Marginal probability distribution, and what is the probability of (a) Pass B given passed A? (b) Pass B given failed A? (c) Fail A given passed B? PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 25 Join Probability Table Subject B Pass B Fail B Sum Sub. A Pass A 0.54 0.06 0.6 Fail A 0.26 0.14 0.4 Sum 0.8 0.2 1 Marginal prob. distributions A A P 0.6 0.4 B B P 0.8 0.2 Example Example 4.8. The exam includes two independent questions. The probability that student’s answer is correct in the A and B question are 0.7 and 0.5. What is the probability that a student (a) Correct in at least one answer? (b) Correct in only one answer? (c) Incorrect in at least one answer? (d) Incorrect in both answers? (e) Joint probability table? PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 26 Problem Problem 4.9. Base on the table, fill the blanks, and build the marginal prob. tables; what is the proportion of: (a) the customer who use services (b) good only buyers in female customers (c) services users only in male customers (d) male in customers who buy goods only (e) female in customers who use services PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 27 Join prob. of Customers Buy goods only (BO) Use Services only (SO) Buy goods and use services (BS) Sum Male: (M) 0.2 0.1 0.15 0.45 Female: (F) 0.15 0.1 0.3 0.55 Sum 0.35 0.2 0.45 1 In General  Union of A1 or A2 or or An is denoted by A1  A2   An  A1, A2,,An are totally mutually exclusive  P(A1A2An) = P(A1) + P(A2) ++ P(An) Example 4.10. There are 3 optional subjects 1, 2, 3. Each of two students B and C randomly chooses one subject, assumed that they are independently. What the probability that they choose the same subject? PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 28 Collectively Exhaustive & Partitions  A1, A2,..., An are Collectively Exhaustive: A1A2 ... An = Ω  A1,A2,...,An are Collectively Exhaustive and Mutually Exclusively  They are Partitions  A and Ā are Partitions PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 29 A1 A2 A3 ... An Ω  A1, A2,..., An are Partitons : P(A1) + P(A2) ++ P(An) = 1 Ex. Flip coin twice; A1 is (2 Heads), A2 is (2 Tails), A3 is (1 Head 1Tail), A4 is (At least one Head).  Whether the following groups are Collectively Exhaustive, Partitions, Complement? (a) A1,A2,A3 (b) A1,A2,A4 (c) A1,A3,A4 (d) A2, A4 PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 30 4.7. Bernoulli Formula Problem 4.11. An quiz has 3 independent questions, the chance for a candidate to answer correctly each question are equally, and be 0.6. What is the probability that a candidate will be (a) Correct in only one answer? (b) Correct in only two answers? (c) Correct in at least one answer? PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 31 Bernoulli Formula  There are independent experiments  The probability that event A occurs in each experiment is equally and denoted by , then probability of A do not occur is – .  Bernoulli Trial, denoted by B(n, p)  The probability that in n experiments, event A occurs in times: , = − PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 32 Example Example 4.12. The test includes 10 independent multiple choice questions. Each question has 4 options with only one correct choice.  A candidate randomly answers all question. What is the probability that (s)he:  (a) Correct in 4 answers  (b) Correct in at least one answer PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 33 3.8. Total Probability – Bayes’ Theorem Example 4.13. There are 2 boxes, Box type I contains 6 white balls and 4 black ones, Box type II contains 8 white balls and 2 black ones. (a) Random choose one box, then random choose one ball. What is the probability that the ball is white? (b) If the chosen ball is white, what is the probability that the chosen box is Type I, Type II? (c) If the chosen ball is black, what is the probability that the chosen box is Type I, Type II ? PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 34 Problem  (White ball) = = (Box I and White ball) or (Box II and White ball) = (B1ÇW)  (B2 ÇW)  = × + × = 0.3 + 0.4 = 0.7  Probability of 0.7 is contributed by 0.3 from Box I and 0.4 from Box II. Therefore Probability that given Ball is white, chosen box is Box type I = . . PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 35 Example  Use table to calculate and analyze Total probability and contribution probability  Example: What happens if there are 3 boxes of type I, and 2 boxes of type II ? PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 36 () (|) (Ç) (|) Box I (B1) 0.5 0.6 0.3 0.3 / 0.7 Box II (B2) 0.5 0.8 0.4 0.4 / 0.7 Sum 1 0.7 1 Bayes’ Theorem  B1,B2,...,Bk are partitions  Total probability: () = ()(|) + ⋯ + ()(|) = ∑ ()(|)  Bayes’ Theorem: Probability of Bi occurs, given A occurred is P(Bi|A) = (|) () PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 37 Bayes’ Theorem  () : prior-probability  (|) : posterior-probability PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 38 1st Stage Prior: P(Bi) 2nd Stage P(A|Bi) P(AÇBi) Posterior P(Bi|A) B1 () ( |) ( ∩ ) ( ∩ ) / () B2 () ( |) ( ∩ ) ( ∩ ) / () ... ... ... ... ... Bk () ( |) ( ∩ ) ( ∩ ) / () Sum 1 P(A) 1 Key concepts  Events, Probability, Conditional probability  Complement, Partitions, Intersection, Union  Independent, Mutually exclusive  Probability Rules, Bernoulli formula  Bayes’ Theorem PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 39 Exercices [1] Chapter 4  (p180) 11, 13  (p184) 18, 19  (p190) 25, 28,  (197) 33, 35,  (204) 41, 42  (208) 46, 50, 52, 57 PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai 40