Summary
Builds on inclusion-exclusion to define disjoint events and partitions. Introduces conditional probability, derives the law of total probability, Bayes’ rule, and the base rate fallacy. Defines independence and pairwise independence.
Example — The Secretary Problem
letters, envelopes. The secretary randomly puts letters into envelopes. What is ?
Solution
Let letter in correct envelope. We want . By inclusion-exclusion:
There are choices of S with equal probability, so
Hence,
Disjoint Events
Definition
Two events and are disjoint if .
A collection of events is disjoint if
This gives a simplification of inclusion-exclusion:
There is nothing to exclude, since all intersections are empty.
Partitions
Definition
A collection of events is called a partition of if:
- disjoint.
Theorem — Law of Total Probability (Intersection Form)
If is a partition of , then event :
Proof
Since partition , we can write . The sets are disjoint (since the are), so by additivity of probability:
Corollary
For two events and :
This is true because and are a partition.
Conditional Probability
Consider two events and of a sample space .
Definition
(probability of B given A) is defined as
under the assumption that .
Example
Rolling a die. . .
Rearranging gives the multiplication rule:
This might seem obvious, but sometimes one side is much easier to calculate than the other.
Law of Total Probability (Conditional Form)
Theorem — Law of Total Probability
Let be a partition of . Then :
Proof
by the intersection form. Substituting gives the result.
Corollary
If are two events:
Base Rate Fallacy
Theorem - Bayes Rule
Let and be two events in .
Proof
.
Example — Disease Testing
A rare disease affects 0.5% of the population. A test is 99% precise (i.e. and ). If a random person tests positive, what is the probability they have the disease?
Solution
By Bayes’ rule:
First, find using the law of total probability:
Therefore:
Despite the 99% precise test, a positive result only means a 33.2% chance of having the disease. This is the base rate fallacy, as the low prevalence dominates.
Independence
Definition
An event is independent of if the info that happened is irrelevant for to happen.
This implies . The latter is less intuitive; it is more natural to say that the probability of something given something else equals its unconditional probability.
We write to denote that and are independent.
Example
Tossing two fair coins. . . These events are independent.
Example
Tossing three coins. . . These are not independent.
Definition
The events are independent if
Question
Is the above equivalent to ?
No — pairwise independence does not imply full independence. The pairwise condition is a special case () of the definition, but higher-order intersections can still fail.
Example
Toss two coins. , , . are pairwise independent but NOT independent:
Definition
are pairwise independent if all pairs of events are independent.
All independent events are pairwise independent, but the converse is not always true, as shown above.
Question
Can an event be independent of itself?
. This holds only if or .
Example
7 people sit at a round table at random. What’s the chance they sit in age order?
By circular permutations, there are distinct seatings. There are 2 favorable arrangements (clockwise or counterclockwise age order).