Summary
Problem
An NBA series (best of 7) was played between the Boston Celtics and LA Lakers (BC-LA). It was played in the following locations: LA-LA-LA-BOS-BOS-BOS-BOS. Teams have a 55% chance of winning home games. What team has an advantage?
Solution
Notice that it doesn’t matter if all 7 games are played (the majority winner is the same regardless). Split into two groups:
- LA games (1-3): BOS wins each with probability
- BOS games (4-7): BOS wins each with probability
BOS wins the series if they win LA games and BOS games with . Since each group’s games are independent, the probability of winning exactly out of 3 LA games is the number of ways to choose which games they win, times . Same idea for out of 4 BOS games.
So Boston has the advantage (~51.6% chance of winning the series).
Definition
A function is called a random variable.
Example
Rolling a die twice. . . . .
Note that ()
Consider a random variable . Fix a number . We can consider the following event:
We can replace with any subset .
Example
We have a machine throwing out (at random) either a banana () or an apple () or an orange (). Every fruit has a price. .
is a map from that is deterministic (fixed), but the actual price you pay is random because the fruit you get is random.
What is the probability that we sell a fruit at 1? .
If we have two R.V. and , and two sets , I can consider the events but also .
Definition
A probability mass function (p.m.f.) of a random variable is defined by
Definition
The cumulative distribution function (c.d.f.) is defined by
Example
. We compute the p.m.f. As soon as , then
If ,
Definition
If is a RV taking only integer values, is a discrete random variable.
Notice that for discrete RVs, we have the property
This is because because () is a partition of (every maps to exactly one integer, so the events are disjoint and cover all of ).
Note
If has p.m.f. , then we often write .
Independence for Discrete Random Variables
Definition
A collection of random variables is called independent if
Proposition
If are independent RVs, then any subcollection of these RVs is a collection of independent RVs.
Proof
It suffices to show any pair, say and , is independent. Need to show
Sum over all possible values of the other variables:
By independence of the full collection:
Factor out the terms (they don’t depend on ):
Each remaining sum is 1 (p.m.f. sums to 1), so we get .
Definition
An infinite sequence of RVs is called independent if for every choice of , the RVs are independent.
Bernoulli & Binomial RVs
Suppose we have a coin with probability of turning up heads, where . Then , , .
Consider the RV with and . Then and .
Definition
A RV is called a Bernoulli() RV if for ,
Now, we toss the -coin times. Let . We showed that
Definition
A RV is called a Binomial() RV if for some , ,
Infinite Sequences of Coin Tosses
Consider a fair coin toss, and let .
What is ? It is an infinite space: we need to consider all possible infinite sequences of coin tosses, e.g. .
If we fix , what is ? We’d get . But we also want , yet every .
The solution to this kind of problem is given using “measure theory”: we define a sub-collection (-algebra) of subsets of where it makes sense to assign a probability. In any case, it’s possible to define probabilities for any event involving a finite number of coin tosses.
Definition
We say that the RVs are i.i.d. (independent and identically distributed) if they are independent and they all have the same distribution.
Geometric & Poisson Random Variables
Consider an infinite sequence of tosses of a -coin. Let . Define if the -th coin is , and if the -th coin is . Then:
Definition
We say that is a geometric RV of parameter if
Definition
We say that is a Poisson RV of parameter if
Proposition (Poisson Limit Theorem)
Fix . Take for every , . Then for every fixed ,
Proof
Since :
As : , , and . So
Example
Suppose at a call center, in each time interval of one second we receive a call with probability . We want to know: what is the probability that we receive calls in one hour?
, . By the Poisson limit theorem, because is approximately .
Joint Probability Mass Functions
Given RVs defined on the same probability space, we consider the function
Definition
The function defined above is called the joint probability mass function (joint p.m.f.).
Definition
The marginal p.m.f. of the RV is defined as
Note that the marginal can be recovered from the joint p.m.f. by summing out all other variables:
This works because .
Example
Fix . Let where . (For example if , .)
We want to determine , i.e. the joint p.m.f. Since the element is uniform:
To compute : for fixed , , the number of pairs s.t. is . Therefore .
So the joint p.m.f. is:
The marginal p.m.f. of :
Similarly, .
Proposition
Let be RVs with joint p.m.f. . Assume that
where each is a p.m.f. Then are independent. Moreover, for all .
Proof
Let be the marginal p.m.f.s of . For any in the range of :
since each is a p.m.f. Similarly for all .
For independence:
PP(X_1 = x_1, ..., X_n = x_n) = f(x_1, ..., x_n) = f_1(x_1) dots f_n(x_n) = PP(X_1 = x_1) dots PP(X_n = x_n) \quad\quad square
Example
Consider a sequence of tosses of a -coin. Let . Then:
where is the p.m.f. of a . So and are independent by the above proposition; that is, they are i.i.d. .
Conditional Probability Mass Functions
Let and be two RVs, and assume for some fixed .
Definition
The conditional p.m.f. of given is defined as
where and (i.e. and ).
Example
, where .
Fix , . Note that and are independent (exercise). Then:
Since , . Therefore:
for all s.t. . This p.m.f. is called the Hypergeometric() distribution. As a free bonus, we get the identity: