Summary
Variance
Definition
For a random variable , we define the variance of as
Theorem
Proof
We can think of variance as a notion of expected distance from the expected value.
Example
. , . , since it is an indicator. Hence, .
Example
. . To make it easier, we compute
so . Hence .
Notice that we employed the useful trick of calculating something simpler: and then found from that. This let us notice that there was a useful double integral.
Example
.
Problem
Compute when , , . Are they the same?
Solution
: and , so
Hence
: Using for ,
Hence
: By definition, the normal distribution has mean and variance , so
They are not all the same. For Poisson, the variance is . For exponential, it is . For normal, it is .
Note
. Prove it.
Covariance
What was nice about Expected Value was that it was linear, but variance is not. Covariance will help us dissect this idea more.
Definition
Given two random variable and , the covariance of and is
Note
Proposition
Let be random variables. are random variables. Set
Then .
Proof
We expand directly from the definition of covariance:
Since expectation is linear,
Therefore
Multiplying these gives
Now take expectation and use linearity again:
In other words, covariance is bilinear, which means it is linear in both variables.
Example
There are a few consequences of bilinearity in covariance.