\[
\begin{aligned}
E[ a Y + b Z ]
&= E[aY] + E[bZ] \\
&= aE[Y] + bE[Z] \\
& \text{ for random variables $Y, Z$ and numbers $a,b$ }
\end{aligned}
\]
There are two things going on here.
To average a sum of two things, we can take two averages and sum.
To average a constant times a random variable, we can multiply the random variable’s average by the constant.
In other words, we can distribute expectations and can pull constants out of them.
In essence, it comes down to the fact that all we’re doing is summing.
Expectations are probability-weighted sums.
And we’re looking at the expectation of a sum.
And we can change the order we sum in without changing what we get.
\[
\small{
\begin{aligned}
\mathop{\mathrm{E}}\qty[ a Y + b Z ]
&= \sum_{y}\sum_z (a y + b z) \ P(Y=y, Z=z) && \text{ by definition of expectation} \\
&= \sum_{y}\sum_z a y \ P(Y=y, Z=z) + \sum_{z}\sum_y b z \ P(Y=y, Z=z) && \text{changing the order in which we sum} \\
&= \sum_{y} a y \ \sum_z P(Y=y,Z=z) + \sum_{z} b z \ \sum_y P(Y=y,Z=z) && \text{pulling constants out of the inner sums} \\
&= \sum_{y} a y \ P(Y=y) + \sum_{z} b z \ P(Z=z) && \text{summing to get marginal probabilities from our joint } \\
&= a\sum_{y} y \ P(Y=y) + b\sum_{z} z \ P(Z=z) && \text{ pulling constants out of the remaining sum } \\
&= a\mathop{\mathrm{E}}Y + b \mathop{\mathrm{E}}Z && \text{by definition}
\end{aligned}
}
\]
Factorization of Products of Independent Random Variables
The expectation of a product of independent random variables is the product of their expectations. \[
\mathop{\mathrm{E}}[YZ] = \mathop{\mathrm{E}}[Y]\mathop{\mathrm{E}}[Z] \qqtext{when $Y$ and $Z$ are independent}
\]
This comes up a lot in variance calculations. Let’s prove it. It’ll be good practice.
\[
\begin{aligned}
\mathop{\mathrm{E}}[YZ] &= \sum_{yz} yz \ P(Y=y, Z=z) && \text{by definition of expectation} \\
&= \sum_y \sum_z yz \ P(Y=y) P(Z=z) && \text{factoring and ordering sums } \\
&= \textcolor[RGB]{17,138,178}{\sum_y y \ P(Y=y)} \textcolor[RGB]{239,71,111}{\sum_z z \ P(Z=z)} && \text{pulling factors that don't depend on $z$ out of the inner sum} \\
& \textcolor[RGB]{17,138,178}{\mathop{\mathrm{E}}[Y]} \textcolor[RGB]{239,71,111}{\mathop{\mathrm{E}}[Z]} && \text{by definition of expectation}
\end{aligned}
\]
The bias of an estimator is the difference between its expected value and the value of the thing it’s estimating.
We haven’t talked about this yet because there hasn’t been any. We’ve been using unbiased estimators.
You can see, looking at the sampling distribution on the right, that bias might cause some problems with coverage.
If we calibrate interval estimates to cover the estimator’s mean 95% of the time
and that’s what we’ve really been doing how often will they cover the thing we’re estimating?
This means biased estimators can cause big problems. We’ll talk about this more later in the semester.
But for now, let’s just show the estimator we’ve been using is, in fact, unbiased.
Claim. The sample mean is an unbiased estimator of the population mean when we use sampling with replacement, sampling without, randomized circular sampling, etc. \[ \mathop{\mathrm{E}}[\hat\mu] = \mu \qqtext{ for} \hat\mu = \frac1n\sum_{i=1}^n Y_i \qand \mu = \frac1m\sum_{j=1}^m y_j \]