5.5 The Distribution of a Linear Combination

• 5.5.1 The Difference Between Two Random Variables
• 5.5.2 The Case of Normal Random Variables
• EXERCISES Section 5.5 (58-74)

The sample mean $\bar{X}$ and sample total $T_o$ are special cases of a type of random variable that arises very frequently in statistical applications.

lineaer combination of rv

Given a collection of $n$ random variables $X_1,\dots ,X_n$ and $n$ numerical constants $a_1,\dots ,a_n$, the rv

$$Y=a_1{X}_1+\cdots +a_n{X}_n=\sum _{i=1}^n{a}_i{X}_i\text{\hspace{1em}(5.7)}$$

is called a linear combination of the $X_i$ ’s.

Example

consider someone who owns 100 shares of stock A, 200 shares of stock B, and 500 shares of stock C. Denote the share prices of these three stocks at some particular time by $X_1,X_2$, and $X_3$, respectively. Then the value of this individual’s stock holdings is the linear combination $Y=100X_1+200X_2+500X_3$.

Taking $a_1=a_2=\cdots =a_n=1$ gives $Y=X_1+\cdots +X_n=T_o$, and $a_1=$ $a_2=\cdots =a_n=1/n$ yields

$$Y=\frac{1}{n}{X}_1+\cdots +\frac{1}{n}{X}_n=\frac{1}{n}\left(X_1+\cdots +X_n\right)=\frac{1}{n}{T}_o=\bar{X}$$

Notice that we are not requiring the $X_i$’s to be independent or identically distributed. All the $X_i$ ’s could have different distributions and therefore different mean values and variances. Our first result concerns the expected value and variance of a linear combination.

Proposition

Let $X_1,X_2,\dots ,X_n$ have

• mean values ${\mu }_1,\dots ,{\mu }_n$, respectively,

• variances ${\sigma }_1^2,\dots ,{\sigma }_n^2$, respectively.

1. Whether or not the $X_i$ ’s are independent,

$$\begin{array}{rl}& E\left(a_1{X}_1+a_2{X}_2+\cdots +a_n{X}_n\right)\\ & =a_{1}E\left(X_1\right)+a_{2}E\left(X_2\right)+\cdots +a_{n}E\left(X_n\right)\\ & =a_1{\mu }_1+\cdots +a_n{\mu }_n\end{array}\text{\hspace{1em}(5.8)}$$

2. If $X_1,\dots ,X_n$ are independent,

$$\begin{array}{rl}& V\left(a_1{X}_1+a_2{X}_2+\cdots +a_n{X}_n\right)\\ & =a_1^{2}V\left(X_1\right)+a_2^{2}V\left(X_2\right)+\cdots +a_n^{2}V\left(X_n\right)\\ & =a_1^2{\sigma }_1^2+\cdots +a_n^2{\sigma }_n^2\end{array}\text{\hspace{1em}(5.9)}$$

and

$${\sigma }_{a_1{X}_1+\cdots +a_n{X}_n}=\sqrt{a_1^2{\sigma }_1^2+\cdots +a_n^2{\sigma }_n^2}\text{\hspace{1em}(5.10)}$$

3. For any $X_1,\dots ,X_n$,

$$V\left(a_1{X}_1+\cdots +a_n{X}_n\right)=\sum _{i=1}^n\sum _{j=1}^n{a}_i{a}_j\mathrm{Cov}\left(X_i,X_j\right)\text{\hspace{1em}(5.11)}$$

Proofs are sketched out at the end of the section. A paraphrase of (5.8) is that the expected value of a linear combination is the same as the linear combination of the expected values

• for example, $E\left(2X_1+5X_2\right)=2{\mu }_1+5{\mu }_2$. The result (5.9) in Statement 2 is a special case of (5.11) in Statement 3; when the $X_i$ ’s are independent, $\mathrm{Cov}\left(X_i,X_j\right)=0$ for $i\ne j$ and $=V\left(X_i\right)$ for $i=j$ (this simplification actually occurs when the $X_i$ ’s are uncorrelated, a weaker condition than independence). Specializing to the case of a random sample $(X_i\text{’s iid})$ with $a_i=1/n$ for every $i$ gives $E\left(\bar{X}\right)=\mu$ and $V\left(\bar{X}\right)={\sigma }^2/n$, as discussed in Section 5.4. A similar comment applies to the rules for $T_o$.

EXAMPLE 5.30