5.2.1 Covariance

When two random variables $X$ and $Y$ are not independent, it is frequently of interest to assess how strongly they are related to one another.

Definition

The covariance between two rv’s $X$ and $Y$ is

$$\begin{array}{rl}& \mathrm{Cov}\left(X,Y\right)\\ & =E\left[\left(X-{\mu }_X\right)\left(Y-{\mu }_Y\right)\right]\\ & =\{\begin{array}{ll}{\sum }_x{\sum }_y\left(x-{\mu }_X\right)\left(y-{\mu }_Y\right)p\left(x,y\right)& \text{if }X,Y\text{ are discrete}\\ {\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }\left(x-{\mu }_X\right)\left(y-{\mu }_Y\right)f\left(x,y\right)dxdy& \text{if }X,Y\text{ are continuous}\end{array}\end{array}$$

That is, since $X-{\mu }_X$ and $Y-{\mu }_Y$ are the deviations of the two variables from their respective mean values, the covariance is the expected product of deviations. Note that $\mathrm{Cov}\left(X,X\right)=E\left[{\left(X-{\mu }_X\right)}^2\right]=V\left(X\right).$

The rationale for the definition is as follows.

• Suppose $X$ and $Y$ have a strong positive relationship to one another, by which we mean that large values of $X$ tend to occur with large values of $Y$ and small values of $X$ with small values of $Y$.
• Then most of the probability mass or density will be associated with $\left(x-{\mu }_X\right)$ and $\left(y-{\mu }_Y\right)$, either both positive (both $X$ and $Y$ above their respective means) or both negative, so the product $\left(x-{\mu }_X\right)\left(y-{\mu }_Y\right)$ will tend to be positive.
• Thus for a strong positive relationship, $\mathrm{Cov}\left(X,Y\right)$ should be quite positive.
• For a strong negative relationship, the signs of $\left(x-{\mu }_X\right)$ and $\left(y-{\mu }_Y\right)$ will tend to be opposite, yielding a negative product.
  • Thus for a strong negative relationship, $\mathrm{Cov}\left(X,Y\right)$ should be quite negative.
• If $X$ and $Y$ are not strongly related, positive and negative products will tend to cancel one another, yielding a covariance near 0.

Figure 5.4 illustrates the different possibilities. The covariance depends on both the set of possible pairs and the probabilities. In Figure 5.4, the probabilities could be changed without altering the set of possible pairs, and this could drastically change the value of $\mathrm{Cov}\left(X,Y\right)$.

Figure 5.4 $p\left(x,y\right)=1/10$ for each of ten pairs corresponding to indicated points: (a) positive covariance; (b) negative covariance; (c) covariance near zero

EXAMPLE 5.15

The following shortcut formula for $\mathrm{Cov}\left(X,Y\right)$ simplifies the computations.

Proposition

$$\mathrm{Cov}\left(X,Y\right)=E\left(XY\right)-{\mu }_X\cdot {\mu }_Y$$

According to this formula, no intermediate subtractions are necessary; only at the end of the computation is ${\mu }_X\cdot {\mu }_Y$ subtracted from $E\left(XY\right)$. The proof involves expanding $\left(X-{\mu }_X\right)\left(Y-{\mu }_Y\right)$ and then carrying the summation or integration through to each individual term.

EXAMPLE 5.16 (Example 5.5 were continued)

It might appear that the relationship in the insurance example is quite strong since $\mathrm{Cov}\left(X,Y\right)=136,875$, whereas $\mathrm{Cov}\left(X,Y\right)=-2/75$ in the nut example would seem to imply quite a weak relationship. Unfortunately, the covariance has a serious defect that makes it impossible to interpret a computed value. In the insurance example, suppose we had expressed the deductible amount in cents rather than in dollars. Then $100X$ would replace $X,100Y$ would replace $Y$, and the resulting covariance would be $\mathrm{Cov}\left(100X,100Y\right)=\left(100\right)\left(100\right)\mathrm{Cov}\left(X,Y\right)=1,368,750,000.$ If, on the other hand, the deductible amount had been expressed in hundreds of dollars, the computed covariance would have been (.01)(.01)(136,875) $=13.6875$. The defect of covariance is that its computed value depends critically on the units of measurement. Ideally, the choice of units should have no effect on a measure of strength of relationship. This is achieved by scaling the covariance.