6.1 Some General Concepts of Point Estimation

Statistical inference is almost always directed toward drawing some type of conclusion about one or more parameters (population characteristics). To do so requires that an investigator obtain sample data from each of the populations under study. Conclusions can then be based on the computed values of various sample quantities. For example, let $\mu$ (a parameter) denote the true average breaking strength of wire connections used in bonding semiconductor wafers. A random sample of $n=10$ connections might be made, and the breaking strength of each one determined, resulting in observed strengths $x_1,x_2,\dots ,x_{10}$ . The sample mean breaking strength $\bar{x}$ could then be used to draw a conclusion about the value of $\mu$ . Similarly, if ${\sigma }^2$ is the variance of the breaking strength distribution (population variance, another parameter), the value of the sample variance $s^2$ can be used to infer something about ${\sigma }^2$ .

When discussing general concepts and methods of inference, it is convenient to have a generic symbol for the parameter of interest. We will use the Greek letter $\theta$ for this purpose. In many investigations, $\theta$ will be a population mean $\mu$ , a difference ${\mu }_1-{\mu }_2$ between two population means, or a population proportion of “successes” $p$ . The objective of point estimation is to select a single number, based on sample data, that represents a sensible value for $\theta$ . As an example, the parameter of interest might be $\mu$ , the true average lifetime of batteries of a certain type. A random sample of $n=3$ batteries might yield observed lifetimes (hours) $x_1=5.0$ , $x_2=6.4,x_3=5.9$ . The computed value of the sample mean lifetime is $\bar{x}=5.77$ , and it is reasonable to regard 5.77 as a very plausible value of $\mu$ -our “best guess” for the value of $\mu$ based on the available sample information.

Suppose we want to estimate a parameter of a single population (e.g., $\mu$ or $\sigma$ ) based on a random sample of size $n$ . Recall from the previous chapter that before data is available, the sample observations must be considered random variables (rv’s) $X_1$ , $X_2,\dots ,X_n$ . It follows that any function of the $X_i$ ’s-that is, any statistic-such as the sample mean $\bar{X}$ or sample standard deviation $S$ is also a random variable. The same is true if available data consists of more than one sample. For example, we can represent tensile strengths of $m$ type 1 specimens and $n$ type 2 specimens by $X_1,\dots ,X_m$ and $Y_1,\dots ,Y_n$ , respectively. The difference between the two sample mean strengths is $\bar{X}-\bar{Y}$ ; this is the natural statistic for making inferences about ${\mu }_1-{\mu }_2$ , the difference between the population mean strengths.

---

DEFINITION

---

A point estimate of a parameter $\theta$ is a single number that can be regarded as a sensible value for $\theta$ . It is obtained by selecting a suitable statistic and computing its value from the given sample data. The selected statistic is called the point estimator of $\theta$ .

In the foregoing battery example, the estimator used to obtain the point estimate of $\mu$ was $\bar{X}$ , and the point estimate of $\mu$ was 5.77 . If the three observed lifetimes had instead been $x_1=5.6,x_2=4.5$ , and $x_3=6.1$ , use of the estimator $\bar{X}$ would have resulted in the estimate $\bar{x}=\left(5.6+4.5+6.1\right)/3=5.40$ . The symbol $\hat{\theta }$ (“theta hat”) is customarily used to denote both the estimator of $\theta$ and the point estimate resulting from a given sample.* Thus $\hat{\mu }=\bar{X}$ is read as “the point estimator of $\mu$ is the sample mean $\bar{X}$ .” The statement “the point estimate of $\mu$ is 5.77” can be written concisely as $\hat{\mu }=5.77$ . Notice that in writing $\hat{\theta }=72.5$ , there is no indication of how this point estimate was obtained (what statistic was used). It is recommended that both the estimator and the resulting estimate be reported.

---

• Following earlier notation, we could use $\hat{\Theta }$ (an uppercase theta) for the estimator, but this is cumbersome to write.

---

EXAMPLE 6.1 An automobile manufacturer has developed a new type of bumper, which is supposed to absorb impacts with less damage than previous bumpers. The manufacturer has used this bumper in a sequence of 25 controlled crashes against a wall, each at $10\mathrm{mph}$ , using one of its compact car models. Let $X=$ the number of crashes that result in no visible damage to the automobile. The parameter to be estimated is $p=$ the proportion of all such crashes that result in no damage (alternatively, $p=P$ (no damage in a single crash)). If $X$ is observed to be $x=15$ , the most reasonable estimator and estimate are

$$\text{estimator}\hat{p}=\frac{X}{n}\text{estimate}=\frac{x}{n}=\frac{15}{25}=.60$$

If for each parameter of interest there were only one reasonable point estimator, there would not be much to point estimation. In most problems, though, there will be more than one reasonable estimator.

EXAMPLE 6.2 Consider the accompanying 20 observations on dielectric breakdown voltage for pieces of epoxy resin first introduced in Exercise 4.89. $\begin{array}{llllllllll}24.46& 25.61& 26.25& 26.42& 26.66& 27.15& 27.31& 27.54& 27.74& 27.94\end{array}$ $\begin{array}{llllllllll}27.98& 28.04& 28.28& 28.49& 28.50& 28.87& 29.11& 29.13& 29.50& 30.88\end{array}$

The pattern in the normal probability plot given there is quite straight, so we now assume that the distribution of breakdown voltage is normal with mean value $\mu$ . Because normal distributions are symmetric, $\mu$ is also the median lifetime of the distribution. The given observations are then assumed to be the result of a random sample $X_1,X_2,\dots ,X_{20}$ from this normal distribution. Consider the following estimators and resulting estimates for $\mu$ :

a. Estimator $=\bar{X}$ , estimate $=\bar{x}=\sum x_i/n=555.86/20=27.793$

b. Estimator $=\tilde{X}$ , estimate $=\tilde{x}=\left(27.94+27.98\right)/2=27.960$

c. Estimator $=\left[\min \left(X_i\right)+\max \left(X_i\right)\right]/2=$ the average of the two extreme lifetimes, estimate $=\left[\min \left(x_i\right)+\max \left(x_i\right)\right]/2=\left(24.46+30.88\right)/2=27.670$

d. Estimator $={\bar{X}}_{\mathrm{tr}\left(10\right)}$ , the $10\%$ trimmed mean (discard the smallest and largest $10\%$ of the sample and then average),

$$\text{ estimate }={\bar{x}}_{\mathrm{tr}\left(10\right)}$$ $$=\frac{555.86-24.46-25.61-29.50-30.88}{16}$$ $$=27.838$$

Each one of the estimators (a)-(d) uses a different measure of the center of the sample to estimate $\mu$ . Which of the estimates is closest to the true value? This question cannot be answered without knowing the true value. A question that can be answered is, “Which estimator, when used on other samples of $X_i$ ’s, will tend to produce estimates closest to the true value?” We will shortly address this issue.

EXAMPLE 6.3 The article “Is a Normal Distribution the Most Appropriate Statistical Distribution for Volumetric Properties in Asphalt Mixtures?” first cited in Example 4.26, reported the following observations on $X=$ voids filled with asphalt (%) for 52 specimens of a certain type of hot-mix asphalt:

74.33	71.07	73.82	77.42	79.35	82.27	77.75	78.65	77.19
74.69	77.25	74.84	60.90	60.75	74.09	65.36	67.84	69.97
68.83	75.09	62.54	67.47	72.00	66.51	68.21	64.46	64.34
64.93	67.33	66.08	67.31	74.87	69.40	70.83	81.73	82.50
79.87	81.96	79.51	84.12	80.61	79.89	79.70	78.74	77.28
79.97	75.09	74.38	77.67	83.73	80.39	76.90

Let’s estimate the variance ${\sigma }^2$ of the population distribution. A natural estimator is the sample variance:

$${\hat{\sigma }}^2=S^2=\frac{\sum {\left(X_i-\bar{X}\right)}^2}{n-1}$$

Minitab gave the following output from a request to display descriptive statistics:

Variable	Count	Mean	SE Mean	StDev	Variance	Q1	Median	Q3
VFA(B)	52	73.880	0.889	6.413	41.126	67.933	74.855	79.470

Thus the point estimate of the population variance is

$${\hat{\sigma }}^2=s^2=\frac{\sum {\left(x_i-\bar{x}\right)}^2}{52-1}=41.126$$

[alternatively, the computational formula for the numerator of $s^2$ gives

$$S_{xx}=\sum x_i^2-{\left(\sum x_i\right)}^2/n=285,929.5964-{\left(3841.78\right)}^2/52=2097.4124]\text{.}$$

A point estimate of the population standard deviation is then $\hat{\sigma }=s=\sqrt{41.126}=$ 6.413.

An alternative estimator results from using the divisor $n$ rather than $n-1$ :

$${\hat{\sigma }}^2=\frac{\sum {\left(X_i-\bar{X}\right)}^2}{n},\text{ estimate }=\frac{2097.4124}{52}=40.335$$

We will shortly indicate why many statisticians prefer $S^2$ to this latter estimator.

The cited article considered fitting four different distributions to the data: normal, lognormal, two-parameter Weibull, and three-parameter Weibull. Several different techniques were used to conclude that the two-parameter Weibull provided the best fit (a normal probability plot of the data shows some deviation from a linear pattern). From Section 4.5, the variance of a Weibull random variable is

$${\sigma }^2={\beta }^2\left\{\Gamma \left(1+2/\alpha \right)-{\left[\Gamma \left(1+1/\alpha \right)\right]}^2\right\}$$

where $\alpha$ and $\beta$ are the shape and scale parameters of the distribution. The authors of the article used the method of maximum likelihood (see Section 6.2) to estimate these parameters. The resulting estimates were $\hat{\alpha }=11.9731,\hat{\beta }=77.0153$ . A sensible estimate of the population variance can now be obtained from substituting the estimates of the two parameters into the expression for ${\sigma }^2$ ; the result is ${\hat{\sigma }}^2=56.035$ . This latter estimate is obviously quite different from the sample variance. Its validity depends on the population distribution being Weibull, whereas the sample variance is a sensible way to estimate ${\sigma }^2$ when there is uncertainty as to the specific form of the population distribution.

In the best of all possible worlds, we could find an estimator $\hat{\theta }$ for which $\hat{\theta }=\theta$ always. However, $\hat{\theta }$ is a function of the sample $X_i$ ’s, so it is a random variable. For some samples, $\hat{\theta }$ will yield a value larger than $\theta$ , whereas for other samples $\hat{\theta }$ will underestimate $\theta$ . If we write

$$\hat{\theta }=\theta +\text{ error of estimation }$$

then an accurate estimator would be one resulting in small estimation errors, so that estimated values will be near the true value.

A sensible way to quantify the idea of $\hat{\theta }$ being close to $\theta$ is to consider the squared error ${\left(\hat{\theta }-\theta \right)}^2$ . For some samples, $\hat{\theta }$ will be quite close to $\theta$ and the resulting squared error will be near 0 . Other samples may give values of $\hat{\theta }$ far from $\theta$ , corresponding to very large squared errors. An omnibus measure of accuracy is the expected or mean square error MSE $=E\left[{\left(\hat{\theta }-\theta \right)}^2\right]$ . If a first estimator has smaller MSE than does a second, it is natural to say that the first estimator is the better one. However, MSE will generally depend on the value of $\theta$ . What often happens is that one estimator will have a smaller MSE for some values of $\theta$ and a larger MSE for other values. Finding an estimator with the smallest MSE is typically not possible.

One way out of this dilemma is to restrict attention just to estimators that have some specified desirable property and then find the best estimator in this restricted group. A popular property of this sort in the statistical community is unbiasedness.

6.1.1 Unbiased Estimators

Suppose we have two measuring instruments; one instrument has been accurately calibrated, but the other systematically gives readings larger than the true value being measured. When each instrument is used repeatedly on the same object, because of measurement error, the observed measurements will not be identical. However, the measurements produced by the first instrument will be distributed about the true value in such a way that on average this instrument measures what it purports to measure, so it is called an unbiased instrument. The second instrument yields observations that have a systematic error component or bias. Figure 6.1 shows 10 measurements from both an unbiased and a biased instrument.

Figure 6.1 Measurements from (a) an unbiased instrument, and (b) a biased instrument

DEFINITION

A point estimator $\hat{\theta }$ is said to be an unbiased estimator of $\theta$ if $E\left(\hat{\theta }\right)=\theta$ for every possible value of $\theta$ . If $\hat{\theta }$ is not unbiased, the difference $E\left(\hat{\theta }\right)-\theta$ is called the bias of $\hat{\theta }$ .

That is, $\hat{\theta }$ is unbiased if its probability (i.e., sampling) distribution is always “centered” at the true value of the parameter. Suppose $\hat{\theta }$ is an unbiased estimator; then if $\theta =100$ , the $\hat{\theta }$ sampling distribution is centered at 100 ; if $\theta =27.5$ , then the $\hat{\theta }$ sampling distribution is centered at 27.5 , and so on. Figure 6.2 pictures the distributions of several biased and unbiased estimators. Note that “centered” here means that the expected value, not the median, of the distribution of $\hat{\theta }$ is equal to $\theta$ .

Figure 6.2 The pdf’s of a biased estimator ${\hat{\theta }}_1$ and an unbiased estimator ${\hat{\theta }}_2$ for a parameter $\theta$

It may seem as though it is necessary to know the value of $\theta$ (in which case estimation is unnecessary) to see whether $\hat{\theta }$ is unbiased. This is not usually the case, though, because unbiasedness is a general property of the estimator’s sampling distribution-where it is centered-which is typically not dependent on any particular parameter value.

In Example 6.1, the sample proportion $X/n$ was used as an estimator of $p$ , where $X$ , the number of sample successes, had a binomial distribution with parameters $n$ and $p$ . Thus

$$E\left(\hat{p}\right)=E\left(\frac{X}{n}\right)=\frac{1}{n}E\left(X\right)=\frac{1}{n}\left(np\right)=p$$

PROPOSITION When $X$ is a binomial rv with parameters $n$ and $p$ , the sample proportion $\hat{p}=X/n$ is an unbiased estimator of $p$ .

No matter what the true value of $p$ is, the distribution of the estimator $\hat{p}$ will be centered at the true value.

EXAMPLE 6.4 Suppose that $X$ , the reaction time to a certain stimulus, has a uniform distribution on the interval from 0 to an unknown upper limit $\theta$ (so the density function of $X$ is rectangular in shape with height $1/\theta$ for $0\le x\le \theta$ ). It is desired to estimate $\theta$ on the basis of a random sample $X_1,X_2,\dots ,X_n$ of reaction times. Since $\theta$ is the largest possible time in the entire population of reaction times, consider as a first estimator the largest sample reaction time: ${\hat{\theta }}_1=\max \left(X_1,\dots ,X_n\right)$ . If $n=5$ and $x_1=4.2,x_2=1.7$ , $x_3=2.4,x_4=3.9$ , and $x_5=1.3$ , the point estimate of $\theta$ is ${\hat{\theta }}_1=\max (4.2,1.7,2.4$ , $3.9,1.3)=4.2$ .

Unbiasedness implies that some samples will yield estimates that exceed $\theta$ and other samples will yield estimates smaller than $\theta$ -otherwise $\theta$ could not possibly be the center (balance point) of ${\hat{\theta }}_1$ ’s distribution. However, our proposed estimator will never overestimate $\theta$ (the largest sample value cannot exceed the largest population value) and will underestimate $\theta$ unless the largest sample value equals $\theta$ . This intuitive argument shows that ${\hat{\theta }}_1$ is a biased estimator. More precisely, it can be shown (see Exercise 32) that

$$E\left({\hat{\theta }}_1\right)=\frac{n}{n+1}\cdot \theta <\theta \left(\text{ since }\frac{n}{n+1}<1\right)$$

The bias of ${\hat{\theta }}_1$ is given by $n\theta /\left(n+1\right)-\theta =-\theta /\left(n+1\right)$ , which approaches 0 as $n$ gets large.

It is easy to modify ${\hat{\theta }}_1$ to obtain an unbiased estimator of $\theta$ . Consider the estimator

$${\hat{\theta }}_2=\frac{n+1}{n}\cdot \max \left(X_1,\dots ,X_n\right)$$

Using this estimator on the data gives the estimate $\left(6/5\right)\left(4.2\right)=5.04$ . The fact that $\left(n+1\right)/n>1$ implies that ${\hat{\theta }}_2$ will overestimate $\theta$ for some samples and underestimate it for others. The mean value of this estimator is

$$E\left({\hat{\theta }}_2\right)=E\left[\frac{n+1}{n}\max \left(X_1,\dots ,X_n\right)\right]=\frac{n+1}{n}\cdot E\left[\max \left(X_1,\dots ,X_n\right)\right]$$ $$=\frac{n+1}{n}\cdot \frac{n}{n+1}\theta =\theta$$

If ${\hat{\theta }}_2$ is used repeatedly on different samples to estimate $\theta$ , some estimates will be too large and others will be too small, but in the long run there will be no systematic tendency to underestimate or overestimate $\theta$ .

Principle of Unbiased Estimation

When choosing among several different estimators of $\theta$ , select one that is unbiased.

According to this principle, the unbiased estimator ${\hat{\theta }}_2$ in Example 6.4 should be preferred to the biased estimator ${\hat{\theta }}_1$ . Consider now the problem of estimating ${\sigma }^2$ .

PROPOSITION

Let $X_1,X_2,\dots ,X_n$ be a random sample from a distribution with mean $\mu$ and variance ${\sigma }^2$ . Then the estimator

$${\hat{\sigma }}^2=S^2=\frac{\sum {\left(X_i-\bar{X}\right)}^2}{n-1}$$

is unbiased for estimating ${\sigma }^2$ .

Proof For any $\mathrm{rv}Y,V\left(Y\right)=E\left(Y^2\right)-{\left[E\left(Y\right)\right]}^2$ , so $E\left(Y^2\right)=V\left(Y\right)+{\left[E\left(Y\right)\right]}^2$ . Applying this to

$$S^2=\frac{1}{n-1}\left[\sum X_i^2-\frac{{\left(\sum X_i\right)}^2}{n}\right]$$

gives

$$E\left(S^2\right)=\frac{1}{n-1}\left\{\sum E\left(X_i^2\right)-\frac{1}{n}E\left[{\left(\sum X_i\right)}^2\right]\right\}$$ $$=\frac{1}{n-1}\left\{\sum \left({\sigma }^2+{\mu }^2\right)-\frac{1}{n}\left\{V\left(\sum X_i\right)+{\left[E\left(\sum X_i\right)\right]}^2\right\}\right\}$$ $$=\frac{1}{n-1}\left\{n{\sigma }^2+n{\mu }^2-\frac{1}{n}n{\sigma }^2-\frac{1}{n}{\left(n\mu \right)}^2\right\}$$ $$=\frac{1}{n-1}\left\{n{\sigma }^2-{\sigma }^2\right\}={\sigma }^2\text{ (as desired) }$$

The estimator that uses divisor $n$ can be expressed as $\left(n-1\right)S^2/n$ , so

$$E\left[\frac{\left(n-1\right)S^2}{n}\right]=\frac{n-1}{n}E\left(S^2\right)=\frac{n-1}{n}{\sigma }^2$$

This estimator is therefore not unbiased. The bias is $\left(n-1\right){\sigma }^2/n-{\sigma }^2=-{\sigma }^2/n$ . Because the bias is negative, the estimator with divisor $n$ tends to underestimate ${\sigma }^2$ , and this is why the divisor $n-1$ is preferred by many statisticians (though when $n$ is large, the bias is small and there is little difference between the two).

Unfortunately, the fact that $S^2$ is unbiased for estimating ${\sigma }^2$ does not imply that $S$ is unbiased for estimating $\sigma$ . Taking the square root invalidates the property of unbiasedness (the expected value of the square root is not the square root of the expected value). Fortunately, the bias of $S$ is small unless $n$ is quite small. There are other good reasons to use $S$ as an estimator, especially when the population distribution is normal. These will become more apparent when we discuss confidence intervals and hypothesis testing in the next several chapters.

In Example 6.2, we proposed several different estimators for the mean $\mu$ of a normal distribution. If there were a unique unbiased estimator for $\mu$ , the estimation problem would be resolved by using that estimator. Unfortunately, this is not the case.

---

PROPOSITION

---

If $X_1,X_2,\dots ,X_n$ is a random sample from a distribution with mean $\mu$ , then $\bar{X}$ is an unbiased estimator of $\mu$ . If in addition the distribution is continuous and symmetric, then $\tilde{X}$ and any trimmed mean are also unbiased estimators of $\mu$ . The fact that $\bar{X}$ is unbiased is just a restatement of one of our rules of expected value: $E\left(\bar{X}\right)=\mu$ for every possible value of $\mu$ (for discrete as well as continuous distributions). The unbiasedness of the other estimators is more difficult to verify.

The next example introduces another situation in which there are several unbiased estimators for a particular parameter.

EXAMPLE 6.5 Under certain circumstances organic contaminants adhere readily to wafer surfaces and cause deterioration in semiconductor manufacturing devices. The article “Ceramic Chemical Filter for Removal of Organic Contaminants” (J. of the Institute of Envir. Sciences and Tech., 2003: 59-65) discussed a recently developed alternative to conventional charcoal filters for removing organic airborne molecular contamination in cleanroom applications. One aspect of the investigation of filter performance involved studying how contaminant concentration in air related to concentration on a wafer surface after prolonged exposure. Consider the following representative data on $x=$ DBP concentration in air and $y=$ DBP concentration on a wafer surface after 4-hour exposure (both in $\mu \mathrm{g}/{\mathrm{m}}^3$ , where $\mathrm{DBP}=$ dibutyl phthalate).

Obs. $i$ :	1	2	3	4	5	6
$x$ :	.8	1.3	1.5	3.0	11.6	26.6
$y$ :	.6	1.1	4.5	3.5	14.4	29.1

The authors comment that “DBP adhesion on the wafer surface was roughly proportional to the DBP concentration in air.” Figure 6.3 shows a plot of $y$ versus $x$ -i.e., of the $\left(x,y\right)$ pairs.

If $y$ were exactly proportional to $x$ , then $y=\beta x$ for some value $\beta$ , which says that the $\left(x,y\right)$ points in the plot would lie exactly on a straight line with slope $\beta$ passing through $\left(0,0\right)$ . But this is only approximately the case. So we now assume that for any fixed $x$ , wafer DBP is a random variable $Y$ having mean value $\beta x$ . That is, we postulate that the mean value of $Y$ is related to $x$ by a line passing through $\left(0,0\right)$ but that the observed value of $Y$ will typically deviate from this line (this is referred to in the statistical literature as “regression through the origin”).

Consider the following three estimators for the slope parameter $\beta$ :

\text{#1:}\widehat{\beta } = \frac{1}{n}\sum \frac{{Y}_{i}}{{x}_{i}}\;\# 2 : \widehat{\beta } = \frac{\sum {Y}_{i}}{\sum {x}_{i}}\;\# 3 : \widehat{\beta } = \frac{\sum {x}_{i}{Y}_{i}}{\sum {x}_{i}^{2}}

The resulting estimates based on the given data are 1.3497, 1.1875, and 1.1222, respectively. So the estimate definitely depends on which estimator is used. If one of these three estimators were unbiased and the other two were biased, there would be a good case for using the unbiased one. But all three are unbiased; the argument relies on the fact that each one is a linear function of the $Y_i$ ’s (we are assuming here that the $x_i$ ’s are fixed, not random):

$$E\left(\frac{1}{n}\sum \frac{Y_i}{x_i}\right)=\frac{1}{n}\sum \frac{E\left(Y_i\right)}{x_i}=\frac{1}{n}\sum \frac{\beta x_i}{x_i}=\frac{1}{n}\sum \beta =\frac{n\beta }{n}=\beta$$ $$E\left(\frac{\sum Y_i}{\sum x_i}\right)=\frac{1}{\sum x_i}E\left(\sum Y_i\right)=\frac{1}{\sum x_i}\left(\sum \beta x_i\right)=\frac{1}{\sum x_i}\beta \left(\sum x_i\right)=\beta$$ $$E\left(\frac{\sum x_i{Y}_i}{\sum x_i^2}\right)=\frac{1}{\sum x_i^2}E\left(\sum x_i{Y}_i\right)=\frac{1}{\sum x_i^2}\left(\sum x_i\beta x_i\right)=\frac{1}{\sum x_i^2}\beta \left(\sum x_i^2\right)=\beta$$

In both the foregoing example and the situation involving estimating a normal population mean, the principle of unbiasedness (preferring an unbiased estimator to a biased one) cannot be invoked to select an estimator. What we now need is a criterion for choosing among unbiased estimators.

6.1.2 Estimators with Minimum Variance

Suppose ${\hat{\theta }}_1$ and ${\hat{\theta }}_2$ are two estimators of $\theta$ that are both unbiased. Then, although the distribution of each estimator is centered at the true value of $\theta$ , the spreads of the distributions about the true value may be different.

Principle of Minimum Variance Unbiased Estimation

Among all estimators of $\theta$ that are unbiased, choose the one that has minimum variance. The resulting $\hat{\theta }$ is called the minimum variance unbiased estimator (MVUE) of $\theta$ .

Figure 6.4(a) shows distributions of two different unbiased estimators. Use of the estimator with the more concentrated distribution is more likely than the other one to result in an estimate closer to $\theta$ . Figure 6.4(b) displays estimates from the two estimators based on 10 different samples. The MVUE is, in a certain sense, the most likely among all unbiased estimators to produce an estimate close to the true $\theta$ .

Figure 6.4 (a) Distributions of two unbiased estimators (b) Estimates based on 10 different samples

In Example 6.5, suppose each $Y_i$ is normally distributed with mean $\beta x_i$ and variance ${\sigma }^2$ (the assumption of constant variance). Then it can be shown that the third estimator $\hat{\beta }=\sum x_i{Y}_i/\sum x_i^2$ not only has smaller variance than either of the other two unbiased estimators, but in fact is the MVUE-it has smaller variance than any other unbiased estimator of $\beta$ .

EXAMPLE 6.6 We argued in Example 6.4 that when $X_1,\dots ,X_n$ is a random sample from a uniform distribution on $\left[0,\theta \right]$ , the estimator

$${\hat{\theta }}_1=\frac{n+1}{n}\cdot \max \left(X_1,\dots ,X_n\right)$$

is unbiased for $\theta$ (we previously denoted this estimator by ${\hat{\theta }}_2$ ). This is not the only unbiased estimator of $\theta$ . The expected value of a uniformly distributed rv is just the midpoint of the interval of positive density, so $E\left(X_i\right)=\theta /2$ . This implies that $E\left(\bar{X}\right)=$ $\theta /2$ , from which $E\left(2\bar{X}\right)=\theta$ . That is, the estimator ${\hat{\theta }}_2=2\bar{X}$ is unbiased for $\theta$ .

If $X$ is uniformly distributed on the interval from $A$ to $B$ , then $V\left(X\right)=$ ${\sigma }^2={\left(B-A\right)}^2/12$ . Thus, in our situation, $V\left(X_i\right)={\theta }^2/12,V\left(\bar{X}\right)={\sigma }^2/n={\theta }^2/\left(12n\right)$ , and $V\left({\hat{\theta }}_2\right)=V\left(2\bar{X}\right)=4V\left(\bar{X}\right)={\theta }^2/\left(3n\right)$ . The results of Exercise 32 can be used to show that $V\left({\hat{\theta }}_1\right)={\theta }^2/\left[n\left(n+2\right)\right]$ . The estimator ${\hat{\theta }}_1$ has smaller variance than does ${\hat{\theta }}_2$ if $3n<n\left(n+2\right)$ -that is, if $0<n^2-n=n\left(n-1\right)$ . As long as $n>1$ , $V\left({\hat{\theta }}_1\right)<V\left({\hat{\theta }}_2\right)$ , so ${\hat{\theta }}_1$ is a better estimator than ${\hat{\theta }}_2$ . More advanced methods can be used to show that ${\hat{\theta }}_1$ is the MVUE of $\theta$ -every other unbiased estimator of $\theta$ has variance that exceeds ${\theta }^2/\left[n\left(n+2\right)\right]$ .

One of the triumphs of mathematical statistics has been the development of methodology for identifying the MVUE in a wide variety of situations. The most important result of this type for our purposes concerns estimating the mean $\mu$ of a normal distribution.

THEOREM

Let $X_1,\dots ,X_n$ be a random sample from a normal distribution with parameters $\mu$ and $\sigma$ . Then the estimator $\hat{\mu }=\bar{X}$ is the MVUE for $\mu$ .

Whenever we are convinced that the population being sampled is normal, the theorem says that $\bar{x}$ should be used to estimate $\mu$ . In Example 6.2, then, our estimate would be $\bar{x}=27.793$ .

In some situations, it is possible to obtain an estimator with small bias that would be preferred to the best unbiased estimator. This is illustrated in Figure 6.5. However, MVUEs are often easier to obtain than the type of biased estimator whose distribution is pictured.

Figure 6.5 A biased estimator that is preferable to the MVUE

6.1.3 Some Complications

The last theorem does not say that in estimating a population mean $\mu$ , the estimator $\bar{X}$ should be used irrespective of the distribution being sampled.

EXAMPLE 6.7 Suppose we wish to estimate the thermal conductivity $\mu$ of a certain material. Using standard measurement techniques, we will obtain a random sample $X_1,\dots ,X_n$ of $n$ thermal conductivity measurements. Let’s assume that the population distribution is a member of one of the following three families:

$$f\left(x\right)=\frac{1}{\sqrt{2\pi {\sigma }^2}}{e}^{-{\left(x-\mu \right)}^2/\left(2{\sigma }^2\right)}-\infty <x<\infty \text{\hspace{1em}(6.1)}$$ $$f\left(x\right)=\frac{1}{\pi \beta \left[1+{\left(\left(x-\mu \right)/\beta \right)}^2\right]}-\infty <x<\infty \text{\hspace{1em}(6.2)}$$ $$f\left(x\right)=\{\begin{array}{ll}\frac{1}{2c}& \mu -c\le x\le \mu +c\\ 0& \text{ otherwise }\end{array}\text{\hspace{1em}(6.3)}$$

The pdf (6.1) is the normal distribution, (6.2) is called the Cauchy distribution, and (6.3) is a uniform distribution. All three distributions are symmetric about $\mu$ . The Cauchy density curve is bell-shaped but with much heavier tails (more probability farther out) than the normal curve. In fact, the tails are so heavy that the mean value does not exist, though $\mu$ is still the median and a location parameter for the distribution. The uniform distribution has no tails. The four estimators for $\mu$ considered earlier are $\bar{X},\tilde{X},{\bar{X}}_e$ (the average of the two extreme observations), and ${\bar{X}}_{\mathrm{tr}\left(10\right)}$ , a trimmed mean.

The very important moral here is that the best estimator for $\mu$ depends crucially on which distribution is being sampled. In particular,

1. If the random sample comes from a normal distribution, then $\bar{X}$ is the best of the four estimators, since it has minimum variance among all unbiased estimators.

2. If the random sample comes from a Cauchy distribution, then $\bar{X}$ and ${\bar{X}}_e$ are terrible estimators for $\mu$ , whereas $\tilde{X}$ is quite good (the MVUE is not known); $\bar{X}$ is bad because it is very sensitive to outlying observations, and the heavy tails of the Cauchy distribution make a few such observations likely to appear in any sample.

3. If the underlying distribution is uniform, the best estimator is ${\bar{X}}_e$ ; this estimator is greatly influenced by outlying observations, but the lack of tails makes such observations impossible.

4. The trimmed mean is best in none of these three situations but works reasonably well in all three. That is, ${\bar{X}}_{\mathrm{tr}\left(10\right)}$ does not suffer too much in comparison with the best procedure in any of the three situations.

More generally, recent research in statistics has established that when estimating a point of symmetry $\mu$ of a continuous probability distribution, a trimmed mean with trimming proportion $10\%$ or $20\%$ (from each end of the sample) produces reasonably behaved estimates over a very wide range of possible models. For this reason, a trimmed mean with small trimming percentage is said to be a robust estimator.

In some situations, the choice is not between two different estimators constructed from the same sample, but instead between estimators based on two different experiments.

EXAMPLE 6.8 Suppose a certain type of component has a lifetime distribution that is exponential with parameter $\lambda$ so that expected lifetime is $\mu =1/\lambda$ . A sample of $n$ such components is selected, and each is put into operation. If the experiment is continued until all $n$ lifetimes, $X_1,\dots ,X_n$ , have been observed, then $\bar{X}$ is an unbiased estimator of $\mu$ .

In some experiments, though, the components are left in operation only until the time of the $r$ th failure, where $r<n$ . This procedure is referred to as censoring. Let $Y_1$ denote the time of the first failure (the minimum lifetime among the $n$ components), $Y_2$ denote the time at which the second failure occurs (the second smallest lifetime), and so on. Since the experiment terminates at time $Y_r$ , the total accumulated lifetime at termination is

$$T_r=\sum _{i=1}^r{Y}_i+\left(n-r\right)Y_r$$

We now demonstrate that $\hat{\mu }=T_r/r$ is an unbiased estimator for $\mu$ . To do so, we need two properties of exponential variables:

1. The memoryless property (see Section 4.4), which says that at any time point, remaining lifetime has the same exponential distribution as original lifetime.

2. When $X_1,\dots ,X_k$ are independent, each exponentially distributed with parameter $\lambda ,\min \left(X_1,\dots ,X_k\right)$ , is exponential with parameter $k\lambda$ .

Since all $n$ components last until $Y_1,n-1$ last an additional $Y_2-Y_1,n-2$ an additional $Y_3-Y_2$ amount of time, and so on, another expression for $T_r$ is

$$T_r=n{Y}_1+\left(n-1\right)\left(Y_2-Y_1\right)+\left(n-2\right)\left(Y_3-Y_2\right)+\cdots$$ $$+\left(n-r+1\right)\left(Y_r-Y_{r-1}\right)$$

But $Y_1$ is the minimum of $n$ exponential variables, so $E\left(Y_1\right)=1/\left(n\lambda \right)$ . Similarly, $Y_2-Y_1$ is the smallest of the $n-1$ remaining lifetimes, each exponential with

parameter $\lambda$ (by the memoryless property), so $E\left(Y_2-Y_1\right)=1/\left[\left(n-1\right)\lambda \right]$ . Continuing, $E\left(Y_{i+1}-Y_i\right)=1/\left[\left(n-i\right)\lambda \right]$ , so

$$E\left(T_r\right)=nE\left(Y_1\right)+\left(n-1\right)E\left(Y_2-Y_1\right)+\cdots +\left(n-r+1\right)E\left(Y_r-Y_{r-1}\right)$$ $$=n\cdot \frac{1}{n\lambda }+\left(n-1\right)\cdot \frac{1}{\left(n-1\right)\lambda }+\cdots +\left(n-r+1\right)\cdot \frac{1}{\left(n-r+1\right)\lambda }$$ $$=\frac{r}{\lambda }$$

Therefore, $E\left(T_r/r\right)=\left(1/r\right)E\left(T_r\right)=\left(1/r\right)\cdot \left(r/\lambda \right)=1/\lambda =\mu$ as claimed.

As an example, suppose 20 components are tested and $r=10$ . Then if the first ten failure times are $11,15,29,33,35,40,47,55,58$ , and 72, the estimate of $\mu$ is

$$\hat{\mu }=\frac{11+15+\cdots +72+\left(10\right)\left(72\right)}{10}=111.5$$

The advantage of the experiment with censoring is that it terminates more quickly than the uncensored experiment. However, it can be shown that $V\left(T_r/r\right)=1/\left({\lambda }^{2}r\right)$ , which is larger than $1/\left({\lambda }^{2}n\right)$ , the variance of $\bar{X}$ in the uncensored experiment.

6.1.4 Reporting a Point Estimate: The Standard Error

Besides reporting the value of a point estimate, some indication of its precision should be given. The usual measure of precision is the standard error of the estimator used.

The standard error of an estimator $\hat{\theta }$ is its standard deviation ${\sigma }_{\hat{\theta }}=\sqrt{V\left(\hat{\theta }\right)}$ . It is the magnitude of a typical or representative deviation between an estimate and the value of $\theta$ . If the standard error itself involves unknown parameters whose values can be estimated, substitution of these estimates into ${\sigma }_{\hat{\theta }}$ yields the estimated standard error (estimated standard deviation) of the estimator. The estimated standard error can be denoted either by ${\hat{\sigma }}_{\hat{\theta }}$ (the over $\sigma$ emphasizes that ${\sigma }_{\hat{\theta }}$ is being estimated) or by $s_{\hat{\theta }}$ .

---

DEFINITION

---

---

EXAMPLE 6.9

(Example 6.2

continued)

---

Assuming that breakdown voltage is normally distributed, $\hat{\mu }=\bar{X}$ is the best estimator of $\mu$ . If the value of $\sigma$ is known to be 1.5, the standard error of $\bar{X}$ is ${\sigma }_{\bar{X}}=\sigma /\sqrt{n}=1.5/\sqrt{20}=.335$ . If, as is usually the case, the value of $\sigma$ is unknown, the estimate $\hat{\sigma }=s=1.462$ is substituted into ${\sigma }_{\bar{X}}$ to obtain the estimated standard error ${\hat{\sigma }}_{\bar{X}}=s_{\bar{X}}=s/\sqrt{n}=1.462/\sqrt{20}=.327$ .

---

(Example 6.1

continued)

---

EXAMPLE 6.10 The standard error of $\hat{p}=X/n$ is

$${\sigma }_{\hat{p}}=\sqrt{V\left(X/n\right)}=\sqrt{\frac{V\left(X\right)}{n^2}}=\sqrt{\frac{npq}{n^2}}=\sqrt{\frac{pq}{n}}$$

Since $p$ and $q=1-p$ are unknown (else why estimate?), we substitute $\hat{p}=x/n$ and $\hat{q}=1-x/n$ into ${\sigma }_{\hat{p}}$ , yielding the estimated standard error ${\hat{\sigma }}_{\hat{p}}=\sqrt{\hat{p}\hat{q}/n}=$ $\sqrt{\left(.6\right)\left(.4\right)/25}=.098$ . Alternatively, since the largest value of $pq$ is attained when $p=q=.5$ , an upper bound on the standard error is $\sqrt{1/\left(4n\right)}=.10$ .

When the point estimator $\hat{\theta }$ has approximately a normal distribution, which will often be the case when $n$ is large, then we can be reasonably confident that the true value of $\theta$ lies within approximately 2 standard errors (standard deviations) of $\hat{\theta }$ . Thus if a sample of $n=36$ component lifetimes gives $\hat{\mu }=\bar{x}=28.50$ and $s=3.60$ , then $s/\sqrt{n}=.60$ , so within 2 estimated standard errors, $\hat{\mu }$ translates to the interval $28.50\pm \left(2\right)\left(.60\right)=\left(27.30,29.70\right)$ .

If $\hat{\theta }$ is not necessarily approximately normal but is unbiased, then it can be shown that the estimate will deviate from $\theta$ by as much as 4 standard errors at most $6\%$ of the time. We would then expect the true value to lie within 4 standard errors of $\hat{\theta }$ (and this is a very conservative statement, since it applies to any unbiased $\hat{\theta }$ ). Summarizing, the standard error tells us roughly within what distance of $\hat{\theta }$ we can expect the true value of $\theta$ to lie.

The form of the estimator $\hat{\theta }$ may be sufficiently complicated so that standard statistical theory cannot be applied to obtain an expression for ${\sigma }_{\hat{\theta }}$ . This is true, for example, in the case $\theta =\sigma ,\hat{\theta }=S$ ; the standard deviation of the statistic $S,{\sigma }_S$ , cannot in general be determined. In recent years, a new computer-intensive method called the bootstrap has been introduced to address this problem. Suppose that the population pdf is $f\left(x;\theta \right)$ , a member of a particular parametric family, and that data $x_1,x_2,\dots ,x_n$ gives $\hat{\theta }=21.7$ . We now use statistical software to obtain “bootstrap samples” from the pdf $f\left(x;21.7\right)$ , and for each sample calculate a “bootstrap estimate” ${\hat{\theta }}^{\ast }$ :

First bootstrap sample: $x_1^{\ast },x_2^{\ast },\dots ,x_n^{\ast }$ ; estimate $={\hat{\theta }}_1^{\ast }$

Second bootstrap sample: $x_1^{\ast },x_2^{\ast },\dots ,x_n^{\ast }$ ; estimate $={\hat{\theta }}_2^{\ast }$

:

Bth bootstrap sample: $x_1^{\ast },x_2^{\ast },\dots ,x_n^{\ast }$ ; estimate $={\hat{\theta }}_B^{\ast }$

$B=100$ or 200 is often used. Now let ${\bar{\theta }}^{\ast }=\sum {\hat{\theta }}_i^{\ast }/B$ , the sample mean of the bootstrap estimates. The bootstrap estimate of $\hat{\theta }$ ’s standard error is now just the sample standard deviation of the ${\hat{\theta }}_i^{\ast }$ ’s:

$$s_{\hat{\theta }}=\sqrt{\frac{1}{B-1}\sum {\left({\hat{\theta }}_i^{\ast }-{\bar{\theta }}^{\ast }\right)}^2}$$

(In the bootstrap literature, $B$ is often used in place of $B-1$ ; for typical values of $B$ , there is usually little difference between the resulting estimates.)

EXAMPLE 6.11 A theoretical model suggests that $X$ , the time to breakdown of an insulating fluid between electrodes at a particular voltage, has $f\left(x;\lambda \right)=\lambda e^{-\lambda x}$ , an exponential distribution. A random sample of $n=10$ breakdown times (min) gives the following data:

$\begin{array}{llllllllll}41.53& 18.73& 2.99& 30.34& 12.33& 117.52& 73.02& 223.63& 4.00& 26.78\end{array}$

Since $E\left(X\right)=1/\lambda ,E\left(\bar{X}\right)=1/\lambda$ , so a reasonable estimate of $\lambda$ is $\hat{\lambda }=1/\bar{x}=1/55.087=$ .018153. We then used a statistical computer package to obtain $B=100$ bootstrap samples, each of size 10, from $f\left(x;.018153\right)$ . The first such sample was $41.00,109.70,16.78,6.31,6.76,5.62,60.96,78.81,192.25,27.61$ , from which $\sum x_i^{\ast }=545.8$ and ${\hat{\lambda }}_1^{\ast }=1/54.58=.01832$ . The average of the 100 bootstrap estimates is ${\bar{\lambda }}^{\ast }=.02153$ , and the sample standard deviation of these 100 estimates is $s_{\hat{\lambda }}=.0091$ , the bootstrap estimate of $\hat{\lambda }$ ’s standard error. A histogram of the $100{\hat{\lambda }}_i^{\ast }$ ’s was somewhat positively skewed, suggesting that the sampling distribution of $\hat{\lambda }$ also has this property.

Sometimes an investigator wishes to estimate a population characteristic without assuming that the population distribution belongs to a particular parametric family. An instance of this occurred in Example 6.7, where a $10\%$ trimmed mean was proposed

for estimating a symmetric population distribution’s center $\theta$ . The data of Example 6.2 gave $\hat{\theta }={\bar{x}}_{\mathrm{tr}\left(10\right)}=27.838$ , but now there is no assumed $f\left(x;\theta \right)$ , so how can we obtain a bootstrap sample? The answer is to regard the sample itself as constituting the population (the $n=20$ observations in Example 6.2) and take $B$ different samples, each of size $n$ , with replacement from this population. Several of the books listed in the chapter bibliography provide more information about bootstrapping.

EXERCISES Section 6.1 (1-19)

1. The accompanying data on flexural strength (MPa) for concrete beams of a certain type was introduced in Example 1.2.

5.9	7.2	7.3	6.3	8.1	6.8	7.0
7.6	6.8	6.5	7.0	6.3	7.9	9.0
8.2	8.7	7.8	9.7	7.4	7.7	9.7
7.8	7.7	11.6	11.3	11.8	10.7

a. Calculate a point estimate of the mean value of strength for the conceptual population of all beams manufactured in this fashion, and state which estimator you used. [Hint: $\sum x_i=219.8$ .]

b. Calculate a point estimate of the strength value that separates the weakest $50\%$ of all such beams from the strongest $50\%$ , and state which estimator you used.

c. Calculate and interpret a point estimate of the population standard deviation $\sigma$ . Which estimator did you use? [Hint: $\sum x_i^2=1860.94$ .]

d. Calculate a point estimate of the proportion of all such beams whose flexural strength exceeds $10\mathrm{MPa}$ . [Hint: Think of an observation as a “success” if it exceeds 10.]

e. Calculate a point estimate of the population coefficient of variation $\sigma /\mu$ , and state which estimator you used.

2. The National Health and Nutrition Examination Survey (NHANES) collects demographic, socioeconomic, dietary, and health-related information on an annual basis. Here is a sample of 20 observations on HDL cholesterol level (mg/dl) obtained from the 2009- 2010 survey (HDL is “good” cholesterol; the higher its value, the lower the risk for heart disease):

35	49	52	54	65	51	51
47	86	36	46	33	39	45
39	63	95	35	30	48

a. Calculate a point estimate of the population mean HDL cholesterol level.

b. Making no assumptions about the shape of the population distribution, calculate a point estimate of the value that separates the largest $50\%$ of HDL levels from the smallest $50\%$ .

c. Calculate a point estimate of the population standard deviation.

d. An HDL level of at least 60 is considered desirable as it corresponds to a significantly lower risk of heart disease. Making no assumptions about the shape of the population distribution, estimate the proportion $p$ of the population having an HDL level of at least 60.

3. Consider the following sample of observations on coating thickness for low-viscosity paint (“Achieving a Target Value for a Manufacturing Process: A Case Study,” J. of Quality Technology, 1992: 22-26): $\begin{array}{llllllll}.83& .88& .88& 1.04& 1.09& 1.12& 1.29& 1.31\end{array}$ $\begin{array}{llllllll}1.48& 1.49& 1.59& 1.62& 1.65& 1.71& 1.76& 1.83\end{array}$

Assume that the distribution of coating thickness is normal (a normal probability plot strongly supports this assumption).

a. Calculate a point estimate of the mean value of coating thickness, and state which estimator you used.

b. Calculate a point estimate of the median of the coating thickness distribution, and state which estimator you used.

c. Calculate a point estimate of the value that separates the largest $10\%$ of all values in the thickness distribution from the remaining $90\%$ , and state which estimator you used. [Hint: Express what you are trying to estimate in terms of $\mu$ and $\sigma$ .]

d. Estimate $P\left(X<1.5\right)$ , i.e., the proportion of all thickness values less than 1.5. [Hint: If you knew the values of $\mu$ and $\sigma$ , you could calculate this probability. These values are not available, but they can be estimated.]

e. What is the estimated standard error of the estimator that you used in part (b)?

4. The article from which the data in Exercise 1 was extracted also gave the accompanying strength observations for cylinders: $\begin{array}{llllllllll}6.1& 5.8& 7.8& 7.1& 7.2& 9.2& 6.6& 8.3& 7.0& 8.3\end{array}$ $\begin{array}{llllllllll}7.8& 8.1& 7.4& 8.5& 8.9& 9.8& 9.7& 14.1& 12.6& 11.2\end{array}$

Prior to obtaining data, denote the beam strengths by $X_1,\dots ,X_m$ and the cylinder strengths by $Y_1,\dots ,Y_n$ . Suppose that the $X_i$ ’s constitute a random sample from a distribution with mean ${\mu }_1$ and standard deviation ${\sigma }_1$ and that the $Y_i$ ’s form a random sample (independent of the $X_i$ ’s) from another distribution with mean ${\mu }_2$ and standard deviation ${\sigma }_2$ .

a. Use rules of expected value to show that $\bar{X}-\bar{Y}$ is an unbiased estimator of ${\mu }_1-{\mu }_2$ . Calculate the estimate for the given data.

b. Use rules of variance from Chapter 5 to obtain an expression for the variance and standard deviation (standard error) of the estimator in part (a), and then compute the estimated standard error.

c. Calculate a point estimate of the ratio ${\sigma }_1/{\sigma }_2$ of the two standard deviations.

d. Suppose a single beam and a single cylinder are randomly selected. Calculate a point estimate of the variance of the difference $X-Y$ between beam strength and cylinder strength.

5. As an example of a situation in which several different statistics could reasonably be used to calculate a point estimate, consider a population of $N$ invoices. Associated with each invoice is its “book value,” the recorded amount of that invoice. Let $T$ denote the total book value, a known amount. Some of these book values are erroneous. An audit will be carried out by randomly selecting $n$ invoices and determining the audited (correct) value for each one. Suppose that the sample gives the following results (in dollars).

Invoice

	1	2	3	4	5
Book value	300	720	526	200	127
Audited value	300	520	526	200	157
Error	0	200	0	0	-30

Let

$\bar{Y}=$ sample mean book value

$\bar{X}=$ sample mean audited value

$\bar{D}=$ sample mean error

Propose three different statistics for estimating the total audited (i.e., correct) value-one involving just $N$ and $\bar{X}$ , another involving $T,N$ , and $\bar{D}$ , and the last involving $T$ and $\bar{X}/\bar{Y}$ . If $N=5000$ and $T=1,761,300$ , calculate the three corresponding point estimates. (The article “Statistical Models and Analysis in Auditing,” Statistical Science, 1989: 2-33 discusses properties of these estimators.)

6. Urinary angiotensinogen (AGT) level is one quantitative indicator of kidney function. The article “Urinary Angiotensinogen as a Potential Biomarker of Chronic Kidney Diseases” (J. of the Amer. Society of Hypertension, 2008: 349-354) describes a study in which urinary AGT level $\left(\mu \mathrm{g}\right)$ was determined for a

sample of adults with chronic kidney disease. Here is representative data (consistent with summary quantities and descriptions in the cited article):

2.6	6.2	7.4	9.6	11.5	13.5	14.5	17.0
20.0	28.8	29.5	29.5	41.7	45.7	56.2	56.2
66.1	66.1	67.6	74.1	97.7	141.3	147.9	177.8
186.2	186.2	190.6	208.9	229.1	229.1	288.4	288.4
346.7	407.4	426.6	575.4	616.6	724.4	812.8	1122.0

An appropriate probability plot supports the use of the lognormal distribution (see Section 4.5) as a reasonable model for urinary AGT level (this is what the investigators did).

a. Estimate the parameters of the distribution. [Hint: Remember that $X$ has a lognormal distribution with parameters $\mu$ and ${\sigma }^2$ if $\ln \left(X\right)$ is normally distributed with mean $\mu$ and variance ${\sigma }^2$ .]

b. Use the estimates of part (a) to calculate an estimate of the expected value of AGT level. [Hint: What is $E\left(X\right)?]$

7. a. A random sample of 10 houses in a particular area, each of which is heated with natural gas, is selected and the amount of gas (therms) used during the month of January is determined for each house. The resulting observations are $103,156,118,89,125$ , $147,122,109,138,99$ . Let $\mu$ denote the average gas usage during January by all houses in this area. Compute a point estimate of $\mu$ .

b. Suppose there are 10,000 houses in this area that use natural gas for heating. Let $\tau$ denote the total amount of gas used by all of these houses during January. Estimate $\tau$ using the data of part (a). What estimator did you use in computing your estimate?

c. Use the data in part (a) to estimate $p$ , the proportion of all houses that used at least 100 therms.

d. Give a point estimate of the population median usage (the middle value in the population of all houses) based on the sample of part (a). What estimator did you use?

8. In a random sample of 80 components of a certain type, 12 are found to be defective.

a. Give a point estimate of the proportion of all such components that are not defective.

b. A system is to be constructed by randomly selecting two of these components and connecting them in series, as shown here.

The series connection implies that the system will function if and only if neither component is defective (i.e., both components work properly). Estimate the proportion of all such systems that work properly. [Hint: If $p$ denotes the probability that a component works properly, how can $P$ (system works) be expressed in terms of $p$ ?]

9. Each of 150 newly manufactured items is examined and the number of scratches per item is recorded (the items

are supposed to be free of scratches), yielding the following data:

Number of scratches per item	0	1	2	3	4	5	6	7
Observed frequency	18	37	42	30	13	7	2	1

Let $X=$ the number of scratches on a randomly chosen item, and assume that $X$ has a Poisson distribution with parameter $\mu$ .

a. Find an unbiased estimator of $\mu$ and compute the estimate for the data. [Hint: $E\left(X\right)=\mu$ for $X$ Poisson, so $E\left(\bar{X}\right)=?]$

b. What is the standard deviation (standard error) of your estimator? Compute the estimated standard error. [Hint: ${\sigma }_X^2=\mu$ for $X$ Poisson.]

10. Using a long rod that has length $\mu$ , you are going to lay out a square plot in which the length of each side is $\mu$ . Thus the area of the plot will be ${\mu }^2$ . However, you do not know the value of $\mu$ , so you decide to make $n$ independent measurements $X_1,X_2,\dots ,X_n$ of the length. Assume that each $X_i$ has mean $\mu$ (unbiased measurements) and variance ${\sigma }^2$ .

a. Show that ${\bar{X}}^2$ is not an unbiased estimator for ${\mu }^2$ . [Hint: For any $\mathrm{rv}Y,E\left(Y^2\right)=V\left(Y\right)+{\left[E\left(Y\right)\right]}^2$ . Apply this with $Y=\bar{X}$ .]

b. For what value of $k$ is the estimator ${\bar{X}}^2-k{S}^2$ unbiased for ${\mu }^2$ ? [Hint: Compute $E\left({\bar{X}}^2-k{S}^2\right)$ .]

11. Of $n_1$ randomly selected male smokers, $X_1$ smoked filter cigarettes, whereas of $n_2$ randomly selected female smokers, $X_2$ smoked filter cigarettes. Let $p_1$ and $p_2$ denote the probabilities that a randomly selected male and female, respectively, smoke filter cigarettes.

a. Show that $\left(X_1/n_1\right)-\left(X_2/n_2\right)$ is an unbiased estimator for $p_1-p_2$ . [Hint: $E\left(X_i\right)=n_i{p}_i$ for $i=1,2$ .]

b. What is the standard error of the estimator in part (a)?

c. How would you use the observed values $x_1$ and $x_2$ to estimate the standard error of your estimator?

d. If $n_1=n_2=200,x_1=127$ , and $x_2=176$ , use the estimator of part (a) to obtain an estimate of $p_1-p_2$ .

e. Use the result of part (c) and the data of part (d) to estimate the standard error of the estimator.

12. Suppose a certain type of fertilizer has an expected yield per acre of ${\mu }_1$ with variance ${\sigma }^2$ , whereas the expected yield for a second type of fertilizer is ${\mu }_2$ with the same variance ${\sigma }^2$ . Let $S_1^2$ and $S_2^2$ denote the sample variances of yields based on sample sizes $n_1$ and $n_2$ , respectively, of the two fertilizers. Show that the pooled (combined) estimator

$${\hat{\sigma }}^2=\frac{\left(n_1-1\right)S_1^2+\left(n_2-1\right)S_2^2}{n_1+n_2-2}$$

is an unbiased estimator of ${\sigma }^2$ .

13. Consider a random sample $X_1,\dots ,X_n$ from the pdf

$$f\left(x;\theta \right)=.5\left(1+\theta x\right)-1\le x\le 1$$

where $-1\le \theta \le 1$ (this distribution arises in particle physics). Show that $\hat{\theta }=3\bar{X}$ is an unbiased estimator of $\theta$ . [Hint: First determine $\mu =E\left(X\right)=E\left(\bar{X}\right)$ .]

14. A sample of $n$ captured Pandemonium jet fighters results in serial numbers $x_1,x_2,x_3,\dots ,x_n$ . The CIA knows that the aircraft were numbered consecutively at the factory starting with $\alpha$ and ending with $\beta$ , so that the total number of planes manufactured is $\beta -\alpha +1$ (e.g., if $\alpha =17$ and $\beta =29$ , then $29-17+1=13$ planes having serial numbers $17,18,19,\dots ,28,29$ were manufactured). However, the CIA does not know the values of $\alpha$ or $\beta$ . A CIA statistician suggests using the estimator $\max \left(X_i\right)-\min \left(X_i\right)+1$ to estimate the total number of planes manufactured.

a. If $n=5,x_1=237,x_2=375,x_3=202,x_4=525$ , and $x_5=418$ , what is the corresponding estimate?

b. Under what conditions on the sample will the value of the estimate be exactly equal to the true total number of planes? Will the estimate ever be larger than the true total? Do you think the estimator is unbiased for estimating $\beta -\alpha +1$ ? Explain in one or two sentences.

15. Let $X_1,X_2,\dots ,X_n$ represent a random sample from a Rayleigh distribution with pdf

$$f\left(x;\theta \right)=\frac{x}{\theta }{e}^{-x^2/\left(2\theta \right)}x>0$$

a. It can be shown that $E\left(X^2\right)=2\theta$ . Use this fact to construct an unbiased estimator of $\theta$ based on $\sum X_i^2$ (and use rules of expected value to show that it is unbiased).

b. Estimate $\theta$ from the following $n=10$ observations on vibratory stress of a turbine blade under specified conditions:

$$\begin{array}{lllll}16.88& 10.23& 4.59& 6.66& 13.68\end{array}$$ $$\text{14.23 19.87 9.40 6.51 10.95}$$

16. Suppose the true average growth $\mu$ of one type of plant during a 1-year period is identical to that of a second type, but the variance of growth for the first type is ${\sigma }^2$ , whereas for the second type the variance is $4{\sigma }^2$ . Let $X_1,\dots ,X_m$ be $m$ independent growth observations on the first type [so $E\left(X_i\right)=\mu ,V\left(X_i\right)={\sigma }^2$ ], and let $Y_1,\dots ,Y_n$ be $n$ independent growth observations on the second type $[E\left(Y_i\right)=\mu ,V\left(Y_i\right)=4{\sigma }^2$ ].

a. Show that the estimator $\hat{\mu }=\delta \bar{X}+\left(1-\delta \right)\bar{Y}$ is unbiased for $\mu$ (for $0<\delta <1$ , the estimator is a weighted average of the two individual sample means).

b. For fixed $m$ and $n$ , compute $V\left(\hat{\mu }\right)$ , and then find the value of $\delta$ that minimizes $V\left(\hat{\mu }\right)$ . [Hint: Differentiate $V\left(\hat{\mu }\right)$ with respect to $\delta$ .]

17. In Chapter 3, we defined a negative binomial rv as the number of failures that occur before the $r$ th success in a sequence of independent and identical success/failure trials. The probability mass function (pmf) of $X$ is

$nb\left(x;r,p\right)=$

$$\left(\begin{array}{c}x+r-1\\ x\end{array}\right)p^r{\left(1-p\right)}^{x}x=0,1,2,\dots$$

a. Suppose that $r\ge 2$ . Show that

$$\hat{p}=\left(r-1\right)/\left(X+r-1\right)$$

is an unbiased estimator for $p$ . [Hint: Write out $E\left(\hat{p}\right)$ and cancel $x+r-1$ inside the sum.]

b. A reporter wishing to interview five individuals who support a certain candidate begins asking people whether $\left(S\right)$ or not $\left(F\right)$ they support the candidate. If the sequence of responses is SFFSFFFSSS, estimate $p=$ the true proportion who support the candidate.

18. Let $X_1,X_2,\dots ,X_n$ be a random sample from a pdf $f\left(x\right)$ that is symmetric about $\mu$ , so that $\tilde{X}$ is an unbiased estimator of $\mu$ . If $n$ is large, it can be shown that $V$ $\left(\tilde{X}\right)\approx 1/\left(4n{\left[f\left(\mu \right)\right]}^2\right)$ .

a. Compare $V\left(\tilde{X}\right)$ to $V\left(\bar{X}\right)$ when the underlying distribution is normal.

b. When the underlying pdf is Cauchy (see Example 6.7), $V\left(\bar{X}\right)=\infty$ , so $\bar{X}$ is a terrible estimator. What is $V$ $\left(\tilde{X}\right)$ in this case when $n$ is large?

19. An investigator wishes to estimate the proportion of students at a certain university who have violated the honor code. Having obtained a random sample of $n$ students, she realizes that asking each, “Have you violated the honor code?” will probably result in some untruthful responses. Consider the following scheme, called a randomized response technique. The investigator makes up a deck of 100 cards, of which 50 are of type I and 50 are of type II.

Type I: Have you violated the honor code (yes or no)?

Type II: Is the last digit of your telephone number a 0 , 1, or 2 (yes or no)?

Each student in the random sample is asked to mix the deck, draw a card, and answer the resulting question truthfully. Because of the irrelevant question on type II cards, a yes response no longer stigmatizes the respondent, so we assume that responses are truthful. Let $p$ denote the proportion of honor-code violators (i.e., the probability of a randomly selected student being a violator), and let $\lambda =P$ (yes response). Then $\lambda$ and $p$ are related by $\lambda =.5p+\left(.5\right)\left(.3\right)$ .

a. Let $Y$ denote the number of yes responses, so $Y\sim$ Bin $\left(n,\lambda \right)$ . Thus $Y/n$ is an unbiased estimator of $\lambda$ . Derive an estimator for $p$ based on $Y$ . If $n=80$ and $y=20$ , what is your estimate? [Hint: Solve $\lambda =.5p+.15$ for $p$ and then substitute $Y/n$ for $\lambda$ .]

b. Use the fact that $E\left(Y/n\right)=\lambda$ to show that your estimator $\hat{p}$ is unbiased.

c. If there were 70 type I and 30 type II cards, what would be your estimator for $p$ ?