Hypothesis Testing For The Mean - Probability Course

← previous next →

8.4.3 Hypothesis Testing for the Mean

Here, we would like to discuss some common hypothesis testing problems. We assume that we have a random sample $X_1$,$X_2$,...,$X_n$ from a distribution and our goal is to make inference about the mean of the distribution $\mu$. We consider three hypothesis testing problems. The first one is a test to decide between the following hypotheses:

$\quad$ $H_0$: $\mu=\mu_0$, $\quad$ $H_1$: $\mu \neq \mu_0$.

In this case, the null hypothesis is a simple hypothesis and the alternative hypothesis is a two-sided hypothesis (i.e., it includes both $\mu \lt \mu_0$ and $\mu>\mu_0$). We call this hypothesis test a two-sided test. The second and the third cases are one-sided tests. More specifically, the second case is

$\quad$ $H_0$: $\mu \leq \mu_0$, $\quad$ $H_1$: $\mu > \mu_0$.

Here, both $H_0$ and $H_1$ are one-sided, so we call this test a one-sided test. The third case is very similar to the second case. More specifically, the third scenario is

$\quad$ $H_0$: $\mu \geq \mu_0$, $\quad$ $H_1$: $\mu \lt \mu_0$.

In all of the three cases, we use the sample mean \begin{align}%\label{} \overline{X}=\frac{X_1+X_2+...+X_n}{n} \end{align} to define our statistic. In particular, if we know the variance of the $X_i$'s, $\textrm{Var}(X_i)=\sigma^2$, then we define our test statistic as the normalized sample mean (assuming $H_0$): \begin{align}%\label{} W(X_1,X_2, \cdots,X_n)=\frac{\overline{X}-\mu_0}{\sigma / \sqrt{n}}. \end{align} If we do not know the variance of the $X_i$'s, we use \begin{align}%\label{} W(X_1,X_2, \cdots,X_n)=\frac{\overline{X}-\mu_0}{S / \sqrt{n}}, \end{align} where $S$ is the sample standard deviation, \begin{align}%\label{} S=\sqrt{\frac{1}{n-1} \sum_{k=1}^n (X_k-\overline{X})^2}=\sqrt{\frac{1}{n-1} \left(\sum_{k=1}^n X^2_k-n\overline{X}^2\right)}. \end{align} In any case, we will be able to find the distribution of $W$, and thus we can design our tests by calculating error probabilities. Let us start with the first case.

Two-sided Tests for the Mean:

Here, we are given a random sample $X_1$,$X_2$,...,$X_n$ from a distribution. Let $\mu=EX_i$. Our goal is to decide between

$\quad$ $H_0$: $\mu=\mu_0$, $\quad$ $H_1$: $\mu \neq \mu_0$.

Example 8.22, which we saw previously is an instance of this case. If $H_0$ is true, we expect $\overline{X}$ to be close to $\mu_0$, and so we expect $W(X_1,X_2, \cdots,X_n)$ to be close to $0$ (see the definition of $W$ above).

Therefore, we can suggest the following test. Choose a threshold, and call it $c$. If $|W| \leq c$, accept $H_0$, and if $|W|>c$, accept $H_1$. How do we choose $c$? If $\alpha$ is the required significance level, we must have

\begin{align} P(\textrm{type I error}) &= P(\textrm{Reject }H_0 \; | \; H_0) \\ &= P(|W| > c \; | \; H_0) \leq \alpha. \end{align} Thus, we can choose $c$ such that $P(|W| > c \; | \; H_0) = \alpha$. Let us look at an example. Example Let $X_1$,$X_2$,...,$X_n$ be a random sample from a $N(\mu,\sigma^2)$ distribution, where $\mu$ is unknown but $\sigma$ is known. Design a level $\alpha$ test to choose between

$\quad$ $H_0$: $\mu=\mu_0$, $\quad$ $H_1$: $\mu \neq \mu_0$.

• Solution
  • As discussed above, we let \begin{align}%\label{} W(X_1,X_2, \cdots,X_n)=\frac{\overline{X}-\mu_0}{\sigma / \sqrt{n}}. \end{align} Note that, assuming $H_0$, $W \sim N(0,1)$. We will choose a threshold, $c$. If $|W| \leq c$, we accept $H_0$, and if $|W|>c$, accept $H_1$. To choose $c$, we let \begin{align} P(|W| > c \; | \; H_0) =\alpha. \end{align} Since the standard normal PDF is symmetric around $0$, we have \begin{align} P(|W| > c \; | \; H_0) = 2 P(W>c | \; H_0). \end{align} Thus, we conclude $P(W>c | \; H_0)=\frac{\alpha}{2}$. Therefore, \begin{align} c=z_{\frac{\alpha}{2}}. \end{align} Therefore, we accept $H_0$ if \begin{align} \left|\frac{\overline{X}-\mu_0}{\sigma / \sqrt{n}} \right| \leq z_{\frac{\alpha}{2}}, \end{align} and reject it otherwise.

Relation to Confidence Intervals: It is interesting to examine the above acceptance region. Here, we accept $H_0$ if \begin{align} \left|\frac{\overline{X}-\mu_0}{\sigma / \sqrt{n}} \right| \leq z_{\frac{\alpha}{2}}. \end{align} We can rewrite the above condition as \begin{align} \mu_0 \in \left[\overline{X}- z_{\frac{\alpha}{2}} \frac{\sigma}{\sqrt{n}} , \overline{X}+ z_{\frac{\alpha}{2}} \frac{\sigma}{\sqrt{n}}\right]. \end{align} The above interval should look familiar to you. It is the $(1-\alpha)100\%$ confidence interval for $\mu_0$. This is not a coincidence as there is a general relationship between confidence interval problems and hypothesis testing problems. Example For the above example (Example 8.24), find $\beta$, the probability of type II error, as a function of $\mu$.

• Solution
  • We have \begin{align} \beta (\mu) &=P(\textrm{type II error}) = P(\textrm{accept }H_0 \; | \; \mu) \\ &= P\left(\left|\frac{\overline{X}-\mu_0}{\sigma / \sqrt{n}} \right| \lt z_{\frac{\alpha}{2}}\; | \; \mu \right). \end{align} If $X_i \sim N(\mu,\sigma^2)$, then $\overline{X} \sim N(\mu, \frac{\sigma^2}{n})$. Thus, \begin{align} \beta (\mu)&=P\left(\left|\frac{\overline{X}-\mu_0}{\sigma / \sqrt{n}} \right| \lt z_{\frac{\alpha}{2}}\; | \; \mu \right)\\ &=P\left(\mu_0- z_{\frac{\alpha}{2}} \frac{\sigma}{\sqrt{n}} \leq \overline{X} \leq \mu_0+ z_{\frac{\alpha}{2}} \frac{\sigma}{\sqrt{n}}\right)\\ &=\Phi\left(z_{\frac{\alpha}{2}}+\frac{\mu_0-\mu}{\sigma / \sqrt{n}}\right)-\Phi\left(-z_{\frac{\alpha}{2}}+\frac{\mu_0-\mu}{\sigma / \sqrt{n}}\right). \end{align}

Unknown variance: The above results (Example 8.25) can be extended to the case when we do not know the variance using the $t$-distribution. More specifically, consider the following example. Example Let $X_1$,$X_2$,...,$X_n$ be a random sample from a $N(\mu,\sigma^2)$ distribution, where $\mu$ and $\sigma$ are unknown. Design a level $\alpha$ test to choose between

$\quad$ $H_0$: $\mu=\mu_0$, $\quad$ $H_1$: $\mu \neq \mu_0$.

• Solution
  • Let $S^2$ be the sample variance for this random sample. Then, the random variable $W$ defined as \begin{equation} W(X_1,X_2, \cdots, X_n)=\frac{\overline{X}-\mu_0}{S / \sqrt{n}} \end{equation} has a $t$-distribution with $n-1$ degrees of freedom, i.e., $W \sim T(n-1)$. Thus, we can repeat the analysis of Example 8.24 here. The only difference is that we need to replace $\sigma$ by $S$ and $z_{\frac{\alpha}{2}}$ by $t_{\frac{\alpha}{2},n-1}$. Therefore, we accept $H_0$ if \begin{align} |W| \leq t_{\frac{\alpha}{2},n-1}, \end{align} and reject it otherwise. Let us look at a numerical example of this case.

Example The average adult male height in a certain country is $170$ cm. We suspect that the men in a certain city in that country might have a different average height due to some environmental factors. We pick a random sample of size $9$ from the adult males in the city and obtain the following values for their heights (in cm): \begin{equation} 176.2 \quad 157.9 \quad 160.1 \quad 180.9 \quad 165.1 \quad 167.2 \quad 162.9 \quad 155.7 \quad 166.2 \end{equation} Assume that the height distribution in this population is normally distributed. Here, we need to decide between

$\quad$ $H_0$: $\mu=170$, $\quad$ $H_1$: $\mu \neq 170$.

Based on the observed data, is there enough evidence to reject $H_0$ at significance level $\alpha=0.05$?

• Solution
  • Let's first compute the sample mean and the sample standard deviation. The sample mean is \begin{align}%\label{} \overline{X}&=\frac{X_1+X_2+X_3+X_4+X_5+X_6+X_7+X_8+X_9}{9}\\ &=165.8 \end{align} The sample variance is given by \begin{align}%\label{} {S}^2=\frac{1}{9-1} \sum_{k=1}^9 (X_k-\overline{X})^2&=68.01 \end{align} The sample standard deviation is given by \begin{align}%\label{} S&= \sqrt{S^2}=8.25 \end{align} The following MATLAB code can be used to obtain these values: x=[176.2,157.9,160.1,180.9,165.1,167.2,162.9,155.7,166.2]; m=mean(x); v=var(x); s=std(x); Now, our test statistic is \begin{align} W(X_1,X_2, \cdots, X_9)&=\frac{\overline{X}-\mu_0}{S / \sqrt{n}}\\ &=\frac{165.8-170}{8.25 / 3}=-1.52 \end{align} Thus, $|W|=1.52$. Also, we have \begin{align} t_{\frac{\alpha}{2},n-1} = t_{0.025,8} \approx 2.31 \end{align} The above value can be obtained in MATLAB using the command $\mathtt{tinv(0.975,8)}$. Thus, we conclude \begin{align} |W| \leq t_{\frac{\alpha}{2},n-1}. \end{align} Therefore, we accept $H_0$. In other words, we do not have enough evidence to conclude that the average height in the city is different from the average height in the country.

What if the sample is not from a normal distribution? In the case that $n$ is large, we can say that \begin{align}%\label{} W(X_1,X_2, \cdots,X_n)=\frac{\overline{X}-\mu_0}{S / \sqrt{n}} \end{align} is approximately standard normal. Therefore, we accept $H_0: \mu=\mu_0$ if \begin{align} \left|\frac{\overline{X}-\mu_0}{S / \sqrt{n}} \right| \leq z_{\frac{\alpha}{2}}, \end{align} and reject it otherwise (i.e., accept $H_1: \mu \neq \mu_0$).

Let us summarize what we have obtained for the two-sided test for the mean.

Table 8.2: Two-sided hypothesis testing for the mean: $H_0: \mu=\mu_0$, $H_1$: $\mu \neq \mu_0$.

Case	Test Statistic	Acceptance Region
$X_i \sim N(\mu, \sigma^2)$, $\sigma$ known	$W=\frac{\overline{X}-\mu_0}{\sigma / \sqrt{n}}$	$|W| \leq z_{\frac{\alpha}{2}}$
$n$ large, $X_i$ non-normal	$W=\frac{\overline{X}-\mu_0}{S / \sqrt{n}}$	$|W| \leq z_{\frac{\alpha}{2}}$
$X_i \sim N(\mu, \sigma^2)$, $\sigma$ unknown	$W=\frac{\overline{X}-\mu_0}{S / \sqrt{n}}$	$|W| \leq t_{\frac{\alpha}{2},n-1}$

One-sided Tests for the Mean:

We can provide a similar analysis when we have a one-sided test. Let's show this by an example. Example Let $X_1$,$X_2$,...,$X_n$ be a random sample from a $N(\mu,\sigma^2)$ distribution, where $\mu$ is unknown and $\sigma$ is known. Design a level $\alpha$ test to choose between

$\quad$ $H_0$: $\mu \leq \mu_0$, $\quad$ $H_1$: $\mu > \mu_0$.

• Solution
  • As before, we define the test statistic as \begin{align}%\label{} W(X_1,X_2, \cdots,X_n)=\frac{\overline{X}-\mu_0}{\sigma / \sqrt{n}}. \end{align} If $H_0$ is true (i.e., $\mu \leq \mu_0$), we expect $\overline{X}$ (and thus $W$) to be relatively small, while if $H_1$ is true, we expect $\overline{X}$ (and thus $W$) to be larger. This suggests the following test: Choose a threshold, and call it $c$. If $W \leq c$, accept $H_0$, and if $W>c$, accept $H_1$. How do we choose $c$? If $\alpha$ is the required significance level, we must have \begin{align} P(\textrm{type I error}) &= P(\textrm{Reject }H_0 \; | \; H_0) \\ &= P(W > c \; | \; \mu \leq \mu_0) \leq \alpha. \end{align} Here, the probability of type I error depends on $\mu$. More specifically, for any $\mu \leq \mu_0$, we can write \begin{align} P(\textrm{type I error} \; | \; \mu) &= P(\textrm{Reject }H_0 \; | \; \mu) \\ &= P(W > c \; | \; \mu)\\ &=P \left(\frac{\overline{X}-\mu_0}{\sigma / \sqrt{n}}> c \; | \; \mu\right)\\ &=P \left(\frac{\overline{X}-\mu}{\sigma / \sqrt{n}}+\frac{\mu-\mu_0}{\sigma / \sqrt{n}}> c \; | \; \mu\right)\\ &=P \left(\frac{\overline{X}-\mu}{\sigma / \sqrt{n}}> c+\frac{\mu_0-\mu}{\sigma / \sqrt{n}} \; | \; \mu\right)\\ &\leq P \left(\frac{\overline{X}-\mu}{\sigma / \sqrt{n}}> c \; | \; \mu\right) \quad (\textrm{ since }\mu \leq \mu_0)\\ &=1-\Phi(c) \quad \big(\textrm{ since given }\mu, \frac{\overline{X}-\mu}{\sigma / \sqrt{n}} \sim N(0,1) \big). \end{align} Thus, we can choose $\alpha=1-\Phi(c)$, which results in \begin{align} c=z_{\alpha}. \end{align} Therefore, we accept $H_0$ if \begin{align} \frac{\overline{X}-\mu_0}{\sigma / \sqrt{n}} \leq z_{\alpha}, \end{align} and reject it otherwise.

The above analysis can be repeated for other cases. More generally, suppose that we are given a random sample $X_1$,$X_2$,...,$X_n$ from a distribution. Let $\mu=EX_i$. Our goal is to decide between

$\quad$ $H_0$: $\mu \leq \mu_0$, $\quad$ $H_1$: $\mu > \mu_0$.

We define the test statistic as before, i.e., we define $W$ as \begin{align}%\label{} W(X_1,X_2, \cdots,X_n)=\frac{\overline{X}-\mu_0}{\sigma / \sqrt{n}}, \end{align} if $\sigma=\sqrt{\textrm{Var}(X_i)}$ is known, and as \begin{align}%\label{} W(X_1,X_2, \cdots,X_n)=\frac{\overline{X}-\mu_0}{S / \sqrt{n}}, \end{align} if $\sigma$ is unknown. If $H_0$ is true (i.e., $\mu \leq \mu_0$), we expect that $\overline{X}$ (and thus $W$) to be relatively small, while if $H_1$ is true, we expect $\overline{X}$ (and thus $W$) to be larger. This suggests the following test: Choose a threshold $c$. If $W \leq c$, accept $H_0$, and if $W>c$, accept $H_1$. To choose $c$, note that \begin{align} P(\textrm{type I error}) &= P(\textrm{Reject }H_0 \; | \; H_0) \\ &= P(W > c \; | \; \mu \leq \mu_0)\\ &\leq P(W > c \; | \; \mu=\mu_0). \end{align} Note that the last inequality resulted because if we make $\mu$ larger, the probability of $W>c$ can only increase. In other words, we assumed the worst case scenario, i.e, $\mu=\mu_0$ for the probability of error. Thus, we can choose $c$ such that $P(W > c \; | \; \mu=\mu_0)=\alpha$. By doing this procedure, we obtain the acceptance regions reflected in Table 8.3. Table 8.3: One-sided hypothesis testing for the mean: $H_0: \mu \leq \mu_0$, $H_1$: $\mu > \mu_0$.

Case	Test Statistic	Acceptance Region
$X_i \sim N(\mu, \sigma^2)$, $\sigma$ known	$W=\frac{\overline{X}-\mu_0}{\sigma / \sqrt{n}}$	$W \leq z_{\alpha}$
$n$ large, $X_i$ non-normal	$W=\frac{\overline{X}-\mu_0}{S / \sqrt{n}}$	$W \leq z_{\alpha}$
$X_i \sim N(\mu, \sigma^2)$, $\sigma$ unknown	$W=\frac{\overline{X}-\mu_0}{S / \sqrt{n}}$	$W \leq t_{\alpha,n-1}$

Note that the tests mentioned in Table 8.3 remain valid if we replace the null hypothesis by $\mu=\mu_0$. The reason for this is that in choosing the threshold $c$, we assumed the worst case scenario, i.e, $\mu=\mu_0$. Finally, if we need to decide between

$\quad$ $H_0$: $\mu \geq \mu_0$, $\quad$ $H_1$: $\mu \lt \mu_0$,

we can again repeat the above analysis and we obtain the acceptance regions reflected in Table 8.4. Table 8.4: One-sided hypothesis testing for the mean: $H_0: \mu \geq \mu_0$, $H_1$: $\mu \lt \mu_0$.

Case	Test Statistic	Acceptance Region
$X_i \sim N(\mu, \sigma^2)$, $\sigma$ known	$W=\frac{\overline{X}-\mu_0}{\sigma / \sqrt{n}}$	$W \geq -z_{\alpha}$
$n$ large, $X_i$ non-normal	$W=\frac{\overline{X}-\mu_0}{S / \sqrt{n}}$	$W \geq -z_{\alpha}$
$X_i \sim N(\mu, \sigma^2)$, $\sigma$ unknown	$W=\frac{\overline{X}-\mu_0}{S / \sqrt{n}}$	$W \geq -t_{\alpha,n-1}$

← previous next →

The print version of the book is available on Amazon.

Practical uncertainty: Useful Ideas in Decision-Making, Risk, Randomness, & AI