Calculating Type I Probability - SigmaZone

t-Test Hypothesis

A t-Test is the hypothesis test used to compare two different averages. There are other hypothesis tests used to compare variance (F-Test), proportions (Test of Proportions), etc. In the case of the Hypothesis test the hypothesis is specifically:

H0: µ1= µ2 ← Null Hypothesis

H1: µ1<> µ2 ← Alternate Hypothesis

The Greek letter µ (read “mu”) is used to describe the population average of a group of data. When the null hypothesis states µ1= µ2, it is a statistical way of stating that the averages of dataset 1 and dataset 2 are the same. The alternate hypothesis, µ1<> µ2, is that the averages of dataset 1 and 2 are different. When you do a formal hypothesis test, it is extremely useful to define this in plain language. For our application, dataset 1 is Roger Clemens’ ERA before the alleged use of performance-enhancing drugs and dataset 2 is his ERA after alleged use. For this specific application the hypothesis can be stated:

H0: µ1= µ2 “Roger Clemens’ Average ERA before and after alleged drug use is the same”H1: µ1<> µ2 “Roger Clemens’ Average ERA is different after alleged drug use”

It is helpful to look at the null hypothesis as the default conclusion unless we have sufficient data to conclude otherwise. In the case of the criminal trial, the defendant is assumed not guilty (H0:Null Hypothesis = Not Guilty) unless we have sufficient evidence to show that the probability of Type I Error is so small that we can conclude the alternate hypothesis with minimal risk. How much risk is acceptable? Frankly, that all depends on the person doing the analysis and is hopefully linked to the impact of committing a Type I error (getting it wrong). For example, in the criminal trial if we get it wrong, then we put an innocent person in jail. For this application, we might want the probability of Type I error to be less than .01% or 1 in 10,000 chance. For applications such as did Roger Clemens’ ERA change, I am willing to accept more risk. I am willing to accept the alternate hypothesis if the probability of Type I error is less than 5%. A 5% error is equivalent to a 1 in 20 chance of getting it wrong. I should note one very important concept that many experimenters do incorrectly. I set my threshold of risk at 5% prior to calculating the probability of Type I error. If the probability comes out to something close but greater than 5% I should reject the alternate hypothesis and conclude the null.