|
| ||
|
|
Reject the null hypothesis (you say there is a significant difference) |
Accept the null hypothesis (you say there is no significant difference) |
|
There really is no significant difference |
|
|
|
There really is a significant difference |
|
|
Hypothesis testing is like a box of chocolates (has someone already used this analogy?) Let's imagine you think that you've found a way of discerning the hidden insides of a chocolate morsel by observing its outer design. In other words, you believe that you have discovered a method of identifying those chocolates with a dark chocolate inside (the good stuff) from those that aren't so good. What is the likelihood that the chocolates you select will consistently be tasty and enjoyable rather than tooth-shattering nut-filled nuggets of discontent?
Now imagine a small child who gets the rare opportunity of selecting a chocolate out of the box. Since this person only gets one chance, they want to make sure they don't end up awful tasting results. They would want us to be really sure that when we claim to have identified a tasty chocolate before biting into it, we are probably correct. They don't want us to make a Type-I error. A Type-I error claims to have found a significant difference between samples when actually no difference exists. They don't want us to claim to have found a tasty chocolate when we have not. That is why we would want to set the level of significance at a small probability level, usually a probability between 5% and 1% (p<0.05 or p<0.01)
If there is a thing called a Type-I error, then there must also be a Type-II error. Taking our method of finding a difference between two samples of chocolates, what if our method of examination did not notice a distinguishable difference when there really is one? In this case, we would have settled for random chance when choosing a chocolate when there really was a method to choose the good from the bad. In this case we would be committing a Type-II error.
