Type 1 and type 2 errors

When statistically testing the results of a comparative study, two types of error can be made. A type 1 error occurs when the null hypothesis (see hypothesis testing) is rejected even though it is true (i.e. there is no difference between treatment groups). A type 2 error occurs when the null hypothesis fails to be rejected by the statistical test even though it is false (i.e. there is indeed a difference between treatment groups). Type 1 (false positive) errors are closely linked to significance level (α): setting a high threshold (low α) means that it is less likely that a significant result, rejecting the null hypothesis of no difference between the groups, will occur when there actually is no difference. By contrast, type 2 (false negative) errors are closely linked to study power (1–β): setting a high threshold (low β) means that it is less likely that the null hypothesis (no difference) fails to be rejected when there actually is a difference between the groups. With stochastic data, it is generally not possible to eliminate both type 1 and type 2 errors. Frequently, a trade-off needs to be made between the two.