Every hypothesis test is a decision — and like every decision, it can be wrong in two different ways. Understanding type 1 and type 2 errors matters because these two mistakes shape how you design a study, choose your sample size, and interpret a p-value.
The idea is simple once you see it clearly. A Type I error is a false alarm: you conclude there is an effect when there is none. A Type II error is a missed signal: you conclude there is no effect when one really exists. Below, both errors are explained in plain English, with a comparison table, a worked example, and practical ways to keep both under control.
What Is a Type I Error?
A Type I error, also called a false positive, happens when you reject the null hypothesis even though it is actually true. In plain English: your test tells you “something is happening here” when, in reality, nothing is.
Imagine a fire alarm going off when there is no fire: the system was designed to be sensitive, so it occasionally cries wolf.
The probability of a Type I error is called alpha (α) — the significance level you choose before running your test. The most common choice is α = 0.05, a 5% chance of a false positive. Setting α = 0.01 makes false positives rarer — but, as we will see, that comes at a price.
Everyday example: A medical test says a healthy patient has a disease. That is a Type I error — a false positive with real consequences, such as unnecessary worry and follow-up testing.
Research example: You test whether a new teaching method improves exam scores. Your p-value comes out at 0.03, so you reject the null hypothesis and claim the method works. But in truth the method makes no difference — the difference you saw was just random variation. You have made a Type I error.
What Is a Type II Error?
A Type II error, also called a false negative, happens when you fail to reject the null hypothesis even though it is actually false. In plain English: there really is an effect, but your test misses it.
Back to the fire alarm: a Type II error is the alarm staying silent while a real fire burns — the system was not sensitive enough to detect a genuine problem.
The probability of a Type II error is called beta (β). Its complement is statistical power (1 − β) — the probability that your test correctly detects a real effect when one exists. Researchers commonly aim for power of 0.80, accepting a 20% chance of missing a real effect.
Research example: You test the same method with only 12 students. The method genuinely improves scores, yet your small sample does not provide enough evidence, your p-value is 0.18, and you conclude the method does not work. You have missed a real effect — a Type II error.
Type I vs Type II Errors: A Comparison Table
Compare the two errors side by side:
| Type I Error | Type II Error | |
|---|---|---|
| Also called | False positive | False negative |
| What happens | You reject a true null hypothesis | You fail to reject a false null hypothesis |
| In plain English | You see an effect that is not really there | You miss an effect that really is there |
| Symbol | Alpha (α) — the significance level | Beta (β) |
| Common value | Often set at 0.05 | Often 0.20 (power of 0.80) |
| Fire alarm analogy | Alarm rings with no fire | No alarm during a real fire |
| Medical test analogy | Healthy person tests positive | Sick person tests negative |
| Courtroom analogy | Convicting an innocent person | Letting a guilty person go free |
Memory trick: Type I = “I” falsely claim something is true (a false positive), while Type II = “I miss” a real effect (a false negative).
A Worked Example: Testing a New Teaching Method
Suppose you test whether a new active-learning method improves statistics exam scores compared with traditional lectures. You set up the hypotheses:
- Null hypothesis (H₀): The new method makes no difference to exam scores.
- Alternative hypothesis (H₁): The new method improves exam scores.
You recruit 60 students, assign 30 to each group, and run an independent-samples t-test with α = 0.05.
Scenario 1 — a Type I error: In reality, the new method is no better than lectures. But by chance, the students in the new-method group happen to score a little higher, and your p-value comes out at 0.04. You reject the null hypothesis and announce that the method works. Your conclusion is wrong — you have made a Type I error.
Scenario 2 — a Type II error: In reality, the new method genuinely improves scores by a meaningful amount. But your sample is small and the scores vary a lot, so your p-value comes out at 0.11. You fail to reject the null hypothesis and conclude the method does not work. You have missed a real effect — a Type II error.
Scenario 3 — the correct decision: The method works, your sample is large enough, and your p-value is 0.01. You reject the null hypothesis, and this time you are right. Notice that the p-value alone never tells you which scenario you are in — only how surprising your data would be if the null hypothesis were true.
The Trade-Off Between the Two Errors
Here is the uncomfortable truth about type 1 and type 2 errors: you cannot reduce one without increasing the other — unless you change something else about your study.
Make alpha stricter (say, from 0.05 to 0.01) and false positives become rarer — but it also becomes harder to reject the null hypothesis at all, so you are more likely to miss a real effect and beta goes up. Loosen alpha to 0.10 and you catch more real effects, but let more false positives through.
Think of a smoke detector: turn the sensitivity up and it catches every fire — but also goes off every time someone burns toast. Turn it down and the toast problem disappears, but a small real fire might go unnoticed. The only way to get fewer mistakes of both kinds is a better detector — and in research, the equivalent is a better study design.
The choice of alpha therefore depends on context. In a courtroom, a false positive (convicting an innocent person) is considered far worse than a false negative, so the standard of proof is “beyond reasonable doubt” — an extremely strict alpha. In exploratory research, researchers sometimes accept a looser alpha so they do not miss a promising lead. Either way, choose your alpha before you see the data, and be able to justify why.
How to Avoid Type I and Type II Errors
You can never eliminate both errors entirely, but these five strategies keep them under control:
1. Run a power analysis before collecting data. Calculate the sample size you need to detect the effect you care about with power of 0.80 before recruiting anyone. Most Type II errors happen because a study was simply too small to detect its own effect.
2. Choose your alpha deliberately, and pre-register it. Decide your significance level before you look at the data — 0.05 is the convention, 0.01 for high-stakes claims — and pre-register it, so you cannot shift the goalposts after seeing the results.
3. Reduce measurement error. Noisy, unreliable measures blur real effects and inflate the risk of a Type II error. Use validated instruments and pilot-test your questionnaire: cleaner measurement reduces both error types at once.
4. Use a larger sample when you can. All else being equal, a larger sample gives you more power without changing alpha — fewer missed effects and no increase in false positives. If you cannot afford a bigger sample, consider a within-subjects design, which usually has more power for the same number of participants.
5. Correct for multiple comparisons. Every additional test you run is another chance for a false positive. If you run twenty tests at α = 0.05, expect about one false positive by chance alone. When you run many tests, use a correction such as Bonferroni — otherwise your Type I error rate quietly balloons.
Common Mistakes Students Make
- Thinking a non-significant result proves the null hypothesis. Failing to reject H₀ is not the same as proving it. A p-value of 0.20 might mean there is no effect — or your study lacked the power to find one. Always report your power alongside a non-significant result.
- Confusing alpha with the probability that your conclusion is wrong. Alpha is the probability of a false positive if the null hypothesis is true. It is not the probability that your particular significant result is a fluke. That distinction trips up even experienced researchers.
- Treating a Type I error as always worse. Which error is worse depends entirely on the stakes. In medical screening, a false negative can be fatal. In a courtroom, a false positive is the nightmare. Always ask: what does each mistake cost here?
Frequently Asked Questions
Which is worse, a Type I or a Type II error?
It depends on the consequences. A Type I error (false positive) is worse when a wrong claim causes harm — for example, convicting an innocent person. A Type II error (false negative) is worse when missing something real causes harm — for example, failing to detect a disease. In student research, examiners mainly want you to understand the trade-off and justify your alpha and sample size.
Can you reduce both errors at the same time?
Yes — but not by changing alpha, which only trades one error for the other. To reduce both, improve the study itself: increase your sample size, use more reliable measures, or use a more powerful design (for example, within-subjects instead of between-subjects).
What is the relationship between power and Type II errors?
Power is one minus the probability of a Type II error: power = 1 − β. Power of 0.80 means β = 0.20 — a 20% chance of missing a real effect of the size you are looking for. The usual way to raise power is a larger sample.
Does a p-value of 0.06 mean I made a Type II error?
Not necessarily. A p-value just above 0.05 means your data were not quite surprising enough to reject the null hypothesis. It could be a Type II error — a real effect your study missed — or there may genuinely be no effect. You cannot tell from the p-value alone. Report the effect size and confidence interval, and let readers judge.
Why is alpha usually set at 0.05?
The 0.05 convention dates back to R. A. Fisher in the 1920s, who suggested it as a convenient threshold — roughly a 1-in-20 chance of a false positive. It is a convention, not a law of nature: some fields use 0.01, and many researchers now report exact p-values and effect sizes instead of relying on a single cut-off. What matters is choosing alpha before seeing your data and being able to explain your choice.
Key Takeaways
- A Type I error is a false positive — rejecting a true null hypothesis. Its probability is alpha (α), usually 0.05.
- A Type II error is a false negative: failing to reject a false null hypothesis. Its probability is beta (β), and power = 1 − β.
- Lowering alpha reduces false positives but raises false negatives — the two errors trade off against each other.
- To reduce both errors, build a better study: adequate sample size, reliable measures, and corrections for multiple testing.
- A non-significant result never proves there is no effect — it may simply mean your study lacked the power to find one.
Once you can spot both errors, you will read research differently — asking not just “was it significant?” but “was the study powerful enough, and was alpha right for the stakes?”