What Is Power Analysis?
Imagine you run a study comparing two teaching methods. You collect data from 20 students, run your test, and get a non-significant result. Does that mean the two methods work equally well? Not necessarily — your study may simply have been too small to detect a real difference. This is the problem power analysis exists to solve.
Power analysis is a calculation you do before collecting data to work out how many participants you need to have a reasonable chance of detecting an effect that is really there. It answers the question every supervisor, examiner, and journal reviewer eventually asks: “How do you know your sample was big enough?”
In plain English: statistical power is the probability that your study will find an effect if the effect genuinely exists. A study with 80% power has an 80% chance of detecting a real effect and a 20% chance of missing it. Think of it like a metal detector: a weak detector (a small sample) misses small objects buried in the sand, while a stronger one finds them. Power analysis is how you decide how strong a detector you need before you start digging.
Statistical Power, Explained Simply
Every statistical test carries two kinds of error. A Type I error is a false alarm: concluding there is an effect when there is none. Researchers control this with the significance level, alpha, almost always set at .05. A Type II error is a miss: concluding there is no effect when one actually exists. The probability of a Type II error is called beta.
Statistical power is simply 1 minus beta. If your chance of missing a real effect is 20%, your power is 80%. The conventional target in most fields is 80% power, though some researchers and funders now ask for 90%. The higher the power you want, the more participants you need.
The Four Ingredients of Power Analysis
Power analysis always involves four ingredients. If you know any three, you can calculate the fourth:
1. Effect size
How big is the difference or relationship you expect to find? A large effect (for example, a drug that cuts symptoms in half) is easy to detect with a small sample. A small effect (a teaching tweak that raises scores by a few points) needs a much bigger sample. Effect size is usually expressed in standardised units such as Cohen’s d for group differences or r for correlations. By convention, d = 0.2 is small, 0.5 is medium, and 0.8 is large — rough guides, not laws. The best effect size estimate comes from previous research on your topic, not from convention.
2. Significance level (alpha)
The threshold for calling a result significant, almost always .05. A stricter alpha (such as .01) demands a larger sample for the same power, because you are making it harder for a result to count as significant.
3. Statistical power (1 − beta)
The probability of detecting a real effect. The standard target is .80, meaning an 80% chance of finding the effect and a 20% chance of missing it.
4. Sample size
The number of participants. This is usually the unknown you are solving for: “How many people do I need?” But if your sample is fixed — say, you can only recruit 60 patients from one clinic — power analysis can tell you the smallest effect you can reliably detect with that sample instead.
These four are locked together. Make the effect smaller, demand more power, or set a stricter alpha, and the required sample size goes up. That trade-off is the whole point: power analysis forces you to make your assumptions explicit before you spend time and money collecting data.
Types of Power Analysis
Not all power analyses happen before data collection. The four main types serve different moments:
| Type | When you use it | Question it answers |
|---|---|---|
| A priori | Before data collection, while planning the study | How many participants do I need? |
| Post hoc | After data collection, usually after a non-significant result | Given my sample and the effect I observed, what was my power? |
| Sensitivity | When your sample size is fixed and cannot change | What is the smallest effect I can reliably detect with this sample? |
| Criterion | When you need an exact significance threshold for your design | What alpha level gives me exactly the power I want? |
A priori analysis is the one reviewers care about most. It shows you planned your sample size deliberately instead of collecting data until the result looked significant — a practice that inflates false positives. Sensitivity analysis is the honest fallback when your sample is genuinely fixed: instead of pretending you chose your sample size freely, you report what your study can and cannot detect. Post hoc power, by contrast, is widely criticised: once you know the result, calculating power from the observed effect size adds no new information, because it is mathematically tied to the p-value you already have. If a reviewer asks for power after the fact, a confidence interval around your effect size is usually more informative.
A Worked Example: How Many Participants Do You Need?
Suppose you want to compare exam scores between students taught with method A and students taught with method B, using an independent-samples t-test. From earlier studies you expect a medium-sized difference, Cohen’s d = 0.5. You set alpha at .05 (two-tailed) and want 80% power. Plug those numbers into G*Power and the answer is about 64 participants per group — 128 in total.
Now watch what happens when the assumptions change:
- Expect a small effect instead (d = 0.2)? You would need roughly 393 participants per group — more than six times as many. Small effects are expensive to detect.
- Want 90% power instead of 80%? With a medium effect, the requirement rises to about 86 per group.
- Expect a large effect (d = 0.8)? You would need only about 26 participants per group.
This is why reviewers push back when a study claims “no difference” with 15 people per group: unless the effect was expected to be enormous, the study was never in a position to find it. The null result is uninformative, not proof that nothing is there.
How to Run a Power Analysis in G*Power
G*Power is a free program that covers almost every common test, and it is the tool most examiners expect to see named in a thesis. The workflow takes about five minutes once you know your test:
- Choose your test family. For a t-test, pick “t tests”; for ANOVA, pick “F tests”; for correlation, pick “exact” under correlation.
- Choose the type of analysis. Select “A priori: Compute required sample size” — the option you want before collecting data.
- Enter your effect size. Use an estimate from previous research where possible. If none exists, G*Power lets you convert from published statistics (for example, from means and standard deviations) using its built-in calculator.
- Set alpha to .05 and power to .80 (or .90 if your supervisor or funder expects it).
- Click Calculate. G*Power reports the total sample size required, and it draws an X–Y plot showing how the required sample changes as the effect size changes — a useful figure for a thesis methodology chapter.
Report the result in your methods section in one sentence, for example: “An a priori power analysis (G*Power) indicated that 128 participants (64 per group) were required to detect a medium effect (d = 0.5) with 80% power at alpha = .05.” That single sentence answers the reviewer’s question before they ask it.
Why Reviewers Ask for Power Analysis
Examiners and reviewers treat this as non-negotiable for practical reasons:
- Underpowered studies waste everyone’s time. A study with 30% power has a 70% chance of missing a real effect. Running it anyway spends participants’ time, researchers’ effort, and often public money on a near-certain shrug.
- Underpowered “significant” results are unreliable. When a small study does find significance, the effect it reports is often exaggerated — a phenomenon sometimes called the winner’s curse. Large, well-powered studies give more trustworthy estimates.
- It is an ethics issue. Ethics committees want to know that the number of people you are asking to take part is justified: enough to answer the question, but not more than necessary. A power analysis is the standard justification.
- It protects you. If your result is non-significant, a prior power analysis lets you argue honestly that the effect is probably small or absent — rather than admitting you simply never had a chance of finding it.
Common Mistakes to Avoid
- Doing power analysis after the fact and treating it as a rescue. Post hoc power based on your observed effect adds nothing beyond the p-value. Plan ahead; if you missed the chance, report confidence intervals instead.
- Basing the effect size on a single small pilot study. Small pilots give wildly unstable effect estimates. Combine pilot data with published literature, or run a sensitivity analysis across a range of plausible effect sizes.
- Forgetting attrition and missing data. If you need 128 complete cases, recruit more than 128 people. Build the expected dropout rate into the target — power analysis assumes complete data, and real studies rarely have it.
- Using a one-tailed test to shrink the sample size. A one-tailed test needs fewer participants, but it is only justified when an effect in the opposite direction is genuinely impossible or meaningless — a claim examiners will challenge.
- Confusing power with the p-value. A p-value describes your data after collection; power describes your design before collection. A study with 80% power can still produce a non-significant result 20% of the time even when the effect is real.
Frequently Asked Questions
What is a good level of statistical power?
The conventional minimum is 80%, meaning a 20% chance of missing a real effect. Many researchers now aim for 90%, especially for expensive or hard-to-repeat studies. Below 80%, a study is generally considered underpowered, and a non-significant result from it is difficult to interpret.
Can I do a power analysis without knowing the effect size?
Yes. If no previous research gives you an effect-size estimate, run a sensitivity analysis instead: fix your sample size and work out the smallest effect you could detect.
Is power analysis only for quantitative research?
Essentially, yes. Power analysis belongs to statistical hypothesis testing, so it applies to quantitative designs — experiments, surveys, and correlational studies. Qualitative research justifies sample size differently, usually through data saturation rather than statistical power.
What is the difference between power analysis and sample size calculation?
They are closely related but not identical. Sample size calculation is one use of power analysis: working out how many participants you need. Power analysis is the broader family of calculations linking effect size, alpha, power, and sample size — it can also tell you the power of a fixed sample or the smallest detectable effect.
Do I need power analysis for a pilot study?
Not in the same way. Pilot studies are for testing procedures, not for hypothesis testing, so a formal power analysis is usually unnecessary. A rough power calculation can still help you decide whether the pilot’s sample is large enough to estimate variability, which is what the pilot is really for.
Key Takeaways
- Power analysis tells you how many participants you need before you collect data — a planning tool, not an afterthought.
- Statistical power is the probability of detecting a real effect; 80% is the conventional minimum, 90% is increasingly expected.
- Four ingredients are locked together: effect size, alpha, power, and sample size. Know any three and you can calculate the fourth.
- Always do an a priori analysis and report it with numbers: the effect size assumed, alpha, target power, software, and resulting sample size.
- Base your effect size on previous research, not on optimism, and add a buffer for dropouts.
- A non-significant result from an underpowered study proves nothing. Power analysis is what keeps you from being in that position.