A power analysis,
step by step.

The six steps

• 1. Name the analysis you will actually run.
• 2. Choose the effect size you need to detect.
• 3. Set α and whether the test is one- or two-sided.
• 4. Set the power you want.
• 5. Solve for the sample size, then adjust it for your design.
• 6. Check how sensitive the answer is to your assumptions.

The running example: a two-group study comparing an intervention with a control condition on a continuous outcome. Every number on this page comes from the StatPower calculator and was checked against R’s pwr package; the covariate figures were also checked by simulation.

1. Name the analysis you will run

A power analysis belongs to a specific statistical test. Plan with the test you will actually use to answer the question, not a simpler stand-in. In the calculator, pick the matching design:

If your analysis is an ANOVA with three or more groups, a regression with several predictors, or a multilevel model, the calculator doesn’t cover it yet; those need a design-specific calculation.

2. Choose the effect size

This is the input that moves the answer most, and the one reviewers question first. The strongest basis is the smallest effect that would matter in practice; prior studies are next, discounted for publication bias; Cohen’s benchmarks are a last resort. Our guide to choosing an effect size covers each option.

Example: the smallest improvement worth acting on is d = 0.40.

3. Set α and the direction of the test

The usual choice is α = .05, two-sided. A one-sided test needs fewer participants (78 per group instead of 100 in the example), but only use it if you commit to it in advance and an effect in the other direction would genuinely not matter. Reviewers are often skeptical of one-sided tests chosen to save sample size.

4. Set the power you want

80% is the common minimum: a one-in-five chance of missing a real effect of the size you planned for. Choose 90% when missing the effect would be costly, for example in a funded trial or a study that is hard to repeat. In the example, 90% power needs 133 per group instead of 100.

5. Solve, then adjust for your design

With d = 0.40, α = .05 two-sided and 80% power, a two-sample t test needs 100 participants per group. That is the number of people you need to analyze. Four adjustments turn it into a realistic plan:

Attrition

Divide by the share you expect to keep. With 15% expected dropout: 100 ÷ 0.85 = 118 per group to recruit. Apply this adjustment last, after the others.

A baseline covariate (ANCOVA)

If you will adjust for a baseline measure, such as a pretest, the covariate removes part of the outcome’s variance and you need fewer participants. With a correlation ρ between covariate and outcome, the required sample shrinks by roughly a factor of 1 − ρ² (Borm et al., 2007). In the example, ρ = 0.5 brings it to about 76 per group, and ρ = 0.7 to about 53. In simulations at those sizes, power came out at 81–82%, so the shortcut errs slightly on the safe side. Only count on this if the adjustment is in your analysis plan and ρ comes from real data.

Clustering

If you randomize or sample whole groups, such as classrooms or clinics, observations within a group are alike and each one carries less information. The design effect is 1 + (m − 1) × ICC, where m is the number of people per group (Kish, 1965). With classrooms of 20 and an ICC of 0.10, the design effect is 2.9: 291 students per group, or about 15 classrooms per group, instead of 100 students. StatPower doesn’t handle clustered designs yet; the DASS multilevel power calculator does.

More than one primary outcome

Testing two primary outcomes at α = .05 each inflates the chance of a false positive. A simple fix is to split α across them (Bonferroni): with two outcomes, test each at .025, which needs 121 per group instead of 100.

6. Check how sensitive the answer is

A single sample size hides how much it depends on your assumptions. Show a small grid instead (per group, α = .05, two-sided):

If the plan only works for the most optimistic cell, it is fragile. Committees and funders take a plan more seriously when it shows you checked.

When the sample size is fixed

Sometimes the sample is set by the setting: one school, one clinic’s patients, an existing dataset. Then turn the question around and report the minimum detectable effect (the calculator’s MDE mode). With 60 per group, α = .05 two-sided and 80% power, the smallest detectable effect is d = 0.52; for the d = 0.40 you hoped for, power is only 58%. Report that honestly and discuss whether an effect of 0.52 is plausible, rather than choosing an effect size to fit the sample.

Mistakes reviewers catch

• Powering a different analysis than the one you run, such as a t test in the plan and a multilevel model in the paper.
• Ignoring clustering when groups were randomized or sampled.
• Forgetting attrition, so the analyzed sample falls short of the target.
• Taking the effect size from a small pilot at face value (see the effect-size guide).
• Computing “observed power” after the study. Power calculated from the effect you found adds nothing beyond the p-value (Hoenig & Heisey, 2001). Report a confidence interval instead.

What to write in your proposal

Name the effect size and its source, the test, α, power, the tool, and each adjustment. Adapt this:

We powered the study to detect d = 0.40, the smallest difference we consider meaningful ([source]). With a two-sided α of .05 and 80% power, a two-sample t test requires 100 participants per group (exact noncentral t; StatPower, statpower.org). Allowing for 15% attrition, we will recruit 118 per group (236 in total).

References

• Borm, G. F., Fransen, J., & Lemmens, W. A. J. G. (2007). A simple sample size formula for analysis of covariance in randomized clinical trials. Journal of Clinical Epidemiology, 60(12), 1234–1238.
• Champely, S. (2020). pwr: Basic functions for power analysis (R package version 1.3-0).
• Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.
• Hoenig, J. M., & Heisey, D. M. (2001). The abuse of power: The pervasive fallacy of power calculations for data analysis. The American Statistician, 55(1), 19–24.
• Kish, L. (1965). Survey sampling. Wiley.
• Lakens, D. (2022). Sample size justification. Collabra: Psychology, 8(1), 33267.