Replication planning · equivalence
Replication Power and Equivalence Planner
Planning a replication and want the option of concluding “no meaningful effect”? A non-significant result cannot show that. This planner sizes a two-group replication so an equivalence test against a bound you justify has a stated chance of succeeding.
For investigators designing a new-data replication with a continuous outcome and two independent groups. All default values are fictional. The planner works under your assumptions; it cannot tell you what bound is meaningful or whether the original effect was real.
Plan an equivalence replication
Enter the bounds and the assumed true difference in standard-deviation units (Cohen’s d). Equivalence is concluded only if the observed difference is significantly above the lower bound and significantly below the upper bound (two one-sided tests).
Planned replication
Sensitivity: analysed n per group
Rows vary the assumed true difference; columns vary the bound by ×0.8, ×1 and ×1.25. The highlighted cell is your plan. Choosing a bound because it gives a convenient n defeats its purpose.
Who this is for and the problem
Teams replicating a published two-group finding often plan for the original effect size. If the original estimate was inflated, that plan is underpowered, and a null result is then uninformative. Planning for equivalence states up front what result would count as “no meaningful effect” and how many participants are needed to show it.
Scope and deliverables
This free planner covers one design and one success criterion: two independent groups, a continuous outcome, equal allocation and a TOST equivalence test. It returns the analysed n per group, enrolment after attrition, exact power, a sensitivity grid and a methods sentence that records your bound rationale.
Not covered here: small-telescopes criteria, Bayes-factor designs, paired or multilevel designs, unequal variances or allocation, and uncertainty in the original estimate. These answer different questions and are not interchangeable with equivalence.
Sample evidence and process
Fictional example: bounds ±0.3 SD, true difference 0, alpha 0.05, 80% power. The planner gives 191 analysed per group; with 10% attrition, enrol 213 per group (426 total). The large-sample normal formula gives about 190.
Checks run on this planner: six sample sizes match an independent R implementation exactly, with power agreeing to within 1e-9. Three simulations of actual two-sample t-tests agree within Monte Carlo error. At the bound itself the rejection rate stays at or below alpha. Download the R recipe (base R only) to reproduce them.
Process: define the outcome and SD scale; justify the bound from prior literature, a cost-benefit argument or the original authors’ stated relevance; enter a defensible assumed difference; review the grid; then register the plan before collecting data.
Limitations
Power assumes independent, normally distributed outcomes, equal variances, equal group sizes and a known true difference. Bounds are in units of the true SD, while the tests use the sample SD. Attrition is a simple inflation that assumes missingness is unrelated to outcome. The planner does not choose or validate a bound, adjust for multiple outcomes or account for an inflated original effect. This is an early release: an independent statistical review has not yet been completed.
Questions
Is this the same as TOSTER or G*Power?
It uses the same test, two one-sided pooled t-tests, with power computed exactly by integrating over the sampling distribution of the SD. Parity with specific package versions has not been checked; use the R recipe to compare.
Why can’t the assumed difference equal the bound?
At the bound, the chance of concluding equivalence is at most alpha however many participants you recruit. Equivalence planning needs a true difference strictly inside the bounds.
What would a scoped pilot cover?
A possible follow-on is a short design memo: a written rationale for the bound, the chosen criterion, a sensitivity analysis and reproducible code for your design. Scope, availability and any fee would be discussed first. No price or validated offer is established here.
Next step
Review the output with your co-investigators. To discuss a scoped design memo, use the DASS contact page and describe the original study, outcome, design and any constraints. Share no participant-level data in an initial inquiry.
Request a scoped pilot