Research concepts
Statistical significance is not effect size
Statistical significance and effect size: Do I know the result's magnitude as well as its p-value?
Source editorial review:
Before interpreting the result
Do I know the result's magnitude as well as its p-value?
- Record the estimated effect, scale and units.
- Keep the uncertainty interval, sample size and analysis assumptions.
- Review how many comparisons were performed and which were planned.
Statistical significance and effect size
A p-value describes data compatibility with a statistical model under its assumptions; it does not directly express magnitude, relevance or the probability that a hypothesis is true. Effect size addresses how much a variable changed, while uncertainty describes precision of that estimate. Read all three alongside study design and measurement units.
A reading case: what to check
If a paper highlights asterisks while obscuring the variable's scale, return to tables or data that make the difference interpretable. A small change can be statistically detectable, while an uncertain one may not cross a threshold. Preserve direction, magnitude and interval where available. Do not replace that context with 'works' or 'does not work.'
What to preserve in the record
Significant does not mean large, reproducible or relevant elsewhere. A result that does not cross the threshold does not establish absence either: read its estimate and precision.
Questions and answers
Do I know the result's magnitude as well as its p-value?
A p-value describes data compatibility with a statistical model under its assumptions; it does not directly express magnitude, relevance or the probability that a hypothesis is true. Effect size addresses how much a variable changed, while uncertainty describes precision of that estimate. Read all three alongside study design and measurement units.
What should the review record preserve?
Significant does not mean large, reproducible or relevant elsewhere. A result that does not cross the threshold does not establish absence either: read its estimate and precision.