Statistical Reporting Standards
Advanced Statistical Methods
Communicating Results With Clarity and Completeness
Reporting standards like APA, CONSORT, STROBE, and PRISMA ensure that statistical results are communicated with sufficient detail for interpretation, replication, and meta-analysis. Effect sizes and confidence intervals complement p-values.
- Journal publication β Meeting reporting guidelines is now required by most top-tier journals
- Regulatory submissions β FDA and EMA require CONSORT-compliant trial reporting
- Meta-analysis β Complete reporting enables inclusion in systematic reviews and evidence synthesis
Good reporting transforms statistical output into useful, actionable scientific knowledge.
Reporting standards are structured guidelines that ensure consistency, transparency, and completeness in how researchers communicate their methods and findings. They exist because vague reporting undermines the ability of readers to evaluate, replicate, and build upon research.
DfStatistical Reporting
Statistical reporting is the practice of communicating research methods, analyses, and results in a way that is complete, transparent, and verifiable β enabling readers to assess the validity of the conclusions and, where possible, reproduce the analyses.
APA Reporting Guidelines (7th Edition)
The American Psychological Association (APA) publication manual specifies detailed standards for reporting statistical results.
APA 7th Edition Requirements
- Effect sizes β always report (Cohen's d, Ξ·Β², r, OR, etc.)
- Confidence intervals β report 95% CI for all estimates
- Exact p-values β report exact values (p = .023), not just "p < .05"
- Test statistics β include test name, df, value, p, effect size, CI
- Descriptive statistics β means, SDs, and Ns for all groups
- Software β cite specific packages, versions, and programming languages
APA Statistical Reporting Format
APA t-test Report Format
Here,
- =t-test statistic
- =Degrees of freedom
- =Exact p-value
- =Cohen's d effect size
- =95% confidence interval bounds
APA Reporting Examples
- t-test: "Participants in the treatment group (M = 82.3, SD = 7.1) scored significantly higher than controls (M = 74.6, SD = 8.4), t(98) = 4.72, p < .001, d = 0.99, 95% CI [4.54, 11.13]."
- ANOVA: "There was a significant main effect of condition, F(2, 87) = 12.34, p < .001, Ξ·Β² = 0.22."
- Regression: "Hours studied predicted exam score, Ξ² = 3.21, SE = 0.54, t(47) = 5.94, p < .001, 95% CI [2.13, 4.29]."
CONSORT Statement
The CONSORT (Consolidated Standards of Reporting Trials) statement is the gold standard for reporting randomized controlled trials (RCTs).
DfCONSORT Checklist Items
- Title and abstract β identify as randomized
- Background β scientific rationale
- Objectives β specific hypotheses
- Trial design β parallel, factorial, etc.
- Participants β eligibility criteria, setting
- Interventions β precisely defined
- Outcomes β primary, secondary, when assessed
- Sample size β how determined
- Randomization β sequence generation, allocation concealment
- Blinding β who was blinded, how
- Statistical methods β pre-specified
- Participant flow β CONSORT diagram
STROBE Statement
The STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guidelines apply to cohort, case-control, and cross-sectional studies.
STROBE Key Items
- Clearly define study design (cohort, case-control, cross-sectional)
- Describe setting β locations, dates, eligibility criteria
- Report variables β definitions, coding, measurement methods
- Present descriptive data β participant characteristics by group
- Report main results with CIs and adjustment for confounders
- Discuss limitations β bias, confounding, generalizability
- Address funding sources and role of sponsors
PRISMA Statement
The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines ensure transparent reporting of systematic reviews.
DfPRISMA Flow Diagram
The PRISMA flow diagram tracks: (1) Identification β records from databases and registers, (2) Screening β records removed and reasons, (3) Eligibility β full-text assessed, (4) Included β studies in qualitative synthesis and quantitative meta-analysis.
Effect Size Reporting
Common Effect Size Measures
Here,
- =Cohen's d (standardized mean difference)
- =Correlation coefficient from t-test
- =Eta-squared (proportion of variance explained)
Effect Size Interpretation (Cohen, 1988)
| Effect Size | Small | Medium | Large |
|---|---|---|---|
| Cohen's d | 0.2 | 0.5 | 0.8 |
| r | 0.1 | 0.3 | 0.5 |
| Ξ·Β² | 0.01 | 0.06 | 0.14 |
Confidence Interval Reporting
Confidence Interval for Mean Difference
Here,
- =Confidence interval
- =Critical t-value for Ξ±/2 with df degrees of freedom
- =Sample variance
- =Sample size
CI Interpretation Common Mistakes
- β "There is a 95% probability the true mean is in this interval" (frequentist CIs do not have probabilistic coverage for a fixed parameter)
- β "If we repeated this study many times, 95% of the constructed intervals would contain the true mean"
- β "The CI shows practical significance" (a narrow CI around a trivial effect is still trivial)
- β Combine CI width with effect size interpretation for practical significance
Python Implementation: APA-Compliant Reporting
APA Statistical Report Generator
import numpy as np
from scipy import stats
import json
class APAReport:
"""Generate APA 7th edition compliant statistical reports."""
@staticmethod
def format_number(value, decimals=2):
"""APA: no leading zero for p-values, effect sizes; zero before decimal for others."""
if abs(value) < 1 and decimals == 2:
return f".{str(round(value, decimals)).split('.')[1]}"
return f"{round(value, decimals)}"
@staticmethod
def t_test_report(group1, group2, group1_name="Group 1", group2_name="Group 2",
alpha=0.05):
"""Generate APA-formatted t-test report."""
n1, n2 = len(group1), len(group2)
m1, m2 = np.mean(group1), np.mean(group2)
sd1, sd2 = np.std(group1, ddof=1), np.std(group2, ddof=1)
# Welch's t-test
t_stat, p_value = stats.ttest_ind(group1, group2, equal_var=False)
df = (sd1**2/n1 + sd2**2/n2)**2 / \
((sd1**2/n1)**2/(n1-1) + (sd2**2/n2)**2/(n2-1))
# Pooled SD for Cohen's d
sp = np.sqrt(((n1-1)*sd1**2 + (n2-1)*sd2**2) / (n1+n2-2))
d = (m1 - m2) / sp
# CI for mean difference
se_diff = np.sqrt(sd1**2/n1 + sd2**2/n2)
t_crit = stats.t.ppf(1 - alpha/2, df)
ci_low = (m1 - m2) - t_crit * se_diff
ci_high = (m1 - m2) + t_crit * se_diff
# Format according to APA
report = {
"descriptive": {
group1_name: {"M": round(m1, 2), "SD": round(sd1, 2), "n": n1},
group2_name: {"M": round(m2, 2), "SD": round(sd2, 2), "n": n2}
},
"test": {
"statistic": f"t({round(df, 1)}) = {round(t_stat, 2)}",
"p_value": f"p = {APAReport.format_number(p_value)}" if p_value >= .001 else "p < .001",
"effect_size": f"d = {APAReport.format_number(d)}",
"ci_95": f"95% CI [{APAReport.format_number(ci_low)}, {APAReport.format_number(ci_high)}]"
},
"narrative": (
f"{group1_name} (M = {round(m1,2)}, SD = {round(sd1,2)}) "
f"{'significantly higher' if m1 > m2 else 'significantly lower'} "
f"than {group2_name} (M = {round(m2,2)}, SD = {round(sd2,2)}), "
f"t({round(df,1)}) = {round(t_stat,2)}, "
f"{'p < .001' if p_value < .001 else f'p = {APAReport.format_number(p_value)}'}, "
f"d = {APAReport.format_number(d)}, "
f"95% CI [{APAReport.format_number(ci_low)}, {APAReport.format_number(ci_high)}]."
)
}
return report
@staticmethod
def anova_report(groups, group_names=None, alpha=0.05):
"""Generate APA-formatted one-way ANOVA report."""
if group_names is None:
group_names = [f"Group {i+1}" for i in range(len(groups))]
f_stat, p_value = stats.f_oneway(*groups)
# Effect size (eta-squared)
all_data = np.concatenate(groups)
grand_mean = np.mean(all_data)
ss_between = sum(len(g) * (np.mean(g) - grand_mean)**2 for g in groups)
ss_total = np.sum((all_data - grand_mean)**2)
eta_sq = ss_between / ss_total
# Report
report = {
"test": f"F({len(groups)-1}, {len(all_data)-len(groups)}) = {round(f_stat, 2)}",
"p_value": f"p = {APAReport.format_number(p_value)}" if p_value >= .001 else "p < .001",
"effect_size": f"Ξ·Β² = {APAReport.format_number(eta_sq)}",
"group_means": {name: round(np.mean(g), 2) for name, g in zip(group_names, groups)}
}
return report
# Example usage
np.random.seed(42)
treatment = np.random.normal(82, 7, 50)
control = np.random.normal(75, 8, 50)
t_report = APAReport.t_test_report(treatment, control, "Treatment", "Control")
print("=== APA T-Test Report ===")
print(t_report["narrative"])
print(json.dumps(t_report["test"], indent=2))
print("\n=== APA ANOVA Report ===")
g1 = np.random.normal(100, 15, 30)
g2 = np.random.normal(108, 15, 30)
g3 = np.random.normal(112, 15, 30)
anova = APAReport.anova_report([g1, g2, g3], ["Control", "Treatment A", "Treatment B"])
print(anova)
Software Citation
APA Software Citation Format
- R: "Statistical analyses were conducted using R (Version 4.3.1; R Core Team, 2023)."
- Python: "Analyses were performed in Python (Version 3.11; Python Software Foundation, 2023) using NumPy (Van der Walt et al., 2011) and SciPy (Virtanen et al., 2020)."
- SPSS: "Data were analyzed using IBM SPSS Statistics (Version 28.0)."
- JASP: "Bayesian analyses were conducted using JASP (Version 0.17.1)."
Sample Size Justification
Sensitivity Power Analysis
Here,
- =Minimum detectable effect size
- =Total sample size
- =Standard deviation
This answers: "Given our sample size, what is the smallest effect we could detect with 80% power?"
Key Takeaways
Summary: Statistical Reporting Standards
- APA 7th Edition requires effect sizes, confidence intervals, exact p-values, and software citation
- CONSORT ensures transparent reporting of randomized controlled trials with flow diagrams
- STROBE provides structured reporting for observational studies (cohort, case-control, cross-sectional)
- PRISMA standardizes systematic review and meta-analysis reporting
- Effect sizes (d, r, Ξ·Β²) quantify practical significance beyond p-values
- Confidence intervals provide information about estimation precision and practical significance
- Exact p-values (p = .023) are preferred over dichotomous reporting (p < .05)
- Software citation with version numbers ensures computational reproducibility