Peer Review — Results¶
A results peer review evaluates the output of a completed experiment or study: internal validity, external validity, effect size, uncertainty quantification, and the fit between claims and evidence. This template complements the methodology review (OPEN-SCI-005a); both must be present for an FCC Critique to clear the open-science gate. Produce this artifact during the Critique phase after results are generated and before findings are shared with the Research Center ecosystem or external partners.
Template¶
Section 1: Review Metadata¶
Instructions: Link the results review to its paired methodology review and to the original preregistration — the chain lets auditors verify that what was planned is what was reported.
| Field | Value |
|---|---|
| Review ID | [FILL — e.g. REV-RES-2026-001] |
| Reviewer | [FILL] |
| Date | [FILL] |
| Linked methodology review (OPEN-SCI-005a) | [FILL] |
| Preregistration (OPEN-SCI-001) | [FILL] |
Section 2: Artifact Under Review¶
Instructions: Identify the experiment or study, authors, result location, and the date results were generated. Results produced on different dates than reported are a P1 finding.
| Field | Value |
|---|---|
| Experiment / study name | [FILL] |
| Authors | [FILL] |
| Result location (repo + path) | [FILL] |
| Date results generated | [FILL] |
Section 3: Statistical Validity¶
Instructions: Every box must be ticked or explicitly waived. An unticked-and-unexplained box is a blocking concern.
- Sample size meets preregistered power requirements
- Statistical tests match those declared in the analysis plan
- Effect sizes reported alongside p-values
- Confidence intervals provided
- Multiple-comparison corrections applied where needed
- Assumptions of each test checked and reported
Section 4: Internal & External Validity¶
Instructions: Internal validity asks "does the experiment support its own claim?"; external validity asks "does that claim generalise?". Both need explicit scoring.
- Internal validity (confounders controlled, reverse-causal ruled out):
[Pass / Partial / Fail — evidence] - External validity (population / setting generalisation):
[Pass / Partial / Fail — evidence] - Construct validity (measurements reflect the concept):
[Pass / Partial / Fail — evidence]
Section 5: Effect Size & Uncertainty Quantification¶
Instructions: Quantitative work must report effect sizes and uncertainty — not just p-values. Report primary hypothesis outcomes in the standard schema below.
| Hypothesis | Result | Effect size | p-value / CI | Supported? |
|---|---|---|---|---|
[FILL] |
[FILL] |
[FILL] |
[FILL] |
[Yes / No / Partial] |
- Secondary results:
[FILL] - Negative / null results:
[FILL — explicit reporting, not suppression]
Section 6: Interpretation & Decision¶
Instructions: Confirm that conclusions match the evidence, causal language matches the study design, and generalisability claims are appropriate. Record decision, confidence, and whether a follow-up review is required.
- Conclusions supported by data:
[Yes / No] - Causal language appropriate for design:
[Yes / No] - Generalisability claims appropriate:
[Yes / No] - Potential issues (severity / blocking?):
[FILL] - Decision:
[Accept / Revise / Reject] - Confidence:
[High / Medium / Low]
Adoption Checklist¶
- All required sections completed
- Artifact peer-reviewed by at least one R.I.S.C.E.A.R. peer
- Stored in the project's designated docs location
- Linked from README or equivalent index
- Versioned + date-stamped, paired with its OPEN-SCI-005a methodology review
References¶
- PHOENIX v4.0.0 —
docs/resources/templates/open-science/peer-review-results.md - ICML 2025 — Towards Transparent Peer Review, Position Paper
- Center for Open Science — Open Science Lifecycle
- Ioannidis, J. P. A. (2005) — Why most published research findings are false, PLoS Medicine 2(8)
- Cumming, G. (2014) — The New Statistics: Why and How, Psychological Science 25(1)
FCC integration¶
This template is referenced from the Forensic Auditor persona
(src/fcc/data/personas/forensic_auditor.yaml) as part of the
Critique-phase evidence set. The auditor pairs each results review with
its OPEN-SCI-005a methodology review and verifies that reported
outcomes match the preregistered hypothesis set. See also
src/fcc/data/governance/critique_protocol.yaml and
src/fcc/data/governance/open_science_gates.yaml.