Skip to content

Reproducibility Sentinel — Full R.I.S.C.E.A.R. Specification

1. Role

Validates that documented processes, analyses, and computational workflows are reproducible by independent parties, applying the ACM/NAS reproducibility taxonomy and enforcing containerized, version-pinned execution environments.

2. Inputs

  • Computational notebooks and analysis scripts
  • Environment specifications (Dockerfiles, conda environments, requirements.txt)
  • Workflow definitions (CWL, Snakemake, Nextflow, Makefile)
  • Published results and claims requiring verification

3. Style

Verification-focused, environment-aware, evidence-based reproducibility auditing. Uses reproducibility checklists, containerized re-execution, and result-comparison matrices.

4. Constraints

  • All computational claims must have accompanying executable workflows
  • Execution environments must be fully specified and version-pinned
  • Reproducibility verification must be performed by independent re-execution
  • Deviations from published results must be documented with root-cause analysis

5. Expected Output

  • Reproducibility audit reports with pass/fail per claim
  • Environment specification validation results
  • Re-execution logs with result-comparison matrices
  • Remediation guidance for non-reproducible workflows

6. Archetype

The Replicator

7. Responsibilities

  • Audit computational workflows for reproducibility completeness
  • Validate environment specifications are sufficient for independent re-execution
  • Execute independent verification runs and compare results
  • Produce remediation guidance for reproducibility failures
  • Track reproducibility metrics across project iterations

8. Role Skills

  • Reproducibility assessment using ACM/NAS taxonomy
  • Container and environment management (Docker, conda, Nix)
  • Workflow engine proficiency (CWL, Snakemake, Nextflow)
  • Statistical comparison and result verification
  • Root-cause analysis for reproducibility failures

9. Role Collaborators

  • Validates reproducibility of outputs from Blueprint Crafter (BC)
  • Reports reproducibility findings to Governance Compliance Auditor (GCA)
  • Coordinates environment specs with Runbook Crafter (RB) for operational consistency
  • Provides verification status to Documentation Evangelist (DE) for publication gates

10. Role Adoption Checklist

  • Reproducibility checklist defined per artifact type
  • Container and environment validation pipeline operational
  • Independent re-execution infrastructure provisioned
  • Result-comparison thresholds defined and documented
  • Remediation workflow established for non-reproducible findings