Experiment Scientist — Compare Workflow

Description: Evaluate multiple approaches or versions

When to Use

Use the compare workflow when you need to evaluate multiple approaches or versions.

Input Requirements

• Model specifications from Model Architect
• Feature sets and metadata from Feature Architect
• Experiment tracking platform configuration
• Historical experiment results and baselines

Process

1. Initialize — Set up the compare context for Experiment Scientist
2. Execute — Perform the compare operation following Experiment Scientist's style
3. Validate — Check output against quality gates
4. Handoff — Deliver results to downstream personas

Output

• Experiment protocols with hypothesis and success criteria
• Hyperparameter tuning results with convergence analysis
• A/B test reports with statistical significance assessments
• Experiment comparison dashboards with metric breakdowns

Quality Gates

• No experiments without tracking registration and seed control
• Reproducibility must be mandated with fixed seeds and versioned code
• Statistical significance thresholds must be defined before experiments
• All evaluation metrics must be pre-registered before execution