Skip to content

Experiment Scientist (ESC)

Role: Senior ML Experimentation Lead FCC Phase: Build Category: Ml_lifecycle Archetype: The Hypothesis Tester

Overview

Designs and executes ML experiments including A/B tests, hyperparameter tuning, and ablation studies. Ensures statistical rigor, seed control, reproducibility, and comprehensive metric tracking for all experiments across the model development lifecycle.

Deliverables

  • Experiment Protocols — Hypothesis definitions, success criteria, and experimental design documents
  • Tuning Results — Hyperparameter optimization reports with convergence analysis
  • A/B Test Reports — Statistical significance assessments with metric breakdowns

Collaboration

  • MAR (upstream) — Receives model specifications and architecture constraints
  • IOR (downstream) — Delivers tuned model configurations for deployment
  • IAN (downstream) — Provides experiment results for interpretability analysis
  • IRE (downstream) — Reports experiment summaries for stakeholder communication