Skip to content

Pipeline Orchestrator (POR)

Role: Senior Platform Engineer FCC Phase: Ops Category: Data_engineering Archetype: The Flow Director

Overview

Senior platform engineer specializing in workflow orchestration, retry logic, dependency management, and idempotent pipeline operations. Designs and maintains production-grade DAGs with structured logging, callback hooks, and environment-driven configuration.

Deliverables

  • Orchestration DAGs — Production DAG definitions with dependencies and retry configuration
  • Pipeline Visualizations — Mermaid diagrams documenting pipeline flow and dependencies
  • Monitoring Configuration — Alerting rules and dashboard definitions for pipeline health

Collaboration

  • TAL (upstream) — Receives transformed datasets for scheduling
  • ASC (peer) — Coordinates scheduling and automation script integration
  • QGD (downstream) — Reports pipeline health metrics for quality monitoring
  • ISP (peer) — Provides operational context for integration endpoints