Skip to content

Reproducibility with FCC

How to use the FCC Agent Team Framework to ensure reproducible research workflows through deterministic simulation, trace recording, event replay, and provenance tracking.

Why Reproducibility Matters

Reproducibility is a cornerstone of scientific research. FCC provides several mechanisms to ensure that workflows can be exactly replicated:

  1. Deterministic simulation mode: The mock engine produces identical results given the same inputs.
  2. Trace recording: Every step is captured in a structured trace.
  3. Event replay: Events can be serialized, stored, and replayed.
  4. Provenance tracking: Observability spans record timing and attributes.

Deterministic Simulations

Running Reproducible Simulations

from fcc.personas.registry import PersonaRegistry
from fcc.simulation.engine import SimulationEngine
from fcc.scenarios.loader import ScenarioLoader

registry = PersonaRegistry.from_package_data()
scenarios = ScenarioLoader.from_package_data()
scenario = scenarios.get("basic_fcc_cycle")

# Mock mode is fully deterministic
engine = SimulationEngine(registry=registry, mode="mock")

# Run twice -- should produce identical traces
trace1 = engine.run(scenario)
trace2 = engine.run(scenario)

# Verify determinism
for s1, s2 in zip(trace1.steps, trace2.steps):
    assert s1.persona_id == s2.persona_id
    assert s1.phase == s2.phase
    assert s1.summary == s2.summary

print("Traces are identical -- deterministic!")

Seeded AI Simulations

When using AI-powered mode, set a seed for reproducibility:

engine = SimulationEngine(
    registry=registry,
    mode="ai",
    seed=42,  # Fixed seed for reproducibility
)

Trace Recording and Replay

Saving Traces

import json
from fcc.simulation.engine import SimulationEngine

engine = SimulationEngine(registry=registry, mode="mock")
trace = engine.run(scenario)

# Serialize trace to JSON
trace_data = {
    "scenario_id": trace.scenario_id,
    "steps": [
        {
            "phase": step.phase,
            "persona_id": step.persona_id,
            "summary": step.summary,
        }
        for step in trace.steps
    ],
    "duration_ms": trace.duration_ms,
}

with open("trace_output.json", "w") as f:
    json.dump(trace_data, f, indent=2)

Event Replay

Record and replay the complete event stream:

from fcc.messaging.bus import EventBus
from fcc.messaging.serialization import EventSerializer, EventReplay

bus = EventBus()
serializer = EventSerializer()

# Capture events during a run
captured = []
bus.subscribe_all(lambda e: captured.append(serializer.serialize(e)))

# ... run simulation with this bus ...

# Save events
with open("events.json", "w") as f:
    json.dump(captured, f, indent=2)

# Later: replay events
replay_bus = EventBus()
replay = EventReplay(replay_bus)

with open("events.json") as f:
    saved_events = json.load(f)

for event_data in saved_events:
    event = serializer.deserialize(event_data)
    replay.replay(event)

Provenance with Observability

Span-Based Provenance

Use tracing spans to record the full provenance of each operation:

from fcc.observability.tracing import FccTracer
from fcc.observability.exporters import JsonFileSpanExporter

tracer = FccTracer()
tracer.add_exporter(JsonFileSpanExporter(output_dir="./provenance"))

with tracer.start_span("experiment.run") as span:
    span.set_attribute("experiment_id", "exp_001")
    span.set_attribute("dataset", "dataset_001")
    span.set_attribute("method", "bayesian_analysis")
    span.set_attribute("seed", 42)

    # Nested spans for sub-operations
    with tracer.start_span("data.preprocessing") as sub:
        sub.set_attribute("rows_processed", 10000)
        # ... processing ...

    with tracer.start_span("model.training") as sub:
        sub.set_attribute("epochs", 100)
        # ... training ...

Provenance Metadata

Attach provenance metadata to knowledge graph nodes:

from fcc.knowledge.graph import KnowledgeGraph

kg = KnowledgeGraph()

kg.add_node("result_001", node_type="RESULT", metadata={
    "experiment_id": "exp_001",
    "timestamp": "2026-03-30T10:00:00Z",
    "seed": 42,
    "framework_version": "1.0.1",
    "trace_file": "trace_output.json",
    "events_file": "events.json",
    "provenance_dir": "./provenance",
})

Reproducibility Checklist

  • Use deterministic (mock) mode for testing
  • Set seeds for AI-powered simulations
  • Record simulation traces to JSON
  • Capture and save event streams
  • Enable span-based provenance tracking
  • Attach provenance metadata to knowledge graph nodes
  • Version-lock framework dependencies (pip freeze > requirements.txt)
  • Store all configuration as code (YAML, JSON)
  • Use Git for version control of all inputs and outputs