Skip to content

Open Science -- Phase 15 Addendum

This addendum extends the Open Science Demo and Phase 14 Addendum with Phase 15 FAIR compliance features that leverage the unified knowledge graph for automated evidence collection and reproducibility verification.

FAIR Compliance with Unified KG Evidence

Overview

Phase 15 adds automated FAIR (Findable, Accessible, Interoperable, Reusable) compliance checking backed by the federated knowledge graph. Each FAIR principle is mapped to evidence nodes in the unified KG, enabling automated scoring and gap analysis.

FAIR Principle Mapping

Principle KG Evidence Source Scoring Method
F1 -- Globally unique identifier Namespace nodes + entity IDs Completeness check
F2 -- Rich metadata Persona dimension profiles Attribute coverage ratio
F3 -- Registered in searchable resource SearchIndex entries Index coverage
F4 -- Metadata includes identifier KG node attributes Field presence check
A1 -- Retrievable by identifier Repository protocol API endpoint verification
A2 -- Metadata accessible ModelFacade.get_full() Response validation
I1 -- Formal knowledge representation KG serializers (OWL/RDF/SKOS) Format availability
I2 -- Uses FAIR vocabularies VocabularyMapping coverage Mapping completeness
I3 -- Qualified references Cross-namespace edges Resolution rate
R1 -- Rich attributes Model card completeness Section coverage

Evidence Graph Integration

Automated Evidence Collection

The FAIR compliance checker walks the unified knowledge graph to collect evidence for each principle:

from fcc.knowledge.builders import build_full_fcc_graph

kg = build_full_fcc_graph()

# Each FAIR principle maps to KG node and edge queries
fair_evidence = {}
for principle in FAIR_PRINCIPLES:
    evidence_nodes = kg.query(node_type=principle.evidence_type)
    fair_evidence[principle.id] = {
        "nodes": len(evidence_nodes),
        "coverage": compute_coverage(evidence_nodes, principle),
    }

Evidence Confidence Scoring

Each evidence item carries a confidence score derived from the KG:

Source Base Confidence Adjustment
Direct KG node attribute 0.95 None
Cross-namespace resolved edge 0.80 +/- resolution confidence
Inferred from related nodes 0.60 +/- inference depth penalty
Manual annotation 1.00 None

Reproducibility Verification

Benchmark Reproducibility

Phase 15 connects CLEAR+ benchmarks to FAIR compliance:

  • Benchmark specs are registered as KG nodes (type: BENCHMARK)
  • Results are linked to the specs with provenance edges
  • Reproducibility score is computed from result consistency across runs

Model Card Completeness

Model cards serve as evidence for FAIR Reusability (R1):

Model Card Section FAIR Principle Evidence Weight
Intended Use R1.1 0.25
Limitations R1.2 0.20
Training Data R1.3 0.30
Ethical Considerations R1.1 0.25

Event Integration

FAIR compliance checks emit events through the EventBus:

Event Type Payload
fair.check.started principles, evidence_sources
fair.principle.evaluated principle_id, score, evidence_count
fair.check.completed overall_score, gaps, recommendations

FAIR Dashboard

The FAIR compliance dashboard shows:

  • Radar chart: All 10 FAIR principles with scores
  • Evidence table: Per-principle evidence items with confidence
  • Gap analysis: Missing evidence and recommended actions
  • Trend chart: FAIR scores across project versions

Tips

  • Run FAIR compliance checks after model card generation to capture maximum evidence coverage
  • Use the evidence graph to trace exactly which KG nodes support each FAIR principle score
  • Connect FAIR events to the compliance pipeline for unified reporting