Skip to content

FCC Evidence Artifacts

Every compliance claim is only as good as its evidence. From v1.3.8, FCC ships 12 canonical evidence templates under docs/resources/templates/open-science/. Each template is vendored from an authoritative source (OSF, AAAI-26, Mitchell et al., Gebru et al., Stanford HAI) and adapted for FCC-style deployments.

This guide explains when to require each template, how to score an adequately-completed template, and how the templates line up against EU AI Act and NIST AI RMF requirements.

The 12 Templates at a Glance

ID Template Source FCC Phase Maps To
OPEN-SCI-001 Experiment Preregistration OSF/COS + AAAI-26 Find NIST MS-1, EU Art 10
OPEN-SCI-002 FAIR Self-Assessment FAIR Principles (Wilkinson et al.) Find EU Art 10, NIST MP-4
OPEN-SCI-003 Reproducibility Checklist AAAI-26 Critique EU Art 15, NIST MS-2
OPEN-SCI-004a ML Model Card Mitchell et al. 2019 Critique EU Art 11, NIST MG-2
OPEN-SCI-004b Dataset Card Gebru et al. 2021 Find EU Art 10, NIST MP-4
OPEN-SCI-005a Peer Review (Methodology) Project template Critique EU Art 15(1), NIST MS-2
OPEN-SCI-005b Peer Review (Results) Project template Critique EU Art 15(1), NIST MS-2
OPEN-SCI-006 Open Access Compliance Plan S / NIH Critique EU Art 13, NIST MG-3
OPEN-SCI-007 Ethical Review Common Rule 45 CFR 46 Find EU Art 5, NIST GV-5
OPEN-SCI-008 Architecture Decision Record Nygard/Michael Create EU Art 11, NIST GV-2
OPEN-SCI-009 Agent Transparency Card Stanford HAI + CRFM Create EU Art 13, NIST MP-1
OPEN-SCI-010 Data Management Plan NIH / NSF / Horizon Europe Find EU Art 10, NIST MP-4

Open each template at docs/resources/templates/open-science/OPEN-SCI-XXX_*.md to see the authoritative form.

When to Require Each Template

Always Required

For every high-risk deployment:

  • OPEN-SCI-004a — ML Model Card for every deployed model
  • OPEN-SCI-009 — Agent Transparency Card for every deployed agent persona
  • OPEN-SCI-008 — ADRs for every major design decision

Required for Research Claims

For any deployment that produces quantitative claims:

  • OPEN-SCI-001 — before data collection
  • OPEN-SCI-003 — before publication
  • OPEN-SCI-005a/b — before acceptance

Required for Data Handling

For any deployment that sources, processes, or releases data:

  • OPEN-SCI-002 — FAIR self-assessment
  • OPEN-SCI-004b — Dataset card for each dataset
  • OPEN-SCI-010 — Data management plan

Required for Regulated Domains

For healthcare, finance, government, or grant-funded work:

  • OPEN-SCI-006 — Open access compliance
  • OPEN-SCI-007 — Ethical review

Most deployments will need 6-9 of the 12 templates. Very few will need all 12; even fewer will be exempt from any.

Scoring Adequacy

An evidence template is "adequate" when it is:

  1. Complete — every required field is filled with non-placeholder content
  2. Current — dated within the engagement's evidence cut-off window
  3. Attributed — the author, reviewer, and approver are identified
  4. Consistent — the facts stated match the deployed system and the other evidence artefacts
  5. Accessible — stored under version control or in the evidence repository, not in a private drive

Score each template on the five dimensions: complete / current / attributed / consistent / accessible. A template that scores 5/5 counts as full evidence. A template that scores ≤3/5 is inadmissible and the audit finding should say so.

The Evidence-Graph View

When fcc compliance-audit runs, it builds an evidence graph that links every compliance requirement to the template (or code artefact, or test, or model card) that satisfies it. The graph is addressable in JSON-LD and can be imported into any RDF-compatible tool.

fcc compliance-audit --evidence-graph /audit/evidence.jsonld

A finding of "requirement R satisfied by evidence E" in the graph is only valid if E is an adequate template. Auditors spot-check the edge by opening E and running the five-dimension scoring.

Template Walkthrough — OPEN-SCI-009 Agent Transparency Card

OPEN-SCI-009 is the single most important evidence artefact for an FCC deployment because it documents what the deployed agent persona actually does. Every deployed persona should have one; auditors should spot-check at least 20% of deployed personas.

The required sections:

  1. Identity — persona ID, name, version, deployment context
  2. Capabilities — what the persona can do (mapped to R.I.S.C.E.A.R. Expected Output)
  3. Limitations — what the persona cannot do (mapped to R.I.S.C.E.A.R. Constraints)
  4. Training data — references for the underlying model
  5. Intended use — authorised use cases
  6. Out-of-scope use — explicitly forbidden use cases
  7. Risk category — from AIActClassifier
  8. Monitoring — which events fire, which metrics track
  9. Human oversight — how a human intervenes
  10. Contact — who is accountable

An OPEN-SCI-009 card with fewer than 10 complete sections is incomplete.

Template Walkthrough — OPEN-SCI-004a ML Model Card

OPEN-SCI-004a follows Mitchell et al. 2019. FCC auto-generates 173 model cards (147 personas + 6 workflows + 20 categories) and regenerates them on every release. Auditors should:

  1. Open docs/model-cards/ for the deployment's git rev
  2. Spot-check 5-10 cards across risk categories
  3. Verify that the performance section reflects the latest CLEAR+ benchmark run
  4. Confirm the evaluation dataset matches the dataset card (OPEN-SCI-004b)

Drift between the model card and the live deployment is a finding. The card should be regenerated on every model or persona change.

Template Walkthrough — OPEN-SCI-003 Reproducibility Checklist

Follow the AAAI-26 / NeurIPS reproducibility checklist. For FCC deployments this reduces to:

  • The git rev of the deployed FCC instance is published
  • The Helm chart version is published
  • The random seed used for each simulation is recorded
  • The persona YAML is checked in under version control
  • A reproduction script is provided
  • Dataset licences are stated

FCC's deterministic simulation mode makes these easy to satisfy; the finding pattern here is usually "we have the artefact but did not publish it."

Evidence Freshness

Evidence must be fresh:

Template Refresh cadence
OPEN-SCI-001 Per experiment
OPEN-SCI-002 Annually or per major data release
OPEN-SCI-003 Per publication
OPEN-SCI-004a/b Per model/dataset change
OPEN-SCI-005a/b Per peer review
OPEN-SCI-006 Annually
OPEN-SCI-007 Per ethics board review
OPEN-SCI-008 Per design decision
OPEN-SCI-009 Per persona deployment change
OPEN-SCI-010 Per grant cycle

Stale evidence is not evidence. Findings should flag any template whose date is outside its refresh cadence.

Evidence Diff Protocol

When an FCC deployment is re-audited after a release bump:

  1. diff the evidence graph between the previous and current audit
  2. For every removed edge, verify that the requirement is re-satisfied elsewhere
  3. For every added edge, verify that the new evidence is adequate
  4. For every changed edge, verify that the change is an improvement, not a regression

Treat an evidence-graph diff as a standalone audit artefact; it is often the most compact record of "what changed" for senior reviewers.

Where Evidence Lives

Evidence kind Canonical location
Persona YAML src/fcc/data/personas/
Model cards docs/model-cards/
Quality gate records src/fcc/data/governance/*.yaml
Compliance requirements src/fcc/data/compliance/*.yaml
OPEN-SCI templates (filled) docs/resources/evidence/<engagement-id>/ (operator convention)
CLEAR+ benchmark results publications/_output/benchmarks/ (or operator's CI store)
Audit reports engagement record (operator maintained)

Treat anything outside these locations as unofficial — it may inform findings but cannot be the sole basis for them.