R-CPV Reconciliation Baseline¶

R-CPV — Research Corpus Parity Value — is the ecosystem's shared measure of how a project's local vocabulary corpus compares, in size and distinctiveness, against the canonical research corpus. This document records FCC's read-side reconciliation baseline: where the authoritative R-CPV values come from, how the measures are defined, what tolerance FCC reconciles against, and what is (and is not) in scope for the FCC-side corpus.

Scope note (v1.9.0). This is a baseline / paper adoption. It documents the contract and the measures and pins the FCC-side corpus in scope. The executable reconcile (a CLI/CI step that fetches the live values and compares them against FCC's computed values) lands in a later phase of the ecosystem re-sync — it is not wired up in v1.9.0.

Authority¶

The Research Center (NEXUS) is the sole R-CPV authority. FCC mirrors and reconciles against it; FCC never authors R-CPV semantics. NEXUS publishes the authoritative corpus-insights surface; every consumer (FCC, CTO, and others) computes its own local value and reconciles it against the NEXUS source.

Live data source¶

The authoritative values come from the NEXUS vocabulary-insights endpoint:

GET http://localhost:2430/v1/kb/vocabulary/insights?kind=&top=

The response payload is:

{
  "num_documents": 0,
  "vocabulary_size": 0,
  "corpus_diversity": 0.0,
  "corpus_type_token_ratio": 0.0,
  "mean_distinctiveness": 0.0,
  "top_idf_terms": []
}

Important operational caveat. RC runs on :2430 as a bare gateway in normal operation, and the /v1/kb/vocabulary/insights endpoint is served only when RC is started with make uat-serve — it is not available on the bare gateway. Any reconcile step must treat the endpoint as optionally-present (see the offline policy below), not as a guaranteed surface.

Measure definitions¶

The R-CPV payload fields are defined as follows (authored by NEXUS; FCC mirrors these definitions verbatim so its local computation matches):

IDF (inverse document frequency) — smoothed:

idf(term) = ln((1 + N) / (1 + df(term))) + 1

where N = num_documents and df(term) = the number of documents containing the term. top_idf_terms is the ranked list of the highest-IDF terms (the rarest, most distinctive terms in the corpus).

corpus_diversity — Pielou evenness: the corpus's Shannon entropy normalised by the maximum possible entropy (ln(vocabulary_size)), yielding a value in [0, 1] where 1 is a perfectly even term distribution.
mean_distinctiveness — the tf-weighted mean IDF: the average IDF across the corpus weighted by term frequency, summarising how distinctive the corpus's terms are overall.
corpus_type_token_ratio — vocabulary_size / total_tokens, the ratio of distinct types to total tokens (a standard lexical-diversity measure).

Reconciliation tolerance and offline policy¶

Tolerance: ±5%. A reconciled FCC value is considered "in parity" when it falls within ±5% of the NEXUS authoritative value for the corresponding measure.
RC-unreachable is always "skipped". FCC is a Tier-1 hermetic framework: when NEXUS is not reachable (no :2430, the uat-serve-only insights endpoint not served, or any connection error), the reconcile result is skipped — it never fails. Offline operation is a first-class, non-failing state; the reconcile is advisory, not a gate.

FCC-side corpus in scope¶

The FCC-side corpus for R-CPV reconciliation is FCC's packaged vocabulary-mapping files under src/fcc/data/objectmodel/ — 34 YAML files, ~298 mapping entries (24 of the files carry mapping lists; the rest are patterns/examples/best-practice docs) as of v1.9.0. These mappings are the cross-project entity-resolution surface FCC ships (consumed via the VocabularyProviderPlugin pattern); they are the corpus over which FCC computes its local IDF / diversity / distinctiveness values when the executable reconcile arrives.

Worked example (CTO)¶

As a cross-project reference point, CTO computes cto.ecosystem.rcpv = 5851 as its own corpus parity value against the same NEXUS authority. CTO's value is cited here purely as a worked example of a consumer reconciling against the R-CPV surface — FCC's own computed value (over its ~298 mapping entries) is a separate quantity that the later executable reconcile phase will produce and pin.

What lands when¶

Capability	Status in v1.9.0
R-CPV authority identified (NEXUS)	Documented (this doc)
Live endpoint + payload contract	Documented (this doc)
Measure definitions (IDF / Pielou / distinctiveness)	Documented (this doc)
±5% tolerance + always-skip-when-offline policy	Documented (this doc)
FCC-side corpus in scope (34 mapping files, ~298 entries)	Pinned (this doc)
Executable reconcile (fetch → compute → compare → report)	Deferred to a later phase