Performance Optimization Guide

Duration: 75 minutes Level: Advanced Module: fcc.personas.registry, fcc.search.incremental, fcc.compliance.memoization, fcc.observability.metrics Pillar: v1.5.0 Pillar F

This tutorial teaches you how to measure, optimize, and verify the hot paths in an FCC deployment. You will learn the three optimization techniques shipped in v1.5.0 (LRU caching for persona lookup, incremental deltas for search, and SHA-256 memoization for compliance evaluation), the benchmark harness that validated each of them, and how to apply the same profile → identify → cache → verify cycle to your own code.

Why optimize?

By the end of v1.4.x, three hot paths had started scaling poorly:

• Persona lookup. PersonaRegistry.from_yaml_directory re-parsed every YAML file on every call. A CLI command that listed personas three times paid the YAML parse cost three times.
• Semantic search index rebuild. Every persona edit invalidated the entire PersonaSearchIndex, causing O(N) embedding regeneration even for a single-row edit in a 164-persona catalog.
• Compliance audit evaluation. The ComplianceAuditor walked 256+ EU AI Act requirements per audit run even when the underlying persona or evidence had not changed.

Pillar F addresses all three without changing any caller-facing API. The common thread is cache-key discipline: each cache uses a stable, content-derived key (a directory mtime, a SHA-256 of evidence, or an event subscription), so cached entries invalidate precisely when the input changes and never otherwise.

The three techniques

Technique	Where	Key idea
LRU cache	fcc.personas.registry	functools.lru_cache keyed on directory mtime. Cold load remains O(N); warm calls are O(1).
Incremental delta	fcc.search.incremental	Subscribe to PersonaUpdated events; re-embed only the changed row.
Memoization	fcc.compliance.memoization	SHA-256 over (persona_id, requirement_id, evidence_sha); skip the evaluator body on cache hits.

All three publish metrics to fcc.observability.metrics: cache_hits_total, cache_misses_total, cache_evictions_total, and a cache_latency_ms histogram. The metrics are the ground truth for whether an optimization is actually firing.

Profile first, optimize second

The optimization cycle is always: profile → identify → cache → verify. Never skip the profile step — cached answers to uncached questions are indistinguishable from bugs.

Profiling with cProfile

import cProfile
import pstats
from fcc.personas.registry import PersonaRegistry

def workload():
    for _ in range(3):
        PersonaRegistry.from_yaml_directory("src/fcc/data/personas")

profiler = cProfile.Profile()
profiler.enable()
workload()
profiler.disable()

stats = pstats.Stats(profiler)
stats.sort_stats("cumulative").print_stats(10)

Look for the cumulative time inside yaml.safe_load — that is the signal that YAML parsing dominates. A warm second call should spend nearly all its time on the cache lookup and almost none on YAML parsing.

Profiling with pyinstrument

For long-running processes, a sampling profiler like pyinstrument is less intrusive:

from pyinstrument import Profiler

p = Profiler()
p.start()
# ... run your workload ...
p.stop()
print(p.output_text(unicode=True, color=False))

Sampling profilers show call-stack trees weighted by wall time, which is what you care about for user-visible latency.

Identify: the benchmark harness

v1.5.0 ships publications/scripts/benchmark_hot_paths.py as a reproducible harness. Run it before you change anything to capture the baseline; run it after to measure the speedup.

python publications/scripts/benchmark_hot_paths.py \
    --runs 50 --warmups 5 --report baseline.json

Typical output for the three hot paths (indicative — measure your own hardware):

Scenario	v1.4.4 median	v1.5.0 median	Speedup
Cold PersonaRegistry.list()	~220 ms	~220 ms	1.0x (no change)
Warm PersonaRegistry.list()	~200 ms	~16 ms	~12x
Single persona edit, full re-index	~1.8 s	~0.4 s	~4.5x
Compliance audit, unchanged persona	~850 ms	~100 ms	~8x

The target was ≥20% improvement on hot-path persona lookups and RAG queries; actual speedups exceeded the gate on every line. Verify your deployment replicates this before you declare victory.

LRU caching the persona registry

The PersonaRegistry.from_yaml_directory classmethod is wrapped in an lru_cache keyed on the tuple (path, dir_mtime). When any YAML under the directory changes, the dir_mtime changes, the lookup key changes, and the next call cold-loads.

from fcc.personas.registry import PersonaRegistry

# First call: full YAML parse
r1 = PersonaRegistry.from_yaml_directory("src/fcc/data/personas")

# Second call: cache hit, milliseconds
r2 = PersonaRegistry.from_yaml_directory("src/fcc/data/personas")

# r1 and r2 are the same registry object — mutating one mutates the other!
assert r1 is r2

The important caveat: cached registries are shared by reference. Callers that mutate the registry break the cache contract. Treat from_yaml_directory output as immutable; if you need to mutate, call registry.copy() and mutate the copy.

Invalidating the cache

Three ways to invalidate:

1. Touch a YAML file. Any write updates dir_mtime → next call cold-loads.
2. Explicit clear — PersonaRegistry.from_yaml_directory.cache_clear(). Useful in tests.
3. Process restart. The cache lives in the module; a new process starts empty.

Never patch a loaded registry's internal dict. The cache key does not track in-memory edits; you will see stale data on the next call.

Incremental search indexing

IncrementalSearchIndex subscribes to PersonaUpdated / PersonaAdded / PersonaRemoved events on the event bus. On each event it re-embeds (or deletes) exactly one row instead of rebuilding the entire index.

from fcc.messaging.bus import EventBus
from fcc.messaging.events import Event, EventType
from fcc.search.incremental import IncrementalSearchIndex
from fcc.personas.registry import PersonaRegistry

bus = EventBus()
registry = PersonaRegistry.from_yaml_directory("src/fcc/data/personas")

# Build the index once, subscribe for deltas
index = IncrementalSearchIndex.from_registry(registry)
index.attach_to_bus(bus)

# Publish a persona update — the index handles it incrementally
bus.publish(Event(
    event_type=EventType.PERSONA_UPDATED,
    source="CLI",
    payload={"persona_id": "RC", "persona": registry.get("RC").to_dict()},
))

# Query the index immediately; the update is reflected
hits = index.search("research", k=5)

Under the hood: the subscriber calls index.upsert(persona), which re-embeds the changed row and updates the search tree in place. A single upsert is O(log N) rather than O(N); for 164 personas that is roughly 4.5x faster than a full rebuild.

When incremental is wrong

If you change your embedding provider (e.g. swap MiniLM for a different model), incremental upserts mix old and new embeddings — which wrecks similarity scores. In that case, do a full rebuild:

index = IncrementalSearchIndex.from_registry(registry)   # fresh full rebuild

Do not rely on the bus to surface the model change — the bus is for data mutations, not code mutations.

Memoizing compliance evaluation

memoize_evaluation is a decorator that wraps an evaluator function. The cache key is a SHA-256 over (persona_id, requirement_id, evidence_sha). When any input changes the hash changes and the evaluator runs; otherwise the cached result is returned and the evaluator body is skipped.

from fcc.compliance.auditor import ComplianceAuditor
from fcc.compliance.memoization import memoize_evaluation
from fcc.personas.registry import PersonaRegistry

registry = PersonaRegistry.from_yaml_directory("src/fcc/data/personas")
auditor = ComplianceAuditor.from_registry(registry)

# First audit on RC — full 256-requirement walk
r1 = auditor.evaluate_persona("RC")

# Second audit on RC, nothing changed — fully cached
r2 = auditor.evaluate_persona("RC")

assert r1.evidence_sha == r2.evidence_sha

The decorator lives on the evaluator, not the auditor entry point, so partial invalidation works: changing one requirement's evidence invalidates only the cache entries keyed to that requirement.

Evidence SHA discipline

The evidence_sha is SHA-256 of the canonical JSON representation of the persona's evidence bundle. If your evidence includes timestamps or nonces, the SHA changes on every call and the cache never hits — a common source of "why did my cache not work" bug reports. Normalize your evidence bundle before hashing: sort keys, drop volatile fields, round timestamps to the day.

Verifying: the metrics layer

All three optimizations publish to fcc.observability.metrics. After a workload, dump the snapshot to see whether caches are firing:

from fcc.observability.metrics import fcc_metrics

snapshot = fcc_metrics.snapshot()
for key in ("cache_hits_total", "cache_misses_total",
            "cache_evictions_total", "cache_latency_ms"):
    print(f"  {key}: {snapshot.get(key)}")

A healthy snapshot for a workload that should be cached:

cache_hits_total:      1243
cache_misses_total:    164     (one per persona, first time)
cache_evictions_total: 0
cache_latency_ms:      {p50: 0.8, p95: 2.4, p99: 5.1}

A red-flag snapshot:

cache_hits_total:      0
cache_misses_total:    1407    (every call missed!)

Zero hits means your cache key is wrong — evidence is drifting, or the key function is seeing fresh inputs on every call. Dump the key function's output across calls to find the rogue field.

A profile → cache → verify cycle end-to-end

Walk through a complete optimization attempt on a CLI command that runs slow:

# 1. Reproduce and measure baseline
import time
from fcc.personas.registry import PersonaRegistry

def workload():
    for _ in range(5):
        r = PersonaRegistry.from_yaml_directory("src/fcc/data/personas")
        _ = [p for p in r.all_personas() if p.category == "core"]

# 2. Baseline
t0 = time.perf_counter()
workload()
baseline = time.perf_counter() - t0
print(f"baseline: {baseline*1000:.1f} ms")

# 3. Clear the registry cache to simulate cold
PersonaRegistry.from_yaml_directory.cache_clear()

# 4. Warm-vs-cold comparison
t0 = time.perf_counter()
PersonaRegistry.from_yaml_directory("src/fcc/data/personas")
cold = time.perf_counter() - t0

t0 = time.perf_counter()
for _ in range(100):
    PersonaRegistry.from_yaml_directory("src/fcc/data/personas")
warm = (time.perf_counter() - t0) / 100
print(f"cold: {cold*1000:.1f} ms   warm: {warm*1000:.2f} ms")

# 5. Check the metrics prove it
from fcc.observability.metrics import fcc_metrics
snap = fcc_metrics.snapshot()
print(f"cache hit ratio: "
      f"{snap['cache_hits_total'] / max(1, snap['cache_misses_total']):.1f}:1")

Expected output: cold load in ~200 ms, warm load in ~2 ms, cache-hit ratio roughly 100:1. That ratio is the certification that the optimization landed.

Tradeoffs and anti-patterns

Tradeoff: memory. Caches hold data. The registry cache is one registry per unique directory path; the search index is one embedding per persona; the compliance memoizer is one result per (persona, requirement, evidence_sha). For a 164-persona, 256-requirement deployment that is ~42,000 entries — manageable. Alarm when cache_size_bytes exceeds your process's memory budget.

Tradeoff: staleness. Caches add a window where stale data is returned. For persona lookups, the window is bounded by directory mtime — a touch on any file invalidates. For compliance, the window is bounded by evidence_sha — which is only as tight as your evidence bundle's normalization.

Anti-pattern: time-based TTLs. Do not add a TTL to any of the three caches. TTLs are a hedge against unknown invalidation conditions. The FCC caches use content-derived keys precisely because content-derived keys are correct; adding a TTL says "we are not confident the key is right."

Anti-pattern: patching cached objects. Cached registries are shared by reference. Mutating one mutates every caller holding the same reference. Treat the output as immutable and copy before mutating.

Anti-pattern: caching non-deterministic evaluators. If the evaluator consults the clock or a random seed, caching returns stale answers indistinguishable from correct ones. Memoize only pure functions of their explicit inputs.

CI integration

Bake the benchmark into CI so regressions are caught at PR time:

# ci/benchmark.sh
set -e
python publications/scripts/benchmark_hot_paths.py \
    --runs 30 --warmups 3 --report /tmp/current.json

python scripts/compare_benchmarks.py \
    --baseline ci/baselines/hot_paths.json \
    --current /tmp/current.json \
    --tolerance 0.20      # fail if >20% slower

compare_benchmarks.py emits a table diff and exits non-zero if any scenario is slower than the tolerance. Commit the baseline after a PR that legitimately slows a path so the gate shifts with reality.

Summary

In this tutorial you learned how to:

• Profile hot paths with cProfile and pyinstrument
• Apply LRU caching to PersonaRegistry.from_yaml_directory, invalidated by directory mtime
• Subscribe an IncrementalSearchIndex to the event bus for O(log N) persona updates
• Memoize ComplianceAuditor.evaluate_persona with SHA-256 keys over (persona_id, requirement_id, evidence_sha)
• Verify cache behavior with fcc_metrics.snapshot()
• Run the benchmark_hot_paths.py harness for reproducible before/after numbers
• Avoid the four common anti-patterns: TTLs, patched cached objects, nondeterministic evaluators, and unmeasured optimization

Next steps

• Commit a baseline benchmark to your CI for each persona edit
• Alarm on cache_hit_ratio below 0.8 in production observability
• Extend the incremental pattern to your own indices using the fcc.search.incremental decorator

Related tutorials

• GraphRAG Deep Dive — benefits from cached persona lookup
• CRDT Multi-User Collaboration — another Pillar F consumer
• Benchmark Interpretation Guide — reading CLEAR+ output
• CLEAR Benchmarking Guide — the CLEAR+ framework

Related references

• ADR-010 Six pillars architecture — docs/decisions/ADR-010_six_pillars_architecture.md
• Six Pillars overview — publications/chapters/arch/six-pillars.qmd
• Performance chapter stub — publications/chapters/ops/performance.qmd
• Source — src/fcc/personas/registry.py, src/fcc/search/incremental.py, src/fcc/compliance/memoization.py
• Metrics — src/fcc/observability/metrics.py