Skip to content

Chapter 32: The Dual-Bus Event Model

New in v1.4.2. Prior to v1.4.2, FCC shipped a single in-process EventBus carrying all 81 event types. As the UX frontend and the Sky Parlour streaming surface matured, two problems emerged: high-volume UX telemetry starved PAOM business-event subscribers, and the two audiences wanted different tracing granularity. This chapter documents the v1.4.2 dual-bus response: two physical EventBus instances — PAOMBus and UXBus — routed by a LaneRouter and instrumented by OpenTelemetry with per-lane tracer providers.

Why two buses

The original EventBus (Chapter 7) is a thread-safe in-process pub/sub with filtering, serialization, and replay. It is correct and has carried every FCC event for 12 minor releases. What it is not is lane-aware: every subscriber sees every event, and back-pressure from one loud category affects unrelated subscribers.

Two use-cases made this hurt:

  1. UX telemetry. The React frontend, the Streamlit dashboards, and the Sky Parlour streaming surface all emit events at roughly 10-100x the rate of the PAOM business workflow. A governance subscriber watching for persona validation failures should not have to filter through a firehose of scroll events.
  2. OTEL tracer granularity. Operators want one tracer provider for business-critical spans (PAOM) and a separate one for UX-interaction spans they can sample more aggressively. A single EventBus forces one tracer for both, which over-indexes sampling on the wrong axis.

The v1.4.2 answer is to split the bus physically. Each bus keeps the exact same API as the pre-v1.4.2 EventBus — the changes are in routing and tracing, not in the subscriber contract.

Lane classification

Every one of the 81 event types is classified into exactly one lane. The classification is a static YAML at src/fcc/data/messaging/lane_classification.yaml and is loaded at bus construction time.

Lane Event families Rate target Retention
PAOM persona., workflow., action., governance., compliance., scenario., simulation.* low volume, high value permanent
UX ui., streaming., dashboard., collaboration_live., notification.* high volume, low durability rolling 1h

Events not listed in either family default to PAOM — the bias is toward durability. Operators who add new event types must classify them explicitly; unclassified events raise a startup-time error under the strict mode enabled in CI.

The LaneRouter

The LaneRouter sits in front of both buses and has exactly one responsibility: given an outbound Event, deliver it to the correct bus. Publishers never touch the router directly; they publish to what looks like the old EventBus interface, and the router interposes transparently.

flowchart LR
    P[Publisher<br/>persona, engine, CLI] --> SH[Back-compat shim<br/>EventBus.publish]
    SH --> LR{LaneRouter}
    LR -->|PAOM lane| PB[PAOMBus]
    LR -->|UX lane| UB[UXBus]

    PB --> PS1[PAOM subscribers<br/>governance, compliance]
    PB --> PT[OTEL tracer<br/>fcc.paom]
    UB --> US1[UX subscribers<br/>dashboards, frontend]
    UB --> UT[OTEL tracer<br/>fcc.ux]

    classDef pub fill:#e3f2fd,stroke:#0d47a1;
    classDef route fill:#fff9c4,stroke:#f57f17;
    classDef bus fill:#c8e6c9,stroke:#2e7d32;
    classDef otel fill:#f8bbd0,stroke:#880e4f;
    class P,SH pub;
    class LR route;
    class PB,UB bus;
    class PT,UT otel;

Routing is O(1) per event: the classification is a dict lookup keyed by event_type. On mismatch (unclassified event in strict mode), the router raises UnclassifiedEventError at construction time, not at publish time.

Back-compatibility shim

Every caller that still imports fcc.messaging.bus.EventBus continues to work. The module exports a shim class whose publish() method delegates to the router. The shim also exposes the legacy subscribe() surface, which fans out to both PAOMBus and UXBus — so legacy subscribers still see every event unless they opt into a lane explicitly.

New code should import fcc.messaging.lanes.PAOMBus or fcc.messaging.lanes.UXBus directly and subscribe to only the lane it cares about. This is the idiomatic form after v1.4.2.

Migration is a one-line change for most subscribers:

# Before v1.4.2
from fcc.messaging.bus import EventBus
bus = EventBus()
bus.subscribe(filter, handler)

# After v1.4.2 (idiomatic)
from fcc.messaging.lanes import PAOMBus
bus = PAOMBus()
bus.subscribe(filter, handler)

There is no urgent deadline to migrate — the shim is permanent. The benefit of opting into a specific lane is reduced noise in the subscriber and (on UXBus) more aggressive tracing sample rates.

OpenTelemetry instrumentation

v1.4.2 also wires the OpenTelemetry SDK in at the lane level. Each bus gets its own TracerProvider with its own OTLPSpanExporter endpoint and its own sampling policy:

  • fcc.paom — 100% sampling by default, exported to the primary OTel collector.
  • fcc.ux — 10% sampling by default, exported to the same collector on a separate service name so dashboards can split.

Every span emitted by either bus carries four canonical attributes:

  • lane — paom or ux
  • event_type — the dotted event name
  • correlation_id — the cross-span correlation id for the parent workflow
  • zachman_cell — the Zachman 6×6 cell (see Chapter 35) the span participates in, if known

The span-attribute discipline is owned by the OTO (OpenTelemetry Orchestrator) persona, new in v1.4.2. OTO owns the per-lane tracer provider configuration, the OTLP export destinations, and the span-attribute schema. The LAR (Lane-Router Architect) persona owns the classification YAML and the router integrity.

UX REST gateway on port 8901

Alongside the dual-bus split, v1.4.2 brings the UX REST gateway online. The gateway listens on port 8901 (distinct from the 8765 WebSocket bridge and the 8900 admin REST port) and exposes a read-only projection of UX events over HTTP for tools that cannot subscribe to a bus directly. The OpenAPI spec is vendored from the Research Center at src/fcc/data/ecosystem/ux_rest_openapi.yaml and CI drift-tests it against the RC upstream by SHA256.

Typical consumers of the gateway:

  • Browser dashboards that cannot hold a WebSocket connection
  • External monitoring agents polling for health
  • Integration tests that want to assert on UX lane state without subscribing

Physical deployment topology

The dual-bus split is visible in the deployment topology. Both buses live in the backend process — there is no separate service — but the OTel exporter fan-out and the UX REST gateway are independently scalable.

flowchart TB
    subgraph Backend["Backend process (port 8765 WS + 8900 admin + 8901 UX)"]
        R[LaneRouter]
        PB[PAOMBus]
        UB[UXBus]
        R --> PB
        R --> UB
    end

    PB --> TPp[Tracer<br/>fcc.paom]
    UB --> TPu[Tracer<br/>fcc.ux]

    TPp --> OC[(OTel Collector)]
    TPu --> OC

    UB --> GW[UX REST gateway<br/>:8901]
    GW --> EC[External consumers]

    OC --> Ja[(Jaeger / Tempo)]
    OC --> Pr[(Prometheus)]

    classDef proc fill:#e3f2fd,stroke:#0d47a1;
    classDef bus fill:#c8e6c9,stroke:#2e7d32;
    classDef otel fill:#fff9c4,stroke:#f57f17;
    classDef ext fill:#f8bbd0,stroke:#880e4f;
    class R,PB,UB bus;
    class TPp,TPu,OC otel;
    class GW proc;
    class EC,Ja,Pr ext;

Migration guide from v1.4.1 EventBus callers

If your code imported EventBus before v1.4.2, the mechanical migration is:

  1. Audit your subscribers. Use grep -rn 'EventBus' src/ to find every subscription. Each is a candidate for lane-specific migration.
  2. Classify by interest. For each subscriber, check whether it cares about PAOM events, UX events, or both. The lane classification YAML is the authoritative reference.
  3. Change the import. Swap from fcc.messaging.bus import EventBus to from fcc.messaging.lanes import PAOMBus (or UXBus).
  4. Verify sampling. If your subscriber relied on seeing every event for audit, ensure it is on PAOMBus — the UX lane samples at 10% for OTel export (though subscription remains lossless within-process).
  5. Update tests. Unit tests should construct the lane-specific bus directly. The shim is fine for integration tests but a specific bus is clearer for unit-level assertions.

Publishers generally do not need to change. The shim EventBus class is a stable wrapper; it is the recommended surface for plugins that want to stay compatible across the migration window.

Observability discipline

OTO's constitution encodes three rules for span attributes:

  1. Every span carries lane and correlation_id. The two form the minimum viable trace.
  2. zachman_cell is added if and only if it is known statically. Do not infer it at runtime; that produces drift.
  3. Never propagate PII in attributes. Attribute values must survive the default OTLP export path without redaction.

Operators who want to tighten further can add a local SpanProcessor that rejects any span missing any of the four canonical attributes. This is enabled in CI for FCC itself and is recommended for production deployments.

Key takeaways

  • v1.4.2 splits the single EventBus into PAOMBus and UXBus, routed by LaneRouter based on a static classification YAML.
  • A permanent back-compat shim at fcc.messaging.bus.EventBus means no pre-v1.4.2 caller breaks.
  • Each lane has its own OTel tracer provider; spans carry lane, event_type, correlation_id, and zachman_cell attributes.
  • The UX REST gateway on port 8901 exposes UX events over HTTP for consumers that cannot subscribe to a bus.
  • The LAR persona owns the classification; the OTO persona owns the tracing discipline.