Packaging Guide¶

This guide explains how the FCC Agent Team Framework packages data files, templates, and other non-Python assets into distributable wheels. It covers the architecture behind _resources.py, the pyproject.toml configuration, and how to verify and extend the packaging setup.

Why Data Lives Inside `src/fcc/`¶

Python wheels only include files that are part of a Python package. Prior to v0.2.0, data files lived at the project root in a data/ directory. This worked for editable installs (pip install -e .) because the entire repository was accessible, but it broke wheel-based installs because the data/ directory was not part of the fcc package tree and was excluded from the built wheel.

The solution: move all data into src/fcc/data/ so it becomes part of the fcc package. This ensures the files are included in every distribution format (sdist, wheel, editable).

src/fcc/
├── __init__.py
├── _resources.py           # Path resolution module
├── py.typed                # PEP 561 marker
├── data/                   # Package data (YAML, JSON)
│   ├── __init__.py
│   ├── personas/           # 9 YAML persona files + dimensions/
│   ├── schemas/            # 4 JSON Schema files
│   ├── workflows/          # 3 workflow graph JSON files
│   ├── scenarios/          # Scenario definitions
│   ├── governance/         # Tags and quality gates
│   └── docs/               # Topic YAML for doc generation
├── templates/              # Jinja2 templates
│   ├── scaffold/           # Project scaffolding templates
│   └── docs/               # 15 docs-as-code templates
└── ... (Python modules)

The `_resources.py` Module¶

The fcc._resources module provides a centralized, installation-mode-agnostic way to locate package data.

Design¶

"""Centralized resource path resolution for FCC package data."""

from __future__ import annotations

import importlib.resources
from pathlib import Path


def _package_root() -> Path:
    """Return the root directory of the fcc package."""
    return Path(str(importlib.resources.files("fcc")))


def get_data_dir() -> Path:
    """Return the path to the package data directory."""
    return _package_root() / "data"

Why `importlib.resources.files()`¶

The module uses importlib.resources.files() (introduced in Python 3.9, stable in 3.10+) instead of __file__-based path resolution. The key advantages:

Works with zipped packages. importlib.resources can extract files from zip archives. __file__ does not exist for packages installed in zip format.
Future-proof. pkg_resources (setuptools) and the older importlib.resources.read_text() API are deprecated.
Consistent across install modes. Returns the correct path whether the package is installed via pip install -e ., pip install fcc-agent-team-ext, or pip install ./dist/fcc_agent_team_ext-0.2.0.whl.

Approach	Editable Install	Wheel Install	Zip Install	Recommended
`Path(__file__).parent / ...`	Works	Works	Breaks	No
`pkg_resources.resource_filename()`	Works	Works	Works	Deprecated
`importlib.resources.files()`	Works	Works	Works	Yes

Available Functions¶

Function	Returns
`get_data_dir()`	`<pkg>/data/`
`get_personas_dir()`	`<pkg>/data/personas/`
`get_schemas_dir()`	`<pkg>/data/schemas/`
`get_workflows_dir()`	`<pkg>/data/workflows/`
`get_scenarios_dir()`	`<pkg>/data/scenarios/`
`get_governance_dir()`	`<pkg>/data/governance/`
`get_docs_data_dir()`	`<pkg>/data/docs/`
`get_templates_dir()`	`<pkg>/templates/`

All functions return pathlib.Path objects. Callers can use standard path operations:

from fcc._resources import get_personas_dir

core_file = get_personas_dir() / "core_personas.yaml"
with open(core_file) as f:
    data = yaml.safe_load(f)

Package Data Configuration in `pyproject.toml`¶

The pyproject.toml file tells setuptools which non-Python files to include in the wheel:

[tool.setuptools.packages.find]
where = ["src"]

[tool.setuptools.package-data]
"fcc" = [
    "data/**/*.yaml",
    "data/**/*.json",
    "templates/**/*.j2",
    "py.typed",
]

How this works¶

packages.find with where = ["src"] tells setuptools to discover Python packages under src/. It finds fcc, fcc.personas, fcc.workflow, etc.
package-data specifies glob patterns for non-Python files to include. The "fcc" key means these patterns are relative to the fcc package directory (src/fcc/).
data/**/*.yaml matches all YAML files in any subdirectory of src/fcc/data/.
data/**/*.json matches all JSON files (schemas, workflows, scenarios).
templates/**/*.j2 matches all Jinja2 templates.
py.typed is the PEP 561 marker file for type checker support.

Common pitfall: MANIFEST.in is not enough¶

MANIFEST.in controls what goes into the sdist (source distribution), not the wheel. For wheel inclusion, you must use package-data in pyproject.toml (or setup.cfg/setup.py). The FCC project does not use MANIFEST.in because the package-data configuration is sufficient for both sdist and wheel.

Wheel Inspection Techniques¶

After building, always verify the wheel contains the expected data files.

Build the wheel¶

python -m build

This creates both an sdist (.tar.gz) and a wheel (.whl) in the dist/ directory.

List wheel contents¶

A .whl file is a ZIP archive. Inspect it with Python:

import zipfile
import pathlib

whl = list(pathlib.Path("dist").glob("*.whl"))[0]
print(f"Wheel: {whl.name}")
print(f"Size: {whl.stat().st_size / 1024:.1f} KB")

with zipfile.ZipFile(whl) as z:
    for name in sorted(z.namelist()):
        print(f"  {name}")

Verify specific data directories¶

import zipfile
import pathlib

whl = list(pathlib.Path("dist").glob("*.whl"))[0]
with zipfile.ZipFile(whl) as z:
    names = z.namelist()

    # Check categories
    personas = [n for n in names if "fcc/data/personas/" in n]
    schemas = [n for n in names if "fcc/data/schemas/" in n]
    workflows = [n for n in names if "fcc/data/workflows/" in n]
    templates = [n for n in names if "fcc/templates/" in n]

    print(f"Personas:  {len(personas)} files")
    print(f"Schemas:   {len(schemas)} files")
    print(f"Workflows: {len(workflows)} files")
    print(f"Templates: {len(templates)} files")

    # Assertions for CI
    assert len(personas) > 0, "No persona data in wheel"
    assert len(schemas) > 0, "No schemas in wheel"
    assert len(templates) > 0, "No templates in wheel"

Use `twine check`¶

twine check dist/*

This validates the distribution metadata (description, classifiers, URLs) but does not inspect file contents. Combine with the Python inspection above for full verification.

Makefile shortcut¶

make release-check

This runs python -m build, twine check dist/*, and the wheel content verification script in sequence.

Adding New Data Files¶

When adding new YAML, JSON, or template files to the package, follow this checklist:

Step 1: Place the file in the correct directory¶

File type	Location
Persona YAML	`src/fcc/data/personas/`
Dimension profile YAML	`src/fcc/data/personas/dimensions/`
JSON Schema	`src/fcc/data/schemas/`
Workflow graph JSON	`src/fcc/data/workflows/`
Scenario JSON	`src/fcc/data/scenarios/`
Governance YAML	`src/fcc/data/governance/`
Doc topic YAML	`src/fcc/data/docs/`
Jinja2 template	`src/fcc/templates/docs/` or `src/fcc/templates/scaffold/`

Step 2: Check the glob pattern¶

Verify that the file extension matches an existing glob in pyproject.toml:

[tool.setuptools.package-data]
"fcc" = [
    "data/**/*.yaml",
    "data/**/*.json",
    "templates/**/*.j2",
    "py.typed",
]

If you are adding a new file type (e.g., .csv, .toml), add a new glob pattern:

"fcc" = [
    "data/**/*.yaml",
    "data/**/*.json",
    "data/**/*.csv",        # New pattern
    "templates/**/*.j2",
    "py.typed",
]

Step 3: Add a `_resources` accessor (if needed)¶

If the new file introduces a new data category, add a function to src/fcc/_resources.py:

def get_new_category_dir() -> Path:
    """Return the path to the new category data directory."""
    return get_data_dir() / "new_category"

For files in existing directories, no change to _resources.py is needed.

Step 4: Build and verify¶

# Clean previous builds
make clean-dist

# Build and verify
make release-check

Check the wheel inspection output to confirm the new files are included.

Step 5: Test the data loads correctly¶

Add a test that loads the new data file via _resources:

from fcc._resources import get_data_dir

def test_new_data_file_exists():
    path = get_data_dir() / "new_category" / "my_file.yaml"
    assert path.exists(), f"Missing data file: {path}"

Testing Packaging End-to-End¶

For comprehensive packaging verification, test the full cycle: build, install from wheel, and verify data access.

Build and install from wheel¶

# Build
python -m build

# Create a clean test environment
python -m venv /tmp/fcc-test-env
source /tmp/fcc-test-env/bin/activate

# Install from the wheel (not editable)
pip install dist/fcc_agent_team_ext-*.whl

# Verify
python -c "
from fcc import __version__
from fcc._resources import get_data_dir, get_personas_dir, get_templates_dir
from fcc.personas.registry import PersonaRegistry

print(f'Version: {__version__}')
print(f'Data dir: {get_data_dir()}')
print(f'Data exists: {get_data_dir().exists()}')

registry = PersonaRegistry.from_yaml_directory(get_personas_dir())
print(f'Personas loaded: {len(registry)}')

templates = get_templates_dir()
template_count = sum(1 for _ in templates.rglob('*.j2'))
print(f'Templates found: {template_count}')
"

# CLI smoke test
fcc --version
fcc --help

# Clean up
deactivate
rm -rf /tmp/fcc-test-env

Install from sdist¶

The sdist test catches issues where files exist on disk but are not listed in the package metadata:

pip install dist/fcc_agent_team_ext-*.tar.gz

CI verification¶

The release workflow (.github/workflows/release.yml) automates this verification. The verify-testpypi and verify-pypi jobs install the published package in clean environments and run smoke tests that load personas and check the CLI.

Build System Requirements¶

[build-system]
requires = ["setuptools>=64.0", "wheel"]
build-backend = "setuptools.build_meta"

The setuptools>=64.0 requirement is important. Older versions of setuptools do not properly support pyproject.toml-only configuration or editable installs with importlib.resources.

Troubleshooting Packaging¶

Data files missing from wheel¶

Symptom: FileNotFoundError when loading personas from a wheel install.

Check 1: Verify the glob patterns in pyproject.toml match the file extensions.

Check 2: Make sure files are under src/fcc/, not in a top-level data/ directory.

Check 3: Inspect the wheel:

python -c "
import zipfile, pathlib
whl = list(pathlib.Path('dist').glob('*.whl'))[0]
z = zipfile.ZipFile(whl)
data = [n for n in z.namelist() if 'fcc/data/' in n]
print(f'Data files in wheel: {len(data)}')
for f in sorted(data):
    print(f'  {f}')
"

Version not updating in built wheel¶

Cause: Stale build cache. Clean and rebuild:

make clean-dist
python -m build

Cause: setuptools caches the version from the egg-info directory. Remove it:

rm -rf src/fcc_agent_team_ext.egg-info
python -m build

`importlib.resources.files()` returns wrong path¶

This can happen if multiple versions of the package are installed. Check:

pip show fcc-agent-team-ext
pip list | grep fcc

Uninstall all versions and reinstall cleanly:

pip uninstall fcc-agent-team-ext -y
pip install fcc-agent-team-ext

Architecture -- Module layout and data flow
Contributing -- Development setup for packaging work
Testing Guide -- Testing infrastructure
Extension Guide -- Adding new data files and templates