Skip to content

Research Project Workflow

This page is a template for planning, executing, and writing up a student research project that uses FCC as an instrument. Appropriate for term papers, senior theses, and early-graduate qualifying papers.

The four phases

  1. Scoping (week 1-2). What question are you asking? Which FCC artifact answers it?
  2. Instrument design (week 3-4). Which personas, scenarios, workflows? What variables are you manipulating?
  3. Runs and analysis (week 5-10). Execute, collect traces, analyze.
  4. Writing (week 11-14). Structure the paper; cite; submit.

Each phase is described below.

Phase 1: Scoping

Write a one-paragraph research question. Good FCC-compatible questions are operational: they can be answered by running scenarios and measuring traces. Examples:

  • "Does adding a Critique persona reduce the rate of out-of-scope responses in an FCC Finder+Creator deployment?"
  • "How does the number of personas in the collaborator list affect message-passing depth?"
  • "Which archetype patterns produce the lowest-latency traces on the CLEAR+ benchmark?"

Bad questions (cannot be answered with FCC as an instrument):

  • "Do LLMs think?" (Too broad; also metaphysical.)
  • "Which model is best?" (Not specific to FCC; use a dedicated benchmark.)

Phase 2: Instrument design

Decide:

  • Provider. Mock for deterministic results; a hosted provider for realistic generative studies. For first papers, use mock and defer generative studies to a follow-up.
  • Personas. Pick 3-10 existing personas, or design 1-3 new ones.
  • Scenarios. At least three variants per experimental condition.
  • Workflow graph. Usually the base 5-node or the extended 20-node.
  • Metrics. From the CLEAR+ suite (Cost, Latency, Efficacy, Assurance, Reliability, Coverage, Explainability). Pick 2-3; do not pick all 7.

Write a methods section before you run anything. Put it in the paper draft. If it is too vague to be written, your design is too vague to be useful.

Phase 3: Runs and analysis

Automate. Do not run scenarios by hand; script them:

for seed in 1 2 3 4 5; do
  fcc run scenarios/my-condition-A.json --provider mock --seed $seed \
    --trace out/condA_$seed.json
  fcc run scenarios/my-condition-B.json --provider mock --seed $seed \
    --trace out/condB_$seed.json
done

Then process the traces with a small Python script. The trace schema is documented in src/fcc/data/schemas/. Produce a tidy CSV of (condition, seed, metric, value) rows. Analyze with any standard tool (pandas, R, etc.).

Phase 4: Writing

A typical FCC-based paper has this structure:

  1. Abstract (150 words). Question, method, finding, implication.
  2. Introduction (1-2 pages). Why the question matters.
  3. Related work (1-2 pages). Cite: FCC itself, prior persona-based systems, the regulation or ethics angle if relevant, the specific domain.
  4. FCC background (0.5-1 page). Brief. Do not re-explain the whole framework; cite the Guidebook and list the specific personas and workflow graph you used.
  5. Methods (1-2 pages). Exactly what you did, reproducibly.
  6. Results (1-2 pages). Quantitative table + one or two figures.
  7. Discussion (1 page). What does the result mean? What are the threats to validity?
  8. Conclusion (0.5 page). Restate contribution.
  9. References.

Citing FCC

A suggested bibliography entry:

INFORMATION COLLECTIVE, LLC. (2026). FCC Agent Team Framework (v1.7.0). [Software]. Licensed MIT. https://github.com/rollingthunderfourtytwo-afk/l2_fcc_agent_team_ext

In text: "We used the FCC framework v1.7.0 (INFORMATION COLLECTIVE, 2026)."

Cite specific artifacts when they matter: "We used persona knowledge-finder (see personas/core/knowledge-finder.yaml)."

Reproducibility checklist

Before you submit:

  • Scenario JSON files committed to your paper's companion repo.
  • Persona YAMLs (any you authored) committed.
  • Exact FCC version (fcc --version) recorded in methods.
  • Provider + model + seed recorded per run.
  • Trace JSON files archived (zip is fine).
  • Analysis script committed.
  • A make replicate target or equivalent one-line command.

Instructors increasingly grade reproducibility directly. Treat it as a first-class deliverable.

Threats to validity

Every FCC-based paper should discuss:

  • Mock determinism. Results with mock may not generalize to hosted LLM deployments. Call this out.
  • Version drift. FCC's persona catalog changes release-to-release. Pin the version.
  • Sample size. Three scenarios per condition is small. Acknowledge.
  • Ecological validity. A laboratory scenario may not reflect production behaviour. Say so.

Explicit discussion of threats raises your grade. It signals scientific maturity.

Example projects that have worked

Three composite sketches drawn from past successful student work:

  • Latency vs. persona count. A junior-year CS paper: run 5-persona, 10-persona, 20-persona versions of the same task and measure latency. Finding: latency was sub-linear because of caching, not super-linear as hypothesized.
  • Constraint effectiveness. An ethics-course final: author two versions of a persona differing only in a single constraint line. Measure the rate at which the trace violates a manually scored policy. Finding: adding one well-crafted constraint reduced violations by a measurable margin.
  • Knowledge-graph coverage. An info-science course final: build a small FCC knowledge graph of a niche topic, query it, and compare coverage to a hand-curated Wikipedia-sourced graph.

Where next

Return to Assignment Patterns for more project shapes, or review Evaluation Rubrics so you know how your work will be graded. When you are ready to author personas with confidence, do the R.I.S.C.E.A.R. Exercise.