Skip to content

Assessment Strategies

Guidelines for assessing student understanding of the FCC Agent Team Framework. This document covers formative assessments, summative projects, rubrics, and self-assessment tools.

Assessment Philosophy

FCC assessments should evaluate both conceptual understanding and practical implementation skills. Students should demonstrate they can:

  1. Explain FCC concepts accurately (knowledge)
  2. Apply FCC patterns to new problems (application)
  3. Design custom personas, workflows, and plugins (synthesis)
  4. Evaluate trade-offs between different FCC approaches (analysis)

Formative Assessments (Ongoing)

Weekly Reading Responses

Assign short written responses (300-500 words) to guidebook chapters:

Week Chapter Prompt
1 Ch. 1: Introduction What problem does FCC solve?
2 Ch. 3: R.I.S.C.E.A.R. Why 10 components instead of fewer?
3 Ch. 4: Persona Dimensions How do dimensions affect persona behavior?
4 Ch. 5: Workflow System Compare FCC workflows to CI/CD pipelines
5 Ch. 7: Event Bus When would you use filtering vs subscribe_all?
6 Ch. 8: Collaboration How do approval gates prevent low-quality output?
7 Ch. 9: Governance Hard-stop vs mandatory: when to use each?
8 Ch. 6: Plugin Architecture Design a plugin type not in the framework
9 Ch. 17: Knowledge Federation How does federation enable cross-project work?

Lab Completion Checks

Use the built-in tutorial tracker to verify lab completion:

from fcc.tutorials.tracker import TutorialTracker

tracker = TutorialTracker(student_id="student_123")
progress = tracker.get_progress()

for module in progress.modules:
    print(f"  {module.name}: {module.completion_pct}%")

Peer Code Review

Pair students for weekly code reviews of lab submissions. Provide a checklist:

  • Code runs without errors
  • Follows FCC conventions (dataclasses, click CLI, _resources)
  • Includes appropriate event bus integration
  • Has at least 3 unit tests
  • R.I.S.C.E.A.R. specs are complete (all 10 fields)

Summative Assessments

Quiz Bank

Use these sample questions for quizzes and exams.

Multiple Choice (Knowledge):

  1. How many components does the R.I.S.C.E.A.R. specification define?
  2. a) 5
  3. b) 7
  4. c) 10 (correct)

  5. d) 12

  6. Which persona type coordinates teams of base personas?

  7. a) Governance persona
  8. b) Champion persona (correct)
  9. c) Integration persona

  10. d) Core persona

  11. How many event types does the FCC event bus support?

  12. a) 20
  13. b) 42
  14. c) 66 (correct)
  15. d) 100

Short Answer (Application):

  1. Describe the difference between a handoff relationship and a feedback relationship in the cross-reference matrix. Give an example of each.

  2. A simulation trace shows 5 steps. Draw the expected FCC phase sequence and explain which personas might be involved at each step.

  3. Explain why the plugin system uses dependency resolution. What would happen without it?

Design Questions (Synthesis):

  1. Design a R.I.S.C.E.A.R. spec for a "Security Auditor" persona. Include all 10 components and explain your choices.

  2. Given a scenario where three teams must collaborate on a document, design a workflow graph with at least 8 nodes. Show the approval gates.

  3. Design an event subscriber that monitors simulation quality. What events would it subscribe to? What metrics would it collect?

Rubric Templates

R.I.S.C.E.A.R. Design Rubric

Criterion Excellent (4) Good (3) Adequate (2) Poor (1)
Role clarity Specific, actionable Clear but broad Vague Missing
Input specification Detailed inputs listed Inputs described Partial Missing
Style consistency Distinct, consistent Mostly consistent Inconsistent Missing
Constraints relevance Domain-specific, practical Relevant Generic Missing
Expected output Measurable, specific Described Vague Missing
Archetype alignment Well-chosen, justified Appropriate Loosely related Missing
Responsibilities Comprehensive, unique Good coverage Partial Missing
Skills specification Specific, verifiable Listed Incomplete Missing
Collaborators defined With relationship types Listed Partial Missing
Adoption checklist Actionable steps Steps listed Incomplete Missing

Workflow Design Rubric

Criterion Excellent (4) Good (3) Adequate (2) Poor (1)
Graph structure Well-connected, no orphans Minor issues Disconnected nodes Invalid
Phase coverage All FCC phases represented Most phases Some phases One phase
Feedback loops Meaningful loops included At least 1 loop No loops N/A
Governance gates Gates at appropriate points Some gates No gates N/A
Documentation Graph + rationale Graph only Partial Missing

Capstone Project Rubric

See the detailed rubric in curriculum-12-week.md, Week 12 section.

Self-Assessment Tools

Concept Mastery Checklist

Students can use this checklist to self-assess their progress:

Beginner Level (Weeks 1-4): - [ ] I can install FCC and run tests - [ ] I can load the persona registry and query personas - [ ] I can explain all 10 R.I.S.C.E.A.R. components - [ ] I can run a mock simulation and read the trace - [ ] I can name at least 5 persona categories

Intermediate Level (Weeks 5-8): - [ ] I can design a custom workflow graph - [ ] I can execute workflow actions through the action engine - [ ] I can subscribe to events and implement filters - [ ] I can create collaboration sessions with approval gates - [ ] I can use the observability layer for tracing

Advanced Level (Weeks 9-12): - [ ] I can build a custom plugin and register it - [ ] I can explain the 3-tier constitution system - [ ] I can build and query a knowledge graph - [ ] I can set up a RAG pipeline - [ ] I can integrate multiple FCC subsystems in a project

Assessment Data in the Framework

The framework includes machine-readable assessment rubrics:

# Load assessment rubrics
from fcc._resources import get_resource_path
import yaml

path = get_resource_path("docs", "assessment_rubrics.yaml")
with open(path) as f:
    rubrics = yaml.safe_load(f)

for rubric in rubrics:
    print(f"  {rubric['name']}: {len(rubric['criteria'])} criteria")