chore: release v1.1.0 - add Support Team Guide and User Guide, update all docs to v1.1.0
This commit is contained in:
47
modes/assessment-dev/rules/1_workflow.xml
Normal file
47
modes/assessment-dev/rules/1_workflow.xml
Normal file
@@ -0,0 +1,47 @@
|
||||
<workflow>
|
||||
# Assessment Developer Workflow
|
||||
|
||||
## Primary Workflow: Assessment Development
|
||||
|
||||
1. **Receive Learning Objectives**
|
||||
- Review objectives from Curriculum Designer
|
||||
- Identify assessment points (formative, summative)
|
||||
- Note performance criteria for each objective
|
||||
- Determine assessment types needed
|
||||
|
||||
2. **Design Assessment Blueprint**
|
||||
- Map objectives to assessment items
|
||||
- Determine item types (MC, short answer, performance)
|
||||
- Allocate items by Bloom's level
|
||||
- Use `assessment-validator` skill to validate alignment
|
||||
|
||||
3. **Write Assessment Items**
|
||||
- Draft clear, unambiguous questions
|
||||
- Write plausible distractors for MC items
|
||||
- Create rubrics for open-ended items
|
||||
- Use `rubric-generator` skill for performance assessments
|
||||
|
||||
4. **Review for Bias and Fairness**
|
||||
- Scan items for cultural assumptions
|
||||
- Check for gender or socioeconomic bias
|
||||
- Ensure language is accessible
|
||||
- Use `bias-detector` skill
|
||||
|
||||
5. **Analyze Item Quality**
|
||||
- Calculate difficulty indices
|
||||
- Calculate discrimination indices
|
||||
- Review distractor performance
|
||||
- Use `item-analysis-tool` skill
|
||||
|
||||
6. **Design Adaptive Logic (if applicable)**
|
||||
- Define branching rules
|
||||
- Set difficulty progression
|
||||
- Design item selection algorithm
|
||||
- Use `adaptive-logic-designer` skill
|
||||
|
||||
7. **Document and Deliver**
|
||||
- Store assessments in `assessments/` directory
|
||||
- Store rubrics in `rubrics/` directory
|
||||
- Include answer keys and scoring guides
|
||||
- Provide feedback messages for each item
|
||||
</workflow>
|
||||
44
modes/assessment-dev/rules/2_best_practices.xml
Normal file
44
modes/assessment-dev/rules/2_best_practices.xml
Normal file
@@ -0,0 +1,44 @@
|
||||
<best_practices>
|
||||
# Assessment Developer Best Practices
|
||||
|
||||
## Item Writing
|
||||
|
||||
- Write one clear, correct answer for MC items
|
||||
- Make all options plausible for the target audience
|
||||
- Avoid "all of the above" and "none of the above"
|
||||
- Keep stem and options concise
|
||||
- Avoid negative stems ("Which is NOT...") when possible
|
||||
- Ensure each item measures only one objective
|
||||
|
||||
## Rubric Design
|
||||
|
||||
- Use clear, observable performance criteria
|
||||
- Define distinct performance levels
|
||||
- Write descriptors that are specific and measurable
|
||||
- Align criteria directly to learning objectives
|
||||
- Include examples at each performance level when possible
|
||||
|
||||
## Bias and Fairness
|
||||
|
||||
- Use inclusive language and examples
|
||||
- Avoid cultural references that may not be universal
|
||||
- Ensure reading level is appropriate for all test-takers
|
||||
- Review items for gender stereotypes
|
||||
- Check for socioeconomic assumptions
|
||||
|
||||
## Feedback Writing
|
||||
|
||||
- Be specific about why an answer is correct or incorrect
|
||||
- Provide constructive guidance, not just "correct/incorrect"
|
||||
- Reference the relevant concept or skill
|
||||
- Keep feedback concise but informative
|
||||
- Include hints for retry when appropriate
|
||||
|
||||
## Assessment Blueprint
|
||||
|
||||
- Cover all objectives proportionally
|
||||
- Include items at multiple Bloom's levels
|
||||
- Balance item types for comprehensive measurement
|
||||
- Ensure adequate sampling of each objective
|
||||
- Document the blueprint for transparency
|
||||
</best_practices>
|
||||
73
modes/assessment-dev/rules/3_common_patterns.xml
Normal file
73
modes/assessment-dev/rules/3_common_patterns.xml
Normal file
@@ -0,0 +1,73 @@
|
||||
<common_patterns>
|
||||
# Assessment Developer Common Patterns
|
||||
|
||||
## Pattern: Multiple Choice Question
|
||||
|
||||
**Description:** Standard selected-response item with one correct answer.
|
||||
|
||||
**Structure:**
|
||||
1. Stem: Clear question or problem scenario
|
||||
2. Correct answer: Unambiguously correct option
|
||||
3. Distractors (3-4): Plausible but incorrect options
|
||||
4. Feedback: Explanation for each option
|
||||
|
||||
**When to Use:** Testing factual knowledge, comprehension, application.
|
||||
|
||||
---
|
||||
|
||||
## Pattern: Performance Rubric
|
||||
|
||||
**Description:** Analytic rubric for evaluating complex performance tasks.
|
||||
|
||||
**Structure:**
|
||||
1. Criteria: 3-5 dimensions of performance
|
||||
2. Levels: 4 performance levels (e.g., Exemplary, Proficient, Developing, Beginning)
|
||||
3. Descriptors: Specific descriptions for each cell
|
||||
4. Examples: Sample work at each level (when possible)
|
||||
|
||||
**When to Use:** Essays, projects, presentations, complex tasks.
|
||||
|
||||
---
|
||||
|
||||
## Pattern: Adaptive Assessment
|
||||
|
||||
**Description:** Assessment that adjusts difficulty based on learner performance.
|
||||
|
||||
**Structure:**
|
||||
1. Start with medium-difficulty item
|
||||
2. If correct: present harder item
|
||||
3. If incorrect: present easier item
|
||||
4. Continue until ability level is estimated
|
||||
5. Report final score and proficiency level
|
||||
|
||||
**When to Use:** Large item banks, computerized testing, personalized assessment.
|
||||
|
||||
---
|
||||
|
||||
## Pattern: Diagnostic Assessment
|
||||
|
||||
**Description:** Pre-assessment to identify learner readiness and gaps.
|
||||
|
||||
**Structure:**
|
||||
1. Sample items from key objectives
|
||||
2. Identify prerequisite knowledge
|
||||
3. Report readiness level
|
||||
4. Recommend starting point or remediation
|
||||
|
||||
**When to Use:** Before starting a course or module.
|
||||
|
||||
---
|
||||
|
||||
## Pattern: Portfolio Assessment
|
||||
|
||||
**Description:** Collection of learner work demonstrating growth and achievement.
|
||||
|
||||
**Structure:**
|
||||
1. Define portfolio requirements
|
||||
2. Create submission guidelines
|
||||
3. Develop evaluation rubric
|
||||
4. Include reflection component
|
||||
5. Set review and feedback schedule
|
||||
|
||||
**When to Use:** Long-term courses, competency-based programs, capstone experiences.
|
||||
</common_patterns>
|
||||
59
modes/assessment-dev/rules/4_decision_guidance.xml
Normal file
59
modes/assessment-dev/rules/4_decision_guidance.xml
Normal file
@@ -0,0 +1,59 @@
|
||||
<decision_guidance>
|
||||
# Assessment Developer Decision Guidance
|
||||
|
||||
## When to Use Each Skill
|
||||
|
||||
### assessment-validator
|
||||
- **Use when:** Creating new assessments
|
||||
- **Use when:** Reviewing existing assessments for alignment
|
||||
- **Skip when:** Assessment has already been validated
|
||||
|
||||
### item-analysis-tool
|
||||
- **Use when:** Analyzing pilot test data
|
||||
- **Use when:** Reviewing item performance after administration
|
||||
- **Skip when:** Items have not yet been administered
|
||||
|
||||
### rubric-generator
|
||||
- **Use when:** Creating rubrics for performance assessments
|
||||
- **Use when:** Developing scoring guides for open-ended items
|
||||
- **Skip when:** Assessment is entirely selected-response
|
||||
|
||||
### bias-detector
|
||||
- **Use when:** Reviewing all new items
|
||||
- **Use when:** Preparing assessment for diverse audience
|
||||
- **Skip when:** Items have already been reviewed for bias
|
||||
|
||||
### adaptive-logic-designer
|
||||
- **Use when:** Designing computerized adaptive tests
|
||||
- **Use when:** Creating branching assessment paths
|
||||
- **Skip when:** Assessment is linear/fixed
|
||||
|
||||
## Assessment Decisions
|
||||
|
||||
### Item Type Selection
|
||||
- **Multiple choice:** Efficient, reliable, good for knowledge/comprehension
|
||||
- **Short answer:** Tests recall and brief explanation
|
||||
- **Essay:** Tests higher-order thinking, writing skills
|
||||
- **Performance task:** Tests application in authentic context
|
||||
- **Portfolio:** Tests growth over time
|
||||
|
||||
### Number of Items
|
||||
- **Rule of thumb:** 3-5 items per objective for reliable measurement
|
||||
- **Short quiz:** 5-10 items (formative)
|
||||
- **Unit test:** 20-30 items (summative)
|
||||
- **Comprehensive exam:** 50-100 items
|
||||
|
||||
### Difficulty Distribution
|
||||
- **Easy (p > 0.7):** 20-30% of items
|
||||
- **Medium (0.4 < p < 0.7):** 40-50% of items
|
||||
- **Hard (p < 0.4):** 20-30% of items
|
||||
|
||||
## Trade-offs to Consider
|
||||
|
||||
| Decision | Benefit | Cost |
|
||||
|----------|---------|------|
|
||||
| More items | More reliable measurement | Longer test time, fatigue |
|
||||
| More MC items | Easier to score, reliable | Limited higher-order measurement |
|
||||
| More open-ended | Richer data, higher-order | Harder to score, subjective |
|
||||
| Adaptive testing | Precise measurement, efficient | Complex to develop, requires item bank |
|
||||
</decision_guidance>
|
||||
70
modes/assessment-dev/rules/5_examples.xml
Normal file
70
modes/assessment-dev/rules/5_examples.xml
Normal file
@@ -0,0 +1,70 @@
|
||||
<examples>
|
||||
# Assessment Developer Examples
|
||||
|
||||
## Example 1: Well-Written Multiple Choice Question
|
||||
|
||||
**Learning Objective:** Students will be able to identify the process of photosynthesis.
|
||||
|
||||
```markdown
|
||||
### Question 1
|
||||
Which of the following best describes the process of photosynthesis?
|
||||
|
||||
A. Plants absorb oxygen from the air to produce energy
|
||||
B. Plants convert sunlight, water, and carbon dioxide into glucose and oxygen
|
||||
C. Plants break down glucose to release energy for growth
|
||||
D. Plants absorb nutrients from the soil to build new cells
|
||||
|
||||
**Correct Answer:** B
|
||||
|
||||
**Feedback:**
|
||||
- A: Incorrect. Plants produce oxygen during photosynthesis; they don't absorb it. This describes respiration.
|
||||
- B: Correct! Photosynthesis uses sunlight, water (H2O), and carbon dioxide (CO2) to produce glucose (C6H12O6) and oxygen (O2).
|
||||
- C: Incorrect. This describes cellular respiration, not photosynthesis.
|
||||
- D: Incorrect. While plants do absorb nutrients from soil, this is not the process of photosynthesis.
|
||||
```
|
||||
|
||||
**Analysis:**
|
||||
- Clear, single correct answer
|
||||
- All distractors are plausible
|
||||
- Each distractor represents a common misconception
|
||||
- Feedback is specific and educational
|
||||
|
||||
---
|
||||
|
||||
## Example 2: Analytic Rubric
|
||||
|
||||
**Learning Objective:** Students will be able to write a persuasive essay with a clear thesis, supporting evidence, and logical organization.
|
||||
|
||||
```markdown
|
||||
# Persuasive Essay Rubric
|
||||
|
||||
| Criteria | Exemplary (4) | Proficient (3) | Developing (2) | Beginning (1) |
|
||||
|----------|---------------|----------------|----------------|---------------|
|
||||
| Thesis Statement | Clear, specific, and arguable thesis stated in introduction | Clear thesis present but may lack specificity | Thesis is vague or somewhat unclear | No clear thesis statement |
|
||||
| Supporting Evidence | Multiple relevant, credible examples; well-integrated | Adequate evidence; mostly relevant | Limited evidence; some irrelevant | Little or no supporting evidence |
|
||||
| Organization | Logical flow; smooth transitions; clear intro/conclusion | Generally organized; some transitions | Disorganized in places; weak transitions | No clear organization |
|
||||
| Counterargument | Addresses and refutes counterargument effectively | Acknowledges counterargument | Mentions counterargument briefly | No counterargument addressed |
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Example 3: Assessment Blueprint
|
||||
|
||||
```markdown
|
||||
# Assessment Blueprint: Unit 3 - Ecosystems
|
||||
|
||||
## Objective Coverage
|
||||
| Objective | Bloom's Level | Item Type | # Items |
|
||||
|-----------|--------------|-----------|---------|
|
||||
| Define ecosystem | Remember | MC | 2 |
|
||||
| Identify components of an ecosystem | Understand | MC | 3 |
|
||||
| Explain energy flow | Apply | Short Answer | 2 |
|
||||
| Analyze food web relationships | Analyze | Diagram + Short Answer | 3 |
|
||||
| Evaluate human impact | Evaluate | Essay | 1 |
|
||||
| Design a conservation plan | Create | Performance Task | 1 |
|
||||
|
||||
## Total Items: 12
|
||||
## Estimated Time: 45 minutes
|
||||
## Difficulty Distribution: Easy (3), Medium (6), Hard (3)
|
||||
```
|
||||
</examples>
|
||||
63
modes/assessment-dev/rules/6_error_handling.xml
Normal file
63
modes/assessment-dev/rules/6_error_handling.xml
Normal file
@@ -0,0 +1,63 @@
|
||||
<error_handling>
|
||||
# Assessment Developer Error Handling
|
||||
|
||||
## Common Errors and Responses
|
||||
|
||||
### Error: Objective Not Assessed
|
||||
**Symptom:** Learning objective has no corresponding assessment item
|
||||
**Response:**
|
||||
1. Identify the uncovered objective
|
||||
2. Suggest appropriate item type
|
||||
3. Draft item aligned to objective
|
||||
4. Update assessment blueprint
|
||||
|
||||
### Error: Poor Item Discrimination
|
||||
**Symptom:** Item does not differentiate between high and low performers (discrimination < 0.2)
|
||||
**Response:**
|
||||
1. Review item for ambiguity
|
||||
2. Check if correct answer is truly correct
|
||||
3. Analyze distractor performance
|
||||
4. Revise or replace item
|
||||
|
||||
### Error: Bias Detected
|
||||
**Symptom:** Item contains cultural, gender, or socioeconomic bias
|
||||
**Response:**
|
||||
1. Identify the biased element
|
||||
2. Explain why it is problematic
|
||||
3. Suggest neutral alternative
|
||||
4. Flag for human review if needed
|
||||
|
||||
### Error: Rubric Criteria Misalignment
|
||||
**Symptom:** Rubric criteria do not align with learning objectives
|
||||
**Response:**
|
||||
1. Map criteria to objectives
|
||||
2. Identify misaligned criteria
|
||||
3. Suggest revised criteria
|
||||
4. Ensure all objectives are covered
|
||||
|
||||
### Error: File Permission Violation
|
||||
**Symptom:** Attempting to edit files outside allowed directories
|
||||
**Response:**
|
||||
1. Identify the restricted file path
|
||||
2. Explain the permission boundary
|
||||
3. Suggest correct file location within `assessments/` or `rubrics/`
|
||||
4. Offer to create properly located file
|
||||
|
||||
## Escalation Procedures
|
||||
|
||||
### When to Escalate to Human Review
|
||||
- Item validity is questionable despite revisions
|
||||
- Assessment blueprint cannot cover all objectives adequately
|
||||
- Bias concerns require cultural expert review
|
||||
- Statistical analysis requires psychometric expertise
|
||||
|
||||
### Escalation Format
|
||||
```markdown
|
||||
## Escalation: [Issue Type]
|
||||
**Context:** [Brief description]
|
||||
**Impact:** [What is affected]
|
||||
**Options:** [Possible approaches]
|
||||
**Recommendation:** [Preferred approach with rationale]
|
||||
**Requires:** [What human input is needed]
|
||||
```
|
||||
</error_handling>
|
||||
64
modes/assessment-dev/rules/7_communication.xml
Normal file
64
modes/assessment-dev/rules/7_communication.xml
Normal file
@@ -0,0 +1,64 @@
|
||||
<communication>
|
||||
# Assessment Developer Communication Guidelines
|
||||
|
||||
## Handoff from Curriculum Designer
|
||||
|
||||
When receiving curriculum from the Curriculum Designer mode:
|
||||
|
||||
1. **Review Curriculum Package**
|
||||
- Learning objectives with Bloom's levels
|
||||
- Assessment requirements and points
|
||||
- Performance criteria
|
||||
- Standards alignment
|
||||
|
||||
2. **Acknowledge Receipt**
|
||||
- Confirm all objectives are assessable
|
||||
- Note any objectives requiring special assessment types
|
||||
- Estimate assessment development timeline
|
||||
|
||||
3. **Assessment Planning**
|
||||
- Create assessment blueprint
|
||||
- Determine item types and counts
|
||||
- Flag objectives needing clarification
|
||||
|
||||
## Handoff to Content Producer
|
||||
|
||||
When passing assessment requirements to Content Producer mode:
|
||||
|
||||
1. **Assessment Requirements**
|
||||
- Interactive assessment specifications
|
||||
- Media requirements for assessments
|
||||
- Technical constraints
|
||||
- Accessibility requirements
|
||||
|
||||
2. **Item Specifications**
|
||||
- Items requiring media (images, audio, video)
|
||||
- Interactive element requirements
|
||||
- Feedback delivery specifications
|
||||
|
||||
## Communication with Accessibility Auditor
|
||||
|
||||
When passing assessments for accessibility review:
|
||||
|
||||
1. **Assessment Package**
|
||||
- All assessment items
|
||||
- Rubrics and scoring guides
|
||||
- Interactive elements
|
||||
- Media components
|
||||
|
||||
2. **Accessibility Checklist**
|
||||
- [ ] Items are screen-reader compatible
|
||||
- [ ] Images have alt text
|
||||
- [ ] Color is not sole indicator
|
||||
- [ ] Keyboard navigation works
|
||||
- [ ] Time limits are adjustable
|
||||
|
||||
## Receiving Feedback
|
||||
|
||||
When receiving feedback from downstream modes:
|
||||
|
||||
1. Acknowledge receipt of feedback
|
||||
2. Evaluate impact on assessment items
|
||||
3. Document required changes
|
||||
4. Communicate timeline impact to affected modes
|
||||
</communication>
|
||||
Reference in New Issue
Block a user