# Cyber AI Learning — Adaptable Lesson Plan

Estimated class time: 75 minutes. Intended for students with basic security concepts; no coding or external AI account is required.

## Topic

From a model score to a defensible security decision.

## Learning outcomes

By the end of the session, students should be able to distinguish a score from ground truth, calculate precision and recall, justify a decision threshold, and explain where an AI-assisted workflow needs independent controls and human review.

## Preparation

- Review the Foundations module on precision and recall.
- Download `synthetic-triage.csv` and `prompt-boundary-exercise.md` from the site.
- Prepare a spreadsheet or use the site's threshold interaction.
- Tell students that all records are synthetic and that the exercise is not a detector benchmark.

## Session plan

| Minutes | Activity | Evidence of learning |
| --- | --- | --- |
| 0–10 | Define the security decision; distinguish observations, scores, and labels | Students identify what the system actually predicts |
| 10–25 | Build the confusion matrix at threshold 0.50 | TP 4, FP 4, FN 2, TN 10; precision 50%, recall approximately 66.7% |
| 25–40 | Compare thresholds 0.30, 0.50, and 0.80 in pairs | A comparison of missed threats and review workload |
| 40–50 | Debate thresholds for scarce analyst capacity versus costly missed threats | A recommendation with explicit assumptions |
| 50–65 | Annotate the prompt-boundary scenario and propose enforceable controls | Students locate the trust boundary and distinguish controls from instructions |
| 65–75 | Exit ticket and discussion of limits | A bounded claim and one unanswered question |

## Assessment

Ask each student to submit:

1. One correctly labeled confusion matrix with precision and recall calculations.
2. A threshold recommendation for a stated operating condition.
3. One false positive and one false negative examined in context.
4. One enforceable application control and a test of that control.
5. One limitation of the evidence.

Suggested rubric: correctness 40%, reasoning 30%, control-and-test alignment 20%, limitations 10%. These are proposed teaching weights, not a certification standard.

## Adaptation

- **Introductory class:** Provide the formulas and manually count ten records before using all twenty.
- **Advanced class:** Ask how class prevalence, calibration, repeated entities, temporal splits, and data drift affect evaluation.
- **Research seminar:** Use `research-worksheet.md` to turn the threshold or review question into a reproducible study.
- **Asynchronous class:** Have students submit the worksheet and a short explanation of a peer's alternative choice.

## Instructor notes

At threshold 0.30: TP 5, FP 8, FN 1, TN 6. At threshold 0.80: TP 2, FP 1, FN 4, TN 13. Predictions use score >= threshold. Precision is undefined if no events are flagged; report that explicitly rather than dividing by zero. A score is not automatically a calibrated probability.

## References

- [scikit-learn model evaluation documentation](https://scikit-learn.org/stable/user_guide.html)
- [OWASP Top 10 for LLM and GenAI](https://genai.owasp.org/initiatives/top-10-for-llm-and-genai/)