Learning that changes what people can do.
▼ ▼ ▼
About
I design learning experiences that change what people can do and build the assessments that show they work.
Over the course of a decade I've designed curricula, courses, and assessments from middle school through college: writing and literature programs, dual enrollment courses aligned to postsecondary outcomes, a published ESL writing curriculum, and a teacher-preparation course and field experience for preservice English educators. I served on the University of Montana's writing curriculum review committee, and my graduate research evaluated learner outcomes for a community writing coaching program.
For five years, I leveraged product strategy and design into real customer outcomes as a product content designer in corporate, helping build fintech products used by millions. User research, cross-functional partnership, system-level guidelines, human-centered design - I did it all, all in support of exceptionally clear user experiences.
I work at the intersection of learning science and design principles to develop end-to-end learning experiences, from diagnosing the problem and exploring divergent solutions to crafting learning outcomes and the experiences to reach them.
Case study
Training content authors for risk mitigation and consistent voice and tone in a regulated industry
Authors publish AI-drafted content without recognizing tone failures, missing risk information, or drafts that need compliance review.
Instructional designer: needs analysis, objectives, assessment and evaluation design, practice design, design document, storyboard, and Rise build.
Sponsor: Director of Customer Education. SMEs: Content Design Lead, Compliance Principal. Reviewer: Support Team Lead.
45-minute scenario-based Rise module, one-page job aid, follow-up practice, and three non-training recommendations.
4 weeks. Articulate Rise (SCORM to LMS), forms for scored edits, WCAG 2.1 AA.
AI-assisted content meeting standards at publication: 22% baseline, 80% target.
Wayline, its voice guidelines, and its content audit are a fictional scenario that supplies the source material, the way a client brief would.
The brief
Wayline, a fictional retail investing app, gave employees an AI writing assistant six months ago. Its content design team supplied the voice guidelines and an audit of 18 AI drafts: 78% missed at least one standard, and a third of routine drafts drifted into regulated language. The request to L&D was training.
I treated the audit as a symptom, not a diagnosis. It showed what was going wrong in published content, not why authors were letting it through or whether training was the right fix.
Discovery
Sources: the audit; interviews with the content design lead, a compliance principal, and the support team lead; a review of the AI assistant's configuration; and observation of the prompt, review, publish workflow.
- Training fixes one of four causes. The other three need changes to the AI tool, the escalation policy, and how authors are measured.
- Topic doesn't predict tier. Authors judged a draft's risk by its subject, but a third of routine drafts contained regulated language. Triage had to be taught as reading for content.
- One draft exposed a policy gap. An operational article was escalated for a single removable phrase. Whether authors may cut incidental phrases was a policy question, so I routed it to compliance before designing practice around it.
Task analysis
- Prompt the assistant and receive a draft.
- Triage the draft by its content, not its topic. The audit shows this step is skipped.
- For lower and medium tiers, edit against the standards.
- For medium tier, check that risks and benefits get balanced treatment.
- For high tier or substantive regulated language, route to compliance without editing.
- Publish, or send for review.
Steps 2, 4, and 5 are where the failures occur. They are judgment tasks, which calls for scenario-based practice rather than information-only training.
Needs analysis
| Cause | Fix | Recommendation | Owner |
|---|---|---|---|
| Authors can't reliably triage drafts or recognize regulated drift | Training | Scenario-based module and job aid | L&D |
| The AI assistant's instructions contain no guidance or guardrails | Tool | Add standards, terminology, and flags for regulated language | Content design, engineering |
| No defined escalation rule or channel | Policy | Define the rule and a single channel | Compliance |
| Author goals count volume only | Incentives | Add a sampled quality measure | Content leadership |
The training won't fix failures introduced by the tool's configuration, unclear escalation before compliance defines the channel, or quality tradeoffs driven by volume-only goals.
Audience
| Audience | Current state | Constraints | Solution |
|---|---|---|---|
| Help center and education authors (~25) | Strong writers; edit for polish, miss tier and risk balance | Deadline-driven, remote, asynchronous | Full module plus job aid |
| Support specialists | High volume; tone slips in problem moments | Little time away from the queue | Job aid and reply templates |
| Product designers | Already trained in content standards | Small group | Job aid; rule communicated directly |
| Marketing | Already routes everything to compliance | High tier by default | Out of scope |
The full design document is a two-page Learning Solution Design Document. Its core sections:
Performance objectives
Given an AI-drafted help or education article and the voice guidelines, the author will:
| Objective | Assessed by | Mastery criterion |
|---|---|---|
| 1 Triage the draft by tier based on its content, not its topic | Classifying 10 unseen drafts | 9 of 10 match the expert key, including every high-tier draft |
| 2 Edit a lower-tier draft to meet every rubric criterion | Rubric-scored edit of 2 drafts | Meets standard on every criterion |
| 3 Rebalance a medium-tier education draft so benefits and risks get balanced treatment | Rubric-scored edit of 1 draft | Comparable detail on risk and benefit; no regulated language remains |
| 4 Apply the threshold rule: cut incidental phrases, route substantive ones | Classifying 6 phrases | 5 of 6 match the key, with no substantive phrase classed as incidental |
| 5 Escalate a high-tier draft without editing it, with a note naming the trigger | Escalation note for 1 draft | Draft unedited; note names trigger and tier |
Solution approach
The module is built around practice, not presentation: learners make the decisions the job requires, on realistic drafts, with feedback that explains the reasoning. Each skill opens with a worked example, then guided and independent practice. Threshold practice pairs incidental and substantive phrases side by side, and under-escalation feedback shows what would have happened if the draft had published. Short follow-up practice at two and four weeks supports transfer.
Module flow, 45 minutes
| Section | Min | Objectives | Activity |
|---|---|---|---|
| Pre-assessment | 5 | 1, 2 | Triage 5 drafts; edit 1 short draft |
| Why this matters | 2 | — | A published draft causes a compliance finding and a customer complaint |
| Triage | 6 | 1 | Expert walkthrough of 2 drafts, then sort 6 by tier |
| Incidental or substantive | 5 | 4 | Contrasting pairs, then classify 8 phrases including borderline cases |
| Editing lower-tier drafts | 7 | 2 | Spot failures in an annotated draft, then edit and compare with a model edit |
| Rebalancing education content | 4 | 3 | Compare versions, then rewrite one risk passage |
| Escalating | 3 | 5 | Branching scenario; write an escalation note from a template |
| Calibration | 3 | 2, 3 | Score a draft with the rubric, then compare with the expert's scores |
| Post-assessment | 10 | All | Items from the assessment blueprint |
Accessibility: drafts are presented as text, not images; sorting activities have a keyboard-accessible alternative; feedback never relies on color alone. Content meets WCAG 2.1 AA.
[Link to design document PDF]
Each major decision, with the options considered and why one was chosen.
| Decision | Options considered | Chosen and why |
|---|---|---|
| Solution mix | Training alone; training plus fixes owned elsewhere | Training for one cause and three non-training recommendations, because training alone can't move the business measure |
| Format | Writing instruction; scenario-based practice | Self-paced scenarios under 45 minutes: the gap is judgment on someone else's draft, and authors are remote and deadline-driven |
| Escalation rule | Conservative: escalate any recommendation or promissory language. Threshold: authors cut incidental phrases in lower-tier content | Routed to compliance as a policy decision, since it determines what practice teaches. Built around the threshold rule compliance adopted |
| Assessment approach | Auto-scored items only; rubric-scored edits | Triage auto-scored in Rise; edits scored outside Rise against an anchored rubric, because Rise can't score free text |
| Support specialists | Full module; job aid | Job aid and reply templates: high-volume work with little time away from the queue |
| Scope cut | Include prompt-writing; exclude it | Excluded: better prompts reduce failures without removing them, and the tool fix addresses the same cause more reliably |
What each choice costs, and how the design handles it.
| Choice | Cost | How it's handled |
|---|---|---|
| Threshold escalation rule | Requires a clear definition of "incidental" and more practice on borderline cases | Working definition approved by compliance before the storyboard; dedicated borderline practice |
| Edits scored outside Rise | Needs human raters and SME time | Anchored rubric and rater calibration; about four SME hours for anchors and calibration |
| Under-escalation fails an item | Pushes learners toward escalating everything, which recreates the review backlog | Over-escalation earns partial credit; unnecessary escalations tracked with a target under 10% |
| Tool fix ships before the training | Delays the training's launch by the 30-day measurement window | Separates the training's effect from the tool's; if the fix slips, attribution shifts to a comparison group |
| 45-minute module | Pre- and post-assessment take 15 of the 45 minutes | Assessments can be separated from the module if one sitting is too long |
| Job aid instead of a module for support | Less practice for support specialists | Reply templates and direct routing rules for high-risk replies |
Assessment
| Assessment | When | Items | Scoring |
|---|---|---|---|
| Pre-assessment | Before the module | Triage 5 drafts; edit 1 draft | Same rubric as post; sets each learner's baseline |
| Practice checks | During | Triage, phrase sorting, edits with model-answer comparison | Formative; feedback only |
| Post-assessment | End of module | Triage 10 unseen drafts; classify 6 phrases; edit 2; rebalance 1; escalate 1 | Against the mastery criteria |
- Anchored rubric. Edits are scored on five criteria at three levels, with a real example anchoring each level so more than one rater can use it.
- Risk-weighted scoring. A correct tier earns full credit, over-escalation partial credit, and under-escalation fails the item.
- Rater calibration. A second rater scores five anchor edits independently; scoring begins when raters agree on at least 4 of 5 criteria per edit.
- Separate item banks. About 20 practice and 20 assessment items, balanced across tiers and failure types.
Evaluation plan
| Level | Measure | Method | Timing | Target |
|---|---|---|---|---|
| 1 Reaction | Relevance and usability | Survey plus one open question | End of module | At least 4 of 5 |
| 2 Learning | Triage accuracy and edit quality | Pre- and post-assessment | Before and after | Mastery criteria |
| 3 Behavior | Rubric scores and escalation accuracy on the job | Monthly content sampling; escalation log review | 30 and 60 days | 80% meet standard; unnecessary escalations under 10% |
| 4 Results | Content meeting standards; regulated drafts escalated; compliance findings | Business measures | 90 days | 80%; 100%; no findings on trained authors' content |
Attribution. The AI tool fix will also improve draft quality, so Level 3 and 4 gains can't be credited to training alone. The fix ships first and is measured for 30 days before the training launches.
| Stage | Reviewers | Status |
|---|---|---|
| Design document | Sponsor, Content Design Lead, Compliance Principal | Drafted |
| Storyboard | Content Design Lead, Compliance Principal | In progress |
| Alpha build | SMEs | Next |
| Pilot | Five or six authors; Support Team Lead for the job aid | Next |
The pilot will measure triage accuracy and rubric scores before and after the module. Findings and the revisions they lead to will be added here.
Artifacts
- Learning Solution Design Document [link]
- Storyboard [in progress]
- Rise module [next]
- Rubric with anchor examples [next]
- Job aid [next]