Learning that changes how people work, think, teach, lead, grow and live
About
I design learning experiences that change what people can do and build the assessments that show they work.
Over the course of a decade I've designed curricula, courses, and assessments from middle school through college: writing and literature programs, dual enrollment courses aligned to postsecondary outcomes, a published ESL writing curriculum, and a teacher-preparation course and field experience for preservice English educators. I served on the University of Montana's writing curriculum review committee, and my graduate research evaluated learner outcomes for a community writing coaching program.
For five years, I leveraged product strategy and design into real customer outcomes as a product content designer in corporate, helping build fintech products used by millions. User research, cross-functional partnership, system-level guidelines, human-centered design - I did it all, all in support of exceptionally clear user experiences.
I work at the intersection of learning science and design principles to develop end-to-end learning experiences, from diagnosing the problem and exploring divergent solutions to crafting learning outcomes and the experiences to reach them.
Case study: Training AI content authors to mitigate risk and edit safely in a regulated industry
Wayline is an invented investing app whose content authors use an AI writing assistant to draft help articles, education content and customer communications.
This case study follows a corporate learning project for this fictional company, built to show how I approach instructional design from start to finish. Its content audit, stakeholders and business data are fabricated, but they are modeled on the risks a regulated financial company faces when AI-drafted content goes out without enough review.
Working as the sole instructional designer, I diagnosed why errors were reaching customers, separated the problems training could solve from those it couldn't, and designed a scenario-based course series with an assessment and evaluation plan. The tabs below follow that process in order, from the problem through how I'd measure results.
Problem
Six months after launching a new AI writing assistant for internal use, the content design team at Wayline performed a content audit across all areas where the AI writing tool was being used: help articles, support replies, education content, and marketing copy.
They discovered that content volume was up approximately 30%, but quality had decreased. Across 160 items only 54% met all voice and tone standards. Of the 160 samples, 18% injected regulatory risk by implying financial advice or assuring outcomes, and missed the required escalation to the compliance department.
That same study identified that employees using the AI tool to draft content usually edited it for conventions and polish, but rarely considered voice and tone or language that needs legal review.
The content team followed up on this study by running customer interviews using those same 160 drafts. They discovered that customers were less likely to trust the Wayline brand after being exposed to content with inconsistencies of language.
Misuse of regulated language carries a significant risk of legal action, the impact of which is difficult to forecast; however, previous legal action based on misleading performance promises cost Wayline fees in the high six figures.
Wayline is a fictional company. This is a fictional scenario devised as source material for an instructional design case study only.
Needs analysis
The content audit proved something was going measurably wrong with AI-generated content at Wayline, but it didn't explain why and how errors were making it past human authors and into production. It also wasn't immediately clear whether additional training was even the right solution for increasing content quality and mitigating risk.
In addition to the main problem outlined in the brief from the content team, I discovered the following problems:
- The AI assistant had minimal built-in guardrails for brand-specific voice and tone guidelines and compliance risk
- Content authors had no formal rules to refer to when making compliance decisions, relying instead on individual judgment
- Authors were rewarded for meeting content volume goals with no incentive related to the quality of that content
I analyzed the root cause of these problems to determine whether training would in fact solve for them.
Root cause analysis
| Problem | Recommendation | Owner |
|---|---|---|
| The AI assistant has minimal guardrails for voice, tone, and compliance risk | Add standards, terminology, and flags for regulated language to the assistant's instructions | Content design, engineering |
| No formal guidelines for compliance and regulatory language for author reference | Specifically define the circumstances that require an escalation to compliance, and how to escalate content when needed | Compliance |
| Authors are rewarded for volume, with no quality incentive | Add a quality metric to author goals and evaluation | Content leadership |
| Authors don't reliably recognize regulated language or issues with voice and tone | Scenario-based training series and job aid | L&D |
Training can provide knowledge and practice recognizing regulated language and voice and tone issues, but it won't:
- Prevent the AI from generating low-quality content in the first place
- Define compliance escalation conditions and how to escalate risky content
- Fix the balance between quality and volume-only content author goals
Task analysis
I segmented the ideal authoring and review process into discrete tasks to find potential failure points. That process is as follows:
- The author prompts the AI assistant and receives a draft.
- The author triages the draft according to the language it uses, not just its topic. According to the content audit, many authors skip this step or perform it incorrectly.
- For content triaged as lower or medium tier, the author edits it against the Wayline voice and tone standards.
- For medium-tier content, the author checks that risks and benefits get balanced treatment. This step, absent in the current process, will ensure risk mitigation without flooding human compliance reviewers.
- For drafts about losses, performance, promotions, or agent trades, or any draft that recommends or promises, route to compliance without editing. Per the audit, not all high-tier content is being sent to compliance for review.
- Publish, or send for review.
Failures are possible anywhere human judgment comes into play: in this case, steps 2, 4, and 5. Scenario-based practice is the best way to norm judgment standards across groups of learners.
Audience analysis
Once I identified what training can actually solve for and why, I revisited the groups at Wayline that author content to pressure-check the training and job aid recommendation. Those findings are as follows:
Help center and education authors
Currently strong writers who edit for writing conventions and polished language, but rarely compliance issues and often Wayline voice and tone. They work remotely and asynchronously, based on strict deadlines, and as such would benefit most from the full three-part series plus a supportive job aid.
Support specialists
Handle a high volume of content in customer-facing contexts, which leads to high variability in voice and tone. They rarely miss the mark when it comes to regulated language because it is already heavily emphasized in their training. They can't take much time away from customer interactions, and would benefit most from a job aid.
Product designers
Already familiar with voice and tone standards thanks to close collaboration with the content design team. They are also a small team, which means a lightweight job aid will support them best.
Marketing writers
Because marketing is a high-risk surface by default, all marketing content must already pass through compliance before publication. This group can be considered out of scope.
Wayline is a fictional company. This is a fictional scenario devised as source material for an instructional design case study only.
Goals
The trust and compliance risks identified in the content audit reveal a very real business need: to elevate risk triage accuracy and voice and tone fidelity by content authors across areas.
Measurable business goals
- Increase the proportion of AI-assisted content that meets voice and tone guidelines to 70% or more at 90 days and 80% or more by 6 months
- Reduce occurrences of regulated language that are published without compliance review to <5% at 90 days and <2% at 6 months without requiring increased legal or compliance headcount due to unnecessary escalations
- Maintain under 10% rate of unnecessary escalations at 90 days and 6 months
| Measure | Baseline (content audit, 160 items) | Target at 90 days | Target at 6 months |
|---|---|---|---|
| AI-assisted content meeting all voice and tone standards at publication | 54% | 70% | 80% or more |
| Content published with regulated language that skipped compliance review | 18% | Under 5% | Under 2% |
| Lower- and medium-tier drafts escalated unnecessarily | Not yet measured; set during the 30-day tool-fix window | Under 10% | Under 10% |
Performance metrics
When reviewing AI-generated content, the author will achieve the following objectives to the corresponding mastery standards:
| Objective | MASTERY STANDARDS |
|---|---|
| Triage the draft by tier based on the language it uses, not its topic | The author accurately triages 9 of every 10 drafts according to the risk tier model with zero misclassification of high-risk content. |
| Edit a lower-tier draft to meet voice and tone standards | The author meets 80% of the voice and tone criteria across 5 sample drafts. Measured on the job through monthly content sampling. |
| Rebalance a medium-tier education draft so benefits and risks get balanced treatment | The author balances risk and benefit with comparable detail in 4 of every 5 medium-tier drafts, and escalates any draft that contains a recommendation or promise. Measured on the job through monthly content sampling. |
| Escalate a high-tier draft without editing it and draft a brief annotation explaining the risk trigger | Every high-risk draft passes to compliance unedited and includes annotation regarding risk trigger. |
Wayline is a fictional company. This is a fictional scenario devised as source material for an instructional design case study only.
Solution and rationale
Based on the root cause, needs analysis, and business goals, I proposed a three-part, self-paced, scenario-based training series for content authors. I also recommended a single one-page downloadable reference, available at the end of Part 3 and shared with support specialists and product designers, moderated follow-up practice at two and four weeks, and three non-training improvements to be handled outside training.
Key decisions and rationale
Strategic solutions
Training alone can't fix a broken model or create compliance rules, so rather than build training for every problem I focused on human failure points and proposed non-training solves for the rest.
Training type
Most authors already write and edit well, making targeted scenario-based practice a more effective and efficient option than direct writing instruction.
Length and delivery
Most authors work remotely under demanding deadlines. I opted for a series of 30-minute async training modules over a single-sitting live training to accommodate bandwidth and appropriate cognitive load.
Video production
I produced Part 1's three videos in Synthesia as voiceovers over AI-generated drafts rather than filming them. Because a change only takes a script edit, I could revise quickly and get Part 1 in front of learners for feedback sooner.
Target learner variability
Learners in some roles, like customer support, already understand compliance triggers and risk mitigation but struggle with voice and tone. Others, such as help content writers, have a better command of voice and tone but overlook regulated language. Rather than deliver training to all roles, I decided it would be more efficient and cost-effective to give customer support a job aid and content templates because voice and tone is low-risk compared to compliance.
Wayline is a fictional company. This is a fictional scenario devised as source material for an instructional design case study only.
Course map
Before developing the course portion of the chosen solution, I mapped each module in document form to allow for quick iteration and SME review. Then, I built the series in Articulate Rise and published it to the company LMS as SCORM. For Part 1's three videos, I wrote each script first, produced the voiceover over AI-generated drafts in Synthesia, and embedded the result in Rise with a code block.
Authors work remotely and on deadlines, so each part needed to be self-paced, 30 minutes long, and accessible to WCAG 2.1 AA.
Guiding principles
Across all three parts, the following learning principles apply:
"Practice mirrors the job" (Clark, Mayer 2023): Example-based scenarios work best when they're close analogues to the real work to be done. In these modules, authors respond to realistic AI drafts to make the same kinds of decisions they make before publishing live content.
Worked examples before practice: Adult learners engage best when they understand the problem and objectives through clear, realistic examples. Each section in the course begins with exposure to worked examples for foundational context and self-directed motivation.
Faded support: Each module provides abundant scaffolds in the early lessons and then strategically removes those supports to build independent reasoning and judgment, which remote and async content authors will need on the job.
Timely and intermittent feedback: In-module feedback explains the reasoning and shows the consequence of risk triage and voice and tone decisions. Learners apply feedback during the week between releases, and short practice sets at two and four weeks space the learning over time.
Course map: three 30-minute parts and a 15-minute post-assessment, about 105 minutes
Objectives key
- Triage the draft by tier based on the language it uses, not its topic
- Edit a lower-tier draft to meet voice and tone standards
- Rebalance a medium-tier education draft so benefits and risks get balanced treatment
- Escalate a high-tier draft without editing it and draft a brief annotation explaining the risk trigger
Part 1
Learners are introduced to the stakes of regulated language and how to triage content into lower, medium, and high tiers.
| Section | Min | Objectives | Activity | Rise approach |
|---|---|---|---|---|
| Introduction | 2 | — | Learners review a real-world incident: Wayline published an AI-drafted education article that implied guaranteed returns. A subsequent complaint and legal matter cost the company money and time. | Synthesia video; text |
| Pre-assessment | 6 | 1, 2 | Learners are presented with 5 AI-generated content drafts. They are asked to sort each into the lower, medium, or high tier and edit one short draft. | Knowledge checks; edit compared with a model reply (in deployment, submitted by form) |
| Lower tier | 5 | 1 | Learners see what makes a draft lower tier and why it's edited rather than escalated. They watch a reviewer spot the problems in a lower-tier draft, such as filler and vague timing, then check their understanding. | Tabs; think-aloud video (Synthesia); knowledge checks |
| Medium tier | 6 | 1 | Learners see what makes a draft medium tier and why its risks and benefits need balanced treatment. They spot vague or missing risk in an education draft where it appears, then check their understanding. | Tabs; knowledge checks |
| High tier | 5 | 1 | Learners see the two things that make a draft high tier: its topic, or a recommendation or promise anywhere in it. They watch a reviewer spot the trigger in a draft, then decide what to do with a promotional email in a short scenario. | Think-aloud video (Synthesia); scenario; knowledge check |
| Mixed practice and check | 6 | 1 | Learners sort a mixed set of drafts by tier and explain their reasoning, then take a five-question quiz on unseen drafts. | Sorting activity; knowledge checks; accordion; quiz lesson (80% to pass) |
Part 2
Learners build on their understanding of tiered risk triage with the Wayline voice and tone guidelines they'll use when editing lower- and medium-tier drafts.
| Section | Min | Objectives | Activity | Rise approach |
|---|---|---|---|---|
| Review | 4 | 1 | Learners sort a lower-, medium-, and high-tier draft and get feedback on each. They then see how good triage paid off: an author's correct escalation kept a promissory draft from publishing and spared the company a compliance review. | Knowledge checks with feedback; text; quote |
| Promotional phrases: cut or escalate | 11 | 1, 4 | Promotional phrases are marketing-style lines an AI draft adds, such as "Investing made easy" or "Start growing your wealth." Learners see a two-part test for these phrases in lower-tier drafts and why it lets them cut harmless filler without sending every phrase to compliance. They compare phrases they can cut with claims that need compliance, then sort eight samples into each. | Process; flip cards; sorting activity; accordion |
| Voice and tone at Wayline | 15 | 2, 3 | Learners see how Wayline's voice stays constant while tone shifts with the situation, and why that shared vocabulary matters for the rubric. They work through the four voice attributes with examples, then check their ability to spot attribute violations and poor tone fit. | Text; tabs; before-and-after comparisons; knowledge checks |
Part 3
Learners apply risk tiers and voice standards to full drafts by editing, rebalancing, and escalating them, then compare their rubric scoring with an expert's.
| Section | Min | Objectives | Activity | Rise approach |
|---|---|---|---|---|
| Introduction | 3 | — | Learners review the tiers and cut-or-escalate decisions, then hear a customer describe a dismissive reply to a delayed deposit. They see why tone isn't polish: a dismissive reply can cost a customer's trust and their account. | Text; video |
| Editing lower-tier drafts | 9 | 2 | Learners see voice and tone problems labeled in an annotated draft and why each one misses the standard. They edit a full draft, then compare it with a model edit to see where their judgment differs. | Labeled graphic; edit in a provided doc; accordion for model edit; rubric self-check |
| Rebalancing education content | 6 | 3 | Learners see why vague hedging isn't the same as stating the risk in education content. They compare unbalanced and balanced versions, rewrite one risk passage, then check their understanding. | Side-by-side comparison; model answer reveal; knowledge check |
| Escalating | 6 | 4 | Learners see in a branching scenario what happens when a high-tier draft is edited instead of escalated. They then write an escalation note from a template that names the risk trigger. | Scenario with branches |
| Calibration | 6 | 2, 3 | Learners score a draft with the rubric and see why consistent scoring holds drafts to one standard across teams. They then compare their scores with an expert's scores and rationale. | Multiple response; expert scoring reveal |
Post-assessment
A standalone 15-minute assessment, taken after Part 3, that measures each objective against its mastery standard and compares with the pre-assessment.
| Section | Min | Objectives | Activity | Rise approach |
|---|---|---|---|---|
| Post-assessment | 15 | All | Triage 10 unseen drafts; edit 1; rebalance 1; escalate 1 with a note | Knowledge checks; edits submitted by form |
Each part runs 30 minutes, so authors can finish it in one sitting. The post-assessment is a separate 15-minute module taken after Part 3, so it has room for enough items to measure each mastery standard. Parts are released a week apart: the spacing supports retention, and Part 2's voice and tone section lands just before Part 3's editing practice. Each part opens with a short introduction that grounds the problem in a case or a customer's experience; in Parts 2 and 3 it also recaps the tiers and the threshold test. Sections unlock in order on first pass; learners can revisit any completed section.
Sequencing rationale
- A case-based introduction to each part: each part opens with a cost or a customer's experience, so authors see why the skill matters before they practice it.
- A lesson for each tier: each lesson shows the problems that define its tier where they appear in a real draft, so authors learn to spot them in context, and an in-flight check catches misconceptions before mixed practice.
- An end-of-part check: Part 1 closes with a scored check on unseen drafts, so the part a learner finishes ends with evidence of the skill it taught.
- Cut or escalate before voice and tone: authors settle what to escalate before learning how to edit what remains.
- Voice and tone before editing practice: gives authors a shared vocabulary for Wayline's standard, which the rubric depends on.
Escalation branching scenario
| Choice | Consequence | Next |
|---|---|---|
| A support specialist forwards an AI draft explaining a customer's agent trade and asks you to "tidy it up." | ||
| Edit the draft and send it back | The draft publishes with an invented reason for the trade; the customer disputes it and compliance opens a review | Feedback, then retry |
| Route it to compliance with a note | Compliance reviews and responds within the review window | Write the escalation note |
| Reply that agent trades aren't your area | The specialist sends the draft as is | Feedback, then retry |
Beyond the series
- Job aid: a single one-page downloadable reference to the tiers and the cut-or-escalate rule, available at the end of Part 3 and shared with support specialists and product designers.
- Follow-up practice: short practice sets at two and four weeks.
- Accessibility: drafts presented as text, not images; keyboard-accessible alternatives for sorting; feedback never relies on color alone; WCAG 2.1 AA.
Wayline is a fictional company. This is a fictional scenario devised as source material for an instructional design case study only.
Assessment
I designed the assessment to measure the judgment the course teaches: deciding a draft's tier, then editing based on the voice and tone guidelines, rebalancing, or escalating it. A pre-assessment early in Part 1 sets each learner's baseline, practice checks give feedback throughout the series, and a 15-minute post-assessment after Part 3 measures each objective against its mastery criteria.
Beyond the course, I tied the evaluation plan to the business goals. On-the-job sampling at 30 and 60 days shows whether authors apply the skills, and the content audit at 90 days and 6 months shows whether quality and compliance improve. Because the AI tool fix ships first, its results become the baseline, so the plan can show what the training adds on its own.
Assessment blueprint
| Assessment | When | Items | Scoring |
|---|---|---|---|
| Pre-assessment | After the introduction in Part 1 | Triage five drafts; edit one draft | Assessment uses the same rubric as the final quiz to set each learner's baseline. |
| Practice checks | Following substantive content throughout the course | Triage, phrase sorting, edits with model-answer comparison | Formative; feedback only |
| Post-assessment | After Part 3; 15 minutes | Triage ten drafts; edit one low-risk draft; rebalance one medium-risk draft; escalate one high-risk draft with a note | Scored against the mastery criteria below |
Mastery criteria
| Objective | Mastery on the post-assessment |
|---|---|
| Triage | At least 9 of 10 drafts sorted into the correct tier, with no high-tier draft marked for editing |
| Edit | The edited lower-tier draft meets at least 4 of the 5 rubric criteria |
| Rebalance | The medium-tier draft gives risks detail comparable to the benefits, and the edit adds no recommendation or promise |
| Escalate | The high-tier draft goes to compliance unedited, with a note that names the risk trigger |
| Part 1 check | At least 4 of 5 unseen drafts triaged correctly (80%), at the end of Part 1 |
Evaluation plan
| Level | Measure and method | Timing | Target |
|---|---|---|---|
| 1 Reaction | Ratings of relevance and usability on a 5-point survey, plus one open question | End of Part 3 | Average rating of 4 or higher |
| 2 Learning | Triage accuracy and edit quality on the pre-assessment and post-assessment | Part 1 and after Part 3 | Every learner meets the mastery criteria |
| 3 Behavior | Share of each author's sampled drafts that meet all voice and tone standards, scored monthly with the rubric | 30 days 60 days | 60% of drafts 70% of drafts |
| Share of each author's escalations that compliance finds unnecessary, from the escalation log | 30 and 60 days | Under 10% | |
| 4 Results | Share of all AI-assisted content that meets all voice and tone standards, from a sampled audit | 90 days 6 months | 70% 80% or more |
| Share of published content with regulated language that skipped compliance review, from the same audit | 90 days 6 months | Under 5% Under 2% | |
| Share of lower- and medium-tier drafts escalated unnecessarily, from the escalation log | 90 days and 6 months | Under 10% | |
| Customer trust, from a repeat of the audit's customer interviews | 6 months | Fewer trust concerns than the baseline interviews |
Wayline is a fictional company. This is a fictional scenario devised as source material for an instructional design case study only.