Mastery Learning in Medical Simulation

Heidi Felix; Kimberly Schertzer

Mastery Learning in Medical Simulation

Free Review Questions

Introduction

Simulation-based education is a broad term that encompasses interactions with mannequins and standardized patients, along with practicing procedural skills. Simulation represents an important facet of undergraduate and graduate medical education and provides an opportunity to practice skills before performing them on patients. The simulation environment represents a perfect opportunity for the deliberate practice of complex skills until mastery is achieved for routine and low-frequency, high-stakes procedures.[1] The process of deliberate practice requires time and motivation, along with goal-oriented feedback at multiple points in time; the process is outcomes-based, not time-based.[2] Mastery learning does not focus on standardizing teaching processes but rather on the process of learning. In doing so, variations in learner outcomes are lessened, leading to improved patient safety and cost-effective care, a vital construct given the continuing shift in healthcare delivery systems.[3] Simulation-based mastery learning represents an opportunity to solve complex drivers identified as areas needed for improvement in patient safety: communication, teamwork, procedural competency, and knowledge of high stakes or rare events.[3]

Curriculum Development

When designing a curriculum encompassing mastery learning concepts, the Angoff method has been widely accepted to identify when mastery is achieved.[4] Traditional education models utilize a minimum acceptable score for passing and moving on to the next level; however, the Angoff method does not account for critical versus noncritical procedural skills. Newer methods, such as the Mastery Angoff method and the Patient-Safety approach, have been proposed, which raises the bar on what should be considered a minimum level of competency.[5] As with introducing any new scoring method for competency, validity evidence with the measurement tool should be utilized to ensure proper assessment of learners.[6][7] Lineberry et al. provide information to consider when developing and implementing assessment tools. Faculty should determine how to use and interpret assessments, gauge the response to feedback, determine the reliability and structure of the assessment, determine the relationship to other variables, and determine the potential intended and unintended consequences of assessment.[7]

Complex procedures and high-stakes clinical scenarios can introduce cognitive overload and psychological stress to learners, especially novices. These can be broken down into micro-skills, introducing each procedure component 1 at a time and building upon it with the next step until learners have reached a level of competency that represents mastery.[1] This process differs from traditional teaching in that no time limit is placed on learning the technique; a faculty member oversees the process and introduces any remediation necessary to achieve mastery.[8] When outlined, a simulation-based mastery learning curriculum focuses on the following: baseline testing, learning objectives that escalate in difficulty level, educational initiatives to meet the learning objectives, a competency level to meet or exceed, periodic formative assessment, a feedback loop, and ability to continually practice until achieving mastery.[9] The feedback loop should be planned out in advance based on the type (feedback versus true debriefing), source (self, peer, video, subject matter expert, simulator data), and timing (in-action, immediate, delayed). Faculty should also set the stage to highlight the selected debriefing process and provide feedback to maintain psychological safety. Feedback and debriefing can be an uncomfortable process.[10]

Procedural Skills Assessment

Mastery learning has been extensively studied for the acquisition of procedural skills. As learners perform these procedures better, patients experience fewer complications and shorter lengths of stay.[4][11] Cost analysis also demonstrates a return on investment when considering the cost of task trainers and faculty time.[11] Mastery learning theory is supported by the body of literature, which demonstrates that learners perform worse over time when there are intervals of time when a certain procedure is not performed.[12]

In 1 study, deliberate practice was shown as an effective method to overcome gender differences in procedural skills. Historically, males have achieved higher scores on the Fundamentals of Endoscopic Surgery. Ritter et al demonstrated that a curriculum focused on mastery learning supported by deliberate practice can eliminate these disparate results virtually.[13]

Medical Decision Making and Leadership Development

Mastery learning has also been studied for clinical skills. A recent study examined its utility in achieving Entrustable Professional Activities (EPAs), core skills that are part of competency measures in undergraduate medical education. The EPAs are various skills that medical students should achieve before entering residency and be able to complete without supervision.[14]

Robinson et al studied the effect of an online training program for detecting melanomas. Their analysis revealed that a mastery training program decreased unnecessary referrals to dermatologists for benign skin lesions and led to an increase in referrals for melanomas.[15]

Another study examined mastery learning for communication skills, specifically difficult conversations like breaking bad news. This is an important method for assessing learners' skills that affect patient-provider relationships.[16]

Clinical Significance

Mastery learning supports competency-based education (CBE). The shift towards CBE derived from the need for supplemental education with the advent of the mandate of reduced working hours in graduate medical education. Residents have subsequently had less exposure to the clinical environment; therefore, a more systematic approach to acquiring skills is necessary.[4] Evidence shows that current case volumes are insufficient for endoscopic surgical skills acquisition during residency programs. Simulation-based mastery learning programs have shown the potential to be employed as a method of filling this gap and lead to higher passing rates on board certification exams.[17]

Skills retention has been shown to wane over time. Mastery learning has been studied to overcome skill degradation and increase patient safety.[2][18][4][18][4][19] However, skill and knowledge degradation can occur even after mastery has been achieved, especially in low-frequency events. A multicenter study showed evidence of degradation starting at 2 months. Subsequently, the authors advocate for frequent retraining before six months after the initial mastery training. However, this was based on a 1-hour training block, and the interval at which re-training should occur may be based on the intensity and length of the original mastery training.[19]

Enhancing Healthcare Team Outcomes

The ability for teams to participate in iterative sessions, especially for resuscitation, positively contributes to patient outcomes. Secondly, deliberate practice can also lend itself to improved confidence for those serving as team leaders in code situations.[20]

Details

References

[1]

Petrosoniak A, Lu M, Gray S, Hicks C, Sherbino J, McGowan M, Monteiro S. Perfecting practice: a protocol for assessing simulation-based mastery learning and deliberate practice versus self-guided practice for bougie-assisted cricothyroidotomy performance. BMC medical education. 2019 Apr 5:19(1):100. doi: 10.1186/s12909-019-1537-7. Epub 2019 Apr 5 [PubMed PMID: 30953546]

[2]

Friederichs H, Marschall B, Weissenstein A. Simulation-based mastery learning in medical students: Skill retention at 1-year follow up. Medical teacher. 2019 May:41(5):539-546. doi: 10.1080/0142159X.2018.1503411. Epub 2018 Oct 17 [PubMed PMID: 30332904]

[3]

Dunn W, Dong Y, Zendejas B, Ruparel R, Farley D. Simulation, Mastery Learning and Healthcare. The American journal of the medical sciences. 2017 Feb:353(2):158-165. doi: 10.1016/j.amjms.2016.12.012. Epub 2016 Dec 16 [PubMed PMID: 28183417]

[4]

Bisgaard CH,Rubak SLM,Rodt SA,Petersen JAK,Musaeus P, The effects of graduate competency-based education and mastery learning on patient care and return on investment: a narrative review of basic anesthetic procedures. BMC medical education. 2018 Jun 28; [PubMed PMID: 29954376]

Level 3 (low-level) evidence

[5]

Barsuk JH, Cohen ER, Wayne DB, McGaghie WC, Yudkowsky R. A Comparison of Approaches for Mastery Learning Standard Setting. Academic medicine : journal of the Association of American Medical Colleges. 2018 Jul:93(7):1079-1084. doi: 10.1097/ACM.0000000000002182. Epub [PubMed PMID: 29465449]

[6]

Østergaard ML, Nielsen KR, Albrecht-Beste E, Konge L, Nielsen MB. Development of a reliable simulation-based test for diagnostic abdominal ultrasound with a pass/fail standard usable for mastery learning. European radiology. 2018 Jan:28(1):51-57. doi: 10.1007/s00330-017-4913-x. Epub 2017 Jul 4 [PubMed PMID: 28677051]

[7]

Lineberry M, Soo Park Y, Cook DA, Yudkowsky R. Making the case for mastery learning assessments: key issues in validation and justification. Academic medicine : journal of the Association of American Medical Colleges. 2015 Nov:90(11):1445-50. doi: 10.1097/ACM.0000000000000860. Epub [PubMed PMID: 26287919]

Level 3 (low-level) evidence

[8]

Liepert AE,Velic AJ,Rademacher B,Blumenfeld AA,Bingman E,O'Rourke AP,Sullivan S, Proficiency development for graduating medical students, using skills-level-appropriate mastery learning versus traditional learning for chest tube placement: Assessing anxiety, confidence, and performance. Surgery. 2019 Jun; [PubMed PMID: 30851948]

[9]

Barsuk JH, Cohen ER, Wayne DB, Siddall VJ, McGaghie WC. Developing a Simulation-Based Mastery Learning Curriculum: Lessons From 11 Years of Advanced Cardiac Life Support. Simulation in healthcare : journal of the Society for Simulation in Healthcare. 2016 Feb:11(1):52-9. doi: 10.1097/SIH.0000000000000120. Epub [PubMed PMID: 26536342]

[10]

Eppich WJ, Hunt EA, Duval-Arnould JM, Siddall VJ, Cheng A. Structuring feedback and debriefing to achieve mastery learning goals. Academic medicine : journal of the Association of American Medical Colleges. 2015 Nov:90(11):1501-8. doi: 10.1097/ACM.0000000000000934. Epub [PubMed PMID: 26375272]

[11]

Schwab B, Teitelbaum EN, Barsuk JH, Soper NJ, Hungness ES. Single-stage laparoscopic management of choledocholithiasis: An analysis after implementation of a mastery learning resident curriculum. Surgery. 2018 Mar:163(3):503-508. doi: 10.1016/j.surg.2017.10.006. Epub 2017 Nov 27 [PubMed PMID: 29191675]

[12]

Baumann LM, Barsness KA. The Case for Simulation-Based Mastery Learning Education Courses for Practicing Surgeons. Journal of laparoendoscopic & advanced surgical techniques. Part A. 2018 Sep:28(9):1125-1128. doi: 10.1089/lap.2017.0656. Epub 2018 Mar 12 [PubMed PMID: 29641367]

Level 3 (low-level) evidence

[13]

Ritter EM, Lineberry M, Hashimoto DA, Gee D, Guzzetta AA, Scott DJ, Gardner AK. Simulation-based mastery learning significantly reduces gender differences on the Fundamentals of Endoscopic Surgery performance exam. Surgical endoscopy. 2018 Dec:32(12):5006-5011. doi: 10.1007/s00464-018-6313-y. Epub 2018 Jul 16 [PubMed PMID: 30014324]

[14]

Salzman DH, McGaghie WC, Caprio TW, Hufmeyer KK, Issa N, Cohen ER, Wayne DB. A Mastery Learning Capstone Course to Teach and Assess Components of Three Entrustable Professional Activities to Graduating Medical Students. Teaching and learning in medicine. 2019 Apr-May:31(2):186-194. doi: 10.1080/10401334.2018.1526689. Epub 2018 Dec 31 [PubMed PMID: 30596271]

[15]

Robinson JK, Jain N, Marghoob AA, McGaghie W, MacLean M, Gerami P, Hultgren B, Turrisi R, Mallett K, Martin GJ. A Randomized Trial on the Efficacy of Mastery Learning for Primary Care Provider Melanoma Opportunistic Screening Skills and Practice. Journal of general internal medicine. 2018 Jun:33(6):855-862. doi: 10.1007/s11606-018-4311-3. Epub 2018 Feb 5 [PubMed PMID: 29404948]

Level 1 (high-level) evidence

[16]

Vermylen JH, Wood GJ, Cohen ER, Barsuk JH, McGaghie WC, Wayne DB. Development of a Simulation-Based Mastery Learning Curriculum for Breaking Bad News. Journal of pain and symptom management. 2019 Mar:57(3):682-687. doi: 10.1016/j.jpainsymman.2018.11.012. Epub 2018 Nov 23 [PubMed PMID: 30472316]

[17]

Ritter EM, Taylor ZA, Wolf KR, Franklin BR, Placek SB, Korndorffer JR Jr, Gardner AK. Simulation-based mastery learning for endoscopy using the endoscopy training system: a strategy to improve endoscopic skills and prepare for the fundamentals of endoscopic surgery (FES) manual skills exam. Surgical endoscopy. 2018 Jan:32(1):413-420. doi: 10.1007/s00464-017-5697-4. Epub 2017 Jul 11 [PubMed PMID: 28698900]

[18]

Lipsky MS, Cone CJ. A review of mastery learning: The roseman model as an illustrative case. Education for health (Abingdon, England). 2018 Jan-Apr:31(1):39-42. doi: 10.4103/1357-6283.239045. Epub [PubMed PMID: 30117471]

Level 3 (low-level) evidence

[19]

Braun L, Sawyer T, Smith K, Hsu A, Behrens M, Chan D, Hutchinson J, Lu D, Singh R, Reyes J, Lopreiato J. Retention of pediatric resuscitation performance after a simulation-based mastery learning session: a multicenter randomized trial. Pediatric critical care medicine : a journal of the Society of Critical Care Medicine and the World Federation of Pediatric Intensive and Critical Care Societies. 2015 Feb:16(2):131-8. doi: 10.1097/PCC.0000000000000315. Epub [PubMed PMID: 25647122]

Level 1 (high-level) evidence

[20]

Rice J, Omron R, Calkins H. Optimizing Mastery Learning Environments: A New Approach to Deliberate Practice for Simulation-based Learning. AEM education and training. 2018 Apr:2(2):77-81. doi: 10.1002/aet2.10083. Epub 2018 Feb 27 [PubMed PMID: 30051072]