Simulation Training and Skill Assessment in Emergency Medicine

Issues of Concern

Cost is the most commonly cited issue of concern regarding simulation teaching.[6] The cost concern comes from a misunderstanding of the goals of simulation. The objectives of the simulation are to provide experience, education, and skill acquisition through simulated environments. These goals do not have to be accompanied by a high cost. Many "simulated environments" exist that are state of the art and do have an enormous cost associated with their construction. Commonly thought of are flight/launch simulators, virtual reality environments, or surgical operating rooms replete with fully functioning mannequins. Additionally, medical simulation centers are becoming more widely built, and within these centers are significant amounts of technology ranging from audio/video recording equipment, interactive presentation platforms, and even holographic projection models. While technology can assist with achieving the learning objectives and does provide a high level of fidelity, simulated teaching objectives do not necessarily require this high fidelity technology to achieve their goal.

Also, learners are provided in a never-ending stream from academic year to year. Unfortunately, inanimate objects do not heal like live tissue, and models have to be reconstructed, and some parts reused. From time to time, a replacement can become more costly. The main cost incurred is usually from purchasing commercially developed training models. The field of simulation has given rise to a significant number of educators that have developed their own "home brews" and recipes that are cost-effective given the disposable and renewable nature of some teaching models. For example, peripheral IV placement is a common procedure, and due to the frequency of the procedure coupled with the volume of learners who require proficiency in the procedure, peripheral IV task trainers are not expressly reusable and require regular replacement. These models do not need to have significant costs associated with their construction and can effectively achieve the goal of the task objectives.

More germane to medical specialties and specifically Emergency Medicine is learning how to think abstractly in preparation for oral board examinations. Oral board examinations are nothing more than simulated cases that require cognitive thought processes, and continued training and preparation for oral board exams have been shown to lead to higher pass rates.[7][8][9] Emergency Medicine residency programs now incorporate simulated oral boards as part of their routine training regimen.[10] Moreover, the oral board cases are constructed around real-life cases and emergencies that the physician will encounter throughout their career. Construction of an oral board case requires essentially no cost other than the instructor's time. Cases and scenarios can be designed for any profession and situation with little to no cost associated and have the benefit of achieving objectives and bridging gaps in knowledge.

Curriculum Development

Resident training is quite a dynamic process. Training in any healthcare profession is quite dynamic. No two residents/students will have the same experience during their shifts, rotations, or entire training experience. Therefore, programs are obligated to ensure each resident has the knowledge and requisite skill to provide life-saving and stabilizing care to any patient that should present to the emergency department in which they work. Given the environmental dynamics relative to the individual programs, departments, and individual experience, simulation has become a more standard way to ensure that each resident has met the requirements to be proficient in their practice of Emergency Medicine. Various researchers over the past millennia have developed several models and educational theories. Knowledge of these theories is paramount in understanding the multiple ways in which people, and in this case, adults, learn. Understanding how adults learn is directly related to the effectiveness of teaching scenarios.

A few widely described educational theories:

• Malcolm Knowles Adult Theory of Andrology:
• Cognitive load is the amount of information the working memory can store at any given time.
• Maintains that working memory has a limited capacity, and instructional design should seek to avoid overloading working memory.
• Suggest that as learners advance to expert levels, the learners develop "schema" or cognitive frameworks that allow the expert to work systematically through problems.
• Sweller's Cognitive Load Theory:[11]
• Adults learners are most engaged when they are involved with the planning, implementation, and evaluation of their learning.
• Experiential based learning platforms are most effective for adult learners.
• Adults care and are more attentive to ideas and subjects that have a direct and immediate bearing on their profession.
• Adults achieve learning through problem-oriented teaching rather than subject-based teaching.
• Kolbs Experiential Cycle Theory:
  • Experiences occur within social settings, and knowledge is organized and constructed based on those real-life social experiences.
  • Therefore, new knowledge should be presented in the context of real-life experiences.
  • Instructors should organize content and subject matter such that they grounded in real-life experiences.
  • The real-life experiences should match the learner's realm of capability, aptitude, and capacity.
  • The quality of the experience directly relates to the learner's ability to apply the newfound knowledge.
  • The newfound knowledge allows the learner to repeat the steps to continue new knowledge acquisition.
• Dewys Speculative Theory:
• Concrete Experience: Learner encounters a new experience or revision of prior encounters.
• Reflective Observation: Learner reflects on variations of their perceptions and knowledge versus the experience or encounter.
• Abstract Conceptualization: Learner's reflection leads to the development of new ideas, concepts, and/or skills.
• Active Experimentation: The learners apply the newly generated ideas, concepts, and/or skills to their world and observes their effect.
• This is a cycle whereby, after active experimentation, the learner has created a new "concrete experience" and thus repeats the steps.
• Dreyfus/Benner Model:
• Novice: Learners are rigid in following the rules, plans, or steps and have not yet learned to exercise discretionary judgment.
• Advanced Beginner: Have advanced, however, are still limited to situational awareness and have not progressed to prioritization of tasks and ideas.
• Competent: The learner begins formulating routines and plans and can now multitask concerning the activity and has generated some preception of ultimate goals.
• Proficient: Learners are now able to understand deviations of "norms" and develop independent holistic views of situations.
• Expert: At this level, the learner has now transcended from normalized views and proceeds with a vision of what could be rather than what is.

When constructing simulation scenarios, the designer must keep in mind these basic education and learning theories as guides to the design of effective scenarios for their adult learners. Simulation scenario objectives should be designed as such to focus on these three broad areas.[12]

• Cognitive: What knowledge will be applied?
• Psychomotor: What actions will be performed, skill, or otherwise?
• Affective: What feelings will be generated?

An objective, however, does not need to encompass all three areas to be effective and may simply focus on one task. Objective designers also must keep in mind the who, what, where, and how so that objectives can be matched properly to the learners. For example:[13]

• Who is the intended audience? (I.E., PGY3 EM residents, First Year Paramedic Students, Practicing ICU Nurses, Commerical Airline Pilots, etc.)
• What would be the behavior or action that the objective is attempting to elicit? (I.E. The EM resident will intubate withing 2 minutes of scenario onset)
• Where are the conditions upon which the scenario/simulation is occurring? (I.E., You are a paramedic called to a multivehicle accident on a busy six-lane highway)
• How in-depth and to what extent is the learner expected to proceed? (I.E., A 78-year-old has witnessed the arrest. The learner is a first-year nursing student. The scenario ends when they call for help)

 Additionally, several aspects have been identified and are commonly employed in and recommended for successful adult learning programs. These include:

• A safe environment is explicitly established. A safe environment allows learners to feel supported and acknowledged.
• The environment allows freedom and creativity and avoids rigid patterns or structures.
• Adult learners are treated as respected and intelligent peers capable of learning and performing to the level of the instructor.
• Learners are inspired to accept responsibility for their learning so that they may achieve professional competence.
• Learner intellect and skill are challenged optimally, meaning challenges should be designed to extend just beyond the learner's present ability.
• Learners are actively engaged in the learning process. Active engagement may include learner-led lectures, encouragement of questions, and dynamic instructor/learner dialogue.
• Learners are encouraged, and mechanisms provided to allow adequate feedback.

Debriefing is critical to successful simulation programs. A debriefing occurs following the simulation scenario. Debriefing differs from feedback as debriefing is a facilitated and active discussion between the learners involved in the scenario and the instructor.[10] Feedback is typically a comparison of a learner's performance to a standardized set of norms. With debriefing, the learner reflects and gains an understanding of their actions to establish insight to change future behavior. With feedback, the learner is simply instructed on what could be done differently. Psychological safety is important and should be established at the onset of the simulation and reaffirmed throughout the exercise. Effective debriefers often use open-ended questions to allow learner reflection. Similarly, active listening and engaging, non-threatening non-verbal communication are directly related to the effectiveness of the debriefer. Validation, support, and summarization of the learner's experience are also helpful tools to facilitate reflection. The debriefer should avoid judgmental statements and generally engage in inquisitive thought-provoking questions.

Clinical Clerkships

Simulation instructors have a wide variety of tools at their disposal to assist with achieving individual learning objectives. Simulation designers must ensure that the fidelity (or realism) and the complexity of the tools chosen match the learning objectives as well as the learner's ability. For instance, first-year medical students would likely not benefit from running codes on high fidelity manakins just as seasoned practicing emergency medicine physicians would likely not benefit from gathering medical histories form standardized patients. Some simulation modalities are listed below:

• Task Trainers: Devices or models used for training a particular task: Can be as mundane as store-bought chicken legs to practice suturing to more advanced such as pericardiocentesis models.[14]
• High Fidelity Manakins: Manakins that closely resemble live patients with technology imbedded to simulate disease or physical exam findings
• Standardized Patients: These are actors trained to represent patients and disease.
• Hybrid Simulations: Simulation that incorporates multiple modalities to achieve objectives. For instance, a standardized patient could present in a hypertensive emergency requiring arterial line placement with a task trainer provided to perform the arterial line.
• Virtual Reality: Simulated virtual environments to train specific actions and tasks.[15][16]
• Games: These are simulation designs where learners are challenged to "win" the game with objectives designed to foster learning. SIM Wars are games designed to inspire competition and foster learning procedural skills.[17][18][19][20]

Procedural Skills Assessment

Below is the ACGME document regarding Emergency Medicine Skills requirements that residents must meet upon graduation. This text was adapted from the ACGME website:[21]

"Emergency Medicine Defined Key Index Procedure Minimums Review Committee for Emergency Medicine The following are key index procedures identified by the Review Committee as essential to the independent practice of emergency medicine (based on the Program Requirements, the Emergency Medicine Milestones, and the Model of the Clinical Practice for Emergency Medicine). Residents are required to perform the minimum numbers indicated for each key index procedure below by the time of graduation from the program. Procedure Minimum

• Adult Medical Resuscitation 45
• Adult Trauma Resuscitation 35
• Cardiac Pacing 6
• Central Venous Access 20
• Chest Tubes 10
• Cricothyrotomy 3 
• Dislocation Reduction 10
• ED Bedside Ultrasound 150
• Intubations 35
• Lumbar Puncture 15
• Pediatric Medical Resuscitation 15
• Pediatric Trauma Resuscitation 10
• Pericardiocentesis 3
• Procedural Sedation 15
• Vaginal Delivery 10

No more than 30 percent of required logged procedures performed in simulated settings can count toward the required minimum, with the exception of rare procedures, namely pericardiocentesis, cardiac pacing, and cricothyrotomy. One hundred percent of these rare procedures may be performed in the lab."

Procedural skills have traditionally been assessed through observation and teaching as the learner is performing the skill.

With simulated skill training, the learner may be evaluated in a formative manner, with a checklist addressing each step of the procedure as well as the associated cognitive steps.

Clinical Significance

Management of emergent medical cases is at the crux and the focal point of emergency medicine resident training. Simulation has the added benefit of allowing medical providers to practice stressful and high-risk scenarios in an environment that is safe for the patient and safe for the practitioners. Almost invariably, the emergency physician is viewed as the leader of the resuscitative (or code) team. Simulation is a natural extension to teach team leadership and communication skills. In code situations, the EM physician is usually assisted by an extensive interdisciplinary team with varying degrees of skill and knowledge. These team members may range from students, paramedics, nurses, respiratory therapists, and medical assistants. Of paramount importance is that the team leader can effectively communicate with each team member and rapidly asses the skill levels and knowledge present within those team members.[23] By designing team-based medical scenarios, instructors are more easily able to teach and assess learner communication and leadership skills. Additionally, reflective scenario debriefing can help address individual, departmental, and or institutional gaps in knowledge or administration of best practice measures. Effective simulation can also allow more formalized processes to be instituted to ensure that these best practices are followed, and gaps in knowledge are addressed.

The American Heart Association (AHA) and American College of Surgeons (ACS) as national professional medical organizations have both incorporated simulation into the certification process for:[22]

• Basic Life Support (BLS)
• Advanced Cardiac Life Support (ACLS)
• Pediatric Advanced Life Support (PALS)
• Advanced Trauma Life Support (ATLS)

For instance, a student or resident learning ACLS may be presented with a "mock" or simulated code in which the patient will arrive in Pulseless Electrical Activity (PEA). The learner would be expected to proceed through the cognitive process of identifying the causes of PEA arrest and working through the treatments of those causes. The instructor may design the case so that any of the causes of PEA arrest is the causative factor, and the case scenario would end when the learner correctly treats the rhythm.[23] ATLS instructors demonstrate systematic approaches to the evaluation of trauma patients with a student-led simulation of life-saving trauma procedures. ATLS culminates with the student performing a wholistic simulated treatment of a trauma patient. In each of these situations, formative assessments are performed to ensure those receiving certifications are proficient and that critical actions are taken. However, in the event a learner performs suboptimally, the learner is allowed time for self-reflection and debrief to reassess their approach before reattempt. Certainly, as a profession, learners experiencing success and/or failure in the simulated environment is preferable and enhances the provider's ability to manage critical patients. 

Pearls and Other Issues

Common Pitfalls encountered when designing simulation scenarios include the following:[24]

• Lack of Realism: The scenario does not match real-world experience, and the learner is disengaged.
• Excessive Complexity: The scenario is so complex that the learner is overloaded cognitively, and the learning objectives are not met.
• Ineffective Scenario: The scenario design is such that the learner has been diverted from the objectives, and performance cannot be ascertained based on the design. 
• Unattainable Objectives: The learners have not reached the requisite level of knowledge, training, or experience to reasonably attain the objectives. 
• Objectives Overload: The scenario incorporates too many objectives, again confusing the learner and overloading the learner cognitively.
• Abbreviated Debriefing: The learners do not have adequate time to process their own performance, learning objectives, and educational points from the scenario.

When designing a simulation scenario, these pitfalls should be kept in mind as the objective of the scenario, whether formative or not, is to assist the learners in achieving the learning objectives rather simply performing the scenario.

Enhancing Healthcare Team Outcomes

All simulation programs have a mandate to keep patients and learners safe. Healthcare simulation education's primary general goal is to enhance the learner's ability to deliver safe, effective, and evidence-based medical care in a compassionate and empathic manner. Overall, medical training simulation's goal is to enhance individual competence, fulfillment, and confidence while enhancing healthcare outcomes throughout departments and healthcare systems. Effective simulation programs adhere to these basic ethical principles:[25]

• Beneficence: Do good.
• Non-maleficence: Do no harm.
• Justice: Equitable treatment.
• Autonomy: Respect the individual right to choose.

While studies have consistently demonstrated that simulation for procedural skills acquisition is effective and allows more repetition of the skill in a safe environment, the establishment of psychological safety again, is of paramount importance. Simulation is stress-inducing, not unlike the stress some students feel during examinations.[26] Instructors must balance active educational objectives against the creation of an environment that impedes the learner's ability to think reflectively. Moreover, instructors must also balance providing overconfidence in learners with learners' realistic limitations of ability. Recording of performances is also stress-inducing for student learners as they would be concerned with the uses of the video. Policies should be developed to address the use and storage of training videos to respect individual autonomy. Standardized patients are also subject to the ethical ethos of simulation and, as such, should be treated in the same way as the learners. Lastly, simulation has the benefit of eliminating the ethical concerns over the use of animal models. The converse question then becomes whether manikins or task trainers are adequate to provide training for dealing with individual human anatomical, functional, and cognitive variables.