Simulation Training and Skill Assessment in Orthopedic Surgery

Function

Surgical education has undergone significant recent changes. The apprenticeship model described as Halsted’s “see one, do one, teach one” model, has been the foundation of surgical training for over a century.[9][10] The principles of this model are based on observation of a procedure, performing the procedure, and then teaching it. The implementation of the European Working Time Directive (EWTD) has resulted in reduced working hours, thus, reduced training hours in which trainees must learn a wide range of skills and adopt new technologies while maintaining patient safety.[11][12] Furthermore, recent media portrayals of medical negligence (such as the Montgomery case and, more recently, the 2018 Bawa-Garba case) have resulted in a lack of patient confidence in the quality of care provided, uncertainty relating to the competency of doctors, and questions of accountability in light of errors.[13][14]

These factors highlight the need to optimize surgical training worldwide. In the United Kingdom (UK), for example, the context of trauma and orthopedic training has shifted from a time-based to a competency-based system of training as laid out by the Royal College of Surgeons and Health Education England. Defining clinical competency is essential to allow validated training, which permits benchmarking of clinical knowledge and skills; this, in turn, helps to meet ever-growing patient expectations.[15] Simulation provides a practical way to improve training while giving consideration to reduced training time and other barriers to on-the-job training.

Issues of Concern

Trainees currently have reduced training opportunities in the operating room (OR) due to service provision demands and arduous on-calls. Simulation shortens the learning curve in surgical training and improves knowledge retention and transference of skills through DP.[16][17] However, access to simulation facilities is a significant barrier and a key area of improvement; this may be due to a lack of resources available, costs, or lack of implementation of simulation-based technology into the surgical curriculum.

Although improved access may increase the uptake of simulation resources for DP- this strategy is based on the premise that the rota will permit simulation training time. Another key area of improvement is the provision of structured simulation training time within the working week. Finally, feedback is essential in supporting behavioral change and preventing internalization of erroneous skills and behaviors; therefore, feedback should be provided by trainers and sought by trainees promptly.[18]

Curriculum Development

A range of simulation modalities exists, from low-fidelity  ‘bench models’ to high-fidelity virtual reality simulators.[12] Fidelity is defined as the accuracy with which a simulation modality represents reality. High fidelity models have been shown to be effective learning tools.[7] Patient-focused simulation is also described in the literature as a means of obtaining patient feedback during surgical skill acquisition, adding realism to the simulation, and allowing practicing non-technical skills such as communication and teamwork.[19] Simulation has been a part of orthopedic training for several years with the use of cadaver models and synthetic bone models to supplement clinical training. However, no formal competency-based simulation training program currently exists in the UK. Arthroscopy simulation using cadaveric specimens or virtual reality (VR) models are also available, and the effectiveness of these in training is the topic of ongoing research. For the simulation to have a significant impact on improving training, it needs to be formally incorporated into the surgical curriculum.

Formative simulation training may be enhanced through self-regulatory learning (SRL); this would also offer a practical strategy to tackle a lack of expert availability. SRL comprises self-awareness, proactiveness to obtain information that will fill gaps in knowledge and internal motivation to monitor and reflect on self-performance, overlapping with key adult learning principles. Increasing the availability of low-fidelity models with high psychological fidelity, such as sawbones, will allow junior orthopedic trainees to practice basic skills such as instrument handling, drilling, and screw insertion in a cost-effective manner.[20] The provision of instructional videos may be sufficient to allow skills practice after working hours when trainers may not be available.[21] Additionally, direct objective feedback from audio-visual (AV) recording of the trainee’s performance and performance data from simulation models will further support formative training that is self-directed. Once a trainee is a self-regulated learner, they may be able to optimize their training through focussing on DP of tasks deemed difficult on low to high-fidelity models and requesting trainer input when needed. Trainees should be made aware of SRL during their early years of surgical training to allow more engagement in self-evaluation and enabling trainees to take ownership of their learning with guidance from clinical supervisors and trainers as needed.  SRL is somewhat embedded in the current surgical curriculum through self-reflection in the e-portfolio.

Current trainee progression is based on an annual review of competence progression (ARCP) following evaluations from colleagues and trainers via Work-Based Assessments (WBAs). WBAs facilitate the provision of regular feedback; however, they have been shown to have a potentially adverse effect on training opportunities due to time taken to carry out and a negative or no overall impact on training.[22] Restructuring how skills are assessed through the implementation of summative simulation-based assessment is an important notion to consider, especially in the context of surgery. Perhaps simulation could also play a potential role in trainee progression and identify those trainees requiring remedial training. More objective research and in-depth cost analysis are necessary before consideration of simulation and technology-enhanced solutions as part of the summative assessment for orthopedic trainees.

Clinical Clerkships

Scheduled Teaching Days

Orthopedic registrars (residents) receive regular teaching; this is often didactic teaching led by senior colleagues or consultants (attendings). To improve learning, we must consider engaging training methods that encourage active participation. Effective simulation-based DP requires structure, and trainees must be facilitated in developing well-defined goals.[23] The concept of a flipped classroom can be utilized here; this will allow the knowledge basics to be delivered online via a virtual learning environment (VLE) or by trainees carrying out private study, such as reading a chapter form a recommended Orthopaedic text prior to their teaching. Trainees will then be expected to utilize the ‘in-class’ face-to-face teaching sessions, now freed up, for simulation enhanced practical skills training using bone models and discussing problems that arose from the online or at-home learning. Some studies suggest that a learning curve plateau may occur after a simulation program; thus, it may be reasonable to offer several sessions throughout the course of training. A structured simulation program incorporating key clinical procedures, appropriate to the year of training, will allow focussed training and enable resources to be rationalized.

On-the-job Training

Trainees should plan educational activities or learning events with trainers well ahead of the training activity such as theatre lists or clinics; this will avoid a top-down approach and stop WBAs being considered summative (rather than a formative) based assessment tool. Also, this will manage trainee perception and reduce the likelihood of WBAs being completed as a tick-box activity.[24][25] Not all consultants will have sufficient dedicated teaching time to supervise simulation-based learning; therefore, junior surgical trainees must be educated in SRL strategies and be able to apply this useful framework. Trainees should undertake self-reflection in the context of SLR and simulation training; this will enable identification of errors and ensure trainees take responsibility to seek feedback and guidance. Feedback, however, remains an invaluable component of effective SRL. All surgical trainers should be informed and utilize feedback tools to give structured feedback; this is discussed further in the ‘Continuing Education’ part of this article.

Procedural Skills Assessment

WBAs are tools used for both formative and summative assessment of trainee performance. Modernising Medical Careers (MMC) brought WBAs into practice in 2007, intending to shift training towards a competency-based model. Formative assessments allow learning through feedback and highlight areas of development, whereas summative assessments determine competence using measurable terms.[26] In the UK, WBAs are a component of the trainee’s Annual Review of Competence Progression (ARCP) which determines whether or not a trainee shows sufficient evidence of competence to allow progression of training; this has the potential of promoting a culture of WBAs being tick-box exercises rather than a tool for personal and professional development.[24] However, feedback is a core element of formative assessments, and if the feedback given to trainees is effective, then it may improve performance, emphasizing the necessity for trainee engagement.[27] Additionally, WBAs are a valuable tool in the concurrent assessment of multiple domains in the clinical environment.[28][29]

Simulation should be incorporated into the early stages of training, such as at the Core Surgical Trainee (CST) level, to evaluate surgical competency and allow more challenging procedural skills development in the OR. Assessment of competence may be achieved through the use of global rating scales or task-specific checklists; the former may be a more reliable method of evaluation; however, the latter may be more appropriate for novices for formative training.[30] The Zwisch model is designed for training in the OR, and it allows assessment of a trainee’s procedural skills through varying stages of supervision required by the trainee.[31] This model is routinely used in current surgical practice, and it may also apply to the distributed simulation setting, which provides the ‘OR experience’ in a safe teaching environment.[3] The distributed simulation techniques are fully immersive and portable, thereby giving the trainer and trainee flexibility in terms of location, accessibility, and providing a viable simulation setup for summative assessment.  

Medical Decision Making and Leadership Development

Peyton’s four-step model is an effective tool in non-technical skills development as well as the transference of technical skills; it comprises of four steps: demonstration, deconstruction, comprehension, and performance.[32] These steps are not only applicable to practical skills; decision making and communication skills can also be improved in a safe and controlled environment through simulation. AV recording of procedures performed using GoPro-type technology or pre-installed camera and microphones in the simulated environment will enable the trainer and trainee to deconstruct their actions and decisions and better understand their areas of development.

Simulation can also be carried out in a team setting to evaluate and improve communication skills, for example, urgent assessment of a patient who has potentially developed acute compartment syndrome may be ‘played out’ in a simulated setting without the need for any advanced technology, and the scenario would include the orthopedic registrar, ward nurse, patient actor and Orthopaedic consultant who the registrar may need to contact. Through observing, conceptualizing, practicing, and performing learned material, trainees are more likely to gain meaning from the learning experience resulting in an increased likelihood of knowledge retention.[16][17]

Continuing Education

Courses such as the Training Orthopaedic Trainers (delivered by the British Orthopaedic Association), Training the Trainers (delivered by the RCSEng, and RCSEd) set out learning outcomes which include the need for trainers to employ technology-enhanced learning (TEL) and simulation where possible. In light of the recent COVID-19 global pandemic, educators have taken to TEL solutions and online platforms, including webinar-based teaching, to educate trainees remotely. The use of such technologies is ideal for adult learners and should be incorporated into training in the post-COVID-19 era. Faculty development days and supervisor training should reflect the use of TEL solutions to enhance locally delivered training. Reflection is already utilized as a learning tool in the current surgical e-portfolio in the UK (Intercollegiate Surgical Curriculum Programme or ISCP); however, faculty should be made aware of newer tools such as online discussion boards which can be implemented to provide a more informal platform for peer and expert discussion and feedback. Training days will be the best way for faculty to practice and learn how to use simulation models. Familiarity with assessment checklists and required competency on simulation models may be achieved through eLearning.

Awareness of educational theory that is relevant to surgical training is invaluable for the delivery of effective teaching. Both faculty and trainees should be aware of a universal model to provide structure to opportunistic clinical teaching. The Briefing, Intraoperative Teaching, Debriefing (BID) model is one such tool which makes operative teaching more deliberate and emphasizes the importance of timely, unambiguous feedback to prevent internalization of erroneous skills and behaviors.[18][33]

Clinical Significance

Simulation in training is already of enormous value in the context of healthcare education, however, unlike other high-risk industry training, the use of simulation training in surgery remains confined to courses which often exceed trainee study budgets and have long waiting lists due to immense demand. High-fidelity modalities such as ‘Sim-man’ are routinely used for Advance Life Support (ALS) courses. Low-fidelity models such as a box-trainers are often used in laparoscopic skills courses; these allow the practice of instrument handling and special awareness.

To optimize orthopedic training using simulation, educators must provide instruction to trainees and implement the need for a level of proficiency in the simulated setting before progression into the OR. Protected time in a simulated environment as part of the trainees’ teaching program with the aim to encourage participation in simulation-based education and engagement with the Dreyfus model of skills development will ensure competence before carrying out skills on a real patient. The Dreyfus model shares the characteristics of trainees at various stages of skills development and highlights the need for trainee assessment from ‘novices’ to ‘advanced beginners.’

Drop-in sessions for clinical skills learning are often offered to medical students to allow DP and experiential learning.[34] Such sessions (in the context of surgical skills) should be provided to surgical trainees, for example, if bone models for practicing ankle open reduction internal fixation (ORIF) or VR simulators for knee arthroscopy are available, then trainees are invited to sign-up and make use of such resources in their own time. Practically, there may be limitations to this approach due to a gap between current and desired trainee competency (The Zone of Proximal Development), highlighting the need for trainer and technical staff availability. Hence, expert feedback is obtainable during planned simulation sessions. Once the trainee is aware of their areas of development, they can pursue DP in their own time using simulation models. The method described has the potential for the trainee to achieve proficiency quicker than traditional methods of training and provides the opportunity to practice skills that may be scarcely encountered in clinical practice.   

Pearls and Other Issues

• Simulation-based education and technology-enhanced learning need to be integrated into the surgical training curriculum.
• Simulation-based proficiency should be incorporated into formative assessments to guide trainees’ level of competence and level of autonomy in the OR.
• Employing simulation to enhance the trainee experience has the potential to improve trainee and public confidence as well as restore public trust.
• Trainers should be suitably qualified in delivering simulation and technology-enhanced training to ensure high-quality training is delivered.
• Provision of structured and timely feedback in all aspects of surgical training is fundamental to trainee development; tools such as the BID model should be considered in all clinical settings.

Enhancing Healthcare Team Outcomes

We must make more use of the extended surgical team to reduce some of the service provision pressure on surgical trainees and allow training time.[35] Specialist nurses and physician associates (PAs) are key team players who can strengthen and support the team to allow the creation of training opportunities for surgical trainees. Certain PAs have specialist interests such as surgery and develop skills to enable them to function at the level of a junior registrar (ST3); hence, they can offer learning opportunities for junior doctors as well as providing them with clinical support. As discussed, simulation allows members of the multidisciplinary team (MDT) to collaborate and practice clinical situations together in a safe environment: this will enable everyone to learn from each other’s experiences through observation and reflection. DP is a more viable training strategy through simulation, allowing surgeons to master procedural and communication skills, inevitably, improving their practice and, in turn, patient care.