Are Multiple Choice Questions A Good Way To Learn Medicine?

Lauren Brewer is a fourth-year medical student at the University of Tennessee Health Science Center.

Dr. Amanda Box is core faculty and research codirector in the University of Tennessee Health Science Center’s emergency medicine program. 

If I were to train for a marathon, and my training was all 5- or 10-milers on my treadmill, I would face some rude awakenings come race day. I don’t need to run a full 26.6 miles every day for practice, but I at least need to simulate some of the conditions. Outdoor runs. Longer runs. Runs in the shoes I’m going to wear on race day. If I were a “real runner,” I would figure out any potential steep inclines and run similar terrains beforehand. The fewer surprises on race day, the better. Much like racing, on test day in medical school, the fewer surprises the better. Most course and board exams have a large multiple-choice question (MCQ) component. Therefore, it would follow that “training” with MCQs is a good way to learn medicine.

Yet medicine is more than just tests and grades. We all know perfect scores don’t automatically translate to good clinical practice. Hence, just 2 years or fewer of medical school is in a formal classroom. We learn on the floor caring for real patients. Repeatedly I’ve heard and experienced that tying a diagnosis to a specific patient is an excellent way to cement it in your mind. On my first day on the wards, I saw a pediatric patient with multiple components of the VACTERL association. Now that association is forever imprinted. Of course, the wards can’t be your first encounter for every concept— that’s not sustainable or reasonable.

So we’re back to that idea of building this base of knowledge. I think the key here is repetition. I learned the basic concept of a pulmonary embolism (PE) in my first year of medical school. I learned about it again in my organ blocks during my second year. I learned it again during my dedicated study for Step 1. I learned it again early in my third year seeing a patient on the wards. At the start of my fourth year, I had a crashing patient in the ED with a saddle embolism, and I placed her central line— I really learned it then! I learned it again preparing for Step 2. I’ll learn it again in residency. I am forgetting and learning new PEs each of these times— but the repetition is strengthening that neural pathway. I now know PEs and their presentations, treatments, etc.

So back to our original question— are MCQs a good way to learn medicine? Based on the retention patterns seen with repetition of information, these MCQs, of which there are tens of thousands available through various resources, are indeed a good way to start solidifying medical knowledge. A quick PubMed search will result in several articles espousing the efficacy of the multiple-choice question. For example, a 2018 study by Pham et al. found that MCQs, when well-written, can test higher-order skills as effectively as short-answer questions[1]. A good MCQ will have “lures”— answers that could at first glance fit the question but are lacking a key defining element upon further inspection. This causes the learner to recall information about various intersecting and contrasting concepts. Reviewing the questions and answers in full, rather than just seeing the correct answer, enhances relearning. Another prospective study published in 2020 found that voluntary completion of supplemental MCQs was correlated with increased scores on student semester exams[2]. There are plenty more studies yielding similar results. Multiple choice questions are beneficial.

There are many excellent online resources for MCQs. My go-to for preparation for the Step and shelf exams has been UWorld, an online database with thousands of MCQs. I appreciate that every question has an explanation, including why wrong answers were wrong. Seeing that the “lure” I chose is similar to yet distinct from the correct answer cements the details further. The combination of answer choices sometimes even highlights similarities I had never noticed but will be mindful of in the future. Sure, I get aggravated when I choose that (wrong) answer that only 1% of people picked— but hey, learning is learning. 

Just like a runner gearing up for the big day, preparation is key. So when we look at the multiple-choice question, the well-formed and appropriately-reviewed MCQ not only readies students to take exams but also enforces true learning as critical reasoning is engaged. It is certainly not a perfect system, but it is a proven system of learning moving us in the right direction, one 40-question block at a time.

[1] Pham, H., Trigg, M., Wu, S., O'Connell, A., Harry, C., Barnard, J., & Devitt, P. (2018). Choosing medical assessments: Does the multiple-choice question make the grade? Education for Health, 31(2), 65. doi:10.4103/efh.efh_229_17

[2] Jud, S. M., Cupisti, S., Frobenius, W., Winkler, A., Schultheis, F., Antoniadis, S., . . . Heindl, F. (2020). Introducing multiple-choice questions to promote learning for medical students: Effect on exam performance in obstetrics and gynecology. Archives of Gynecology and Obstetrics, 302(6), 1401-1406. doi:10.1007/s00404-020-05758-1

 -------------------------------------

Consolidated opinion of Steve Leslie, MD; Associate Professor, Division of Urology, Creighton University; Nick Lorenzo, MD; AOA Graduate and Co-Founder of eMedicine; Jesse Cole, MD, AOA Graduate and Co-Founder of eMedicine; and S. Huntly Plantz, Associate Professor of Emergency Medicine (retired), AOA and Co-Founder of eMedicine, and author of 38 textbooks, including several multiple-choice question books.

For instructors, multiple-choice questions allow quick scoring, and it is easier to create multiple versions of the exam. Statistics are available to allow item analysis to reveal how well a question discriminates between students that know the material and those that don’t. Grading is objective. Further, a large variety of questions can be addressed in a single exam allowing for broad testing of student knowledge. For these reasons and many others, multiple-choice questions have become the defacto standard in medical school, residency, and board exams. But the major question that is often not considered is multiple choice questions a good way to learn medicine? And if so, how should students best learn from questions?

Our opinion is that they can be an excellent way to become a better physician if they are case-based and require levels of understanding and thought to get to the answer and go beyond accessing the mere memorization of facts and details more typical of 1st order questions. Students that tend to do very large numbers of scenario case-based questions that simulate patient evaluation and care tend to develop a pattern of thinking necessary to practice medicine.

Well-written advanced questions move medical students and residents into higher-order thinking that includes sorting through information, evaluation, and the application to simulated patients. In the practice of medicine, patients generally present with a convoluted story, exam findings, and tests that need to be sorted through to get to the appropriate answer. Good multiple-choice questions grow students' ability to quickly sort through the information at hand and come to the appropriate conclusion.

Just like a careful reading of a question’s results in the correct answer, careful listening to a patient’s story, a targeted physical exam, and focused testing generally leads to the appropriate conclusion for patient care. As a result, multiple-choice examinations are often used by basic science and clinical clerkships to evaluate a student’s knowledge base and ability to decipher patient presentations and come to the correct conclusion.

Students that become skilled at answering multiple-choice questions tend to become better throughout medical school and residency, constantly applying what they have learned to real-world patients. Students that develop good test-taking techniques, in theory, have the acumen to equally develop similar patient evaluation techniques.

When we mentor students, our suggestion, after years of taking exams and developing study material to help students, is to approach questions with the mindset that you are dealing with a real patient. As you progress in your training, the emphasis of knowledge will shift from foundational science to diagnosis, management, and communication. 

As students advance in their education, testing from USMLE® 1-3 and COMLEX® 1-3, then to specialty boards, will gradually become reflective of real-life tasks. This will be seen in the clinical vignettes. In the practice of medicine, patients present with signs and symptoms, you complete a work-up, make a diagnosis, and eventually offer therapeutic options. The Step exams gradually move students further and further along in this process. We emphasize the word practice because just like the more patients you see, your skills improve; the same goes for taking multiple-choice questions; the more questions you practice, the more likely you are to be successful.

We believe the best way to prepare for USMLE® Steps 1-3, COMLEX® 1-3, and then specialty boards are to do as many questions as possible. Studies have shown the more questions you do, the higher you will score. If they tend to be case-scenarios questions, we believe this transfers into helping you become a better clinician. So the trick is to make it part of your daily routine to do questions! We believe you should start doing practice questions as early as possible in your training. It is better to start completing questions on a regular basis earlier rather than later. Further, you want to get them wrong.  It is when you get questions wrong and rethink your thought processes that you will learn the most. 

When looking for question banks, you will encounter first, second, and third-order questions. First-order questions are straightforward, such as “what test should you order?” Second- and third-order questions have intermediary steps, such as figuring out the suspected underlying pathology, then applying the information to the next step. Second and third-order questions require you to think more and more like a physician, and we believe you should search for databases that offer more of these challenging questions because they are closer to real-world medicine and will help you develop the skills you need to become a successful clinician.

Virtually all of the question banks available to students offer clinical scenario-based questions. The challenge is, more practice questions usually means more money. For years, students have had little option but to purchase multiple databases, which can quickly become very pricy. For example, BoardVitals® has approximately 1500-3000 Step 1-3 questions, generally for $200 for short-term access for each examination. Kaplan® provides 1000-3000 questions for Steps 1-3 also for about $200 for each examination. UWorld® has been around since we were students, is expensive with 1500-4000 Step 1-3 questions and for short term access, approximately $400.

There is a relatively new database built and created by academicians to help students learn. Thirty years ago, several young academic physicians founded www.emedicine.com, a database of free textbook chapters that has grown to be used over 70 million times a month. These same founders, now much older and perhaps a little wiser, set out on a mission to revolutionize medical education and created, StatPearls®, which is a database of 9,000 PubMED indexed review articles linked to a giant database of nearly 75,000 case-based practice questions with four teaching points assigned to 500 different board exams as well as all clerkship and elective rotations. 

The StatPearls system is designed to allow students to move from practice questions in the basic sciences USMLE® and COMLEX® Step 1, to more clinical science-based questions for USMLE® and COMLEX® Step 2 and 3, and then on to specialty and subspecialty board exams. Because the questions are all written by academic clinicians, they tend to be very challenging case-based scenarios similar to real-life patients. For USMLE® 1-3 and COMLEX® 1-3, students have access to 13,000-17,000 questions for each exam and subsequently 1,000-6,000 specialty and subspecialty questions for each elective and clerkship rotation. 

Students can elect to purchase the StatPearls learning management system for as little as $100-$200 a year. The database is not for those that are faint of heart or less motivated,  questions are challenging, but the students that start early in their training, and commit to daily regular practice questions, will have a major advantage over students that do a smaller number of questions that may be less challenging. Essentially, struggling through all the StatPearls questions covers every conceivable topic potentially found on exams, so students that do them all are essentially overprepared.

Perhaps one of the best reasons to use a large comprehensive database such as StatPearls or purchasing several smaller databases is overcoming anxiety and fatigue in test-taking. The more questions you do, the less anxious you will become on test day. You will be used to doing larger numbers of questions, so the battle fatigue of 8 hours of testing will seem like a piece of cake when you have been doing 12 hours of practice testing a day on weekends and during school holidays. Test anxiety is overcome, students become more composed, and fatigue and anxiety gradually fade.

We believe students should realize that multiple-choice questions are a fact of life throughout their medical career; they have a strong potential to make you a better clinician if applied clinically. As a result, you might as well strive to get very good at them. Students that share in our belief of “Illegitimi non-carborundum” or “Don’t let the bastards grind you down” by doing thousands of questions and getting really good at it will most likely succeed on tests, and more importantly, succeed in life, becoming great physicians and surgeons.

------------

Matthew Scott, UTHSC MD Candidate, Class of 2024

As a first-year medical student, it is easy to look back at the standardized tests that landed me here: the ACT, the SAT, and then the MCAT. At the same time, I look forward and see that USMLE© Step 1 and Step 2 are on the horizon. Despite these five exams differing in content and level of difficulty, there is one common thread: they are all multiple choice. In short, my peers and I have been raised on multiple-choice tests and have more important ones ahead of us, so from my perspective, multiple-choice questions are just a part of life. However, what if multiple-choice questions are not the best way to go about learning medicine? Are they the law of the land because they are the ideal method for learning or because that is just the way things have always been?

Written Short Answer Essays

In order to consider that multiple-choice might not be the way to go, after all, there must be a competing option on the table to compare it to, and that option is a writing-based format such as short answer or essays. Before answering if multiple choice is unequivocally the better choice, it is important to assess why multiple-choice is the preferred one at this time. While the multiple-choice format may or may not be best for learning, it is undoubtedly the most efficient way to assess large volumes of information in a fair and equitable manner. I cannot imagine the time and personnel that would be involved in the grading of my M1 class’s exams if they were in a written format. Furthermore, that is only for a class of 170 medical students, so the scale of the operation that would be necessary to grade written versions of nationwide tests like USMLE© Step 1 or USMLE© Step 2 CK would be mind-boggling. Aside from this logistical concern, there is also the issue that there are already challenges to multiple-choice questions on my M1 class’s exams, so one would conclude that there would be a significant uptick in challenges to grades if the tests were written and thus inherently less objectively scored than a multiple-choice test. In short, the obstacle that is the logistics of changing everything to a written format seems insurmountable for a number of reasons, so it is clear why no one is rushing to switch from multiple-choice.

However, I think the idea of a written format for exams still merits discussion for one simple reason: the end goal of all these exams is to become a physician and serve patients, and there will not be any multiple-choice options when attempting to identify what is wrong with a patient. It is more life-like to have a prompt describing the findings of a history, physical exam, lab work, and imaging that asks the student to write what is wrong with the patient and what the plan of action should be. Even before getting to the point of being a practicing physician, M3’s, M4’s, and residents are verbally quizzed by their superiors, and there are no pre-determined options to choose from. Thus, written tests are much more conducive with preparing students to practice medicine compared to multiple-choice tests. Therefore, if written tests seem to be a better reflection of the reality of medicine, is there a reason besides the logistics why multiple-choice is still dominant in standardized tests as well as for tests in the pre-clinical years? 

Multiple Choice Questions

Perhaps the answer lies in the adage of “if it ain’t broke, don’t fix it”. Despite multiple-choice having been present every step of the way in the education of practicing physicians, there has not been any national crisis of doctors being inept and unprepared to aid their patients. Without a sign of something being acutely wrong due to the current testing format, there is little reason to make alterations given that the age-old recipe is still producing high-quality physicians. This raises an important question: How is it that students are becoming competent physicians even though they are being tested throughout their education in a manner that does not reflect the real world? On one hand, in spite of the questions being multiple-choice, students are still learning the details and overarching concepts in order to be able to answer those questions, and the questions are complex in nature to the point where a mastery of the relevant material is necessary to be able to correctly interpret and answer the questions. On the other hand, the rotations during the clinical years serve a myriad of purposes, and among them is the opportunity to begin to translate all of those concepts that were learned for the multiple-choice exams into caring for patients and intelligently discussing their care with attendings. Additionally, those clinical years are a time to build upon and solidify the foundation of knowledge that is years in the making due to habitual studying for assessments as well as to test one’s command of the information in a way that is far superior to written tests that are meant to simulate real patient encounters but fall short of the real deal. 

Are multiple-choice questions the best way to learn medicine? No, they are not ideal. Are multiple-choice questions a good way to learn medicine? Yes. They are vastly advantageous from a logistics standpoint, students are soaking up as much information as physically possible just as they would regardless of the test format, and medical school is arranged in such a way that multiple-choice exams are complemented by time spent with patients and attendings to better accomplish what a written exam would offer. Thus, multiple-choice questions are here to stay.

How Multiple Choice Questions Help Students Learn to Evaluate and Treat Patients

Given that medical students are stuck with multiple-choice questions, the final question that must be answered is this: How can students take advantage of this format and utilize it to become the best physicians possible? The beauty of multiple-choice questions is that there are myriad practice questions out there to use in order to prepare oneself to succeed on standardized tests that are formatted the same way. A prime example of this is the prep that students do for the USMLE© Step exams. Whether it is StatPearls© with over 15,000 questions, or more costly options with fewer questions like uWorld©, Princeton©, and Kaplan©, these question banks contain thousands of questions that are in a case-presentation format, which mimics what is seen with the USMLE© Step exams and to a certain extent, mimics how patients present in practice.

These practice questions are the key to success, and students that fully commit to utilizing them are usually rewarded with high scores. I am stressing the idea of being fully committed because students that excel on USMLE© Step 1 or USMLE© Step 2 typically do 10,000+ unique questions, which means setting up a schedule of completing questions for a few hours each day for a long time leading up to the exam. The best part of using such a vast number of case-presentation questions is that students are exposed to nearly everything that can go wrong in the course of practicing medicine and learn how to respond appropriately in each scenario. Additionally, this style of questions pairs well with clinical rotations because both serve to rewire students’ brains in favor of thinking like a physician, which sets students up for success in residency and their careers thereafter. In culmination, multiple-choice questions are the law of the land for a variety of reasons, and medical students can take advantage of this by utilizing thousands of practice questions that will prepare them to excel on standardized tests as well as in a clinical setting.