Research on Test-Taking Errors: Why Medical Students Miss Questions They Know

There's a common assumption among Step 1 students: if you got a question wrong, you need to study that topic more. Sometimes that's true. But a growing body of research in medical education suggests that a significant portion of wrong answers on board exams have nothing to do with content knowledge.

The real issue? Students can't accurately identify why they're missing questions, which means they keep applying the wrong fix.

The Self-Assessment Problem

A 2024 study published in BMC Medical Education(Knof, Berndt & Shiozawa) looked at 426 first-semester medical students and found that only 18.5% accurately assessed their own performance. Over a third (35.5%) overestimated how well they did, while 46% underestimated.

The pattern was consistent with the Dunning-Kruger effect: low performers grossly overestimated their abilities, while high performers slightly underestimated. A separate review in the Journal of Graduate Medical Education (Rahmani, 2020) found that low-performing medical trainees rated themselves 30-40 percentile ranks higher than peer assessments indicated.

What does this mean for Step 1 prep? If you're relying on how you “feel” about a topic to decide whether to study it, the research says you're probably wrong. Objective data from practice exams and question banks is far more reliable than self-assessment.

The Answer-Changing Data

“Go with your gut.” “Don't change your answer.” Almost every student has heard this advice. But the research consistently shows it's wrong.

Kruger, Wirtz, and Miller (2005) published a landmark study in the Journal of Personality and Social Psychology identifying what they called the First Instinct Fallacy. The belief that your first answer is usually right persists because changing a correct answer to a wrong one is emotionally painful and highly memorable. Changing a wrong answer to a right one barely registers. So students remember the times they “should have stuck with their gut” while forgetting the times a change saved them.

The actual data tells a different story. Bauer, Kopp, and Fischer (2007) studied 79 third-year medical students and found that answer changes were 2.2 times more likely to go from wrong to right than right to wrong. Specifically: 48.2% of changes were wrong-to-right, 21.6% were right-to-wrong, and 30.2% were wrong-to-wrong. Each student gained an average of 2.5% on their score from answer changes alone.

The largest study on this topic looked at USMLE Step 2 CK specifically. Ouyang et al. (2019) analyzed 27,830 examinees and found that answer changes were 1.48 times more likely to be beneficial than harmful. Even more interesting: higher-performing examinees had a better ratio (1.58 vs 1.28 for lower performers). Their ability to recognize and correct errors was a measurable skill, not luck.

The takeaway from the research is clear: if you have a specific reason to change your answer, change it. The data is on your side.

Test Anxiety: What Actually Matters

Anxiety is real and widespread in medical education. A 2024 systematic review in Medical Science Educator(Williamson, Wright & Beck Dallaghan) found that anxiety and depression affect 1 in 3 medical students, and suicidal ideation affects 1 in 10.

But here's the uncomfortable finding from that same review: none of the commonly recommended anxiety-reduction interventions (deep breathing, hypnosis, dog therapy, fish oil supplementation) resulted in improved exam performance.

A 2022 study in Psychological Science(Theobald, Breitwieser & Brod) helps explain why. They found that test anxiety did NOT independently predict exam scores when students' knowledge was controlled for via mock exams. Instead, what happened was that anxious students learned less during their study period. The anxiety didn't lower their score on test day. It lowered the quality of their preparation in the weeks before, which is what actually hurt their performance.

This has a practical implication: the most effective “anxiety intervention” for Step 1 may not be breathing exercises. It may be structured preparation and accurate, objective self-assessment so you know where you actually stand going into the exam.

What the Research Suggests

Pulling these studies together, a few evidence-based principles emerge:

Don't trust your self-assessment. Only 18.5% of medical students accurately gauge their own performance. Use objective data (NBME scores, QBank percentages) to drive your study decisions.
Change answers when you have a reason. The First Instinct Fallacy is well-documented. In the largest USMLE study (n=27,830), answer changes were 1.48x more likely to help than hurt.
Address anxiety through preparation, not just coping techniques. Research shows anxiety impairs studying quality more than test-day performance. Structured preparation with objective progress tracking may be more effective than relaxation techniques alone.
Categorize your errors. Not every wrong answer is a content gap. Understanding why you missed a question (not just what the right answer was) is the first step to fixing the right problem.

References