For better multiple-choice tests, avoid tricky questions, study finds

October 1, 2018 by Gerry Everding, Washington University in St. Louis
Credit: CC0 Public Domain

Multiple-choice tests and quizzes are an effective tool for:

a) assessing a student's mastery of facts and concepts;
b) helping students learn and retain facts and concepts.

While some educators might see this as a trick question, the correct answer appears to be:

c) all of the above, suggests new research from Washington University in St. Louis.

"Although people often think about multiple-choice tests as tools for assessment, they can also be used to facilitate learning," said Andrew Butler, a cognitive psychologist in Arts & Sciences who studies the brain processes behind learning and recall. "The act of retrieving strengthens memory for that information, leading to better long-term retention, and changes the representation of the information, creating deeper understanding."

Butler's study, published in the September issue of Journal of Applied Research in Memory and Cognition, offers straightforward tips for constructing multiple-choice questions that are effective at both assessing current knowledge and strengthening ongoing learning.

Among key findings, educators should never include trick questions or offer "all of the above" or "none of the above" options among the list of possible answers.

Research on the format of multiple-choice questions is important, Butler noted, because the tests are widely used throughout the world, especially in the United States where they originated as part of early efforts to measure intelligence.

Fueled in the beginning by the need for an efficient way to measure characteristics of World War I soldiers and booming student enrollments, multiple-choice tests now influence important life decisions in areas such as college placement, workplace hiring, career advancement and even online dating.

As an associate professor in the Departments of Education and of Psychological & Brain Sciences, both in Arts & Sciences, Butler conducts research that explores the malleability of memory—the and mechanisms that cause memories to change or remain stable over time.

Taking any form of has the potential to alter our understanding of a topic, he said, because the process of recalling information requires important details to be freshly reconstructed from related memories.

While multiple-choice testing, especially repeated testing, has the potential to strengthen our recall, a poorly formatted test question can have the opposite effect, Butler said. Such an ill-formed question can muddy our recollection of the correct answer and reinforcing memories for inaccurate "distractor" answers, he added.

Butler's research review confirms that proper question formatting and presentation are critical to creating effective multiple-choice tests. It also suggests that many widely used multiple-choice tests still include lots of questions that fail to comply with research-based best practices.

"Fortunately, the best practices for creating multiple-choice tests that effectively assess understanding are much the same as those for supporting student learning," Butler said.

Butler's study explains the cognitive science behind five research-based recommendations for crafting more effective multiple choice questions:

  • Create questions with simple formats. Complex question-and-answer formats have become popular as educators look for ways to test for deeper understanding and higher-level learning. Examples include offering answers such as "A and B, but not C," or allowing repeated answers until correct choice is made. Such formats may be detrimental to assessment since guessing is encouraged and processing may focus more on parsing question context and less on recalling correct information. Some complex formats, such as confidence-rated answers, may offer benefits, but more research is needed.
  • Create questions that engage "real world" cognitive processes. To truly test abilities, questions must be structured so correct answers require use of the specific cognitive processes necessary to address similar problems in the real world. Questions that require higher-order thinking will enhance learning and improve future performance. For example, a multiple-choice question could require a test-taker to contrast two concepts (Which of the following is a way in which hawks differ from eagles?) or analyze a set of conditions to make a decision (Given a patient symptoms, which of the following diagnoses is most likely?).
  • Avoid using "none of the above" and "all of the above" as answer choices. When "none-of-the-above" is correct, students may not need to retrieve correct information to answer the question and they are exposed a lot of incorrect information. Using "all of the above" exposes students to a lot of correct information, but answers may be more obvious, robbing students of potential learning that comes from recall processing. Both question types can be detrimental to accurate assessment and potential benefits to learning are small.
  • Use three plausible response options. Question difficulty increases with each answer option offered. Students who correctly answer more difficult questions may learn more from rising to the challenge, but questions that offer too many plausible answers can have a negative effect on both learning and assessment. Use the Goldilocks principle: not too many, or too few.
  • Make the test challenging, but not too difficult. Create tests that are hard enough to reveal how well students know the material, but easy enough that a majority (80 percent) get a passing grade. Retrieving information and answering questions correctly reinforces learning; failing to correctly may strengthen memories for misinformation. Challenge students, but allow them to succeed.

Finally, because multiple choice expose students to lots of plausibly presented false information, it's important for students to review answers after grading is completed. Feedback enables test-takers to correct errors and avoid internalizing incorrect information. It also strengthens learning around correct answers that were low-confidence guesses at test time.

"One takeaway from these recommendations is that the most effective multiple-choice items get students to think in ways that are productive for learning and enable valid measurement of whether they have acquired the desired skills and knowledge," Butler said. "To maximize both effectiveness and efficiency, it is also best to keep the process of answering multiple-choice items simple—added complexity often has a negative effect on both learning and assessment."

Explore further: Researchers make five recommendations for standardized test designers

More information: Andrew C. Butler. Multiple-Choice Testing in Education: Are the Best Practices for Assessment Also Good for Learning?, Journal of Applied Research in Memory and Cognition (2018). DOI: 10.1016/j.jarmac.2018.07.002

Related Stories

Quick learners remember more over time

September 10, 2018

Healthy adults who learn information more quickly than their peers also have better long-term retention for the material despite spending less time studying it, a new study from psychologists at Washington University in St. ...

'Exam Roulette' could quell essay-induced anxiety

June 21, 2018

For many students, essay tests are a source of dread and anxiety. But for professors, these tests provide an excellent way to assess a student's depth of knowledge and critical-thinking skills. At the American Physiological ...

Recommended for you

University choice and achievement partly down to DNA

October 18, 2018

Research from King's College London has shown for the first time that genetics plays a significant role in whether young adults choose to go to university, which university they choose to attend and how well they do.

3 comments

Adjust slider to filter visible comments by rank

Display comments: newest first

antialias_physorg
not rated yet Oct 01, 2018
Multiple choice tests are not a good way to test anything. They give the false impression of mastery, but students can get by by eliminating 'implausible' answers and picking the one answer left (or simply eliminating as many as possible and then just playing the odds and the psychology of the test maker who will rarely do things like make a test that will have the answer a) 10 times in a row.

*Mastery* of anything is only demonstrated by training someone and then giving them a new task which they have to figure out on their own - with NO aid given. Try that with someone who has just passed a multiple choice test. They will fail to a scary degree.

because the tests are widely used throughout the world, especially in the United States...

And the quality of the United States education system should give a hint as to how sensible multiple choice tests are

Multiple choice (in whatever of the in the article mentioned forms) tests recall - not understanding.
antialias_physorg
5 / 5 (1) Oct 01, 2018
Make the test challenging, but not too difficult. Create tests that are hard enough to reveal how well students know the material, but easy enough that a majority (80 percent) get a passing grade.

No. Do not give a free pass for people who don't understand (or to teachers who are inept at teaching) just because you want to pass 80% of the class. That's a sure fire way of sending people on to college or into the job market that are woefully unprepared.

What does the author think schools do? Give a few years of feel-good environment until the shock of real life destroys these people? That is doing them a massive disservice.
Nik_2213
not rated yet Oct 01, 2018
Another problem is that the answers may be too narrow in scope.
What if the student is well-read ? What if the questions are over-simplified ?
Due care, please.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.