December 2010

Using Multiple-choice Tests in Science Education

As science teachers, our most important goal should be to guide students in developing their ability to apply fundamental scientific principles and logical reasoning to formulate and test scientific hypotheses, as well as the analytical and critical reasoning skills to interpret the results. Unfortunately, the pressures of large class sizes and doing more with less frequently induces instructors to employ multiple-choice tests as their primary, and sometimes exclusive, format for assessing student progress. (Titled "Distracters or Detractors? What’s the Catch?" in print version.)


Synthesizing versus Selecting

Perhaps the most important transition that any student of science must make is the leap from consumer to producer. Most students majoring in science, technology, engineering and math areas have the capacity to become quite adept at absorbing and utilizing existing information and methodologies. Although such technical knowledge and proficiency generally is sufficient to obtain a degree and become gainfully employed as a member of the supporting cast, becoming an originator of new facts, new concepts and new technologies requires the capacity to engage in creative and analytical thought on a routine basis.

"If Charles Darwin had been trained in a system overly reliant on multiple-choice testing, would he ever have conceived of natural selection?"

I generally like to initiate a discussion on the nature of science by asking, “How can a scientist have confidence in the answer they obtain when asking a question that never has been raised before?” Because students are all about getting the “correct” answer, this question places the issue in a very practical and familiar context. How do you prepare for an exam or complete a homework assignment in a course for which no assigned text or answer key exists, and no authority figure is available to confirm or deny the validity of your answer?

As science teachers, our most important goal should be to guide students in developing their ability to apply fundamental scientific principles and logical reasoning to formulate and test scientific hypotheses, as well as the analytical and critical reasoning skills to interpret the results. It therefore would appear reasonable to expect STEM curricula to be replete with exercises and questions that challenge students to draw upon these skills to synthesize an original response. Unfortunately, the pressures of large class sizes and doing more with less frequently induce instructors to employ multiple-choice tests as their primary, and sometimes exclusive, format for assessing student progress.

I Thought You Were Trying to Trick Me

Unfortunately, picking from a list of prepared responses falls well short of providing students with a chance to exercise their abilities to draw upon their knowledge and reasoning skills to generate an answer whose origins are indigenous to their own intellect. Moreover, in attempting to increase the sophistication of the thought processes required to select the correct answers on a multiple-choice test, instructors often exacerbate the situation by choosing inappropriate “distracters” (the alternative answers from which the student must select).

One form of distracter that can do more harm than good is to append some mysterious condition such as, “Choose the most correct answer from the choices given below.” What is the definition of “most correct”? Doesn’t the perception of which of the correct answers is the “most” correct depend upon (frequently unmentioned) circumstances? Another common distracter is the “parade of permutations”: a), b), c); both a) and b); b) and c) but not a); none of the above or all of the above. Both of these rubrics share the property of transforming normally correct answers into incorrect ones. In the first case, one can select a perfectly correct answer and receive no credit because it was not the “most” correct. Similarly, if both a) and b) are correct, a student who recognizes that a) is correct or b) is correct, but not that both are, receives the same zero score as a person who marked c) or “none of the above” even though the former responses betray greater insight into the correct answer.

The use of deception or misdirection as a means for differentiating amongst the top performers in a class can be counterproductive at a number of levels. First, such a format tends to reward the memorization of minutiae and the ability to recognize the subtle semantic cues indicating that a particular question has some “twist” to it. While these qualities may correlate to some degree with a student’s capacity for critical thinking and intellectual synthesis, differentiation through deception neither encourages nor stimulates the development of these higher-level skills. When instructors become overly reliant on differences in form rather than substance when constructing a multiple-choice examination, some students become alienated from science, seeing it as a “game” they can never “win.”

Trust, but Verify

"When students are fed a steady diet of 'trick' questions and distracters that rely on disguise rather than substance, an insidious form of conditioning occurs."

As a research mentor, I always am on the alert for the occurrence of what now is popularly referred to as a “teachable moment”. In my best Socratic style, I will begin asking the student questions, “How can a protein bind to an ion-exchange column in the presence of the large excess of counter ions present in a low-salt loading buffer?” “How does our laboratory refrigerator work?” If the student encounters difficulty with my opening question, I emulate my mentors and ask a follow-up question that generally is more focused and simple than the initial one. “When a gas is allowed to expand, where does the energy to support this process come from?” With frightening regularity, I encounter students unable to formulate the answer to even a simple question or who provide bizarrely complex answers. When, in my frustration, I ask the student why they are having so much trouble answering even extremely basic questions, an increasingly common response is, “The question was too simple. I thought you were trying to trick me.”

When students are fed a steady diet of “trick” questions and distracters that rely on disguise rather than substance, an insidious form of conditioning occurs. To protect themselves against being fooled by the various forms of misdirection and camouflage encountered among the distracters, many students soon learn to consider every question to be a trick question until proven otherwise. This adaptive mechanism can have insidiously unfortunate consequences. The practice of habitual skepticism requires that students condition themselves to distrust their powers of observation, to view their initial reasoning as suspect and to reject their instincts, as these constitute the prime targets of the deceptive distracter.

Tragically, such conditioned skepticism selects against several of the most important attributes of the successful scientist— the ability to correctly utilize and trust in logic, reason, the experimental method, careful observation and measurement. If Charles Darwin had been trained in a system as overly reliant on multiple-choice testing as most contemporary high schools and universities, would he ever have conceived of the process of natural selection? Or, upon first perceiving patterns of adaptation in the animals and plants around him, would he simply have rejected this as an anomaly arising from some catch hidden below the surface?

The multiple-choice examination will be with us for as long as “do more with less” remains the order of the day in public education. However, it is important that the menu of potential answers that we ask students to select from differ from one another in substance, with no attempt to hide that substance under some semantic disguise. If multiple possibilities are correct, tests should be constructed such that students can select amongst individual entities instead of selecting the one correct combination. Will this result in some score compression, as the culling effect of deceptive distracters and convoluted questions is lost? Perhaps initially, but this is a point that can, and should, be addressed by adding questions that are more challenging in substance rather than form.

Peter J. Kennelly (pjkennel@vt.edu) is a professor and head of the department of biochemistry at Virginia Polytechnic Institute and State University. He also is chairman of the ASBMB Education and Professional Development Committee.

found= true1102