Writing Distractors for Multiple Choice Questions
by Steven B. Just
Everyone knows what a multiple-choice question
looks like, but not everyone knows how to name the parts.
For the record:
- The part that contains
the question is called the stem
- The choices from which
the student chooses are, not surprisingly, called the choices
- The choice that is correct
is called, again not surprisingly, the correct answer (or
- The incorrect choices
are called the distractors
So a standard four choice multiple-choice question
looks like this:
Writing the correct answer is easy of course
but writing the distractors is something of a science. Why
should writing distractors be a science? For a few reasons:
- Poorly written distractors
can invalidate a question
- You can control the
question difficulty through the distractors
- The frequency with which
the students choose each distractor is meaningful
Writing Valid Questions
There are many do’s and don’ts
for writing valid questions. Several refer to writing distractors:
- Do not use "All of the
Above" as a choice.
- Make the distractors
plausible as correct answers.
- Do not do write complex
distractors that require high level logical thinking. You
are testing the question posed in the stem, not the student’s
logical reasoning ability.
- All of the choices (correct
answer and distractors) should be of roughly the same length.
- Order the distractors
(and the correct choice), if this is relevant to the question.
(For example, for a question with numeric choices, the choices
should be presented in ascending order.)
Controlling Question Difficulty
How well you write the distractors controls
the difficulty of the question. If all the distractors are
implausible as correct answers, then the student doesn’t
even need to know the correct answer to get the question right—he/she
merely needs to eliminate the obviously incorrect choices.
If there are three distractors and two are implausible and
one is plausible, even the student with no knowledge of the
correct answer can still eliminate two of the choices and
then has a 50/50 chance of guessing correctly. So the more
plausible the distractors are as correct answers, the more
difficult the question.
One of the most meaningful post-exam statistics
available to test administrators is the choice distribution.
For each question the choice distribution shows the percentage
of students who selected each choice. For the easiest questions
100%, or close to 100%, of the students will choose the correct
answer, which doesn’t provide a lot of information.
But for the more difficult questions you will see higher percentages
for each of the distractors. If you have written the distractors
properly (i.e. as plausible choices—see above) this
gives you a lot of information about where your students have
misconceptions about the material—information that you
can then use for meaningful remediation.