News and events

“Writing Distractors for Multiple Choice Questions”
by Steven B. Just

Everyone knows what a multiple-choice question looks like, but not everyone knows how to name the parts. For the record:

• The part that contains the question is called the stem
• The choices from which the student chooses are, not surprisingly, called the choices
• The choice that is correct is called, again not surprisingly, the correct answer (or best response)
• The incorrect choices are called the distractors

So a standard four choice multiple-choice question looks like this:

Stem

Distractor1

Distractor2

Correct Answer

Distractor3

Writing the correct answer is easy of course but writing the distractors is something of a science. Why should writing distractors be a science? For a few reasons:

• Poorly written distractors can invalidate a question
• You can control the question difficulty through the distractors
• The frequency with which the students choose each distractor is meaningful

Writing Valid Questions

There are many do’s and don’ts for writing valid questions. Several refer to writing distractors:

• Do not use "All of the Above" as a choice.
• Make the distractors plausible as correct answers.
• Do not do write complex distractors that require high level logical thinking. You are testing the question posed in the stem, not the student’s logical reasoning ability.
• All of the choices (correct answer and distractors) should be of roughly the same length.
• Order the distractors (and the correct choice), if this is relevant to the question. (For example, for a question with numeric choices, the choices should be presented in ascending order.)

Controlling Question Difficulty

How well you write the distractors controls the difficulty of the question. If all the distractors are implausible as correct answers, then the student doesn’t even need to know the correct answer to get the question right—he/she merely needs to eliminate the obviously incorrect choices. If there are three distractors and two are implausible and one is plausible, even the student with no knowledge of the correct answer can still eliminate two of the choices and then has a 50/50 chance of guessing correctly. So the more plausible the distractors are as correct answers, the more difficult the question.

Choice Distributions

One of the most meaningful post-exam statistics available to test administrators is the choice distribution. For each question the choice distribution shows the percentage of students who selected each choice. For the easiest questions 100%, or close to 100%, of the students will choose the correct answer, which doesn’t provide a lot of information. But for the more difficult questions you will see higher percentages for each of the distractors. If you have written the distractors properly (i.e. as plausible choices—see above) this gives you a lot of information about where your students have misconceptions about the material—information that you can then use for meaningful remediation.