Can’t afford a full-fledged simulation? You can still recreate the learner’s world in your materials, even if your only tool is the lowly multiple-choice question.
Let’s say you’re writing materials for people who create custom pet hedgehogs using genetic engineering. You might be tempted to write a question like the one below.
It’s a good idea to include parrot genes in a custom hedgehog.
- True
- False (correct)
How could you make this question more realistically reflect the learner’s world?
First ask yourself, “Why does the learner need to know this fact?”
Then write a question that tests both the learner’s knowledge of the fact and their ability to apply it in the real world. “What if…” questions come in handy for this.
What would happen if you included parrot genes in a custom hedgehog?
- The hedgehog would become depressed by its inability to fly.
- The hedgehog would charm customers with its bright plumage.
- The hedgehog would develop an offensive vocabulary.
The “before” question tests only basic knowledge. The “after” question tests the same basic knowledge and at the same time tests whether learners understand why the fact is important.
Of course, make sure your materials answer the question.
Obviously, the “after” question has more shades of meaning and suggests that the materials contain more information than simply, “It’s a bad idea to include parrot genes.”
I’m a big advocate of writing questions before you write the main materials. So if you want your low-tech materials to simulate real life, you might:
- Write questions that learners must be able to answer in order to fully understand (not just perform) their jobs. These probably aren’t fact-based questions but instead are “What if” and “Why” questions.
- Then write the materials that will precede the questions, assuming you’re doing the standard “tell, then test” approach. Also consider putting the questions first, then explaining why the correct answer is correct.
What are some other ways to give basic questions more depth? Are simple fact checks ever appropriate?
Cathy,
Great post! The multiple choice question is so common that anything we can do to improve it is good news! And all too often it is followed by “That’s right” or “Correct”. In my observation, learners who see they have answered the question correctly don’t bother to read the remaining feedback. They immediately click the next button.
Cathy, I’m a strong believer in “write the questions first,” and not questions for their own sake. In one excruciating project related to EEO complaints, the client’s first cut at important questions included knowing the dates of legislation like the Voting Rights and Americans With Disabilities acts.
What was the real-world simulation? Two main clusters: examples (and non-examples) of discriminatory behavior, and how to initiate and follow an appeal.
To satisfy the highly detailed types, we put a summary of the legislation in a reference section. Even then, we tried to position it to say behaviors like X and Y are discriminatory based on the Z Act of 19whenever.
The military systems approach to training (SAT) model that I learned many years ago had us developing all the evaluation tools before any design activities began. It works.
It’s good to hear that writing the evaluation first has many other supporters. Every team I’ve been on has wanted to write the evaluation activities last, which probably seems familiar and “right” because that’s how most of us were taught in school.
It’s a lot easier to keep information overload under control if we first write realistic activities that will help learners create the necessary change in their jobs, and then we identify the minimum information we need to provide for the learners to complete the activities.
We gotta get on the same team.
Cathy,
I really like this article. I’m actually developing a training for managers to teach them how to Create Pivot Tables and analyze data. In designing the quiz questions, I thought long and serious about true/false questions. And, to me I thought the “True and False” is not enough to prove overall learning. So, I decided to go with the “Hot Spots” design. I ask a questions … Click on the correct objects that allow you to design the format of your Pivot Table” for example. Learners get to choose any of four objects. However, I like your example as well.
I really appreciate you and Tom’s blogs. They’ve help me as an Instructional Designer “Big Time.” Keep feeding good information to this industry. We need specialist like you two. BTW: your blog on Objectives is HOT.
Kathie, thanks for your comment, and I’m glad the blogs are helpful. Another advantage of your “hot spots” approach is that it has the learner physically interact with the software (or a screenshot of it) instead of just thinking abstractly about it.
Yeah look up research by Wouter van Joolingen and Ton de Jong. There is a close connection between learning with simulations and performance on “What if” type questions.
Questions set the context for the learners.
Often times, the right question or the attempt to ask the question is the most important step.
Our role as trainers starts by asking the leaner to ask to themselves their own questions. And provide the learner the resource to find answers. But never, if we can help, not provide the answers.