What’s particularly cool about this is that a lot of research on retrieval practice uses identical questions and the research has sometimes been criticized for being primarily rote retrieval as opposed to deeper learning. But this shows a benefit for related material, indicating that there is a benefit for understanding that goes beyond question-answer memorization. And that benefit (in this study) is 10% – a full letter grade.
Students also responded to a survey about the pre-tests. There were some interesting results to the survey.
· Students indicated that they took the pretests pretty seriously even though they weren’t for a grade. Students did have to put their names down in order to connect their scores with their final exam grades, so perhaps that raised the stakes for them a bit.
· On average, students believed they got more than half of the pretest answers correct. This is somewhat surprising. At the time this survey was taken, students have already sat through the lectures and gotten the correct answers, yet they are overestimating how well they did. Still, there was quite a delay between the pretests and this response, so we wouldn’t expect it to be perfectly accurate.
· Students indicated that they used a strategy on the pretest, either a process of elimination or picking an item with familiar terms. This indicates that students had some knowledge before taking the pretest, which allowed them to consider answer choices instead of purely guessing.
· About half of the students indicated that they didn’t think about the (ungraded) pretests after they took them, but the other half reported using the pretests as a kind of study tool. They looked up the answers, studied those items, or talked with their friends about them.
· Interestingly, almost half of the students reported that they engage in some kind of pretesting on their own. They completed quizzes provided by the instructor or those in the textbook before coming to class.
Here’s my take:
There are several possible reasons why pretesting worked in this study.
1) Students paid more attention to the pretested material during the lecture.
2) The pretest activated prior knowledge (some of them are clearly doing a lot of prework), and allowed them to encode the new information more deeply.
3) They were doing a lot of studying of the pretested information outside of class.
4) There are some great spaced retrieval effects going on. That is, students saw the material before lecture, they took a quiz on it during the pretest, then later they reviewed or quizzed themselves on that same material again during self-study.
Of those reasons, three of them depend on students working on their own in a strategic way, outside of class. While there is clearly more research needed in this area to disentangle these possibilities, I feel pretty confident in telling educators that this is a promising technique for certain students. My concern is that these might be the students who need it the least. If they are already doing extra strategic studying that better prepares them for lectures and exams, those aren’t the students I’m worried about. And if this effect requires all that extra work, there’s less that we, as educators, can do about it.
But there are some things. For example, making those pretests worth a very small amount of the total grade and encouraging students to prepare for them could extend this benefit to students who otherwise wouldn’t do that prework. If we provide quizzes in advance of lectures for students to take at home and then provide the pretest questions for students to take home and use as study we might guide them to more effective study strategies. After all, it’ll be difficult for them to practice retrieval if they have to create their own materials.
In short, this data is promising and folks could consider using pretests in their classes. My advice might be to make these pretests a combination of old and new material, retrieval practice and pretest, and to explicitly tell our students how to effectively study for both better scores on pretests and long-term retention.
