A number of studies with university students have examined whether retrieved knowledge can be applied to novel contexts. Butler (2010) was one of the first studies to explore this. The study showed that testing participants on new knowledge domain questions (wing structure for military aircrafts) after quizzing them on different knowledge domain questions (wing structure of birds) resulted in better performance on the final test compared to the restudy materials group (17). Another study showed that studying the same apply questions at both the initial quiz and final test improved performance on the final test for these items compared to participants that restudied by highlighting the materials. However, crucially, this finding did not replicate when the apply questions on the final test differed to the initial quiz (18). Therefore, this suggests that the similarity of the retrieved materials is important when considering the effectiveness of retrieval practice.
This point is further extended by a recent study by Agarwal (2019), who, across three experiments, asked university students and school-aged pupils to retrieve information using fact questions, higher-order questions, and a mix of question types. The results showed whilst higher-order and mixed quizzes improved higher-order test performance (i.e. apply, analyse, evaluate and create questions), fact quizzes only improved factual knowledge questions. This study suggests two key takeaways; first, that retrieval practice performance is based on the similarly of the retrieved information. Second, when studying or teaching you do not need to use factual questions before higher-order questions in order to help build critical thinking skills. Instead, you need to practice retrieval for that specific skill (7). An outline of this study can be found in this recent Learning Scientists’ blog post.
Final comments and a few words of caution
So the evidence so far indicates that spaced learning and retrieval practice are both beneficial for learning critical thinking – hooray! But what do these findings actually mean for learning and teaching practice?
When considering how to embed spaced learning, it means that when we teach critical thinking (which should be all the time!) we should stagger the review process – keep revisiting the content and build on it across multiple sessions. The same process applies if a student is studying. Likewise, with retrieval practice, quizzing ourselves on higher-order materials is beneficial for the development of critical thinking. So don’t just use quizzes for factual knowledge but also use them for those higher-order skills. Importantly, the fact that we observe positive effects of spaced learning and retrieval practice on critical thinking suggests that these strategies are not just improving the specific factual concepts that have been studied but are having wider effects on how information is learned (19).
The above all sounds very promising, and it is, but a few cautionary notes should be noted when considering the practical applications:
Firstly, whilst spaced learning is an effective strategy for improving critical thinking, it also depends what content is being taught and reviewed as a part of this strategy. If the pedagogy of the sessions is not designed to promote higher-order thinking then it cannot be expected that spacing this learning will facilitate critical thinking. This is why careful attention needs to be paid to how critical thinking is taught (20, 21). For an excellent discussion on this topic, see Dr. Althea Need Kaminske’s Learning Scientists’ blog post on “Can we teach critical thinking?”.
Secondly, studies have only looked at certain critical thinking skills (mainly apply), so it is yet to be established whether spaced learning and retrieval practice supports all types of critical thinking skills (i.e., apply, analyse, evaluate and create).
Finally, understanding the effects of learning strategies for critical thinking will always be a little ‘hazy’ simply because of the challenges associated with defining and assessing critical thinking, so therefore we should expect that the benefits of any learning strategy may not be as strong for critical thinking compared to factual knowledge (19).
(1) Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14, 4–58.
(2) Pashler, H., Bain, P. M., Bottge, B. A., Graesser, A., Koedinger, K., McDaniel, M., & Metcalfe, J. (2007). Organizing instruction and study to improve student learning. Washington, DC: National Center for Education Research, Institute of Education Sciences, U.S. Department of Education.
(3) McDaniel, M.A., & Masson, M.E.J. (1985). Altering memory representations through retrieval. Journal of Experimental Psychology: Learning, Memory, and Cognition, 11, 371–385.
(4) Karpicke, J. D., & Roediger, H. L., III. (2007). Repeated retrieval during learning is the key to long-term retention. Journal of Memory and Language, 57, 151–162.
(5) Roediger, H. L., III, & Karpicke, J. D. (2006a). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1, 181–210.
(6) Roediger, H. L., III, & Karpicke, J. D. (2006b). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17, 249–255.
(7) Agarwal, P. K. (2019). Retrieval practice & Bloom’s taxonomy: Do students need fact knowledge before higher order learning? Journal of Educational Psychology, 111, 189-209.
(8) Foot‐Seymour V, Foot J, Wiseheart M. (2019). Judging credibility: Can spaced lessons help students think more critically online? Applied Cognitive Psychology, 1-12.
(9) Bellaera, L., Debney, L., & Baker, S. (2016). An intervention for subject comprehension and critical thinking in mixed academic ability university students. The Journal of General Education, 65, 264-282.
(10) Griggs, R. A., Jackson, S. L., Marek, P., & Christopher, A. N. (1998). Critical thinking in introductory psychology texts and supplements. Teaching of Psychology, 25, 254 –265
(11) Bloom, B. S. (Ed.), Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). The taxonomy of educational objectives: The classification of educational goals (Handbook 1: Cognitive domain). New York, NY: David McKay Company.
(12) Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin, 132, 354-380.
(13) Ebbinghaus, H. Memory: A contribution to experimental psychology. Ruger, HA.; Bussenius, CE.; Hilgard, ER., translators. New York: Dover Publications; 1964. (Original work published in 1885).
(14) Vlach, H. A., & Sandhofer, C. M. (2012). Distributing learning over time: The spacing effect in children’s acquisition and generalization of science concepts. Child Development, 83, 1137-1144.
(15) Vlach, H. A., & Gluckman, M., Vlach, H., & Sandhofer, C.M. (2014). Spacing simultaneously promotes multiple forms of learning in children’s science curriculum. Applied Cognitive Psychology, 28, 266-273
(16) Kapler, I. V., Weston, T., & Wiseheart, M. (2015). Spacing in a simulated undergraduate classroom: Long-term benefits for factual and higher-level learning. Learning and Instruction, 36, 38-45.
(17) Butler, A. C. (2010). Repeated testing produces superior transfer of learning relative to repeated studying. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36, 1118–1133.
(18) Wooldridge, C. L., Bugg, J. M., McDaniel, M. A., & Liu, Y. (2014). The testing effect with authentic educational materials: A cautionary note. Journal of Applied Research in Memory & Cognition, 3, 214–221.
(19) Wiseheart, M., Küpper-Tezel, C., Weston, T., Kim, A.S.N., Kapler, I.V., and Foot-Seymour, V. (2019). Enhancing the quality of student learning using distributed practice. In J. Dunlosky & K. Rawson (Eds.), Cambridge handbook of cognition and education (pp. 550-584). New York: Cambridge University Press.
(20) Abrami, P. C., Bernard, R. M., Borokhovski, E., Wade, A., Surkes, M. A., Tamim, R., & Zhang, D. (2008). Instructional interventions affecting critical thinking skills and dispositions: A stage 1 meta-analysis. Review of Educational Research, 78, 1102-1134
(21) Abrami, P. C., Bernard, R. M., Borokhovski, E., Waddington, D. I., Wade, C. A., & Persson, T. (2015). Learning strategies for teaching students to think critically: a meta-analysis. Review of Educational Research, 85, 275-314.
