Retrieval practice and transfer learning

The simplest instantiation of the Testing effect is elicited when students test themselves on a prompt which they’ll later be tested on (i.e. identically). So: say you memorize a thousand prompts about a topic. What we really care about is not just your ability to answer those specific questions, but how that practice affects your utilizable understanding of the topic. What can you actually do with it? If I ask you an open-ended question involving the knowledge you practiced, can you marshall that knowledge—i.e. Transfer learning.

In the mnemonic medium

My coarse impression from the last few years of observations (2024-04) is that knowledge reinforced by the Mnemonic medium is pretty brittle; doesn’t transfer very well.

• The mnemonic medium could include prompts which support near transfer
• Idea-centric memory system
• How might the mnemonic medium support readers in building more complex understanding?

Experimental evidence

• Pan, S. C., & Rickard, T. C. (2018). Transfer of test-enhanced learning: Meta-analytic review and synthesis. Psychological Bulletin, 144(7), 710–756
  • This meta-analysis of transfer effects in retrieval practice finds an overall average transfer effect of d=0.4 relative to non-practice interventions, moderated in particular by response congruency, initial accuracy, and presence of elaborated retrieval during practice.
• Butler, A. (2010). Repeated Testing Produces Superior Transfer of Learning Relative to Repeated Studying. Journal of Experimental Psychology. Learning, Memory, and Cognition, 36, 1118–1133.
  • small experiments with adapted Wikipedia passages demonstrate retrieval practice improves performance on transfer-like questions relative to re-studying by a similar margin to verbatim questions
• Agarwal, P. K., Nunes, L. D., & Blunt, J. R. (2021). Retrieval Practice Consistently Benefits Student Learning: A Systematic Review of Applied Research in Schools and Classrooms. Educational Psychology Review.
  • in this broader systematic review of Retrieval practice, the authors note that “Effect sizes were generally smaller for experiments with rephrased questions compared to experiments with verbatim or repeated questions.” The effect looks smallish—perhaps d~0.1-0.2.
• Samani, J., & Pan, S. (2021). Interleaved Practice Enhances Memory and Problem-Solving Ability in Undergraduate Physics. Npj Science of Learning, 6.
  • Interleaved practice, which in this instance would involve both Retrieval practice and the Spacing effect, produces markedly improved results on a delayed transfer test.
• Karpicke, J. D., & Blunt, J. R. (2011). Retrieval Practice Produces More Learning than Elaborative Studying with Concept Mapping. Science, 331(6018), 772–775
  • Study primarily designed to assess Concept mapping vs. Retrieval practice, but assessment involves inference questions requiring transfer; on these questions, students in the retrieval practice condition performed better than all other conditions (repeated studying, concept mapping), and fairly well in absolute terms.
• Agarwal, P. (2019). Retrieval Practice & Bloom’s Taxonomy: Do Students Need Fact Knowledge Before Higher Order Learning? Journal of Educational Psychology, 111, 189–209
  • In some conditions, students practice with factual questions and are asked to answer higher-order test questions. These students’ performance is ind istinguishable from those who don’t engage in retrieval practice at all.
• O’Day, G. M. (2019). Improving Problem Solving with Retrieval-Based Learning \[Unpublished master’s thesis, Purdue University\]
  • Testing effect found for near transfer but not far transfer on complex problem-solving task.
• Cerdán, R., Vidal-Abarca, E., Martinez, T., Gilabert, R., & Gil, L. (2009). Impact of question-answering tasks on search processes and reading comprehension. Learning and Instruction, 19(1), 13–27
  • high-level Adjunct questions produce much more transfer learning than low-level ones

To read

• Agarwal, P. K., Bain, P. M., & Chamberlain, R. W. (2012). The value of applied research: Retrieval practice improves classroom learning and recommendations from a teacher, a principal, and a scientist. Educa- tional Psychology Review, 24,437– 448. http://dx.doi.org/10.1007/ s10648-012-9210-2
• Johnson, C. I., & Mayer, R. E. (2009). A testing effect with multimedia learning. Journal of Educational Psychology, 101(3), 621–629. https://doi.org/10.1037/a0015183
• McDaniel, M. A., Howard, D. C., & Einstein, G. O. (2009). The Read-Recite-Review Study Strategy: Effective and Portable. Psychological Science, 20(4), 516–522. https://doi.org/10.1111/j.1467-9280.2009.02325.x
• McDaniel, M. A., Thomas, R. C., Agarwal, P. K., McDermott, K. B., & Roediger, H. L. (2013). Quizzing in middle-school science: Successful transfer performance on classroom exams. Applied Cognitive Psychol- ogy, 27,360–372. http://dx.doi.org/10.1002/acp.2914
• Agarwal, P. K. (2019). Retrieval practice and Bloom’s taxonomy: do students need fact knowledge before higher order learning? Journal of Educational Psychology, 111, 2 189–209.
• Butler, A. C., Black-Maier, A. C., Raley, N. D., & Marsh, E. J. (2017). Retrieving and applying knowledge to different examples promotes transfer of learning. Journal of Experimental Psychology: Applied, 23,433–446. http://dx.doi.org/10.1037/xap0000142
• Carpenter, S. K. (2012). Testing enhances the transfer of learning. Current Directions in Psychological Science, 21,279–283. http://dx.doi.org/10 .1177/0963721412452728
• Rohrer, D., Taylor, K., & Sholar, B. (2010). Tests enhance the transfer of learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36,233–239. http://dx.doi.org/10.1037/a0017678
• van Eersel, G. G., Verkoeijen, P. P., Povilenaite, M., & Rikers, R. (2016). The testing effect and far transfer: The role of exposure to key infor- mation. Frontiers in Psychology, 7, 1977. http://dx.doi.org/10.3389/ fpsyg.2016.01977
• Balch, W. R. (1998). Practice versus review exams and final exam performance. Teaching of Psychology, 25, 181–185. http://dx.doi.org/10 .1207/s15328023top2503_3
• Bjork, E. L., Little, J. L., & Storm, B. C. (2014). Multiple-choice testing as a desirable difficulty in the classroom. Journal of Applied Research in Memory & Cognition, 3, 165–170. http://dx.doi.org/10.1016/j.jarmac .2014.03.002
• McConnell, M. M., St-Onge, C., & Young, M. E. (2015). The benefits of testing for learning on later performance. Advances in Health Sciences Education, 20, 305–320. http://dx.doi.org/10.1007/s10459-014-9529-1
• McDaniel, M. A., Anderson, J. L., Derbish, M. H., & Morrisette, N. (2007). Testing the testing effect in the classroom. European Journal of Cognitive Psychology, 19, 494 –513. http://dx.doi.org/10.1080/ 09541440701326154
• McDaniel, M. A., Wildman, K. M., & Anderson, J. L. (2012). Using quizzes to enhance summative-assessment performance in a web-based class: An experimental study. Journal of Applied Research in Memory & Cognition, 1, 18–26. http://dx.doi.org/10.1016/j.jarmac.2011.10.001