Testing effect

When you try to recall some detail from memory, that act strengthens your memory of that detail. When exploited as a learning activity, this is called “retrieval practice.”

Experiment has demonstrated this effect even when the correct answer is not provided, and even when the test-taker is given “open-book” access to find a correct answer. The effect has been demonstrated in many fields and at many age levels.

This suggests a significantly different role for tests. In typical classrooms, teachers and students imagine that learning happens during lectures, or while reading the material. The tests are there to assess that learning. But in fact, the tests themselves are an important part of the learning process.

Versus other study activities

Retrieval practice leads to more durable long-term memory than simply studying material by e.g. re-reading, despite the fact that students will be less successful during practice itself (Roediger, 2006). Related: Desirable difficulties, after Bjork.

It also seems to lead to more durable memory (Karpicke and Smith, 2012) and improved learning performance in general (Karpicke and Blunt, 2011) relative to Elaborative encoding-based practice alone.

Format

The testing effect is generally measured to be more pronounced for production tests (short answer, essay) than for discrimination (multiple choice / true-false) (e.g. Kang et al, 2007). This may be due to the Generation effect.

References

Reviews

Basic Research and Implications for Educational Practice. Perspectives on Psychological Science, 1(3), 181–210. https

Branwen, G. (2009). Spaced Repetition for Efficient Learning. Retrieved December 16, 2019, from https://www.gwern.net/Spaced-repetition

Commentary

Roediger, H. L., & Karpicke, J. D. (2018). Reflections on the Resurgence of Interest in the Testing Effect. Perspectives on Psychological Science, 13(2), 236–241. https://doi.org/10.1177/1745691617718873

Primary research

\/\/doi.org\/10.1080\/09541440601056620

\/\/doi.org\/10.1126\/science.1199327

\/\/doi.org\/10.1016\/j.jml.2012.02.004

\/\/doi.org\/10.1037\/edu0000367

Backlinks

• Spaced repetition memory system
  • A spaced-repetition memory system combines the Testing effect and the Spacing effect to enable efficient memorization of many thousands of facts (Spaced repetition memory systems are extremely efficient). Some people also use them for a broader set of tasks (see below). Spaced repetition memory systems make memory a choice, but they’re not just for rote facts: Spaced repetition memory systems can be used to develop conceptual understanding.
• Spaced repetition memory systems can be used to develop conceptual understanding
  • It’s worth noting, too, that the cogpsy basis of spaced repetition (the Testing effect) has been demonstrated for conceptual knowledge as well as for declarative knowledge (see Karpicke and Blunt, 2011).
• Generation effect
  • This effect is sometimes equated with the Testing effect: when students are asked to produce answers from memory (as in free recall tests), some part of the impact on recall may be due to the processing involved in generating the answers. But experiments by Karpicke and Zaromb (2010) suggest that the two effects are distinct and can be to some extent isolated. People who studied by trying to generating targets ended up with poorer recall than people who study by attempting to recall.
• Evergreen note maintenance approximates spaced repetition
  • This isn’t an efficient spaced repetition, memory-wise: you’re not really taking advantage of the Testing effect. But it does take advantage of something like the Generation effect, and it may be a useful lens to think about managing your attention across the corpus of ideas you accumulate over time. Specifically, this practice will encourage you to repeatedly give attention to past ideas which seem relevant to your present work. And because you may find yourself revising your notes on those past ideas, the attention may be quite effortful.
• Nate Kornell
  • Professor of cognitive psychology at Williams College. A student of Robert A. Bjork. Lots of research around Spacing effect and Testing effect.
• Spaced repetition memory prompts should ensure reviewers must retrieve answers from memory
  • A Spaced repetition memory system depends on the Testing effect: you’ll remember something longer if you have to actually retrieve it from memory than if you’re simply re-exposed to it. So when writing prompts, make sure that the prompt requires the reader to actually retrieve the memory you want to reinforce.
• §Taking knowledge work seriously (Stripe convergence talk, 2019-12-12)
  • Testing effect
• 2020-11-03
  • Testing effect
• 2020-11-01 – 2020-11-07
  • Better understanding of Testing effect and Generation effect, read a bunch of papers around that. Incrementally better understanding of Elaborative encoding. A nice review paper:
• 2019-12-16
  • 8:42: wrote Spaced repetition memory systems make memory a choice and Testing effect
• [[Karpicke, J. D., & Blunt, J. R. (2011). Retrieval Practice Produces More Learning than Elaborative Studying with Concept Mapping. Science, 331(6018), 772–775.

https://doi.org/10.1126/science.1199327]]
* Experiment comparing concept mapping (invoking Elaborative encoding) vs. retrieval practice (invoking Testing effect) on memory. Like Karpicke and Smith (2012), this paper is also interested in whether the effects of retrieval practice seem to be due to elaborative encoding, or whether there’s some other process going on.

• [[Karpicke, J. D., & Smith, M. A. (2012). Separate mnemonic effects of retrieval practice and elaborative encoding. Journal of Memory and Language, 67(1), 17–29.

https://doi.org/10.1016/j.jml.2012.02.004]]
* Is the Testing effect produced by Elaborative encoding? When you review prompts in a Spaced repetition memory system, you give yourself an opportunity to form new connections based on your current context and the different thoughts you have during the review session.

• [[Karpicke, J. D., & Zaromb, F. M. (2010). Retrieval mode distinguishes the testing effect from the generation effect. Journal of Memory and Language, 62(3), 227–239.

https://doi.org/10.1016/j.jml.2009.11.010]]
* Are the Testing effect and the Generation effect simply the same thing? They’re often equated, and indeed, I’d equated them in my own notes. This paper describes a series of experiments which attempt to demonstrate that a different underlying mechanism drives the two phenomena, and that retrieval practice has a greater impact on memory than generation.

• Preparing for FOO Camp talk 2020
  • Testing effect
• [[Roediger, H. L., & Karpicke, J. D. (2006). The Power of Testing Memory: Basic Research and Implications for Educational Practice. Perspectives on Psychological Science, 1(3), 181–210.

https://doi.org/10.1111/j.1745-6916.2006.00012.x]]
* A thorough review paper on the Testing effect. This paper covers the history of experiment on the test in laboratory settings and more practical classroom settings and reviews some more recent research by the authors.

• [[Rohrer, D., Dedrick, R. F., Hartwig, M. K., & Cheung, C.-N. (2020). A randomized controlled trial of interleaved mathematics practice. Journal of Educational Psychology, 112(1), 40–52.

https://doi.org/10.1037/edu0000367]]
* Interleaved practice creates a natural context for the Spacing effect and Testing effect: