Roelle, J., & Berthold, K. (2017). Effects of incorporating retrieval into learning tasks: The complexity of the tasks matters. Learning and Instruction, 49, 142–156

The authors demonstrate that making Adjunct questions function as Retrieval practice (i.e. via closed-book testing) is a double-edged strategy: closed-book practice improves recall, but it also lowers immediate performance. When adjunct questions are complex, the impact on immediate performance is enough to produce an inverted Testing effect. i.e: “a closed-book implementation style can substantially moderate the superiority of high complexity adjunct questions.” This is an interesting answer to How complex should tasks be for test-enhanced learning?.

n.b: students weren’t given knowledge of results or any other kind of feedback on the adjunct questions. I’d guess these findings would be attenuated if they were.

Q. Main findings?
A. Open-book study produces more forgetting, but enables better initial performance on high-complexity questions, and so open-book/high-complexity produced the best performance on delayed questions. Closed-book questions (no feedback) produced the same performance irrespective of complexity. Low-complexity/open-book performed worst.


See also Agarwal, P. K., Karpicke, J. D., Kang, S. H. K., Roediger, H. L., & McDermott, K. B. (2008). Examining the testing effect with open- and closed-book tests. Applied Cognitive Psychology, 22(7), 861–876.

Last updated 2023-09-15.