How complex should tasks be for test-enhanced learning?

If one plans to exploit the Testing effect for Test-enhanced learning (e.g. in a Spaced repetition memory system), should you test yourself on lots of simple items or higher-level tasks, to promote Transfer learning?

Commonplace practice typically advises the former (Spaced repetition memory prompts should usually focus on one atomic unit), but the balance of research is more ambivalent:

• Best practice task complexity matches test task complexity; i.e. low-complexity practice outperforms high-complexity practice on low-complexity tasks (see also Transfer-appropriate processing):
  • Jensen, J. L., McDaniel, M. A., Woodard, S. M., & Kummer, T. A. (2014). Teaching to the Test…or Testing to Teach: Exams Requiring Higher Order Thinking Skills Encourage Greater Conceptual Understanding. Educational Psychology Review, 26(2), 307–329. https://doi.org/10.1007/s10648-013-9248-9
  • Agarwal, P. (2019). Retrieval Practice & Bloom’s Taxonomy: Do Students Need Fact Knowledge Before Higher Order Learning? Journal of Educational Psychology, 111, 189–209
• High-complexity practice outperforms low- for test items of both low and high complexity:
  • Roelle, J., & Berthold, K. (2017). Effects of incorporating retrieval into learning tasks: The complexity of the tasks matters. Learning and Instruction, 49, 142–156

There’s also van Gog, T., & Sweller, J. (2015). Not New, but Nearly Forgotten: The Testing Effect Decreases or even Disappears as the Complexity of Learning Materials Increases. Educational Psychology Review, 27(2), 247–264, which claims that the testing effect doesn’t apply to complex tasks, but this seems to be at best only sometimes true.

See also: Retrieval practice and transfer learning