Pan, S. C., & Rickard, T. C. (2018). Transfer of test-enhanced learning: Meta-analytic review and synthesis. Psychological Bulletin, 144(7), 710–756

Meta-analytic review of the empirical literature around Retrieval practice and transfer learning. Includes studies which perform the classic three-phase retrieval practice protocol (initial study, practice test, final recall test).

The analysis examines many different types of possible transfer:

• test format (e.g. cued recall to free recall or multiple choice)
• stimulus-response rearrangement (e.g. cloze deletions erasing a different portion in multi-term facts or term-definitions)
• untested materials seen during initial study
• application and inference questions: on the test, subjects must apply what they learned or make inferences
• problem-solving skills: using a multi-step procedure in a new problems (drawing on worked examples or medical diagnoses)
• mediator and related word cues (e.g. studying (mother, child), then being asked to recall (father, ??)

Grand weighted mean effect size for transfer of test-enhanced learning: d = {0.4}

They find a medium-large effect (d=0.58) for transfer across initial and final test formats.

Moderators

The authors coded a number of candidate moderators for transfer effects.

Response congruency (i.e. practice and final tests having substantially overlapping answers) was the most substantive: “When there was no response congruency between the initial and final tests, the weighted effect size was d = 0.28…; if response congruency held, the estimated effect size increased by d = 0.30… yielding an estimated effect size of d = 0.58.”

They also find that a higher initial accuracy was associated with greater likelihood of transfer. (increased d by 0.0058 for each 1pp in initial test).

The last significant moderator they found was the presence of elaborated retrieval practice, which accounts for a d=0.23 boost. Here elaborated retrieval practice refers either to broad encoding methods (subjects instructed to think of all related information, to construct a detailed explanation, multiple questions involving higher-order Bloom’s taxonomy levels, etc) or to post retrieval processing of elaborative feedback.

This figure summarizes the three effects combined:

And this one describes the effect of initial test performance alone in more detail. It’s interesting that the transfer effect disappears completely for particularly poor performance:

It’s interesting to note that response congruency may take many forms, in terms of mechanistic retrieval pathways:

Compared to direct testing effects

In general, transfer effects are less potent than the corresponding direct Testing effect—which makes sense.

Q. Three substantive moderating factors?
A. Initial accuracy, response congruency, and elaborated retrieval practice.

Q. What two categories of design properties are included in the category of “elaborated retrieval practice”?
A. Broad encoding methods, post-processing of elaborative feedback

Q. What’s meant by “broad encoding methods”?
A. Explicit instruction to think of all possibly relevant information, constructing a detailed explanation of one’s response, multiple tasks about the same detail including multiple levels of Bloom’s taxonomy, etc