Pan & Rickard - 2018 - Transfer of test-enhanced learning

referenceArchiveUUID: "eb360fc4-c164-46b4-8d47-4e18a2ae9583"
fullTitle: "Transfer of test-enhanced learning: Meta-analytic review and synthesis."
authors: ["Steven C. Pan","Timothy C. Rickard"]

Meta-analytic review of the empirical literature around Retrieval practice and transfer learning. Includes studies which perform the classic three-phase retrieval practice protocol (initial study, practice test, final recall test).

The analysis examines many different types of possible transfer:

• test format (e.g. cued recall to free recall or multiple choice)
• stimulus-response rearrangement (e.g. cloze deletions erasing a different portion in multi-term facts or term-definitions)
• untested materials seen during initial study
• application and inference questions: on the test, subjects must apply what they learned or make inferences
• problem-solving skills: using a multi-step procedure in a new problems (drawing on worked examples or medical diagnoses)
• mediator and related word cues (e.g. studying (mother, [child]), then being asked to recall (father, ??)

Grand weighted mean effect size for transfer of test-enhanced learning: d = {0.4} ☉

They find a medium-large effect (d=0.58) for transfer across initial and final test formats.

Moderators

The authors coded a number of candidate moderators for transfer effects.

Response congruency (i.e. practice and final tests having substantially overlapping answers) was the most substantive: “When there was no response congruency between the initial and final tests, the weighted effect size was d = 0.28…; if response congruency held, the estimated effect size increased by d = 0.30… yielding an estimated effect size of d = 0.58.”

They also find that a higher initial accuracy was associated with greater likelihood of transfer. (increased d by 0.0058 for each 1pp in initial test).

The last significant moderator they found was the presence of elaborated retrieval practice, which accounts for a d=0.23 boost. Here elaborated retrieval practice refers either to broad encoding methods (subjects instructed to think of all related information, to construct a detailed explanation, multiple questions involving higher-order Bloom’s taxonomy levels, etc) or to post retrieval processing of elaborative feedback.

This figure summarizes the three effects combined:

And this one describes the effect of initial test performance alone in more detail. It’s interesting that the transfer effect disappears completely for particularly poor performance:

It’s interesting to note that response congruency may take many forms, in terms of mechanistic retrieval pathways:

Compared to direct testing effects

In general, transfer effects are less potent than the corresponding direct Testing effect—which makes sense.

---

Q. Three substantive moderating factors?
A. Initial accuracy, response congruency, and elaborated retrieval practice. ☉

Q. What two categories of design properties are included in the category of “elaborated retrieval practice”?
A. Broad encoding methods, post-processing of elaborative feedback ☉

Q. What’s meant by “broad encoding methods”?
A. Explicit instruction to think of all possibly relevant information, constructing a detailed explanation of one’s response, multiple tasks about the same detail including multiple levels of Bloom’s taxonomy, etc ☉