Agarwal, P. (2019). Retrieval Practice & Bloom’s Taxonomy: Do Students Need Fact Knowledge Before Higher Order Learning? Journal of Educational Psychology, 111, 189–209

This paper from Pooja Agarwal questions the sequentiality of Bloom’s taxonomy, contra e.g. Deep understanding requires detailed knowledge of fundamentals. Provides evidence around How complex should tasks be for test-enhanced learning?.

Methodologically, this paper includes an interesting application of Bloom’s taxonomy to authoring Retrieval practice questions: it gives a scheme for creating retrieval practice questions for apply, analyze, evaluate, create.

The authors hoped to find evidence which can distinguish between three possible frameworks which might apply: Desirable difficulties, after Bjork, Transfer-appropriate processing, and “foundation of factual knowledge” (i.e. sequentialist Bloom’s taxonomy).

Experiment 1

• Within-subjects design of 48 people with a 4x2 design (study once, study twice, fact quiz, higher order quiz x fact test, higher order test) in varying orders. Two sessions: study, then test (two days later).
  • At test time, the question stems were rephrased and multiple-choice alternatives were reordered.
• Delayed test performance ~10pp improved for the quiz conditions vs. study conditions.
• Findings consistent with Transfer-appropriate processing and contrary to the other two:
  • Experiment 1 finds retrieval practice of higher-order questions indistinguishable from re-study condition on delayed fact test; likewise for retrieval practice of factual questions on a delayed higher-order test.
  • “In sum, initial retrieval practice enhanced final test performance, but only when the initial quiz type (fact or higher-order) matched the final test type (fact or higher order, respectively).”

Experiment 2

• Designed to explore impact of mixing factual and higher order question practice
• Another within-subjects design of 48 students, this time with practice conditions varying between one higher order quiz, two higher order quizzes, two fact quizzes, two mixed quizzes. Same basic design otherwise, same 2-day delay.
• Findings still most consistent with Transfer-appropriate processing; the mixed condition had the highest average result on the final tests but matched conditions (fact–fact, higher-order–higher-order) had the highest separate scores.

Experiment 3

• Attempted replication with different demographic (sixth graders) and content (world history) in an authentic classroom environment.
• 3x2 within-subjects design (higher-order quiz, mixed quiz, nonquizzed x fact test, higher-order text) in similar interleave fashion to previous experiments
• Findings consistent with previous two experiments, though interestingly, mixed quizzes produced better higher-order test scores than higher-order-only quizzes.
  • Effects were quite large, shifting final scores from roughly a D to roughly an A

Q. Finding regarding the notion that factual information supports transfer to higher-order learning?
A. Students who practiced with factual quizzes didn’t do better than students who simply re-studied on delayed higher-order quizzes.

Q. Which theoretical framework relating factual and higher-order knowledge was best supported by this study?
A. Transfer-appropriate processing

Q. Which practice modality produced the highest average results (across test types) in the latter two experiments?
A. Mixed

Q. Population for experiments 1 and 2?
A. Undergrads.

Q. Purpose of experiment 2?
A. Explore benefits of mixing factual and higher-order quiz questions in practice.

Q. Population for experiment 3?
A. Sixth graders.

Q. Purpose of experiment 3?
A. Replicate first two experiments with middle school kids.

Q. In experiment 2, which practice modalities produced the highest test results?
A. Matching modalities: factual quizzes for factual tests, higher-order quizzes for higher-order tests.

Q. In experiment 3, how did higher-order quiz practice fare on factual questions?
A. Same as non-quizzed.

Q. In experiment 3, which practice modality fared best on higher-order test questions?
A. Mixed (surprisingly)