Rothkopf, E. Z. (1966). Learning from Written Instructive Materials: An Exploration of the Control of Inspection Behavior by Test-Like Events. American Educational Research Journal, 3(4), 241–249

Ernst Rothkopf investigates the effect of asking “adjunct questions” while people read. He compare the effect of asking the questions before a given passage with the effect of asking after; he also varies whether students are told the correct answers after answering (en bloc).

Testing effect vs. facilitation effects: Critically, he also distinguishes between the effects of these questions on subsequent performance on those questions, and on other questions about different details in the passage. Here he’s trying to distinguish between the direct instructional effects of practice (i.e. Testing effect) and the more general facilitative effects of being asked questions while one reads. He verifies that there’s no transfer between these two sets of questions by letting a separate group of students practice the adjunct questions and then testing them on the “unrelated” questions (both pre- and post-practice). He finds that mere practice on the adjunct questions does not improve performance on the “unrelated” questions. He concludes that any performance improvement in the reading experiment must come from “general facilitation effects”, rather than direct testing effects of the adjunct questions. (This is a really nice approach!)

Direction reference group (DRG): He’s also interested in comparing the influence of adjunct questions to that of “hortatory care-inducing directions”—i.e. reminding people to read slowly and carefully (sort of like controlling for a Hawthorne effect). So one group (DRG) got those directions but no questions.

Findings

• Post-section adjunct questions (SA) and DRG both performed much better than the control on the general test, by similar margins. Adjunct questions improve comprehension of related but untested content
  • Interestingly, the group which was given the correct answers (SAA) didn’t quite reach statistical significance (but N was only 21; I think there was probably an effect, probably of similar size).
  • DRG spent much more time reading than SAA. SA read more slowly, perhaps because the answers weren’t given.
• But SAA performed about twice as well as the DRG on those questions on a post-test.
• Rothkopf concludes that that even if the facilitative effects are the same between SA and DRG, adjunct questions are a “two-for-one”: they both produce the facilitative effects and also reinforce specific knowledge.

Q. Main question?
A. Do “adjunct questions” given while reading facilitate reading efficacy, apart from the direct practice effects on the tested information?

Q. Summarize the findings.
A. Post-section adjunct questions and “hortatory care-inducting directions” produced a substantial improvement on untested questions about a passage. Pre-section questions didn’t. Adjunct questions also massively boosted performance on a post-test with those questions, particularly when answers were given on the post-section tests.

Q. How did the author distinguish between the direct instructional effect of the adjunct questions and the general facilitative effect?
A. The “general test” questions were on other material from the passage that didn’t overlap with the adjunct questions. The author verified this by letting a separate group practice the adjunct questions without reading the passage, and observed no improvement on the “general test” questions.

Q. Why does the author argue adjunct questions are valuable even if “hortatory care-inducing directions” produced the same effect on the general test?
A. The adjunct questions also produced massively improved performance on the specific knowledge tested by the questions.

Q. What was the point of the “hortatory care-inducing directions”?
A. To compare the effects of asking adjunct questions to simply telling students to read slowly and carefully.