What is the causal impact of the mnemonic medium’s review sessions on reader retention?

Quantum Country readers reliably develop detailed retention, but that data is correlative. What’s the causal impact?

We’re currently running a large experiment on this question: holding out an entire review set

An informal experiment in 2019Q3 demonstrated a promising causal effect of the review sessions. We removed 8 questions from a set of users who did their reviews as normal, then reinserted them into their sessions after 2 weeks.

Compared to their peers, who had been reviewing those 8 questions during that 2 week period, the readers with delayed reviews saw their accuracy drop by 4pp, whereas the control readers’ accuracies increased by 7pp. 40% of the readers with delayed reviews saw their performance decline; by contrast, every control reader’s performance remained the same or improved.