What is the causal impact of the mnemonic medium’s review sessions on reader retention? An informal experiment in 2019Q3 demonstrated a promising causal effect of review sessions in Quantum Country. We removed 8 questions from a set of users who did their reviews as normal, then reinserted them into their sessions after 2 weeks.
Compared to their peers, who had been reviewing those 8 questions during that 2 week period, the readers with delayed reviews saw their accuracy drop by 4pp, whereas the control readers’ accuracies increased by 7pp. 40% of the readers with delayed reviews saw their performance decline; by contrast, every control reader’s performance remained the same or improved.