Quantum Country readers reliably develop detailed retention, but that data is correlative. What’s the causal impact?
We’re currently running a large experiment on this question: holding out an entire review set
An informal experiment in 2019Q3 demonstrated a promising causal effect of the review sessions. We removed 8 questions from a set of users who did their reviews as normal, then reinserted them into their sessions after 2 weeks.
Compared to their peers, who had been reviewing those 8 questions during that 2 week period, the readers with delayed reviews saw their accuracy drop by 4pp, whereas the control readers’ accuracies increased by 7pp. 40% of the readers with delayed reviews saw their performance decline; by contrast, every control reader’s performance remained the same or improved.