Performance in-essay and in first review strongly predict a Quantum Country reader’s future performance on that question

If a reader remembers an answer in the context of the essay, they have an 8% chance of having multiple lapses in the next half year; if they forget, that rises to 30%. But the overall correlation’s not that great: In-essay Quantum Country reader performance partially predicts first review performance.

Performance in the first review session is a better predictor: only 5% of traces remembered at first review have multiple lapses in the following half year, rising to 34% when forgotten in first review. (This shift is congruent with In-essay per-question accuracies on Quantum Country don’t effectively predict which questions drive long-term lapses (but first repetition accuracies do))

Users who remember a question successfully both in-essay and in their first review session have only a 3% chance of forgetting that question in the next half year; 4% in the next year.

Slicing per-question (20210322120153), we see a strong predictive effect across all questions except the top few % “easiest,” which have too few lapses to measure. 25th/50th/75th %ile likelihood ratios across all questions: 2.3 / 4.0 / 7.1 (20210322120552)

More data

Measuring number of lapses in half-year traces, for traces which last at least half a year, unconditioned. 20210322102651:

• 71% of traces have 0 lapses
• 16% of traces have 1 lapse
• 6% of traces have 2 lapses
• 3% of traces have 3 lapses
• 1% has 4, 1% has 5, 1% has 6, 1% has >6 (max 48).

Extending that to a year, there’s very little change:

• 71% of traces have 0 lapses
• 15% of traces have 1 lapse
• 6% of traces have 2 lapses
• 3% of traces have 3 lapses
• 1% has 4, 1% has 5, 1% has 6, 1% has 7-8, 1% has > 8 (max 51).

Conditioned on a successful in-essay repetition (back to half a year, to get more samples):

• remembered
  • 81% of traces have 0 lapses
  • 11% have 1 lapse
  • 4% have 2 lapses
  • 1% has 3, 1% has 4, 1% has 5, 1% has > 5 (max 38).
• forgotten
  • 47% of traces have 0 (additional) lapses
  • 23% have 1 more lapse
  • 11% have 2 more lapses
  • 7% have 3
  • 4% have 4, 3% have 5, 1% has 6, 1% has 7, 1% has 8, 1% has 9-10, 1% has > 10 (max 48).

To put this another way: 30% of traces forgotten in-essay will have multiple future lapses; only 8% of traces remembered in-essay will.

Conditioned on a successful first review:

• remembered
  • 83% of traces have 0 lapses
  • 12% of traces have 1 lapse
  • 3% of traces have 2 lapses
  • 1% has 3, 1% has > 3 (max 22)
• forgotten
  • 43% of traces have 0 (additional) lapses
  • 23% have 1 more lapse
  • 13% have 2 more
  • 8% have 3 more
  • 4% have 4 more, 3% have 5, 2% have 6, 1% has 7, 1% has 8, 1% has 9-10, 1% has >10 (max 48)

To put this another way: 34% of traces forgotten in first review will have multiple future lapses; only 5% of traces remembered at first review will.

Conditioned on in-essay repetition and first review both successful (20210322120946):

• 90% of traces have 0 lapses
• 7% have 1
• 1% has 2, 1% has 3, 1% has >3 (max 22).

So a user-question pair with two successful first repetitions has only a 3% chance of multiple lapses in the first half-year. 4% in the first year.