2021-04 Quantum Country schedule experiment

Having now discovered that Quantum Country users seem to forget most prompts quite slowly and that Quantum Country users rarely forget after demonstrating five-day retention, we should experiment with some much more aggressive schedules for Quantum Country.

In terms of actually designing the schedules, this is a good reference: Spitzer, H. F. (1939). Studies in retention. Journal of Educational Psychology, 30(9), 641–656. Some desiderata:

  • can plot a forgetting curve for the first repetition (and weakly, the second)
  • can compare multiple schedules at time T, with different prior repetition numbers
  • can compare different ISIs for a given RI

Schedules:

  • A: [20] 1 week / 3 weeks / 3 months / 9 months

  • A’: [5] 1 week / 6 weeks / 4 months / 1 year

  • B: [20] 2 weeks / 6 weeks / 4 months / 1 year

  • B’: [5] 2 weeks / 3 months / 9 months

  • C: [20] 1 month / 3 months / 9 months

  • C’: [5] 1 month / 6 months / 1 year

  • D: [20] 2 months / 4 months / 1 year

  • D’: [5] 3 months / 6 months / 1 year

  • Same-time comparisons:

    • 1 month: A vs. C
    • 2 months: B vs. D
    • 4 months: A vs. C
    • 6 months: B vs. D
    • 13 months: A vs. C
    • 18 months: B vs. D
  • Initial forgetting curve points: 1 week / 2 weeks / 1 month / 2 months / 3 months

  • ISI / RI combinations:

    • ISI: 1/2 weeks; RI: 6 weeks
    • ISI: 2 weeks, 3 weeks, 1 month; RI: 3 months
    • ISI: 6 weeks, 2 months; RI: 4 months
    • ISI: 1 month, 3 months; RI: 6 months

Language to be updated

  • [x] Description of schedule in first essay
    • QCVC
    • “and you’ll receive reminders each day (or few days)”
    • “Then continue to follow the procedure described just above, taking a few minutes each day to complete your review sessions, prompted by the reminders you’ll be sent”
    • Search:
    • “in the days and weeks ahead”
    • Teleportation:
    • “in the days and weeks ahead”
    • “and you’ll receive reminders each day (or few days)”
  • [x] Description of schedule (and coordinates) in coachMarkContents.ts (will require passing in schedule)
  • [ ] firstReviewEmail.ts
  • [ ] possibly: secondReviewEmail.ts
  • [ ] welcomeEmail.ts
  • [ ] possibly: finishedLevelMessages.ts
  • [ ] firstReviewSessionMessages.ts

Lapses

Motivated by Quantum Country users who forget in-essay exhibit sharp forgetting curves, I think I’d like to make forgotten prompts (either in-essay or relative to the first level) repeated after 1 day.

I’m little concerned about jumping from 1 day to these other minimum intervals, but I have enough span across the schedule variants that I’ll at least get to see what happens and really understand it.

Implementation-wise, the key “weird” thing that needs to happen here is that when you complete a needsRetry prompt with no bestInterval, we need to go to this minimum level instead of the first “real” level.

TODO

  • [x] Review essays for language which would be problematized by schedules
  • [x] Review email language
  • [x] Review end of session language
  • [x] Review coach mark language
  • [x] Make plan for lapses
  • [x] End retry experiment (or, well, I limited it to old users)
  • [x] Create new schedules
    • [x] Check in on “default”… where is that used?
  • [x] Disable modified intervals for new schedules
  • [x] Disable new enrollments for efficacy2 experiment
  • [x] Create experimental condition for schedules
  • [x] Use experimental condition throughout codebase (i.e. where aggressiveStart condition is used)
  • [x] Update review marmking logic for interval floor
  • [x] Update in-essay language
  • [x] Update coach marks
    • [x] Make progress indicator render correctly with < 6 markers
  • [x] Check card rendering
  • [ ] Make sure first-second-level coach mark won’t show in basement review session
  • [ ] Update emails
  • [ ] Update review session summaries
  • [ ] Test and debug
Last updated 2023-07-13.