Longer initial intervals produce much lower completion rates (among 2021 QCVC readers)

The 2021-04 Quantum Country schedule experiment varies the initial interval from 1 week to 2 months. Readers with those much longer intervals were much less likely to complete a full first repetition of all the questions they’d collected:

• aggressiveStart: 10081 registered -> 2076 completed in-essay (21%) -> 979 completed first review (47%)
• 1 week: 316 registered -> 62 completed in-essay (20%) -> 22 completed first repetition (35%)
• 2 weeks: 301 registered -> 59 completed in-essay (20%) -> 22 completed first repetition (37%)
• 1 month: 313 registered -> 69 completed in-essay (22%) -> 14 completed first repetition (20%)
• 2 months: 329 registered -> 74 completed in-essay (22%) -> 5 completed first repetition (7%)

20220118105427—includes only readers who registered at least 4 months ago, “completion” defined as 103+ cards of 112

Here, “first review” means any first review session, potentially including a one-day-interval “make-up session” added in the 2021-04 experiment for questions forgotten in-essay.

I believe the 1 and 2 week conditions’ completion rates are lower than aggressiveStart because the 1-day “make-up sessions” will mean the user will be asked to complete an additional session or two before they have a chance to run through all the cards they remembered. With aggressiveStart, if a user forgot a question in-essay, they’d review it again 5 days later, same as if they remembered it.

Here are the deciles of the number of sessions required to complete a first review:

• aggressiveStart: 2 / 3 / 3 / 4 / 4 / 5 / 5 / 6 / 7 / 8 / 16
• 1 week: 4 / 4 / 4 / 5 / 5 / 5 / 6 / 7 / 8 / 10 / 11
• 2 weeks: 2 / 4 / 4 / 4 / 5 / 6 / 6 / 7 / 8 / 9 / 10

Note that 8 deciles require at least 1 more review in these conditions, compared to aggressiveStart. The rest:

• 1 month: 3 / 4 / 4 / 4 / 4 / 5 / 5 / 6 / 7 / 8 / 12
• 2 months: 3 / 4 / 4 / 4 / 4 / 5 / 5 / 6 / 8 / 9 / 9

These don’t seem meaningfully different from 1/2 weeks, so I don’t think this explains the differences in completion rates.

Here are the rates for users completing a full first set of delayed reviews (or, in the case of aggressiveStart, a full set of 2-week reviews) ==wait, but the numbers for 1 and 2 weeks are higher than the numbers above—this can’t be right== 20220120121511:

• aggressiveStart: 659 (32% of those who completed the essay)
• 1 week: 29 (39%)
• 2 weeks: 27 (41%)
• 1 month: 14 (18%)
• 2 months: 6 (7%)

So in general, people who complete a full first repetition are overwhelmingly likely to finish that first repetition, i.e. to repeat all the prompts they initially got wrong and had to make up.