In 2019H1, 29% of QCVC readers who finished the in-text level finished the 1 month level

Summary: P(finished 1 month level | finished in-text level) ~= 29% for QCVC readers registering from March through June of 2019. That is extremely high! It suggests that a huge fraction of serious readers do indeed end up memorizing all the material.

The mnemonic medium aspires to make it almost effortless to remember what you read. A weaker version of that claim is: “people who really do want to remember the details of a mnemonic essay, and who are willing to spend some effort on it, can usually remember all those details.”

It’s hard to know which people really do want to remember the details of QCVC, but a decent upper bound on that set of people is “the set of people who ever correctly answered 95% of questions in QCVC.” That is: “the set of people who finished the in-text level on QCVC.” Many of those people don’t really want to remember all the details, but most people who do want to remember all the details are in that set. So this is test for seriousness is not very specific, but it’s fairly sensitive.

Now: how many people who reached the in-text level in QCVC ended up reaching the 1-month level?

The answer’s remarkably high. About 29% of users who reach the in-text level of QCVC eventually reached the 1-month level, among users registering between 2019/03/01 and 2019/06/31. (see logs.levelAttainment and logs.levelAttainmentAggre, as well as Level-over-level conversion rates, by registration month, now on the dashboard).

It’s worth considering what those users actually had to do. Because our old batching and schedule were pretty inefficient, this takes most such users 20-30 sessions (see Level completion events; change schedule to “original”). Reaching the 1 month level with our old schedule required answering at minimum 560 questions, but the real 25th/50th/75th %iles for these users are 800/848/903 questions answered (Query).

That’s really wild! Simply reading the essay and answering all the questions is not an enormous investment. Answering almost a thousand questions, as lightweight as we try to make that, is an enormous investment. It’s truly astonishing to me that so many readers make it all the way from one milestone to the other!

Furthermore, 29% is a lower bound: among these cohorts, another 8% have reached the 2-week level and are still actively reviewing. Many of those users will ultimately reach 1 month, likely pushing the final fraction to around 1/3. (2020/08/14 update: it reached 31%).

A reasonable interpretation of all this is something like: “a large fraction of people who really do want to remember the details of QCVC, and who are willing to spend some effort, do actually end up remembering all those details.”

Update: for 2020Q4, 183 readers finished the in-text level, and 46 of them reached 1-month: that’s about 25%, with another few percentage points of users still pending. So