Michael Nielsen has suggested that the review sessions of a Spaced repetition memory system don’t just help you remember things: it orchestrates your repeated attention over time across hundreds of tiny tasks, too many to manage by hand. Systems like these are a form of programmable attention.
The concept and term come from Michael Nielsen, in conversations from ~2018—I’ve forgotten the details, unfortunately.
Michael Nielsen explores a method for deepening one’s understanding of a mathematical result, to the point that one becomes so familiar with the pieces that you can almost see the result directly. The method is expressed in terms of writing Spaced repetition memory system prompts, but as he notes near the end of the piece, the SRS format isn’t really necessary to the process. It just provides some pressure to piece apart the ideas into fine-grained atoms. (Note-writing does something similar for me, at a coarser grain: Evergreen notes should be atomic) But the advantage of writing these pieces as SRS prompts, of course, is that one will remember the results.
Michael Nielsen suggests that we should Prefer “memory system” to “spaced repetition system”, because spaced repetition is one idea among many which will make for a powerful memory system. But I wonder if even “memory” is too narrow a frame. What I’m really doing is creating a daily routine for practice. Systematized, orchestrated attention given to ideas I find interesting (Programmable attention). Often, that really is memory-centric: I’m just maintaining certain declarative facts I’ve learned. But when I’m using these systems well, I’m often making new connections, thinking new thoughts, putting ideas to use. My instinct is that the more I do the latter, the more interesting and valuable those sessions get.
First version published in September, 2018—coincidentally around the time Michael Nielsen and I started working on Quantum Country. As of 2022-02-15, doesn’t appear to be under active development: most recent changes were in Feb, 2019.
Fearful that I’ll lose the respect of others (particularly Michael Nielsen, apparently) because I get wrapped up in coding instead of generating "real" insight
Very briefly, we dug into another set of feelings. I’ve been working on the mnemonic medium for almost four years now. It’s truly “mine” now in all the relevant ways… but it’s incontrovertibly true that the original core idea was Michael Nielsen’s.
Michael Nielsen reports that in later chapters of Quantum Country, he included less review than he might have otherwise (“Recall that the Hermitian…”), because as an author he can rely on the reader knowing those details.
2023-02-13: last night Michael Nielsen sent me How to make memory systems widespread? - Michael Nielsen and it really threw me into a tailspin. I felt betrayed, judged, threatened, undermined, etc. I thought we were friends / collaborators—why would he write something which casually purports to answer the question I’ve been working on for years? Is he just trying to demonstrate that I’m incompetent? Of course, in fact, he’s trying to capture an unusual framing of the problem that we’ve discussed at length before. It’s not a damning critique of my essential competence; it’s a generous contribution to the field. As soon as I read the piece, of course, I felt enormously better.
In 2019, my colleague Michael Nielsen and I created a prototype Mnemonic medium—a medium which makes it much easier for people to remember what they read, instead of rapidly forgetting all but the gist. We built on the work of cognitive scientists, who have identified a formulaic set of steps you can take to reliably remember something. The medium integrates those steps into prose, and automates them, so that the reading experience naturally reinforces readers’ memories.
Think about how I felt having a heart-to-heart with Michael Nielsen a couple walks ago. I asked him about his reticence over the pandemic, about whether he felt any kind of resentment towards me. He confessed that he felt really bad about his behavior and that it was really just a problem of feeling incredibly low-energy and somewhat depressed all this time. That felt great! And I think the vulnerability brought us closer. This conversation could have happened sooner, and it would have been better for both of us. But I waited because I was afraid.
I don’t think this necessarily contradicts the ideas that Michael Nielsen and I have written about Insight through making: I still need insight in order to invent. A powerful system systematizes some powerful insight. It basically always begins with a bespoke, N-of-1, ad-hoc activity—NLS with that lab’s practices; Mathematica with Wolfram’s research; text interpolation interfaces with Robin Sloan’s writing experiments. As I write in Focus on power over scale for transformative system design, it’s going to be very difficult to find the most powerful idea in the context of building a fully-general system. Much easier to do it in the context of a string of small prototypes I can throw out and recreate. Cultivating that insight (from the domain, from the system) requires some scientist-like practices. But (for me, anyway), it’s driven by wanting to make, rather than by raw curiosity about the world.
To quote from Michael Nielsen’s “Principles of effective research”, an effective vision offers “{clarity} about {what one wants to achieve}, {why one wants to achieve it}, and {how to go about achieving it}.”
Deepening over time: present-day memory systems let me save a lossy snapshot of my understanding at the time I wrote some prompts about a particular topic. It’s about preservation, maintenance. But a more powerful frame is connection and depth, rather than mere preservation, as Michael Nielsen has pointed out (e.g. in Building a better memory system - Michael Nielsen). It would be much more interesting and much more valuable to use that review session time to think new thoughts about the material, to push it further, to notice new connections, to reconnect with it emotionally.
For example, Michael Nielsen and I started working on the Mnemonic medium on these terms. He was working full time on writing the text of QCVC, and I was just thinking about it on a couple evenings per week. We made a ton of creative progress. Though this is not an ideal example: I was in the critical path for actual execution work.
A better example, perhaps, can be found in Michael Nielsen’s collaborations with Patrick Collison and Kanjun Qiu, on their two co-authored essays. In these, Michael drove the project and the writing, while his collaborators offered regular deep high-context conversation. They also did independent research and development on subtopics of interest to bring to those conversations. But they mostly avoided the critical path, and they could scale up or down their involvement somewhat flexibly.
Another angle here is to reflect on Michael Nielsen’s years spent trying to work on a cognitive medium: depending on how you count projects like Deep Learning and Neural Nets, he started in roughly 2013, or perhaps 2015. He’d publish something every year or two, and the intervening period was filled largely with reading and prototypes, waiting for the next important-enough project idea to strike. I’m not sure he’d regard those years as terribly valuable or productive, but in that time he did become one of the most important intellectual contributors in that field, and he did create some important contributions (most notably Quantum Country, but also notably the AI piece with Shan Carter and Thought as a Technology).
Of course, the best colleagues and collaborators actively avoid this trap! One of my favorite Michael Nielsen behaviors is that if he hears me talking about some idea that seems fairly banal, he’ll deliberately tug at the places where I’m straining to reach past typical interpretations.
There are lots of novel research and arts funding programs which offer fellowships and grants. The trouble is that there isn’t an off-ramp from these programs into a subsequent viable structure of funding institutions. If the funding period is relatively short—like two years—the grant recipient may not really lean into the desired new way of thinking because they know, in the back of their mind, that they have to keep their future options open with older funding models. Michael Nielsen calls these concerns “shadows of the future.”
Michael Nielsen proposes a practice of “deliberate interrogation”: when you find someone who behaves or achieves things which totally break your models, seem unimaginable to you, you can learn much by interrogating them quite precisely:
2023-06-27: Michael Nielsen told me that he thinks my writing has gotten to a place where it seems like a real shame to keep it private: I’m thinking about this problem from a space that no one else in the world is, and “almost every sentence is extremely interesting.”
Q. Jonathan Blow’s comments about people not actually wanting to be successful suggest what reflection question about a potential action? (via Michael Nielsen)
I had an extended conversation with Michael Nielsen today about 2022-12-27 Patreon letter - Towards impact through intimacy in my memory system research. It was a bracing discussion—extremely generous and thoughtful… but ultimately critical and challenging. He bookended the discussion by pointing out that in fact the best thing for me to do may be to completely ignore what he’s saying. Integrated progress in some direction can be quite valuable, and he’s skeptical of “outsiders’” ability to criticize effectively.
“I wish GPT4 had never happened” - a friend in India has a job as an annotator of biotech papers; author was able to reproduce their work in a fraction of the time with GPT-4; fears they will lose their job. (viaMichael Nielsen
This week was tough for me. No clear next step for my projects. Project - collaboration with Delta Academy seems like it’s clearly not going to work. Conversation with Michael Nielsen reminded me of Focus on power over scale for transformative system design. I’ve been trying for a couple years now to find a good context to do that, but I’ve not yet succeeded in setting one up. I feel I’ve let myself flinch away from this crucial observation—finding other things to do in absence of being able to do what I really need to do. It sounds a little melodramatic, but I guess I feel like I’m failing.
More helpful, but emotionally challenging, shoves this week from Michael Nielsen. I find myself strongly agreeing with everything he’s said in Building a better memory system - Michael Nielsen and in our conversation together. But several parts also feel quite threatened by his suggestions and by my reactions. To enumerate:
In conversation with Michael Nielsen on Sunday, he mentioned the value of finding “difficult” ways to engage with certain social situations. For instance, instead of going to a Neighborhood meetup, spending hours in conversation with Jason Benn about his goals and influences. That resonated. When I attend events like the essay-writing event, I want to attend with some sense in mind of what I’d like to get out of it.
I really can’t tell if I’m fixating far too much on this problem our not. I suspect that Michael Nielsen would say that getting this right is basically the only thing that really matters. But I can also imagine him saying: well, can you just iterate on Quantum Country? Or just use “what people seem to want, as indicated by their behavior” as a proxy? There’s a pragmatic lens one could apply here, I suppose. And if I keep flailing in the way I’ve been flailing, I probably should figure out how to apply that lens.
It was lovely to see Michael Nielsen this past Sunday—it’s been more than two months. One particular nudge he left me with (incidentally, I suppose): how might I use my intermediate outputs to more effectively create long-tail community for myself? Should I move to publishing this stuff publicly (e.g. with a several month delay)? Should I create a mailing list for other memory augmentation system designers? I think I should probably make those letters public.
Great conversations about my plans with Rob Ochshorn and Michael Nielsen, plus some good comments from Joe Edelman. Lots more to reflect on as a result… plans aren’t ready for action.
The month was also quite difficult emotionally, with respect to my work. I opened the month still somewhat reeling from a conversation with Michael Nielsen in which he uprooted many of premises, and closed the month with another which did the same, even more deeply (though perhaps more tentatively)—see 2022-12-26. The latter was particularly disappointing in part because I’d spent the preceding several weeks thinking through 2022-12-27 Patreon letter - Towards impact through intimacy in my memory system research, and I was moderately pleased with where I ended up. But it wasn’t enough… Michael and Rob and Joe (but particularly Michael) are really the only ones in my life who consistently see far past where I’m seeing, push me to think further. I’m grateful for it, but it’s still tough in the moment. I’d like it to become less tough in the moment—a reflection on Ling Lam‘s push in our session earlier this month to become more “centered”. It hurts mostly because I’m worried what others are thinking about me.
Bujold (viaMichael Nielsen): “All great human deeds {both consume and transform their doers}. … A false destiny may be spotted by the fact that it {consumes without transforming, without giving back the enlarged self}.”
A lovely lunch supporting Michael Nielsen today. James Scholz had a mental health crisis; tried to support him but felt fairly unable to provide what was needed.
Michael Nielsen suggests that rituals play an important role in the Jewish sabbath: gathering the whole family together, lighting a candle, saying a prayer.
I’d like to start the morning with some journaling. My interactions with Michael Nielsen the past few days have been bracing, and I’m feeling somewhat down on myself and my projects.
What’s holding me back? I think it’s Michael Nielsen’s voice in my head, e.g. via Building a better memory system - Michael Nielsen. He’d say that automating prompt creation isn’t terribly exciting. In Quantum Country, we “automated” prompt creation by way of having experts author them. That’s even better than having an LLM do it (modulo possible interface distinctions, discussed in 2022-10-14 Patreon letter - Lessons from summer 2022’s mnemonic medium prototype). And… how well did it work? How well do those people understand QC? What’s most important to help them learn more deeply? Is what’s needed here to simply automate what we did with QC, but perhaps with more flexibility (The mnemonic medium should give readers control over the prompts they collect)? Michael would say no: what’s needed is to “increase the depth of understanding being encoded”, to “raise the emotional stakes”, to “strongly connect it to meaningful projects”, etc.