Many bloggers and “life-hackers” have made a full-time job of suggesting how you should organize your journal, or how you should most effectively Write about what you read to internalize texts deeply. We should take this advice seriously insofar as those practices have helped the authors achieve meaningful creative work: “Better note-taking” misses the point; what matters is “better thinking”
But most people who write about note-taking don’t seem particularly accomplished in their own fields, whatever those may be. In fact, most such writers aren’t applying their notes to some exogenous creative problem: their primary creative work is writing about productivity. These writers offer advice on note-taking to help scientists and executives with the challenges of their work, but the advice was developed in a context disconnected from those external realities. There are two related problems here: Effective system design requires insights drawn from serious contexts of use, and Powerful enabling environments usually arise as a byproduct of projects pursuing their own intrinsically meaningful purposes.
Luhmann, by contrast, barely wrote about his Zettelkasten: he focused on his prolific research output, then published a couple small essays about his practices near the end of his career.
I’m not quite guilty of this problem myself, but I certainly slip into this behavior for weeks at a time. This is a cautionary note. Related: The most effective readers and thinkers I know don’t take notes when reading.
Scrappy prototypes are great: they allow scrappy iteration and quick evaluation. But many critical insights will only emerge in the context of a serious creative problem that’s not about the system itself. This is a key claim of Insight through making.
That sounds like standard practice: of course systems have to be evaluated! But most system designers don’t take “serious” seriously: Tool-makers usually lack connection to a serious context of use.
Observing how your theories (represented in systems) interact with reality can yield insights which help improve your theories. The character of those insights will depend on the context in which the system was used. If the system isn’t used seriously, the insights will be more like those which a pure theorist could have seen. Those were possible without actually building a system.
Pixar’s a good example of an organization which creates serious contexts of use, which in turn drive system design: Pixar’s movies and technology development act as coupled flywheels.
Common challenges:
Related theory:
Matuschak, A., & Nielsen, M. (2019). How can we develop transformative tools for thought? Retrieved December 2, 2019, from https://numinous.productions/ttft
Concretely: suppose you want to build tools for subject X (say X = differential geometry). Unless you are deeply involved in practicing that subject, it’s going to be extremely difficult to build good tools. It’ll be much like trying to build new tools for carpentry without actually doing any carpentry yourself. This is perhaps part of why tools like Mathematica work quite well – the principal designer, Stephen Wolfram, has genuine research interests in mathematics and physics. Of course, not all parts of Mathematica work equally well; some parts feel like toys, and it seems likely those are the ones not being used seriously internal to the company.
Brooks, F. P., Jr. (1994). The Computer Scientist as Toolsmith II ACM Allen Newell Award Lecture. SIGGRAPH.
It aims us at relevant problems, not just exercises or toy-scale problems.
It keeps us honest about success and failure, so that we don’t fool ourselves so easily.
It makes us face the wholeproblem, not just the easy or mathematical parts. In computational geometry, for example, we can’t avoid the cases of collinear point triples or coplanar point quadru- ples. We can’t assume away ill-conditioned cases.
Facing the whole problem in turn forces us to learn or develop new computer science, often in areas we otherwise never would have addressed.
Besides all of that, it is just plain fun to look over the shoulders of those discovering how proteins work, or designing submarines, or fabricating on the nanometer scale.
Software-based researchers often strive to build systems containing high-level ideas that are likely to generalize, since those make for more compelling academic papers. However, we believe that trying to be too general actually hinders scale and sustainability. To build long-lasting software that can organically grow a large userbase, one must instead start specific.
In 2009 we created Python Tutor with a very specific goal in mind: to provide a convenient way for students and instructors (such as ourselves) to walk through Python code step-by-step and see the values of variables.
People making Tools for thought often say they’re trying to help people do math, or make art, or whatever. But in reality, the people making these tools are rarely connected very deeply with the actual creative practices they’re trying to amplify. The work is often more of a tech demo or a toy or a “sandbox.” As we wrote in How can we develop transformative tools for thought?: “Tools for writing that aren’t used by actual writers. Tools for mathematics that aren’t used by actual mathematicians.” Deep down, such system designers are generally developing a system for its own sake—not because there’s some creative problem they’re desperately trying to solve.
Such tools might look mathematical (or whatever) on their surface, but they’re not seriously trying to answer hard problems in those domains—often because the creators don’t actually know what those problems are or understand how to solve them. Effective system design requires insights drawn from serious contexts of use
Startups and technologists often fool themselves by working with and talking to early adopters who are mostly interested in playing with new technologies. But it’s not just a “tech industry” problem: academic tool-makers generally evaluate their systems in artificial settings with artificial data, against artificial validation criteria.
It’s hard to resolve this issue: Great tool-makers are often not great tool-users, and vice-versa
Matuschak, A., & Nielsen, M. (2019, October 0). How can we develop transformative tools for thought? https://numinous.productions/ttft
Interface matters to me more than anything else, and it always has. I just never realized that. I’ve spent a lot of time over the years desperately trying to think of a “thing” to change the world. I now know why the search was fruitless — things don’t change the world. People change the world by using things. The focus must be on the “using”, not the “thing”. Now that I’m looking through the right end of the binoculars, I can see a lot more clearly, and there are projects and possibilities that genuinely interest me deeply.