Scrappy prototypes are great: they allow scrappy iteration and quick evaluation. But many critical insights will only emerge in the context of a serious creative problem that’s not about the system itself. This is a key claim of Insight through making.
That sounds like standard practice: of course systems have to be evaluated! But most system designers don’t take “serious” seriously: Tool-makers usually lack connection to a serious context of use.
Observing how your theories (represented in systems) interact with reality can yield insights which help improve your theories. The character of those insights will depend on the context in which the system was used. If the system isn’t used seriously, the insights will be more like those which a pure theorist could have seen. Those were possible without actually building a system.
Pixar’s a good example of an organization which creates serious contexts of use, which in turn drive system design: Pixar’s movies and technology development act as coupled flywheels.
Matuschak, A., & Nielsen, M. (2019). How can we develop transformative tools for thought? Retrieved December 2, 2019, from https://numinous.productions/ttft
Concretely: suppose you want to build tools for subject X (say X = differential geometry). Unless you are deeply involved in practicing that subject, it’s going to be extremely difficult to build good tools. It’ll be much like trying to build new tools for carpentry without actually doing any carpentry yourself. This is perhaps part of why tools like Mathematica work quite well – the principal designer, Stephen Wolfram, has genuine research interests in mathematics and physics. Of course, not all parts of Mathematica work equally well; some parts feel like toys, and it seems likely those are the ones not being used seriously internal to the company.