Biomolecular design in Realtalk - Bret Victor

Bret Victor gave this talk at 2022-07 Foresight Workshop - Designing Molecular Machines on his latest evolution of Dynamicland. His system is now embedded within Shawn Douglas’s lab. His portion of the talk follows a talk from Shawn about a DNA origami project he’s pursuing; his demo shows how that project might be carried out in Realtalk.

In asides at Foresight Institute’s November 2022 Vision Weekend, Shawn noted that the lab’s day-to-day is still carried out in a traditional fashion. This demo system isn’t really how they’re working in practice, as of that time. But it’s becoming more real. As Bret says, “These are just prototypes… Their purpose is to explore ideas for what Shawn’s lab could look like in a couple years.”

Cleaner version of the video: YouTube

Some questions

• How does/should statefulness work in Realtalk?
  • e.g. for the PDB search card, Shawn types onto it, and then moves the keyboard away. Presumably, to reset it, you flip the paper over. Or… maybe the text must be deleted? At 16:20, Bret appears to backspace several times.
• Moreover: how does/should stateful events work?
  • e.g. the mock test tube “picks up” a chemical when it’s close to something. But presumably it doesn’t reset when it’s removed from view, because Bret proposes later that the test tube could persist information about its contents.
• How broad is the role of simulation? Is this mostly useful for a science lab? How might it be useful for more everyday circumstance?
• How does the vision of malleability here differ from Webstrates, Engraft?
• How should we think about AR?

New things since 2020-era Dynamicland

• Use of a laser pointer as a cursor of sorts
  • to spin the 3D molecule on the table
  • and to scrub through the DNA sequence—surprisingly high resolution!
  • 20:59: scrubbing through small multiples
  • feels like a return to his 2011-era explorables work; likewise the 22:15 protocol preview
  • 22:06: lasering names of reagents to see information
• Using a program’s whisker as a high-resolution pointer
  • 16:47: he points the “permute program”’s whisker at a spot on the GFP’s linker
  • 17:17: splice program points to a spot on BurrH where the GFP should be spliced in
• “Mock test tubes”, “mock gel rack”, “mock electron microscope”, etc—connects critically to the simulation key idea

Key ideas

• physicality
  • using macro-scale, embodied sense-making: positioning the protein structures using hands, on the table
  • swapping out scale bar, 18:30
  • 17:47: Shawn using Post-Its to draw on top of the projections
  • 22:10: the system knows where reagents are physically located
  • 22:20: “at each step, a projected line shows exactly what to pipette where”
  • “molecules in test tubes are physical things; we’d like our computation out in the physical world, where the science is happening”
• simulation (“computational modeling”)
  • show the consequence of contemplated biological procedures, in realtime, in various linked representations, to allow rapid and natural iteration
  • positioning and splicing proteins (with annotated sequences)
  • simulated gels for synthesis
  • routing structures for DNA origami blocks
  • 20:03: mock racks, mock plates for stocks, showing flows from one into the next “an auto-designed folding protocol”
  • liveness!
  • 20:55: simulated ESM micrographs, showing realtime expected changes in response to concentration changes “the entire analysis is updated live”
  • “because it’s simulated data, we already know the right answer, so we can evaluate how well our algorithms are doing, and get everything debugged in simulation before going through real runs”
• sociality (“togetherness”)
  • Shawn and Bret, working together at the table, reaching over each other, using physical materials
  • “what if this was what a group meeting was like?”
• malleability (“pervasive programmability”)
• transparency
  • in the Vision Weekend version of this talk, at 21:18, Bret noted: “these programs to pick and classify particles would usually be some incredibly messy Matlab script on some grad student’s computer, and no one else would have it or be able to run it again”
• computability
  • 22:48: pulling a tube out of freezer years later, should be able to see the protocol which generated it, “the data we collected, the paper we later published… all that could be attached to the physical test tube”