Unlocking progress
It’s always been true that the bottleneck to human progress was not the pace of computation on the planet, but the fidelity of interface between humans and tools. The fastest jet will only take you to the wrong destination quicker if the pilot cannot steer precisely.
This is only going to become truer as computation and knowledge available to each human accelerates upwards.
Interface is an under-invested common good for which innovation is hard to create, even more challenging to defend in the market, and progress is essential to our future.
New representations simplify complex knowledge tasks
Better numeric notation made complex arithmetic trivial. Better data display makes complex statistical reasoning trivial. Better diagrams make reasoning about complex particle physics tractable.
I think what we'll find in retrospect when we look back at the success of language models from the future is that neural language models work so well because they learn a new way to represent knowledge in high dimensional latent spaces, and it's not doing anything particularly remarkable algorithmically, but the representations neural models use internally to work with knowledge is far superior to text/token sequences that humans use, and that's where it derives its magic — complex thought becomes simple in the right representation.
I also have a speculative sense that as we understand these models better, we are going to continue to be surprised by how much of the model's computational capacity is allocated to simply translating semantics between the human (token) representations and its own, continuous latent representations of meaning.
More in Notational intelligence (2023).
The depth of a romance is directly proportional to the fraction of Taylor Swift's discography lived through.
"Authoring" vs "Assembling"
In the case of generative AI, arguing that these systems "assemble" more than "author" makes for provocative bit, but as is usually the case with these things, I think these are two ends of a spectrum. authoring is perhaps just assembly with smaller more fundamental atoms/assembly in a different dimension.
This also goes back to composition, a core aspect of any language or semantics that I've been thinking a lot about. Tokens (words; atomic units of text) and features (atomic units of meaning; latent dimensions) compose ("assemble") differently, but I think we can think of both of them as assembly.
An excerpt from an internal doc on AI:
My belief is that, once the dust settles, humans are still working — still creating, still wanting, still debating. But that our time and energy will take us farther with the new tools and collaborators we'll have created along the way. We hope these tools will help our best work become even better, while letting us delegate away the minutiae of the meta-work that takes us away from it.
I've spent a lot of time over the years desperately trying to think of a "thing" to change the world. I now know why the search was fruitless — things don't change the world. People change the world by using things. The focus must be on the "using", not the "thing". Now that I'm looking through the right end of the binoculars, I can see a lot more clearly, and there are projects and possibilities that genuinely interest me deeply.
— Bret Victor
This is the humanist view of AI that I think should fuel our work. This is what we should try to see, where we should aim, when we build with AI, as extensions of our human agency rather than amplifiers of the mechanics of our labor.
On this, I loved what Maya added:
...the AI should gesture, not instruct, us. "Gesturing" preserves our agency, "instructing" infringes upon it.
I want to build interfaces that lets the AI gesture us into a better future without infringing upon our agency.
I feel like the concept of a "device" is constraining. There are no devices in the physical world. Only materials and objects. Devices intermediate us and our things to our detriment.
Three phases of creative production
(Excerpt from some old research notes.)
- Gathering raw materials
- Through reading and consuming media, listening to others, thinking aloud in conversations, or just pure contemplation, the user gathers a bundle of ideas and related experiences that start to point to some insight or further idea.
- A lot of creative value comes from this very unstructured stage of creation where ideas are gathered without any goal directed work. This I call "simmering".
- Active thinking
- In the active thinking phase, the user looks at the gathered raw materials and tries to form structure out of them to produce something that's coherent and clear and perhaps even actionable.
- This implies and requires a "sense-making" process, of going from unstructured information to emerging a structure out of it.
- In the process of active thinking, even more raw materials may be gathered, perhaps in a goal-directed way. Information may also be discarded or deemed no longer relevant.
- Production
- For the user to create some creative work/output from ideas or insights gathered during active thinking, it must be crystallized into a form that can facilitate communication. This I call "drafting" or "production" and is a writing/communication process more than a thinking process. It is about serializing thought into media.
- Production may often also serve the role of communicating with oneself, i.e. when taking notes for my future self, to remember my past thoughts.