Linus's stream

Invite some chaos into your life. Makes it more likely to be worthwhile.

"Authoring" vs "Assembling"

In the case of generative AI, arguing that these systems "assemble" more than "author" makes for provocative bit, but as is usually the case with these things, I think these are two ends of a spectrum. authoring is perhaps just assembly with smaller more fundamental atoms/assembly in a different dimension.

This also goes back to composition, a core aspect of any language or semantics that I've been thinking a lot about. Tokens (words; atomic units of text) and features (atomic units of meaning; latent dimensions) compose ("assemble") differently, but I think we can think of both of them as assembly.

An excerpt from an internal doc on AI:

My belief is that, once the dust settles, humans are still working — still creating, still wanting, still debating. But that our time and energy will take us farther with the new tools and collaborators we'll have created along the way. We hope these tools will help our best work become even better, while letting us delegate away the minutiae of the meta-work that takes us away from it.

I've spent a lot of time over the years desperately trying to think of a "thing" to change the world. I now know why the search was fruitless — things don't change the world. People change the world by using things. The focus must be on the "using", not the "thing". Now that I'm looking through the right end of the binoculars, I can see a lot more clearly, and there are projects and possibilities that genuinely interest me deeply.

— Bret Victor

This is the humanist view of AI that I think should fuel our work. This is what we should try to see, where we should aim, when we build with AI, as extensions of our human agency rather than amplifiers of the mechanics of our labor.

On this, I loved what Maya added:

...the AI should gesture, not instruct, us. "Gesturing" preserves our agency, "instructing" infringes upon it.

I want to build interfaces that lets the AI gesture us into a better future without infringing upon our agency.

A map is a specific view of a world that lets the user orient and navigate

— Dani B.

I feel like the concept of a "device" is constraining. There are no devices in the physical world. Only materials and objects. Devices intermediate us and our things to our detriment.

Three phases of creative production

(Excerpt from some old research notes.)

  1. Gathering raw materials
    • Through reading and consuming media, listening to others, thinking aloud in conversations, or just pure contemplation, the user gathers a bundle of ideas and related experiences that start to point to some insight or further idea.
    • A lot of creative value comes from this very unstructured stage of creation where ideas are gathered without any goal directed work. This I call "simmering".
  2. Active thinking
    • In the active thinking phase, the user looks at the gathered raw materials and tries to form structure out of them to produce something that's coherent and clear and perhaps even actionable.
    • This implies and requires a "sense-making" process, of going from unstructured information to emerging a structure out of it.
    • In the process of active thinking, even more raw materials may be gathered, perhaps in a goal-directed way. Information may also be discarded or deemed no longer relevant.
  3. Production
    • For the user to create some creative work/output from ideas or insights gathered during active thinking, it must be crystallized into a form that can facilitate communication. This I call "drafting" or "production" and is a writing/communication process more than a thinking process. It is about serializing thought into media.
    • Production may often also serve the role of communicating with oneself, i.e. when taking notes for my future self, to remember my past thoughts.

humans are but agents of writing.

writing crystallizes too early and corrupts too often but we trade off agency and flexibility for the clarity of thought we get when we channel it onto the page.

Remember to imagine and craft the worlds you cannot live without, just as you dismantle the ones you cannot live within.

— Ruha Benjamin

Perhaps every science must start with metaphor and end with algebra; and perhaps without the metaphor there would be no algebra.

Max Black, via Ivan

Even when we extrapolate arbitrarily far into the future of very large, very capable ML models, as long as they are built on the current paradigm (Internet-scale self-supervision followed by fine-tuning for desired behavior), I think being Extremely Fucking Good at anything will still remain valuable, regardless of field.

At the upper echelon of any craft you're a tastemaker more than a skilled hand, and taste is like entropy — there is only so much you can distill from a limited dataset into a model before it starts losing fidelity.

Human preference data will always be a scarce resource.