There are no original ideas; only infinitely varied remixes.

This itself isn't a new idea. All thinking is creative recombination. But language model-based creative tools of today all require us to start each idea from scratch, at an empty text box.

When I think within my own mind, I build upon snippets of conversations and bits of quotes from stories I've heard, and recombine them to produce something new. The ingredients with which I think are not words or tokens, but pieces of ideas, abstract blobs of concepts and quotes from my memory. It seems obvious that we should be able to work with language models similarly, at least for creative use cases, by recombining pieces of our experience rather than typing into a mostly empty rectangle.

Instead of instructing with words or prompting with keywords, I want to bring in a passage from my favorite author and say "what does this make you think of?" I want to smash two different paragraphs about creativity from my notes together inside a neural network and see what ideas fall out. I want to paint over a model-generated image with a brush I conjured from the color palette of my favorite photograph. I want to control these models with ideas and experiences plucked from my memory, not tokens and words.