As an extension to my note on media-native programming languages, I wanted to note: Oak's standard library contains built-in support for not only parsing and rendering JSON, but also Markdown content via the json and md libraries. Markdown support is built into the language tooling, and it means when I write Oak programs, I feel like Oak "speaks Markdown" natively as an information format. It means there's no additional development effort for Oak programs to really support Markdown -- supporting it is about as easy as supporting plain text, at least in rendering, processing, and parsing. As a result, most tools I'm building with Oak where Markdown support makes sense, like this stream site, support Markdown.
I think this demonstrates one successful case study of a language that built in support for unconventional data formats, and benefited in the kinds of software and tools built from it.
QR codes for software distribution
I love how lightweight and error-resistant QR codes are. You can stick them anywhere and display them nearly everywhere. What if we could pack small pieces of contextually relevant software, like to-do apps, chat clients, little maps, and restaurant reservation systems into big QR codes to distribute software? Instead of giving you a download link, I just give you a QR code to scan, and your phone loads the program directly from the code and starts executing.
I want to imagine a future where software isn't necessarily "installed" and "uninstalled", but ephemeral like web apps and situationally delivered to our devices. Little micro pieces of functionality, quickly loaded and unloaded, scooped up by our devices from little tags we can touch in the physical world.
It would be so cool to be able to share a game I made with a friend, by giving them a QR code they can scan whenever they want to play it. No install, no download. Just load it up from the code!
Everything new is also old -- this reminds me of loading up software from floppy disks! But I think it can be different. QR codes can be so much more versatile and ubiquitous, so much more error-resistant, infinitely cheaper, and if delivered on screens, forever changeable. It's also decentralized, in a way. If I give you a QR code, you don't need to rely on some app store being online to use that new program.
One way I can imagine this working is some lightweight "base" app or virtual machine installed on your phone, that can load and execute very compact bytecode from a QR code.
Media-native programming languages
Modern programming languages are very good at handling strings. Not only do they have built-in representations of strings in the common string "type", they have built-in support (into the language or as standard libraries) for searching within strings, comparing them, slicing them, combining them, and various other useful operations. As a result, most software we use today all expect us to enter text data. They speak the language of "text".
By contrast, modern tools handle images and audio only reluctantly. Images and audio are the native I/O types of the human mind, if you will -- it's much higher-bandwidth, and much more closer to "the organs" even if they're farther from "the metal" of the computer.
What if we could build into programming languages the same capabilities for working with rich media, as we've done for strings? What if OCR and speech to text, seeking and searching for objects or strings within video, photos, and audio, were all as easy as photo.findAll(:car) or audio.transcribe({ lang: 'en_us' }), built into your compiler? I think it would usher in a whole new age of software tools that let us interact with them in richer, more organic ways. If reading text from an image was as easy as reading text out of a binary buffer, how many more tools would let us take pictures to capture information?
You might say, "this sounds like a huge amount of complexity, Linus! No sane PL would ever do this!" But we've done this for text, because the tradeoffs are worth it -- Go ships out of the box with rich support for full UTF-8 text. This wasn't always the case. C, for example, has no native string type -- C works with bytes and characters, in the same way that current programming languages work with pixels and audio file buffers.
I submit to you: it doesn't have to be this way! We can create a world where we can program with rich visual and sonic information with the same ease with which we work with text. That day can't come quick enough.