Over the weekend, I've once again found myself going down a rabbit hole of dynamic language compiler design. I was reminded of two approaches that grabbed my interest from a pedagogical perspective, because it simplifies and makes approachable the typically revered task of compiler development for a high level language.
First, A nanopass framework for commercial compiler development by Keep and Dyvbig presents a way to incrementally develop a compiler by dividing the task of compilation into very many small "nanopasses", each of which lowers one high level abstraction into a lower one.
Second, An Incremental Approach to Compiler Construction by Ghuloum slices up the task of compiler development in the other direction, by starting with a minimal compiler that lowers numeric constants to assembly, then in a couple dozen incremental steps adds small abstractions like functions and stacks, closures, and GC on top.
Both of these works have implementations in Scheme, the first in nanopass-framework-scheme and the second in namin/inc. I'm hoping to write a compiler for Klisp, which is a Scheme in many ways, by following the latter paper soon. Should be fun.
Finally got around to reading Chris Voss's Never Split the Difference to the end this weekend, after having started it early this year and then forgotten about it. It's a very tactical, pedagogical book, so it'll appeal to those who resonate with that style of learning. It reminds me a lot of Never Eat Alone, in all the good ways. It teaches you to think about the aspects of human relationships and communication that are normally in the shadows, and chief among them is the main conceit of the book: effective communication is mostly thoughtful, curious listening.
I've been thinking more recently about the see-through-ness of tools. Tools are see-through when the abstractions they provide don't unnecessarily cover up what's going on under the hood, i.e. when it's not too "magical" to be understood.
Often tools and abstractions (software libraries, algorithms) seem magical and easy to work with at first glance, but fall apart under its own complexity when you need to dig deeper into its inner workings to fix a bug or customize it to your specific usage. I think there are ways to design tools so that they're less susceptible to this kind of failure mode, though, where it's possible to peer beneath the abstraction layers provided by the tool without getting into a tangly mess you don't understand. I want to build more tools that empower users to peer inside and understand and modify, and fewer tools that operate only within the safety of its own walled garden.
At this point, I've been using GitHub Copilot at my day job long enough for me to feel used to it under my fingers, and have a rough sense of what kind of code Copilot is good at generating correctly. While writing some Oak code the other day in Vim (my editor for anything non-TypeScript/React), I even found myself missing Copilot's little grey type-ahead hints to stamp out some repetitive bit of code it surely would have written for me.
My current feeling on Copilot is that it doesn't really compete with me as a developer, but does make my job slightly easier, reducing a thousand very small paper cuts in a way that makes a difference in my work. At the best of times, it feels like pair-programming. At worst, I spend a few extra seconds code-reviewing and rejecting a suggestion. Contrary to what Hacker news seems to suggest, I've found the overall UX pretty solid and rarely distracting. Is it indispensable? No. But would I use it if given the choice? Sure.
My first update to the stream :)
Building this in pure Oak was not trivial — a good bit of yak-shaving writing a date/time library, polishing up various existing standard library functions for better performance and fit, fixing a couple of errors in the interpreter itself and the oak fmt code formatter... and some good old performance work. But I'm excited to start using it to share some real updates!