Updates on 2023/5/18

Perhaps every science must start with metaphor and end with algebra; and perhaps without the metaphor there would be no algebra.

Max Black, via Ivan

Even when we extrapolate arbitrarily far into the future of very large, very capable ML models, as long as they are built on the current paradigm (Internet-scale self-supervision followed by fine-tuning for desired behavior), I think being Extremely Fucking Good at anything will still remain valuable, regardless of field.

At the upper echelon of any craft you're a tastemaker more than a skilled hand, and taste is like entropy — there is only so much you can distill from a limited dataset into a model before it starts losing fidelity.

Human preference data will always be a scarce resource.