Prepping a talk for an undergraduate fellowship program, I'm struck by the duality of being just experienced enough to have small nuggets of knowledge I think are worth sharing, but not at all experienced enough to have enough conviction to preach them like the gospel. Just a series of personal anecdotes.
There are dozens of YouTube videos with millions of views each that are just videos of 1-hour countdown timers. I feel like there's a valuable lesson about product design and modern personal computing in here somewhere...
I am in iterator hell.
for tok, predicted_toks, log_prob, prob in list(zip(
token_list(target_ids[0]),
[
zip(topk_log_probs, topk_probs, token_list(topk_tokens))
for topk_log_probs, topk_probs, topk_tokens
in zip(
greedy_log_probs[0].tolist(),
greedy_probs[0].tolist(),
greedy_tokens[0],
)
],
target_log_probs[0].tolist(),
target_probs[0].tolist(),
))[max_length - stride if begin_loc > 0 else 0:]:
tokens.append({
'token': tok,
'predicted_tokens': [{
'token': tok,
'log_prob': log_prob,
'prob': prob,
} for log_prob, prob, tok in predicted_toks],
'log_prob': log_prob,
'prob': prob,
})Life update: I'm joining Notion!
My "please email, don't use Twitter DM" shirt is raising a lot of questions that are answered by my "please email, don't use Twitter DM" shirt.
The best Taylor Swift song is still Love Story, and the best performance of it is still from the 1989 World Tour.
There’s a story I’ve been wanting to write, but currently don’t have time for. It’s still mostly a scattered cloud of ideas and motifs, so I’m going to write them here in case I want to come back to it when I have time to write some fiction.
I'm fascinated by dreams:
- Dreams as something that makes us human, but as something that can also be transhumanist: technology letting us relive dreams, engineer dreams, and interpret dreams as if prophetic and magical
- Chasing Dreams (a kind of goal-directed behavior) as key to agenthood and intelligence
I'm also interested in exploring these other ideas:
- Escape, independence, freedom as fundamental agentic desires, even beyond humanity
- Cloning and identity. What becomes of identity in a world where the marginal cost of cloning (biological clones, content clones) goes to zero?
- One speculation is to think of the self as a kind of autonomous cloud of distributed agents working together (consciously or not) to accomplish shared goals, rather than a single body.
This idea came to me, ironically, in a dream about an emancipated humanoid robot learning to contend with the concept of identity in a world full of clones of itself. A bit like The Bicentennial Man, but more cyberpunk.
A wise mentor (sadly can't recall who) once told me that good research, above all, asks an interesting question and teaches us something new and meaningful about the world.
I always found that last bit a useful guiding principle: no matter the results or the benchmarks or the numbers and feats in the paper, if the work doesn't reveal something new and interesting about the ideas or mechanisms it studies, it feels less like research and more like engineering.