Posit: Higher intelligence is being able to learn an arbitrary class of patterns/functions in-context, what a less intelligent system needs to be directly optimized ("trained") to learn.

For example:

• Larger transformers can in-context learn things smaller models need to be trained to know
• Extrapolating, super-human general intelligence will be able to learn in-context things that take humans lots of iterations to learn.
  • The set of things that take humans many iterations to learn includes the ability to autoregressively predict (imitate) another human. Thus, a sufficiently intelligent AGI will be able to mimic any human with arbitrary accuracy by learning from "in-context" observations.

In-context learn "arbitrary class of functions" here seems pretty profound. It doesn't just include language, but also models of environments and policies (Gato), arbitrary linear functions, etc.