Insiders Weekly #6

On emergent abilities, and why GPT-4 still can't reason.

Hey folks!

Welcome to another issue of Insiders Weekly, the subscriber-only newsletter where I share my views on recent events in the world of artificial intelligence and computer science.

In this issue, as usual, I’ll tell you what’s in the oven and the near-term plans. Then I’ll continue on the topic we discussed last week regarding the limitations of current machine learning, especially large language models. In this issue, I will discuss the idea of “emergent abilities” and give my two cents on why it happens and what it means. Then I’ll give a couple more formal reasons why any language model in the current generation can’t really reason and what it would take to get us there.

Let’s get onto it.