Every post on the blog this month is on the theme of agent reliability, anchored on the second edition of Mostly Harmless AI — 50% off during early access — where the history below is the full ~8,000-word opening chapter, with 70+ references and all the scenes this post had to cut. You can also read the whole book online for free. More at the end.

Seventy years ago, two men sat in two different rooms and disagreed about what a thinking machine should look like. Neither of them has been proven right. Both have been proven half-right, several times, in alternation, for the whole of my lifetime and most of yours.

I think the entire history of AI is that one argument, still going.

The first camp wanted to build minds out of rules. Feed the machine enough knowledge in a logical enough form, and reasoning falls out of the logic. It called itself a lot of things over the decades — symbolic AI, knowledge-based systems, good old-fashioned AI — but its home is rationalism. The second camp wanted to build minds out of examples. Feed the machine enough data, in any messy form whatsoever, and behavior falls out of the statistics. It also kept renaming itself — connectionism, machine learning, deep learning — but its home is empiricism. Same goal, a machine that does what intelligent people do. Sixty years of disagreement about how.

Here’s the ending, spoiled early, because this isn’t a thriller. The argument did not produce a winner. It produced a marriage. The chatbots, the image generators, the agents writing code while you sleep — none of them is one side beating the other. They’re both sides, finally forced to share a workshop. Let me walk you through how we got there. Fast.

Subscribe now

Both seeds, one summer

They were planted within five years of each other. In 1943, McCulloch and Pitts wrote down a neuron as a weighted sum with a threshold — twelve pages, the seed of the empiricist branch. In 1950, Turing refused to define thinking and proposed a behavioral test instead, a question both camps could chase. In 1956, ten people spent a summer at Dartmouth, coined the phrase artificial intelligence, and planned to crack language and reasoning in a few months. (We are still working on it.) In 1957, Rosenblatt built the Perceptron, the first machine that learned from examples, and the New York Times announced it would soon walk, talk, and be conscious of its own existence.

Two foundational myths, in the ground, in the same decade. The rest is which one got watered.

The rationalists win the first round

And they win it convincingly. In the 1950s and 60s compute is tiny and data, in the sense of millions of labeled examples, does not exist.

What you can do is write a program that does something specific and inspect every step of it. So the symbolic camp gets the better results and the better tools. Newell and Simon’s theorem-prover. McCarthy’s LISP. Weizenbaum’s ELIZA — four pages of pattern-matching that understood nothing, and that people confided in anyway. (Hold onto ELIZA. The field will relearn that exact lesson about six more times.) Winograd’s SHRDLU, fluent and thoughtful inside a closed world of colored blocks.

The catch was always the world. SHRDLU’s blocks could all be known, listed, reversed. The real world has rain, and grandparents, and the smell of coffee, and you cannot list it. In the closed world of symbols, symbols were enough. The next decade was about discovering, painfully, that the world is not closed.

The cost of winning too hard

In 1969, Minsky and Papert published Perceptrons and proved a single-layer network can’t compute XOR.

The proof was correct. It was also narrow — they admitted multi-layer networks could do it, nobody just knew how to train them yet. But the field was hungry for a verdict, and it read the book as one. Funding for neural networks collapsed. Rosenblatt died two years later in a boating accident, on his 43rd birthday. The algorithm that would resurrect his branch didn’t arrive at scale until 1986. Seventeen years of silence.

Modern AI runs on the work of people who weren’t born when Minsky and Papert published. The reason their work came so late is that the field they’d return to had been kept near-dead for two decades. The symbolic camp’s victory was real. The field paid for it. It will pay that bill again.

The rationalist trap

Through the 1970s and 80s the symbolic branch found something that made money: expert systems.

MYCIN matched infectious-disease specialists. XCON saved DEC tens of millions a year. The thesis was clean and seductive — intelligence is rules plus facts; hire the expert, extract the rules, ship the system. And these systems were legible. You could read every rule, audit the reasoning, fix the wrong line. (Your favorite large language model cannot do this. We’ll come back to that another day.)

Two problems killed it. Common sense turns out to be unrepresentable in rules — birds fly, except penguins, except baby penguins, except dead ones — and the rules contradict each other faster than you can write them. And then there’s Cyc: in 1984 Doug Lenat set out to hand-encode all of common-sense knowledge, estimated ten years, and is still at it forty-two years later. It is the most thoroughly humbling monument in the history of cognitive science.

By the late 80s the money dried up and the Second AI Winter set in. The field was tired of the rationalists.

The empirical rebellion

We’re on 1986: backpropagation, in Nature, multi-layer networks are finally trainable.

Then the empiricist branch spends fifteen years not scoring one big win but a thousand small ones. Support vector machines. Random forests. Boosting. Statistical methods quietly eating one application after another, including the symbolic camp’s home turf — language, where IBM’s speech team found that every time they fired a linguist, the system improved.

Why now? Three things are moving together, slowly. Compute grew. The internet started producing data in volumes nobody had imagined. And the methods were simple enough to scale with both.

In 2019 Richard Sutton would name this The Bitter Lesson: across seventy years, the general method that scales with compute beats the clever hand-engineered one, every time. It’s bitter because it tells researchers their hard-won taste gets steamrolled by someone with more GPUs. It is mostly right.

The thing that complicates it is the thing symbolic AI was good at all along — but I’m getting ahead of myself.

The earthquake

Now jump to September 2012.

AlexNet — eight layers, two gaming GPUs, a couple of training tricks — drops the ImageNet error rate ten points below the nearest hand-engineered system. A ten-point gap isn’t an improvement. It’s a different category of result. Within six months every computer-vision lab on Earth has pivoted. AlexNet is, by a wide margin, the single most consequential paper in modern AI.

Then it cascades, almost too fast to track. Sequence-to-sequence translation. GANs. Atari from raw pixels. In 2016 AlphaGo beats Lee Sedol at a game with more board positions than there are atoms in the universe — and almost nobody notices that inside it is a deep network (empiricist) wrapped around a tree search (symbolic). The marriage is already there, in 2016, hiding in plain sight. In 2017, “Attention Is All You Need” introduces the Transformer, and every model in your chat window today descends from that one paper.

The crown jewel nobody talks about

The most consequential AI system of the modern era is not a chatbot. It doesn’t write poems. It’s in the bloodstream of structural biology.

Predicting a protein’s 3D shape from its amino-acid sequence is a fifty-year-old problem. The rationalist approach — simulate the physics — was beautiful and almost completely intractable. For twenty years the field’s hardest benchmark plateaued at a score around 40. In 2020, DeepMind’s AlphaFold 2 scored above 92 on that exact tier. The grand challenge was, for practical purposes, solved. Hassabis and Jumper got the 2024 Nobel in Chemistry for it — the only AI work so far to produce a Nobel-level scientific breakthrough.

Read the citation. It isn’t about AI as a technology. It’s about a problem that got finished while the people whose careers were defined by it slept. The chatbots get the headlines. The image generators get the lawsuits. The protein folder got the world. Remember that the next time someone wants to tell AI is ChatGPT.

The synthesis

Now the marriage. The word agent did not come from machine learning. It came from classical, symbolic AI in the 1970s and 80s: a system that perceives its environment, deliberates, picks an action, acts, observes, loops. The architecture was right. The brain was missing. Pure symbolic computation could never model a world with grandparents and coffee in it, so the agent shell sat there for decades, structurally correct and operationally empty. Cyc, again, is the long sad proof.

The empiricists borrowed the same word in the 2000s — in reinforcement learning, an agent is a learned policy. DQN was an agent. AlphaGo was an agent. A new brain, slotted into the old shell. Spectacular, and narrow. An AlphaGo cannot make you a sandwich.

In 2024 the cognition slot gets filled a third time, by a general-purpose reasoning language model. The shell is still the seventy-year-old symbolic frame: perceive, deliberate, pick an action with a name and a meaning — read_file, run_tests, send_email — act, observe, loop. The brain is now an LLM. From the empiricist side the system inherits flexibility: it has read enough of the world that you don’t have to tell it what a file is, or what an angry customer sounds like. From the symbolic side it inherits structure: the actions have names, the consequences are bounded, the trajectory is auditable. The model can hallucinate; the system can’t run rm -rf unless somebody wired that action in and granted it.

The 1970s symbolic agent could never reason. The 2010s RL agent could never generalize. The 2026 agent does both — badly, often clumsily, but for the first time at the same time. You can watch this happen most clearly in software development right now: a language model at the core, a harness of tools around it, a test suite as the verifier, a human reviewing the diff. All four layers, on a laptop, today. Software is the canary. The same pattern is already moving toward research, then education, then everything whose feedback loops are fast enough.

So here is the closing claim, the techno-pragmatist version of the whole story. The seventy-year argument did not produce a winner. It produced three layers: a learning substrate that absorbed the written record of humanity, a symbolic shell that makes it accountable, and a human frame that decides what the whole thing is for. The first two are engineering. The third is the only one that was ever really about us.

The synthesis exists. What we do with it is still up to us.

Until next time, stay curious.

This post is the speedrun — the book’s ~8,000-word opening chapter compressed down to its spine. The full version in the second edition of Mostly Harmless AI has every scene with its characters, the eras this post skipped, 70+ references, and the agentic stack the rest of the book then takes apart mechanism by mechanism. It’s the book I wish someone had handed me when I was trying to make sense of the noise — and it’s 50% off while it’s in early access. You can also read the whole thing online for free in a custom reader I built and am rather proud of: dark mode, font controls, progress tracking, offline support, the works.

Get it (50% off)

And if you want the whole catalog of everything I’ve written, plus everything I’m going to write, that’s the Compendium. One purchase, in perpetuity.

Let’s kick off 2026 with a thought experiment.

You’ve all heard the Simulation Hypothesis by now, right? The idea that we might just characters in a high-res version of The Sims running on a supercomputer in some teenager’s basement in another dimension.

Well, this is not that.

The simulation hypothesis is a fun thought experiment, and it’s certainly great for selling movie tips, but I think it misses the point entirely. It presumes an outside that we can never reach. But we don’t need to posit any external observer, a higher-dimensional teenager in a basement. What if the universe isn’t inside a computer; but it is itself a computer.

This is a serious metaphysical position held by several prominent thinkers known as pancomputationalism, which basically means everything there is, is just computation. In this and a follow up article, I want to run with this argument and stretch it to its ultimate implications. Not necessarily because I believe it (I will tell you exactly what I believe at the end) or because I want you to believe it.

No, I just honestly think this is a very intellectually engaging argument to discuss, and even if it ends up not being true in its ultimate form (which we shall see), it presents some pretty damn cool ideas–and some pretty horrific ones. So even if just as a fun thought experiment, I invite you to ponder on it for a while.

You may be surprised by what you discover.

The Universe as a Universal Computer

To understand pancomputationalism, we have to go back to the 1980s, to a physicist-turned-polymath named Stephen Wolfram staring at a computer screen.

Wolfram was playing with something called Cellular Automata—incredibly simple programs that live on a grid of pixels. You start with a line of black and white squares and give them a rule: “If your neighbor is black and you are white, turn black in the next step.” It sounds like a toy or perhaps something you’d learn in a mid-school basic coding class.

CAs where not Wolfram’s invention. We can trace them back to Stanislaw Ulam and the polygenious John von Neumann back in the 1940s (because of course anything you find interesting in late 20th century was actually invented by von Neumann!) but it wasn’t until the 70s when John Conway brought them to the public’s attention with a series of puzzles about a specific form of 2-dimensional CA called Game of Life.

The following picture shows a “Glider Gun”, one of the most interesting patterns that answers one of Conway’s original questions: the existence of configurations that ran forever without looping (you can see how the small thingies that are thrown diagonally—called gliders—keep moving forever).

The reason these are interesting, among other things is because depending on the initial configuration, there are several, qualitatively distinct developments a cellular automaton can follow. Wolfram classified them into 4 more-or-less well-defined classes.

Classes 1 and 2 corresponding to different ways in which the initial configuration converges to a stable configuration (like the Glider Gun) that may or not keep evolving, but in a very controlled, predictable way. Some of these eventually stop, others keep going forever, but always in a very structured form.

Class 3, in contrast, is chaotic, random, unpredictable. In fact, several configurations in this category are thought to be cryptographically-secure random generators—the strongest kind of randomness you can get from a deterministic set of rules.

And then there is Class 4, a category of CA configurations that exhibit such a degree of complexity in their evolution that Wolfram conjectured most if not all configurations in this category where capable of general-purpose computation! Eventually, it was shown that extremely simple CAs do possess this trait, including one of Wolfram’s original CAs, Rule 110, and several configurations in Conway’s Game of Life.

The reason this is interesting for our discussion, is because it led some thinkers to ponder whether this kind of complexity born out of simplicity could be the explanation behind all of physics. The first one is perhaps Konrad Zuse, a German engineer who built the first programmable computer (the Z3) during World War II.

In his 1969 book Rechnender Raum (Calculating Space), he proposed that the universe is a discrete lattice. He argued that the infinities that plague quantum physics could be solved if we just admitted that space has a minimum pixel size. In Zuse’s vision, space isn’t an empty stage where things move around. Space is the computer itself. An electron isn’t a particle moving through the vacuum; it’s a propagating pattern on the spatial grid. Think of it like a glider in Conway’s Game of Life: the pixel doesn’t move, the information does.

This is the heart of Pancomputationalism. The idea that, deep down, the Universe is not a metaphorical clockwork machine but an actual digital computer, though not one made of transistors, of course. Wolfram takes this to an extreme, by positing all of physics can be derived from a simple computational model, a hypergraph of nodes and edges that change according to some specific rules. In this theory—which is perhaps the most advanced form of pancomputationalism—the underlying computational lattice is so intricate that extremely simple processes like one electron jumping from an energy state to another would require several trillions of operations.

But the details matter less than the general idea: if there are some extremely simple mechanism (like the Rule 110 CA) that can support general computation, then universal computers most be abundant in the Universe. As an example, the DNA molecule must be capable of universal computation!

So why not take this argument to its logical end? Perhaps this is all there is! Perhaps universal computation is the ultimate level of complexity. If that’s the case, then the whole Universe is ultimately just a very complex computer.

Time for some counter-arguments

Of course, the idea that the universe is just a computer—or more specifically, a Turing machine—is a hard pill to swallow for many. If the cosmos is indeed a digital lattice, then everything within it should be effectively computable. But nature has a way of throwing us curve balls that look suspiciously like hacks of the system. Processes that seem to require more than universal computation to work out.

Let’s check two of the most famous counter-arguments to pancomputationalism.

Black Holes and Hypercomputers

One of the most persistent challenges to pancomputationalism is the idea of Hypercomputation. The argument goes like this: one of the simplest ways to show something is more complex than a Turing machine is to show it can solve problems known to be computationally undecidable. The most famous of these is the Halting problem–the question of whether a given arbitrary computation will end up eventually.

This was proven by Alan Turing himself to be generally undecidable. That is, you cannot predict whether a computer program will eventually halt or run forever just by looking at its source code. You can do it for some programs (because the code has no loops, for example, or for some contrived mathematical property), but you cannot devise a single, universal rule—a computer program—that tells you without doubt whether another arbitrary program stops.

But you can simulate it! If you want to know whether a program ends, just run it. If it ends, then eventually you will know it. The problem is, if it doesn’t end, you will never know! You will just wait indefinitely, because you can never say “Ok, I’ve seen enough”, the program might just end next iteration.

Ok, so here’s the twist. This presumes each iteration takes a finite amount of time, so an infinite number of iterations must take an infinite time, so you cannot outwait it. But what if we could make each iteration take less and less time? Like, the first iteration takes 0.5 seconds, the next one 0.25 seconds, and so on, each iteration taking half of the previous one? If we had this magical computer that accelerates as it computes, we could finish an infinite amount of iterations in a finite time!

This is called hypercomputation, and is an extension of something called a hypertask to the realm of computational complexity. Hypertasks are like taking limits in real life. You perform each step slightly faster, in such a way that the infinite sum of smaller and smaller times converges to something finite. Much like Zeno’s paradox, but each step is not smaller, it’s just faster.

Can we have hypercomputers in the real world? Well, some physicists suggest that the extreme gravity of a special, theoretic kind of black holes could allow for supertasks. Imagine an observer orbiting a black hole while a computer falls into it. Because of the way gravity warps time, the falling computer, in its own timeframe, would spend eternity counting towards infinity. But we would observe all its infinite lifetime compressed into a finite amount of time.

This would be crazy! Imagine we want to know whether, e.g., Riehmann hypothesis—the most important unsolved problem in pure mathematics—is true. We would just make a program that would check all infinite zeros of the Zeta function, run it in a computer, and throw it into a black hole. As the computation stretches to infinity, we would know definitely whether any zero has real part different to 1/2, because the program would check them all in finite time!

We could solve almost any open mathematical problem this way!

Now, this does require some stretch of imagination, some bizarre physics that many don’t agree actually exists. These are not regular black holes, but a special kind of black hole that has never been observed. And of course we have to issue of actually building a computer than can last forever! But nevertheless, just proving the physical possibility of hypercomputation–even if impossible in engineering terms–would undermine the whole idea of pancomputationalism.

The Mind vs. The Machine

Then there is the challenge from the human side of the fence. Roger Penrose, in his seminal The Emperor’s New Mind, famously argued that human consciousness is fundamentally non-algorithmic.

His argument leans heavily on Gödel’s Incompleteness iheorem. Penrose claims that because we can “see” that a Gödelian statement is true—even though it’s unprovable within its own formal system—our brains must be doing something that transcends simple computation. He actually suggests that this “something” is a quantum gravity effect happening in the tiny structures of our brain called microtubules.

Critics like Scott Aaronson are quick to point out that even if our brains use quantum effects, that doesn’t necessarily make them hypercomputational. (A quantum computer, after all, is still just a computer, it just has a different complexity class). But Penrose’s argument hits on a deep-seated intuition: that phenomena such as love, consciousness, or creativity aren’t just very long strings of 1s and 0s. If this is true, then not only is the Universe itself uncomputable, but actually the most interesting human traits, perhaps even the root of intelligence itself, would be forever outside the reach of regular computers—no superintelligent AI for you, Sam Altman.

The Universe Hardware Spec

But for now, let’s run with the original argument. If we do assume the Universe is doing something akin to general computation, we can ask the next logical question? How long has this been going on? Not in years, of course–we kinda know that already–but in computational steps. How big is the Universal Hardware?

The question might seem bogus at first, but underneath this seemingly crazy assumption we will find some profound truths about the nature of computation. To even start to answer this we must link two seemingly separate realms: the digital and the physical.

We often think of information as something ethereal—software is just math, right? But in the 1960s, Rolf Landauer showed all us wrong. He realized that information is fundamentally tied to thermodynamics. Specifically, Landauer’s Principle states that any logically irreversible operation—like erasing a bit of information—must produce a specific amount of heat.

This isn’t the regular heat you feel when your computer works for too long, of course. That is just a byproduct of imperfect engineering–moving pieces with friction, electrical wiring with imperfect insulation, etc. No, this is much more fundamental. Even if you remove all sources of energy loss, the fact that two bits of information become one (when you and two boolean variables for example) means that some amount of energy is released, increasing the entropy of the system.

Thinking has a literal energy cost.

This gives us a maximum performance for any given physical computer: Bremermann’s Limit. Just take the total energy you input into the system, assume there are no extra losses, and you will still deplete that energy just doing computation. So, if the Universe is a computer, it must have this physical limit. After all, there is just so much energy in the Universe. How fast is the cosmic CPU?

If you take 1 kg of matter and turn it into the Ultimate Laptop, it can perform at most 1.36 × 10⁵⁰ bits per second. That’s a staggering number—far beyond anything we can currently build—but it’s not infinite. It is a hard, physical ceiling. There is no “overclocking” the universe beyond the speed of light and the Planck constant.

When you scale this logic up to the entire observable universe, you get a spec sheet for reality that is both terrifying and humbling.

Physicist Seth Lloyd calculated that since the Big Bang, the entire universe has performed at most 10¹²⁰ operations on roughly 10⁹⁰ bits. To put that in perspective, there are some 10⁸⁰ elementary particles in the observable universe, so each particle must be represented by something like 10¹⁰ bits. Huge, yes, but finite.

Where does this leave us?

If the Universe is indeed a computer, that has been running for a finite amount of time, and has thus performed a finite amount of computation, what can we say about the laws of physics. Are there things forever beyond the realm of Science.

This is question I want to answer next, but that will take us on a tour around some of the most important theoretical concepts in computer science. In the end, we will see that a Digital Universe has some damn horrifying constraints, but it also possesses some beautiful freedoms we can only dream of in an otherwise uncomputable Universe.

But that is a story for another Friday!

Leave a comment

You can’t scroll through a tech feed these days without tripping over a prophecy: the AI bubble is about to burst, and a long, cold “AI Winter” is coming. The narrative is as seductive as it is simple. The current frenzy around Generative AI, we’re told, is a speculative mania. When the inflated expectations inevitably collide with reality and the firehose of investment capital slows to a trickle, the whole enterprise will be exposed as a grand fiasco. We’ll discover, the skeptics say, that it was all a cuento.

And let’s be clear: they’re not entirely wrong about the first part. The expectations are inflated. A correction is not just likely; it’s necessary.

But here’s my thesis: the idea that this correction will lead to another AI Winter—a catastrophic freeze comparable to the funding droughts of the 1970s and 80s—is a fundamental misreading of the landscape. I will argue that what we are heading for is not a collapse, but a normalization—what I will call an AI autumn.

The inevitable deflation of the hype won’t reveal a failed technology. Instead, it will reveal a technology that has already, quietly and irrevocably, proven its utility and woven itself into the fabric of our digital lives.

This isn’t a story about a bubble bursting; it’s about a revolutionary technology finally growing up. But let’s be clear: growing up can be a painful process. The normalization I’m describing won’t be a gentle, seamless transition. An industry built on unsustainable economics and AGI-or-bust promises can still face maybe not a brutal winter, but a significant autumn, even if the underlying technology continues to thrive.

Subscribe now

Anatomy of the Hype (Or Why the Skeptics Have a Point)

Before we can talk about the future, we have to be honest about the present. The current AI landscape feels like a bubble because, in many ways, it is one. This isn’t to say the technology is vaporware; far from it. The frenzy is built on a kernel of genuinely astonishing progress. But that kernel has been buried under an avalanche of speculative capital and quasi-religious prophecy.

The promises are, to put it mildly, grandiose. Tech leaders, flush with unprecedented investment, speak of replacing vast swaths of the workforce and ushering in an era of unimaginable productivity. Every incremental improvement is framed as another step on the inexorable march toward Artificial General Intelligence. This narrative is then amplified by a chorus of accelerationists and futurists who speak of the Singularity not as a distant sci-fi concept, but as an imminent event. It’s a powerful and compelling story, and it’s fueling a gold rush.

But back on planet Earth, the story is more complicated. For every breathless demo, there are practical and theoretical roadblocks that the hype conveniently ignores. The most glaring is the hallucination problem. These models, by their very nature, invent things. We’ve managed to reduce the frequency, but we haven’t eliminated the phenomenon, and there are compelling theoretical arguments that we may never be able to. This isn’t just a bug; it’s a feature of the architecture, a fundamental crack in the foundation of trust.

This technical limitation then crashes headfirst into the corporate world’s messy reality. Most companies, lured by the promise of easy productivity gains, are discovering a massive adoption gap. They lack the clean data, the streamlined processes, and the technical expertise to reliably integrate these powerful but flawed tools. It’s no wonder, then, that an astonishing number of corporate AI projects—some estimates say as high as 85%—are quietly failing to deliver a return on investment. Sky-high promises plus messy, difficult reality is the classic recipe for a bubble.

Perhaps the most potent dose of reality, however, is coming from the frontier models themselves. We’re witnessing a classic case of diminishing returns. The leap in capability from GPT-3 to GPT-4 was so profound it felt like a paradigm shift, leading many to draw a straight line on the progress graph and conclude that GPT-5 would be knocking on AGI’s door. That hasn’t happened.

The newest models are better, certainly, but the improvement is incremental, not awe-inspiring. It strongly suggests we’re hitting the ceiling of what the current paradigm can do. Experts like Yann LeCun and François Chollet argue persuasively that to progress further, we need fundamentally new approaches—paradigms that have yet to be invented. This pushes the dream of AGI firmly back into the realm of long-term research, not the foreseeable future.

Compounding this is a simple fact: the economics of frontier AI are fundamentally broken. The cost to train a single model like GPT-4 is north of $100 million. The data center infrastructure required to support the industry’s ambitions will require an estimated $5.2 trillion by 2030.

Unsurprisingly, this has created a severe profitability crisis. In 2024, OpenAI reportedly lost approximately $5 billion on $9 billion in revenue, with inference costs alone accounting for a multi-billion dollar loss. This isn’t a business model; it’s a venture-subsidized science experiment, and it’s hitting a hard physical wall with an energy grid that cannot keep up.

Furthermore, we must recognize that this isn’t just another tech bubble. The investment flowing into AI is qualitatively different from, say, funding for a better SaaS tool or a more efficient database. A significant portion of this capital is a high-stakes, geopolitical bet on the imminent arrival of AGI. The valuations of the frontier labs are not based on their current, money-losing products; they are based on the promise of creating a literal god-in-a-box.

Whether Sam Altman and company believe or not is beyond the point. This dream of AGI is driving market valuations, and when the market finally digests that we are hitting a paradigm ceiling—a point this article has already made—the withdrawal of that ‘AGI-or-bust’ capital won’t be a gentle correction. It will be a sudden, violent repricing that could vaporize billions in paper wealth overnight.

What Will Happen When the Bubble Bursts?

So, given the inflated expectations and technical ceilings, what happens when the hype recedes? I don’t really like to make predictions, and much less about the future. It’s damn hard. But I think we can outline a possible, perhaps even probable near future. I want to draw an analogy here and claim we will see not a true AI winter, but something close to an AI autumn.

An AI autumn is an economic event. It’s a period of massive financial correction, characterized by layoffs, hiring freezes, startup failures, and a freeze in venture capital. It’s painful for the people and companies in the field. An AI winter, on the other hand, is a crisis of relevance of the core technology. It’s when the technology itself proves to be a dead end, progress stalls, and the world moves on.

To be as blunt as I can, I do believe a severe autumn for the AI industry is not just possible; it’s likely. The current economics are unsustainable, as we’ve seen. But the central argument of this article is that this painful industrial correction will not trigger a catastrophic winter, which would be far worse. No, AI is here to stay, and here is why.

First, we can’t ignore the relentless democratization of compute. The idea that cutting-edge AI will forever be the exclusive domain of billion-dollar data centers is a historical fallacy. We are already seeing an explosion of highly capable open-source models that can run on local, consumer-grade hardware. What requires a professional-grade, 10,000 dollars GPU today will run on your laptop in two years, and on your phone two years after that.

This trajectory completely decouples the utility of AI from the subsidized business models of a few large companies. The capability is escaping the lab and becoming part of the background radiation of computing.

Second, even if the progress of frontier models were to stop dead in its tracks today—which it won’t, but it will likely continue to decelerate—we still have a decade’s worth of technological breakthrough that most of the world has not even begun to properly digest. The current adoption gap isn’t a sign of inevitable failure; it’s a sign that the technology has advanced far faster than our institutions can keep up.

A slowdown in R&D investment won’t cause a retreat. Instead, it will trigger a necessary and healthy shift in focus from pure research to practical implementation, integration, and process refinement. This is what maturity looks like. The frantic sprint to invent the future will become the marathon of actually building it.

Most importantly, this shift will not trigger a true AI winter because we are simply far beyond the point where Artificial Intelligence can disillusion us. It is already a proven technology, woven so deeply into our digital infrastructure that a true winter is no longer possible.

Why We Won’t See Another AI Winter

Let’s start with Generative AI itself. Even with all its flaws, its core utility is now undeniable. The previous AI winters occurred when promising lab demos failed to translate into real-world applications. That is not the situation today.

A significant percentage of the global population—some conservative estimates say around 10%— now uses these tools not as novelties, but as integrated parts of their daily work. It’s the assistant that transcribes a meeting and pulls out action items, summarizes a sprawling email thread you don’t have time to read, and helps you rephrase a blunt message into a diplomatic one. Online search is quickly becoming the playground for generative AI, and online search is by far the most profitable business in the Internet Era.

The genie is out of the bottle; people are not going to suddenly stop using a tool that demonstrably saves them time, just because its creators promised it would become a god.

But perhaps the world of software development is an even more potent example. There’s a lot of noise about irresponsible “vibe coding,” where novices generate code they don’t understand, creating an unmaintainable mess. This is a real problem, but it’s a problem of skill, not a failure of the tool.

For experienced developers, these assistants are transformative. The mythical “10x productivity” boost is largely a myth, but a consistent 1.5x to 2x multiplier is very real. I’ve seen it in my own projects. Code assistants act as the new IntelliSense, handling the mind-numbing boilerplate and letting me focus on the architectural challenges. I may now only write 20% of the final characters in the codebase, but I am still the author of 90% of the critical ideas. This is not a crutch; it’s leverage.

And beyond these consumer-facing applications lies an even larger world of traditional machine learning that is indispensable to modern science and industry.

From drug discovery and genomic sequencing in biotech to predictive maintenance and supply chain optimization in manufacturing, decades of successful applications of AI in the industry today delivers billions of dollars in quantifiable value. Their success is measured in efficiency gains and scientific breakthroughs, not hype cycles.

But the more fundamental point is this: the debate over a “Generative AI” bubble distracts from the fact that the broader field of AI has already won its place. We haven’t had a true AI winter since the 1990s because AI stopped being a distinct, speculative field and became the foundational plumbing of the modern world. The search engine that found this article? That’s AI. The recommendation algorithm that determines your social media feed? AI. The logistics network that delivered your last package, the facial recognition that unlocks your phone, the voice transcription that takes your meeting notes—it’s all AI. Not Generative AI (for the most part), but AI nonetheless.

The line between computer science and AI has become so blurred that it’s practically meaningless. To talk about an AI winter today is like talking about an Internet winter in 2005. The technology is simply too embedded to fail.

However, as we’ve argue, there will be some painful correction. That much is, I think, almost undeniable. If that’s indeed the case, here are some optimistic arguments for why it may all be for the better in the end.

The Renaissance of AI Research

When the unsustainable hype collides with this resilient foundation, a fundamental law of economics reasserts itself: there is no free lunch. An AI autumn is the inevitable trade-off for a period of unchecked exuberance. A wave of consolidation will wash away unprofitable startups, and the market’s strategic focus will pivot from “bigger is better” to efficiency.

But this period of commercial cooldown has a powerful, if counter-intuitive, silver lining: a renaissance of real research. History shows us that AI’s greatest winters have been fertile ground for its most important breakthroughs. The hype recedes, and with it, the noise. The crushing pressure for short-term commercial returns is replaced by the intellectual freedom to tackle fundamental, long-term challenges.

Many of the core technologies fueling today’s boom were born in the quiet of previous winters. The backpropagation algorithm, popularized by Geoffrey Hinton in the 1980s, was refined during a period of deep skepticism about neural networks. Most famously, the Long Short-Term Memory (LSTM) architecture, which was a cornerstone of natural language processing for decades, was developed by Hochreiter and Schmidhuber in 1997, the absolute heart of the last AI winter.

The coming autumn will trigger a similar cycle. As the brightest minds are freed from the scaling hype, the real work on the next generation of AI can begin. We are already seeing the intellectual seeds of this shift. AI pioneers are openly discussing the deep limitations of current models. Yann LeCun is championing his Joint Embedding Predictive Architecture (JEPA) as a path toward “world models” that learn abstract representations of reality.

The field of Neuro-Symbolic AI, which fuses neural nets with structured logic, is experiencing a surge in interest. These are not incremental improvements; they are explorations of entirely new paradigms.

Conclusion: No Retreat, Just Normalization

So, where does that leave us? The coming correction is not an apocalypse; it’s a maturation. The frantic, gold-rush energy will dissipate, and in its place, something far more durable will emerge. The deflation of the hype bubble will not send talent fleeing the field or cause us to abandon the tools we’ve built. Instead, it will mark the end of the beginning.

The great irony is that the very thing that guarantees AI’s long-term survival—its commoditization into reliable ‘plumbing’—is what makes the current industry valuations so precarious. Plumbing is a low-margin, utility business, not a world-dominating monopoly. This disconnect between utility and valuation is the financial fault line where the industrial earthquake will hit. The era of breathless, revolutionary promises will give way to the slow, difficult, and necessary work of integration.

This is the natural lifecycle of any transformative technology. It moves from a speculative curiosity to a reliable, if sometimes challenging, part of the professional toolkit. Generative AI will not become the all-knowing oracle we were promised, but it has already secured its place as a uniquely powerful tool for thought, creation, and productivity.

The question was never really if AI would change the world; the underlying technology has been doing that for decades. The real question is how we manage the transition. This industrial autumn will be cushioned, to some extent, by geopolitical reality. The race between the US and China ensures that a certain level of state-sponsored R&D will continue, preventing a total 1980s-style collapse.

But for the people working in the field, the transition will still be jarring. The future of AI isn’t a simple story of success or failure. It’s the messy, often painful process of separating a world-changing technology from the unsustainable industry that’s driving it, and going back to drawing board, back to building new and even cooler stuff.

I've spent the last few years trying to make sense of the noise around Artificial Intelligence, and if there's one feeling that defines the experience, it's whiplash. One week, I'm reading a paper that promises AI will cure disease and unlock unimaginable abundance; the next, I'm seeing headlines about civilizational collapse. This dizzying cycle of AI springs, periods of massive investment and hype, followed by the chilling doubt of AI winters isn't new. It's been the engine of the field for decades.

After years of this, I've had to develop my own framework just to stay grounded. It’s not about being an optimist or a pessimist; it’s about rejecting both extremes. For me, it’s a commitment to a tireless reevaluation of the technology in front of us; to using reason and evidence to find a path forward, because I believe we have both the power and the responsibility to shape this technology’s future. That begins with a clear-eyed diagnosis of the present.

One of the most useful diagnostic tools I've found for this comes from computer scientist Melanie Mitchell. In a seminal paper back in 2021, she identified what she claims are four foundational fallacies, four deeply embedded assumptions that explain to a large extent our collective confusion about AI, and what it can and cannot do.

My goal in this article isn't to convince you that Mitchell is 100% right. I don't think she is, either, and I will provide my own criticism and counter arguments to some points. What I want is to use her ideas as a lens to dissect the hype, explore the counterarguments, and show why this intellectual tug-of-war has real-world consequences for our society, our economy, and our safety.

Deconstructing the Four Fallacies

For me, the most important test of any idea is its empirical validation. No plan, no matter how brilliant, survives its first encounter with reality. I find that Mitchell’s four fallacies are the perfect tool for this. They allow us to take the grand, sweeping claims made about AI and rigorously test them against the messy, complicated reality of what these systems can actually do.

Fallacy 1: The Illusion of a Smooth Continuum

The most common and seductive fallacy is the assumption that every impressive feat of narrow AI is an incremental step on a smooth path toward human-level Artificial General Intelligence (AGI). That is, that intelligence is a single, unidimensional metric on a continuum that goes from narrow to general.

We see this everywhere. When IBM's Deep Blue beat Garry Kasparov at chess, it was hailed as a first step towards AGI. The same narrative emerged when DeepMind's AlphaGo defeated Lee Sedol. This way of thinking creates, according to Mitchell, a flawed map of progress, tricking us into believing we are much closer to AGI than we are. It ignores the colossal, unsolved challenge known as the commonsense knowledge problem—the vast, implicit understanding of the world that humans use to navigate reality.

As philosopher Hubert Dreyfus famously said, this is like claiming that the first monkey that climbed a tree was making progress towards landing on the moon. Well, in a sense, maybe it is, but you get the point. We didn't get to the moon until we invented combustion rockets. Climbing ever taller trees gets us nowhere closer, it's just a distraction. In the same sense, mastering a closed-system game may be a fundamentally different challenge than understanding the open, ambiguous world.

But here's the nuance. While beating Kasparov isn't a direct step to having a conversation, the methods developed can be surprisingly generalizable. The architecture that powered AlphaGo was later adapted into MuZero, a system that mastered Go, chess, and Atari games without being told the rules.

Furthermore, can we really call a Large Language Model narrow in the same way? Its ability to write code and summarize text feels like a qualitative leap in generality that the monkey-and-moon analogy doesn't quite capture.

This leaves us with a forward-looking question: How do recent advances in multimodality and agentic AI test the boundaries of this fallacy? Does a model that can see and act begin to bridge the gap toward common sense, or is it just a more sophisticated version of the same narrow intelligence? Are world models a true step towards AGI or just a higher branch in a tree of narrow linguistic intelligence?

Fallacy 2: The Paradox of Difficulty

We have a terrible habit of projecting our own cognitive landscape onto machines, assuming that what's hard for us is hard for them, and what's easy for us is easy for them. For decades, the opposite has been true.

This is Moravec's Paradox, named after the roboticist Hans Moravec, who noted it's easier to make a computer exhibit adult-level performance on an IQ test than to give it the sensory and motor skills of a one-year-old.

This explains why we have AI that can master the ridiculously complex game of Go, while a fully self-driving car remains stubbornly just over the horizon. The "easy" things are built on what Mitchell calls the "invisible complexity of the mundane." This paradox causes a chronic mis-calibration of our progress and priorities, leading us to be overly impressed by performance in formal domains while underestimating the staggering difficulty of the real world.

Of course, some would argue this isn't a fundamental barrier, but a temporary engineering hurdle. They’d say that with enough data and compute, the "invisible complexity" of the real world can be learned, just like the complexity of Go was.

From this perspective, the problem isn't one of kind, but of scale. This forces us to ask: as sensor technology and robotics improve, are we finally starting to overcome Moravec's Paradox? Or are we just discovering even deeper layers of complexity we never knew existed?

Fallacy 3: The Seduction of Wishful Mnemonics

Language doesn't just describe reality; it creates it. In AI, we constantly use anthropomorphic shorthand, saying a system "learns," "understands," or has "goals." Mitchell argues this practice of using "wishful mnemonics" is deeply misleading, fooling not just the public but the researchers themselves.

When a benchmark is called the "General Language Understanding Evaluation" (GLUE) and a model surpasses the human baseline, headlines declare that AI now understands language better than humans. But does it?

The term "stochastic parrot" was coined as a powerful antidote, reframing what LLMs do as sophisticated mimicry rather than comprehension. This isn't just a semantic game, Mitchell argues; it creates a flawed mental model that leads to misplaced trust, encouraging us to deploy systems in high-stakes situations where a lack of true understanding can have serious consequences.

A fair critique is that these terms are a necessary cognitive shorthand. At a certain level of complexity, a system's emergent behavior becomes functionally indistinguishable from "understanding," and arguing about whether it really understands is an unprovable philosophical distraction.

But that still leaves a crucial question: can we develop a more precise, less anthropomorphic vocabulary to describe AI capabilities? Or is our human-centric language the only tool we have to reason about these new forms of intelligence, with all the baggage that entails?

Fallacy 4: The Myth of the Disembodied Mind

This is the most philosophical, and in my opinion, the most important fallacy. It's the deep-seated assumption that intelligence is, like software, a form of pure information processing that can be separated from its body.

This "brain-as-computer" metaphor leads to the belief that AGI is simply a matter of scaling up compute to match the brain's raw processing power. It's challenged by Mitchell and many others with the thesis of embodied cognition, a view from cognitive science which holds that intelligence is inextricably linked to having a body that interacts with the world. If this is correct, then our current approach may just be creating ever-more-sophisticated systems that are fundamentally brittle because they lack grounded understanding.

This is where we hit the great intellectual battle line in modern AI. The primary counterargument can be framed in terms of Rich Sutton's famous essay, "The Bitter Lesson," which argues that the entire history of AI has taught us that attempts to build in human-like cognitive structures (like embodiment) are always eventually outperformed by general methods that just leverage massive-scale computation.

From this viewpoint, embodiment isn't a magical prerequisite for intelligence; it's just another fiendishly complex problem that will yield to more data and processing power.

This tension poses a critical question for the future: do multimodal models that can process images and text represent a meaningful step toward solving the embodiment problem? Or are they just a more sophisticated version of the same disembodied mind, a brain in a slightly larger digital vat?

What is Intelligence, Really?

As we dig into these fallacies, a deeper pattern emerges. They aren't just four isolated mistakes; they're symptoms of a fundamental schism in how the AI world thinks about intelligence itself. Again, my goal isn't to pick a side but to avoid falling prey to cheap heuristics or ideological banners, and instead evaluate which of these paradigms gives us a more useful map of reality.

On one side, you have what I’ll call the Cognitive Paradigm, championed by thinkers like Mitchell and her mentor, superstar AI researcher and philosopher Douglas Hofstadter. This view sees intelligence as a complex, integrated, and embodied phenomenon. It assumes that the things we associate with human intelligence—common sense, emotions, values, a sense of self—are likely inseparable components of the whole, emerging from rich interaction with a physical and social world.

From this perspective, the path to AGI requires a deep, scientific understanding of these integrated components, not just more processing power.

On the other side is the Computationalist Paradigm, which is the implicit philosophy behind many of today's leading labs, and best captured by The Bitter Lesson. This posits that the biggest breakthroughs have always come from general methods that leverage massive-scale computation—in other words, from scaling things up.

In this paradigm, intelligence is a more abstract, substrate-independent quality of optimization. Problems like embodiment aren't fundamental barriers; they are just incredibly complex computational tasks that will eventually be solved by ever-larger models and ever-faster chips.

Of course, it's not a perfect binary. Most researchers are pragmatists, like me, working somewhere in the messy middle. But these two paradigms represent the poles of the debate, and the tension between them defines the entire field. It shapes which research gets funded, which systems get built, and ultimately, which vision of the future we are collectively racing toward.

Why This Debate Matters

This debate isn't just an academic parlor game. These fallacies have a massive ripple effect across society because they obscure a fundamental rule of technology and economics: there's no free lunch, only trade-offs.

The hype generated by fallacious thinking isn't just an innocent mistake; it's the fuel for a powerful economic engine. The intense competition between tech giants, the flood of venture capital, and the geopolitical AI race all depend on a constant narrative of imminent, world-changing breakthroughs. This political economy of hype forces us into a series of dangerous trade-offs.

First, we trade long-term progress for short-term hype.

The fallacies create an unstable, boom-and-bust funding cycle. During an AI spring, capital flows to projects that can produce impressive-looking demos, often based on narrow benchmarks. This starves the slow, methodical, foundational research needed to solve the hard problems like common sense and reasoning. The result is a field that lurches from one hype bubble to the next, leaving a trail of abandoned projects and unfulfilled promises that trigger the inevitable AI winter.

Second, we trade public trust for market excitement.

The cycle of over-promising and under-delivering is deeply corrosive. When we use wishful mnemonics to describe a system that "understands," and it then fails in spectacular, nonsensical ways in the real world, it breeds public anxiety and skepticism. Recent studies show the public perceives AI scientists more negatively than almost any other field, specifically because of a perceived lack of prudence. This isn't a vague feeling; it's a direct reaction to the unintended consequences of deploying brittle, overhyped systems.

Finally, and most critically, we trade responsible validation for speed to market.

This is where the consequences become most severe. Believing a system is on a continuum with general intelligence, or that it truly "understands" language, leads to its premature deployment in high-stakes domains.

When a mental health chatbot, which is fundamentally, at least today, a sophisticated pattern-matcher, gives harmful advice to a person in crisis, it’s a direct result of these fallacies. When we over-rely on brittle systems in healthcare, finance, or autonomous vehicles, we are making a dangerous bet, trading real-world safety for the illusion of progress.

Conclusion

So where does this leave us? The value of Mitchell's fallacies isn't just in spotting hype, but in exposing the deep, productive tension between these two powerful ways of thinking about intelligence. We can't ignore the fallacies, but we also can't deny the incredible, world-altering power of the scaling paradigm that fuels them.

Mitchell in her paper compares modern AI to alchemy. It produces dazzling, impressive results but it often lacks a deep, foundational theory of intelligence.

It’s a powerful metaphor, but I think a more pragmatic conclusion is slightly different. The challenge isn't to abandon our powerful alchemy in search of a pure science of intelligence. The goal, at least from a pragmatist point of view, should be to infuse our current alchemy with the principles of science, to make scaling smarter, safer, and more grounded by integrating the hard-won insights about how intelligence actually works.

The path forward, I believe, requires more than just intellectual humility. It also requires a willingness to synthesize these seemingly opposed worldviews, and a commitment to a tireless reevaluation of the technology before us. The ultimate question is not if we should choose the path of scaling or the path of cognitive science, but how we can weave them together to guide the raw power of our modern AI alchemy with the deep understanding of a true science of intelligence.

What if the most significant revolution of our time isn't just about faster computers or smarter algorithms, but about a fundamental re-wiring of how we experience reality itself? In an age often polarized between unbridled techno-optimism and dire predictions of collapse, I believe a more nuanced, pragmatic approach is essential. We must look beyond the hype and the fear to understand the true nature of the technological shifts unfolding around us.

In this essay, I will argue that we are not merely witnessing the rise of powerful new tools; we are entering the Era of Artificial Intelligence, a profound societal transformation where AI increasingly mediates our relationships with objective reality and, crucially, with each other.

To understand the magnitude of this shift, consider how past technological eras have reshaped human existence. For the purpose of this article, I'll define an "era" as a period marked by a fundamental shift where the majority of relationships, both with objective reality and among people themselves, become mediated by a specific technology stack.

Let's look at a few examples.

First, consider what we may call the Era of Electricity. Slowly first, but increasingly accelerated, our interactions with the physical world were completely transformed. Light, heat, power, all mediated by electric currents. Work in factories and farms became mediated by electricity, then transportation, then entertainment and education. Electricity even changed how we sleep. As people started staying outside after dark, we changed from a sleep cycle, often biphasic, that mirrored the behavior of the Sun, to our monophasic cycle that is completely artificial—we go to sleep way before its dark, and we wake up often before the Sun comes out.

Another example is the Era of Computers. Our engagement with reality became even more profoundly mediated, not just by physical devices, but by the logic and processing power of silicon. From personal computing to industrial automation, the computer became the invisible hand guiding our interaction with information and machinery. Think how many jobs started to function exclusively with a computer mediating between our intentions and their results. We started to use computers to pay for things, where before a simple exchange of paper was enough. Even our language changed with the advent of computers.

The Internet Era took this mediation a step further, fundamentally altering human relationships. We started flirting over the internet, ordering food, arguing about politics, and making friends over the Internet. Our social fabric itself became woven through digital threads. All our life is intricately connected to the Internet, there is almost no interaction between ourselves and with the real world that doesn't happen through some sort of online application or platform. Even this interaction between me, the author, and you, the reader, is only possible because the Internet mediates. We wouldn't know each other otherwise.

Now, Artificial Intelligence stands poised to inherit this mantle. In the remaining of this article I want to argue why AI is already becoming the primary mediator of most meaningful human interactions, and will continue to do so in the near future. And while no one can deny the progress of technology brings incredible benefits---and I'm the first to acknowledge it and embrace the immense power that AI grants---it is also undeniable that this mediation brings some pretty scary consequences, some which are already unfolding.

Subscribe now

AI as the Ultimate Mediator

AI's mediating power extends across every facet of our lives, subtly, yet profoundly, shaping how we perceive, interact with, and relate to the world and each other.

Consider how AI is already changing our perception and understanding of the world. In medicine, AI analyzes X-rays and MRIs, often detecting anomalies like tumors with greater speed and accuracy than the human eye. It mediates how doctors and patients perceive health conditions.

In industry, AI monitors machinery to predict failures, mediating our understanding of the physical world's future state. AI even helps us perceive the planet's health by processing satellite imagery to track deforestation or pollution.

Think about your phone's camera: when you take a photo, AI often automatically adjusts the lighting, focus, and even smooths skin, mediating how you perceive the scene or person you're photographing. It's not just capturing reality; it's enhancing or altering it before you even see the final image.

Beyond perception, AI is increasingly mediating our interaction and control over reality. Autonomous vehicles, from cars to drones, are changing how we physically move through and interact with our environment.

Smart home systems, managed by AI, mediate our control over our immediate living spaces. In manufacturing, AI-driven robots optimize production lines, mediating how we create and manipulate physical goods. When you ask a voice assistant like Alexa or Google Home to play music or turn off lights, AI is mediating your control over your environment through voice commands, rather than you physically interacting with switches or devices.

Many modern cars also have AI-powered features like adaptive cruise control or lane-keeping assist, mediating your interaction with the road by taking over some aspects of driving.

And then there's the realm of creation and augmentation. Generative AI, like Midjourney or ChatGPT, creates art, music, and text from simple prompts, mediating our creative output and expanding what's possible.

AI accelerates scientific discovery by analyzing vast datasets, mediating our ability to discover and augment knowledge. When you use an AI assistant or just an innocent grammar checker like Grammarly, AI is mediating your writing process and creative output, suggesting changes to make your sentences clearer or more impactful. Many video editing apps now have AI features that can automatically cut scenes or add background music, mediating your creative process in video production.

Our daily consumption of information is also heavily mediated by AI. Algorithms curate our news feeds, suggesting articles and entertainment based on past behavior, mediating what information and experiences we consume from the vast digital reality. Think about your Netflix recommendations; AI is constantly analyzing your viewing habits to mediate what shows and movies you see, often leading you down rabbit holes of similar content.

But the impact of AI on human relationships is even more profound. Communication itself is being redefined. Real-time language translation, powered by AI, enables seamless conversations across linguistic divides, mediating interpersonal understanding. AI-powered chatbots handle customer service, mediating interactions between businesses and consumers. And in a more intimate, and perhaps concerning, development, AI companions are emerging, offering conversational interaction and emotional support, mediating personal and emotional relationships in ways we're only beginning to comprehend.

Have you ever used a dating app? Or just Facebook or LinkedIn? AI algorithms are mediating who you see as potential partners by suggesting matches based on your profile and preferences, fundamentally changing how many people initiate relationships, romantic or otherwise.

When you send an email and your email client suggests auto-completing your sentences or offers quick replies, AI is mediating your written communication, subtly influencing your phrasing and the very nature of your interaction with the person at the other side of that email.

Consider how ride-sharing apps like Uber or Lyft connect drivers and passengers; AI mediates who you interact with for transportation, replacing traditional hailing or dispatch systems. Many online gaming platforms also use AI to match players for competitive games, mediating who you collaborate or compete with.

Finally, AI is beginning to mediate our very identity and self-perception. Deepfakes and synthetic media, generated by AI, create realistic images and videos of people, blurring the lines of authenticity and mediating how we perceive others and ourselves online. Personalized digital avatars, often AI-assisted, mediate our online identity.

Think about the filters you use on social media that can alter your appearance in photos or videos; AI is mediating how you present yourself to others online and, in turn, how you might perceive your own image. AI is starting to mediate even your relationship with your own mind.

The Perils of AI Mediation

The immediate benefits of Artificial Intelligence are undeniable. AI offers unprecedented efficiency, personalized experiences, and the ability to solve complex problems that have long eluded us. This potential for human flourishing is immense, provided we make conscious choices about its use.

However, this era, like all technological advancements, comes with no free lunch. While the benefits are compelling, they are inextricably linked to profound, often insidious, risks. To focus solely on these gains would be to miss the critical trade-offs inherent in this new era. As AI imposes more layers of mediation between us and our world, we risk a dangerous detachment from immediate reality, from each other, and ultimately, from our own agency.

This is where the prescient critique of Guy Debord, articulated in his seminal 1967 work The Society of the Spectacle, becomes chillingly relevant. Debord argued that modern capitalist society had transformed "all that was once directly lived" into "mere representation." The spectacle, for Debord, was not just a collection of images, but a social relationship mediated by images, where appearances replaced authenticity, and passive consumption supplanted active participation.

He warned of a world where reality itself was subsumed by its image, leading to profound alienation and a loss of genuine human experience. If Debord's mid-20th century world was a spectacle, the Era of AI threatens to usher in a Hyper-Spectacle—a phenomenon of mediation so pervasive and sophisticated that it would dwarf anything he could have imagined. I think there are three fundamental, distinct concerns about this hyper-spectacular reality that we should analyze in depth.

Disconnection from Immediate Reality

In the Hyper-Spectacle of AI, our direct engagement with reality is increasingly replaced by an algorithmic illusion. When AI curates all our information, news, and even sensory input, we risk living in an algorithmic filter bubble on steroids. Imagine an AI-powered AR overlay that filters out undesirable elements of reality, presenting a highly curated, potentially distorted, and ultimately unreal version of the world.

This is the ultimate triumph of Debord's "mere representation", where the mediated reality becomes more compelling, more perfectly tailored, than the actual reality. We become passive consumers of AI-generated solutions, outsourcing our direct engagement with the physical world and its challenges. This leads to a loss of practical skills and a diminished sense of agency, as our capacity for direct experience atrophies.

Think about how many people rely solely on their phone's GPS for directions, even in familiar areas. AI mediates their perception of their physical surroundings, often leading them to ignore street signs or landmarks, and potentially diminishing their spatial awareness. If the GPS is wrong, they're lost because they haven't engaged directly with the real world.

Similarly, when you watch a heavily AI-edited video or listen to an AI-generated song, you might be consuming something that feels real but was never "created" by a human in the traditional sense, mediating your experience of art and authenticity.

Then comes the black box issue—where AI systems make critical decisions based on opaque algorithms—which means we benefit from outcomes without understanding why or how. The "truth" presented by the AI is accepted without question, becoming an unchallengeable spectacle. This opaque authority reinforces the illusion, as the underlying mechanisms of control are hidden, and the mediated outcome is presented as objective fact, further separating us from the direct understanding of cause and effect.

For example, many people now rely on AI-powered smart assistants to answer questions or provide information without ever checking the source. If the AI gives a subtly incorrect or biased answer, it's often accepted as fact, mediating their understanding of truth without critical engagement.

Or consider a smart refrigerator that automatically reorders groceries based on AI predictions of your consumption; while convenient, it mediates your direct engagement with your food choices and shopping habits, potentially leading to a loss of awareness about what you're actually consuming or spending.

Disconnection from Each Other

The Hyper-Spectacle extends its reach into our most fundamental human connections, fostering a simulated relationship that replaces authentic interaction. The rise of sophisticated AI companions, so convincing they fulfill emotional or social needs, poses a chilling question: what happens to genuine human interaction?

The "relationship" with the AI is a performance, a simulation of connection, a perfect example of Debord's social relationship mediated by images where the image of connection supplants its substance. This risks eroding empathy, social skills, and the capacity for deep, complex human relationships, leading to profound loneliness in a hyper-connected, yet isolated, world.

Think about how many people prefer to text or message rather than call or meet in person. Even without AI, the digital mediation of communication can lead to a simulated connection where nuances of tone, body language, and spontaneous interaction are lost, potentially eroding deeper social skills. Now add an ever-present, never-angry chatbot that can replace all your friends and family.

Moreover, AI-driven algorithms optimize content delivery to maximize engagement, often by showing us more of what we already agree with. This fuels algorithmic polarization and echo chambers, making shared understanding and collective action incredibly difficult. Different groups live in entirely different "spectacles" of reality, curated by algorithms, making genuine dialogue and the bridging of divides increasingly impossible.

The spectacle here is not just what we see, but who we see and how we see them, fragmenting society into isolated, algorithmically-defined bubbles. If your social media feed is constantly showing you content from only one political viewpoint, AI is mediating who you see and what opinions you encounter, making it harder to understand or empathize with opposing views. You're living in an algorithmically-defined bubble.

Servitude to Embedded Ideologies

Perhaps the most insidious risk in the Hyper-Spectacle is the subtle, yet pervasive, servitude to ideologies embedded within these mediating layers. AI systems new and old, when trained on biased historical data, can perpetuate and amplify existing societal inequalities in areas like hiring, lending, or criminal justice. This ideology of past discrimination gets baked into the future, presented as objective truth by AI.

This is the spectacle's ultimate power: presenting its biased outcomes as neutral, objective reality, subtly reinforcing existing power structures and values without overt coercion.

Imagine an AI-powered resume screening tool that, because it was trained on historical hiring data, subtly favors candidates from certain demographics or educational backgrounds, even if those biases aren't explicitly programmed. This mediates who gets professional opportunities, reinforcing past inequalities.

To make things even worse, AI systems are increasingly designed to subtly influence our choices—what to buy, what news to read, whom to vote for—through personalized recommendations and persuasive interfaces. Our choices feel free, but they are meticulously guided by an unseen hand, a spectacle of autonomy that masks underlying control.

This is the essence of Debord's critique of commodity fetishism extended to every aspect of life: our desires and decisions are manufactured and mediated, not genuinely our own. When you're shopping online, AI-powered "recommended for you" sections or "customers also bought" suggestions are constantly mediating what products you see and are encouraged to buy, often leading you to spend more or buy things you didn't initially intend to. Your choice feels free, but it's heavily influenced.

When the development and deployment of powerful AI systems are concentrated in the hands of a few corporations or governments, it gives them unprecedented control over information, resources, and even human behavior. The AI-mediated world becomes a spectacle designed and controlled by an elite, shaping narratives and realities to their advantage, further cementing our role as passive spectators in a world we no longer truly govern.

The Future is Not Predetermined

This Era of AI is not a distant future; it is here, and it is accelerating at a pace that leaves little room for avoidance. We cannot, and perhaps should not, try to stop it. But we absolutely must shape its trajectory. The technology is inevitable, but its direction and impact are not predetermined. This requires a proactive, collective effort, guided by reason and evidence, not by fear or blind faith.

I have absolutely no idea how to "fix" the problem of AI mediation. But I'm sure there are at least three critical approaches we have to embrace, lest we become mere spectators of this hyper-spectacular reality.

First, we must demand regulation that keeps choice firmly in the hands of the people. This requires advocating for the principles of algorithmic choice. We should always be able to determine when an algorithm is being used to guide our choices, and crucially, we must have the agency to understand, challenge, and more importantly, select which algorithms we let influence ourselves. Just as we choose our food or news sources—because you do, right?—, we should demand the right to choose our algorithmic lenses.

This means not only having, e.g., options to turn off algorithmic recommendation in Netflix or Twitter, but also to connect our own algorithmic filters and recommendation engines to replace the built in functionality. To achieve this, platforms need to embrace open protocols for algorithmic recommendation, for example, such that anyone can implement a different feed sorting procedure for Twitter. Platforms like Mastodon and Bluesky are pioneering this approach, but without explicit regulation there is no hope the largest platforms will follow.

It is clear then that we need robust institutional oversight. This means proactive, informed government regulation to set standards and enforce accountability. But it also means empowering non-governmental watchdogs, independent organizations, academia, and civil society, to monitor, critique, and advocate. Their role is crucial in preventing overreach, ensuring fairness, and keeping powerful AI entities in check. This collective vigilance is essential to prevent the concentration of power that fuels the spectacle.

And finally, beyond regulation and oversight, a more fundamental and personal shift is needed. We must consciously and actively strive to reclaim our direct experience of reality and shed the algorithmic lenses that increasingly mediate our lives. This means cultivating critical awareness of when and how AI is shaping our perceptions, fostering genuine human connections that bypass algorithmic curation, and seeking out unmediated experiences.

It means valuing authentic engagement over passive consumption, and actively participating in the world rather than merely observing its mediated representation. This is a call to intellectual and experiential rebellion against the Hyper-Spectacle, a commitment to living directly rather than through the image.

The Era of AI is upon us. It promises to mediate nearly every aspect of our lives. The question is not if it will mediate, but how. Answering this in a way that encourages human flourishing demands an unending, tireless reevaluation of our position towards technology. There are no banners, no ideology we can trust blindly. Techno-optimists and AI-doomers are both wrong, because they believe the outcome of technology is predetermined. And that makes us, by definition, spectators.

So here is my call to action for you today. Dare to stop living inside this spectacle of reality for a minute, put down your phone, close your laptop, or turn off your display. Look up, the world is still out there, waiting for you. Call one of your friends and ask them out for a pizza or a coffee or a beer—or all of them for a triple bonus. Hugh your kids if you have any, and your parents if you're lucky to still have them around. Go outside, feel the warmth of the sun in your face, the smell of the city, or the countryside, or the ocean, the roughness of dirt or pavement or grass under your bare feet. If it hurts just a little bit, even better.

Dare to choose how you want experience the actual, raw, unmediated world. The choice, I believe, is still yours. But not for long.

PS: Remember you can get Mostly Harmless AI while in early access at a reduced price. We are now running a special offer that gives you the PDF and EPUB version of the book as it currently stands, plus guaranteed access to all future editions for just $5.

Get Mostly Harmless AI ($5)

For centuries, we humans have been captivated by the idea of a thinking machine. This isn’t some modern tech obsession; the dream of automatons and artificial minds is woven through our myths and philosophies. But the formal quest to build one began only in the mid-20th century, and its history has been a dramatic back-and-forth between two core, seemingly antagonistic approaches.

One path, rooted in the logic of rationalism, sought to build intelligence from the top down by programming explicit rules and symbols. The other, inspired by the biological empiricism of the brain, tried to create it from the bottom up by allowing machines to learn patterns from data and experience.

This chapter explores the history of Artificial Intelligence (AI) through the lens of this great intellectual tug-of-war. It is a journey through distinct eras, each defined by which philosophy was dominant, what external factors like computing power and data availability enabled its rise, and how these forces have finally begun to converge, leading us to the powerful tools we have today.

In the appendix of this book, we will present a detailed chronology of the most important milestones in the history of artificial intelligence.

Subscribe now

The Foundational Era (1940s - 1960s)

The dawn of AI was a time of immense optimism, where the very concept of a “thinking machine” was formalized. Even before the field had a name, its philosophical and theoretical groundwork was being laid. In his seminal 1950 paper, “Computing Machinery and Intelligence,” Alan Turing proposed the Turing Test, setting a profound, long-term goal: to create a machine whose conversation was indistinguishable from a human’s. In parallel, the work of Warren McCulloch and Walter Pitts in 1943 on the first mathematical model of an artificial neuron planted the seeds of the connectionist dream—the idea that intelligence could emerge from simple, brain-like units.

When the field was officially christened at the Dartmouth Workshop in the summer of 1956, the symbolic, logic-based paradigm took the lead. Researchers believed that human thought could be mechanized, and the primary task was to build systems that could manipulate symbols according to formal rules. This vision was solidified by the creation of the LISP programming language in 1958, a tool perfectly suited for this symbolic manipulation.

Yet, in that same year, the connectionist counterpoint took physical form. Frank Rosenblatt developed the Perceptron, the first artificial neural network that could learn to classify patterns on its own, offering a tangible, bottom-up alternative to pure logic.

The public imagination was quickly captured by early demonstrations of AI’s potential. The Unimate (1961), the first industrial robot, showed that machines could perform physical labor. Shakey the Robot (1966) took this a step further, becoming the first mobile robot to perceive its environment and reason about its own actions. Joseph Weizenbaum’s ELIZA (1964), a simple chatbot that simulated a psychotherapist, revealed how easily humans could attribute intelligence and understanding to a machine. But this initial optimism soon collided with reality.

The ambitious promises of creating true intelligence went unfulfilled, and in 1969, the publication of the book Perceptrons by Marvin Minsky and Seymour Papert delivered a critical blow. By rigorously detailing the mathematical limitations of simple neural networks, the book effectively starved the connectionist school of funding, ushering in the first “AI Winter” and ensuring that the symbolic approach would dominate the field for the next decade.

The Knowledge Era (1970s - 1980s)

With connectionism on the back burner, the field regrouped around a more pragmatic goal: instead of trying to create general intelligence, researchers focused on capturing and mechanizing human expertise in narrow domains. This led to the golden age of expert systems, the first commercially successful form of AI. The core idea was to interview a human expert, painstakingly encode their knowledge into a vast set of “if-then” rules, and use a reasoning engine to produce solutions.

This approach yielded impressive results. SHRDLU (1972) was a landmark natural language program that could understand and respond to commands about a simulated world of blocks, showcasing a new level of sophistication for symbolic AI. Expert systems like MYCIN (1972) could diagnose blood infections as accurately as junior doctors, while others like DENDRAL and PROSPECTOR found success in chemistry and geology. This culminated in the first true commercial boom, as companies like Digital Equipment Corporation used the XCON system (1980) to configure complex computer orders, saving millions of dollars. The ambition of this paradigm reached its peak with the Cyc project (1984), a monumental effort to manually encode all of human common sense knowledge into a single, massive database.

While the symbolic school reigned, a connectionist undercurrent continued to flow. In Japan, Kunihiko Fukushima’s work on the Neocognitron (1980) created a hierarchical, multi-layered neural network for visual recognition that was the direct ancestor of the architectures that would dominate computer vision decades later. And in 1986, the popularization of the backpropagation algorithm provided an efficient method for training these deeper networks, solving a critical problem that had plagued the field for years.

However, the symbolic paradigm’s dominance was destined to end. Expert systems were incredibly brittle; they were expensive to build, nearly impossible to update, and would fail completely if faced with a situation not explicitly covered by their rules. The hype, fueled in part by Japan’s ambitious Fifth Generation Computer Systems project (1982), once again outpaced reality. When the specialized hardware market collapsed in 1987, the field plunged into its second “AI Winter,” leaving the promise of AI unfulfilled once more.

The Internet Era (1990 - 2011)

The end of the second AI winter was not driven by a single algorithmic breakthrough, but by two external forces that changed everything: the public launch of the World Wide Web in 1991 and the invention of the Graphics Processing Unit (GPU) in The web began generating an unimaginable ocean of data—text, images, and user interactions. The GPU, particularly after the release of NVIDIA’s CUDA platform in 2007, provided a way to perform the massive parallel computations needed to learn from that data. These two catalysts—data and computation—created the perfect conditions for the statistical, learning-based paradigm to finally thrive.

Before deep learning took hold, this new environment fueled the rise of “shallow” machine learning. Algorithms like Support Vector Machines (SVMs) (1995) became dominant, and open-source libraries like scikit-learn (2007) made them accessible to a wide audience. This approach had a massive real-world impact, powering the recommender systems of companies like Amazon and sparking global competitions like the Netflix Prize (2006). The infrastructure to handle this new scale was built in parallel, with Google’s MapReduce (2004) providing the blueprint for big data processing.

During this time, foundational work in reinforcement learning was also bearing fruit. TD-Gammon (1992) showed that a program could teach itself to play backgammon at a superhuman level, and the textbook by Sutton & Barto (1998) codified the field for a new generation. The seeds for the coming deep learning revolution were being sown with the invention of key architectures like LSTMs (1997) and LeNet-5 (1998), while the creation of the massive ImageNet dataset (2009) provided the high-quality benchmark that would soon ignite it.

AI also became a tangible part of public life. The symbolic paradigm had its last great public triumphs with Deep Blue’s victory over Garry Kasparov in chess (1997) and Watson’s win on Jeopardy! (2011). But the future belonged to learning-based systems. Dragon NaturallySpeaking (1997) brought continuous speech recognition to consumers. Competitions like the DARPA Grand Challenge (2004) spurred the development of autonomous vehicles. Consumer products like the Roomba (2002) and Microsoft’s Kinect (2010) brought robotics and computer vision into millions of homes. With the launch of Siri in 2011, a conversational AI assistant was finally in everyone’s pocket.

The Deep Learning Era (2012 - 2018)

If the Internet Era set the stage, 2012 was the year the curtain rose on the deep learning revolution. In October, a deep convolutional neural network called AlexNet, trained on GPUs using the ImageNet dataset, shattered all previous records in the annual image recognition competition. This “ImageNet Moment” proved the overwhelming superiority of deep, data-driven learning and kicked off a Cambrian explosion of breakthroughs.

This period was a stunning validation of what AI researcher Rich Sutton would later call The Bitter Lesson: that general methods leveraging massive computation almost always outperform approaches that rely on hand-crafted human knowledge. The field progressed at a breathtaking pace. In natural language processing, Word2Vec (2013) provided a powerful way to represent the meaning of words as vectors. In generative AI, Generative Adversarial Networks (GANs) (2014) introduced a novel way to create stunningly realistic synthetic images. New architectures like ResNet (2015) allowed for the creation of networks hundreds of layers deep, solving a fundamental barrier to scale.

These new techniques allowed AI to achieve superhuman performance in increasingly complex domains. Deep Q-Networks (DQN) (2013) learned to master classic Atari games directly from pixels, and in a landmark event in March 2016, AlphaGo defeated Lee Sedol, the world’s greatest Go player. The revolution was powered by a new generation of open-source tools like TensorFlow (2015) and PyTorch (2016) that democratized deep learning, as well as specialized hardware like Google’s Tensor Processing Units (TPUs) (2016). The era culminated with the invention of the Transformer architecture in 2017.

However, 2018 marked a turning point. In March, the Cambridge Analytica scandal revealed how machine learning algorithms, fed by the personal data of millions of Facebook users, had been used for political manipulation, sparking a global reckoning over data privacy and the ethics of AI.

That same month, though, the scientific community formally recognized the field’s impact, awarding the Turing Award to Geoffrey Hinton, Yann LeCun, and Yoshua Bengio for their foundational work. The Turing Award–aptly named after the most important figure in the history of Computer Science as a whole, let alone Artificial Intelligence–is the most prestigious academic award in computing, akin to the Nobel Prize.

The Generative Era (2019 - Present)

The current era is defined by the application of the Transformer architecture at an unprecedented scale. By training these models on vast swaths of the internet, researchers discovered that quantitative leaps in size and data could lead to qualitative leaps in capability, resulting in models with emergent generative and reasoning abilities that have captured the world’s attention.

The first sign of this new power came with GPT-2 in 2019, whose ability to generate coherent text was so advanced that its release was initially staged due to safety concerns. Its successor, GPT-3 (2020), demonstrated that massive scale could unlock “few-shot” learning, the ability to perform tasks it was never explicitly trained on.

Soon, this generative power was applied beyond text to images with DALL-E (2021) and to code with GitHub Copilot (2021). But the cultural tipping point arrived in November 2022 with the release of ChatGPT. Its simple, conversational interface made the power of Large Language Models (LLMs) accessible to millions, sparking a global phenomenon and a new wave of investment.

This boom was accompanied by a powerful open-source counter-movement. The release of the image-generation model Stable Diffusion (August 2022) and Meta’s Llama models (2023) democratized access to powerful foundation models, sparking a “Llamaverse” of community-driven innovation. The field is now a global race, with major competitors like Anthropic’s Claude (2023), Google’s multimodal Gemini (December 2023), and China’s DeepSeek R1 (January 2025) demonstrating capabilities on par with the best proprietary systems.

Perhaps the most profound impact of this new era has been in science. In November 2020, AlphaFold 2 solved the 50-year-old grand challenge of protein folding, a breakthrough of such significance that its creators were awarded the Nobel Prize in This demonstrated that AI could be a tool not just for automating tasks, but for accelerating fundamental scientific discovery. The road ahead now points towards more autonomous, “agentic” systems, where the AI transitions from a single-response tool to a collaborator capable of executing complex, multi-step tasks on our behalf.

Conclusion

We’ve journeyed through decades of ambition, breakthroughs, and tough realizations. What we’ve seen is a constant back-and-forth, a dynamic dance between two powerful ideas: the precise, rule-based, inflexible logic of symbolic AI and the adaptable, pattern-based, unreliable power of statistical AI. This dance, as we’ve explored, often mirrors the philosophical tension between rationalism and empiricism.

Today, AI stands at a fascinating crossroads. The purely statistical systems that define the generative era have achieved incredible feats. Yet, we are beginning to see diminishing returns. With GPT-4 having been a high point, many newer models have made only incremental progress, suggesting that simply scaling the existing paradigm may not be enough to achieve the next level of intelligence. This has led some to speculate that we may be on the brink of a third “AI Winter,” as the hype once again outpaces the reality of the technology’s capabilities.

The recent focus on reasoning models and agentic systems seems capable of fueling the statistical hype a bit longer, but a growing number of researchers are realizing we may not achieve Artificial General Intelligence (AGI) purely by scaling. This brings us to a crucial realization: the future of AI likely isn’t about one approach winning out over the other, but about intelligently combining them. The inherent limitations of today’s models—their unreliability and lack of true reasoning—have sparked a renewed interest in the long-neglected symbolic paradigm. Hybrid approaches, particularly neuro-symbolic AI, which seek to integrate the pattern-matching strengths of neural networks with the rigorous logic of symbolic systems, hold immense potential for creating the next breakthrough.

Whether a winter is coming or not, it is indisputable that AI has already had a profound impact on society and will continue to do so. The history of this field is far from finished. It is a living, breathing, civilization-wide project with the potential to transform society for the better or, some believe, to become our ultimate doom. Everyone has a place here: technologists, yes, but also humanists, economists, historians, artists, and policymakers. The next few years promise to be extremely exciting, and you can be a part of shaping what comes next.

Thanks for reading! Below you’ll find the full expanded timeline. Please let me know if you think I missed something or made any mistakes. All feedback is appreciated!

PS: Claim your copy of Mostly Harmless AI for only $5 in the link below.

Get Mostly Harmless AI ($5)

Appendix: A Chronology of Artificial Intelligence (1956-2025)

This timeline details the key breakthroughs, conceptual shifts, and landmark achievements in the field of Artificial Intelligence, tracing its path from a niche academic discipline to a transformative global technology.

The Foundational Era (1940s - Late 1960s)

• (Science) 1943: The First Artificial Neuron is proposed by Warren McCulloch and Walter Pitts, laying the theoretical foundation for connectionism.

• (Science) October 1950: Alan Turing publishes “Computing Machinery and Intelligence,” introducing the Turing Test.

• (Social) Summer 1956: The Dartmouth Workshop is held, where John McCarthy coins the term “Artificial Intelligence,” formally establishing the field.

• (Tech) 1958: Frank Rosenblatt develops the Perceptron, the first artificial neural network capable of learning.

• (Tech) 1958: John McCarthy develops the LISP programming language, which becomes the standard for symbolic AI.

• (Product) 1961: The Unimate industrial robot begins work on a General Motors assembly line.

• (Product) 1964: Joseph Weizenbaum creates the chatbot ELIZA at MIT.

• (Tech) 1966: The Stanford Research Institute (SRI) develops Shakey, the first mobile robot to reason about its own actions.

• (Social) 1969: The publication of Perceptrons by Marvin Minsky and Seymour Papert marks the beginning of the first “AI Winter.”

The Knowledge Era (1970s - 1989)

• (Tech) 1972: Terry Winograd develops SHRDLU, a groundbreaking natural language understanding program.

• (Tech) 1972: The logic programming language Prolog is created by Alain Colmerauer and Philippe Roussel, becoming a key tool for symbolic AI.

• (Tech) 1972: Stanford University develops the MYCIN expert system for medical diagnosis.

• (Science) 1974: Marvin Minsky publishes his influential paper on “Frames” theory, a new paradigm for knowledge representation.

• (Tech) Late 1970s: Expert systems like DENDRAL (for chemistry) and PROSPECTOR (for geology) demonstrate success in specialized scientific domains.

• (Science) 1980: Kunihiko Fukushima develops the Neocognitron, an early hierarchical neural network that is the direct ancestor of modern Convolutional Neural Networks (CNNs).

• (Product) 1980: Digital Equipment Corporation begins using the XCON expert system, marking a high point for commercial AI.

• (Social) 1982: Japan’s Ministry of International Trade and Industry begins the Fifth Generation Computer Systems project, a massive initiative to build a new generation of computers based on logic programming, sparking competitive AI investment worldwide.

• (Tech) 1984: The Cyc project is initiated by Douglas Lenat, an ambitious attempt to manually encode all of human common sense knowledge into a single knowledge base.

• (Science) 1986: The backpropagation algorithm is popularized by Geoffrey Hinton, David Rumelhart, and Ronald Williams.

• (Social) 1987: The collapse of the LISP machine market signals the start of the second “AI Winter.”

The Internet Era (1990 - 2011)

• (Social) August 1991: The World Wide Web project is released to the public, creating the infrastructure for the data explosion that would fuel modern AI.

• (Science) 1992: Gerald Tesauro develops TD-Gammon, a backgammon program that trains to a superhuman level using reinforcement learning, a landmark for the field.

• (Social) 1995: Stuart Russell and Peter Norvig publish “Artificial Intelligence: A Modern Approach,” which becomes the leading textbook in the field for decades.

• (Science) 1995: The Support Vector Machine (SVM) algorithm is popularized by Corinna Cortes and Vladimir Vapnik.

• (Social) May 1997: IBM’s Deep Blue defeats world chess champion Garry Kasparov.

• (Science) 1997: Sepp Hochreiter and Jürgen Schmidhuber invent the Long Short-Term Memory (LSTM) network.

• (Product) 1997: Dragon NaturallySpeaking is released, becoming the first widely available continuous speech recognition software for consumers.

• (Product) September 1998: Google is founded, and Amazon patents its item-to-item collaborative filtering, marking the start of large-scale data-driven AI applications.

• (Science) 1998: Richard Sutton and Andrew Barto publish “Reinforcement Learning: An Introduction,” a seminal textbook that codifies the field.

• (Tech) November 1998: Yann LeCun and his team develop LeNet-5, a pioneering Convolutional Neural Network (CNN).

• (Tech) August 1999: NVIDIA releases the GeForce 256, marketed as the world’s first Graphics Processing Unit (GPU).

• (Product) November 2000: Honda unveils its ASIMO humanoid robot, a landmark in robotics and motion planning.

• (Product) September 2002: iRobot releases the Roomba, the first commercially successful autonomous home robot.

• (Tech) 2004: Google publishes its paper on MapReduce, a programming model for processing massive datasets that becomes foundational to big data infrastructure.

• (Social) March 2004: The first DARPA Grand Challenge for autonomous vehicles is held, sparking a new wave of research in self-driving technology.

• (Social) October 2006: The Netflix Prize competition is launched, galvanizing research in recommender systems.

• (Science) 2006: Geoffrey Hinton develops Deep Belief Networks, introducing effective strategies for unsupervised layer-wise pre-training.

• (Tech) June 2007: NVIDIA releases CUDA, a parallel computing platform that allows developers to use GPUs for general-purpose processing.

• (Tech) June 2007: David Cournapeau develops scikit-learn as a Google Summer of Code project.

• (Science) 2009: A Stanford team led by Andrew Ng publishes a paper showing that GPUs can make training deep neural networks 10-100 times faster.

• (Tech) 2009: The ImageNet dataset is created by Fei-Fei Li’s team at Stanford.

• (Product) November 2010: Microsoft releases the Kinect, a consumer device that brings sophisticated real-time computer vision into millions of homes.

• (Social) February 2011: IBM’s Watson wins the quiz show Jeopardy!.

• (Product) October 2011: Apple integrates Siri into the iPhone 4S, making conversational AI assistants a mainstream consumer product.

The Deep Learning Era (2012 - 2018)

• (Social) April 2012: Coursera is founded, and Andrew Ng’s Machine Learning course begins to democratize AI education.

• (Science) June 2012: The Google Brain “Cat Neuron” project demonstrates that a neural network can learn high-level concepts from unlabeled data.

• (Social) October 2012: AlexNet, a deep CNN trained on GPUs, wins the ImageNet competition by a massive margin, officially kicking off the deep learning revolution.

• (Tech) 2013: Google researchers led by Tomas Mikolov release Word2Vec, a highly efficient method for creating word embeddings that revolutionizes NLP.

• (Science) December 2013: DeepMind publishes its work on Deep Q-Networks (DQN), demonstrating an AI that can learn to play Atari games at a superhuman level from raw pixels.

• (Science) June 2014: Ian Goodfellow and his colleagues introduce Generative Adversarial Networks (GANs), sparking a revolution in generative AI for images.

• (Science) December 2015: A team at Microsoft Research introduces Deep Residual Networks (ResNet), allowing for the training of much deeper neural networks.

• (Tech) November 2015: Google releases the TensorFlow open-source library, making deep learning more accessible.

• (Social) March 2016: Google DeepMind’s AlphaGo defeats world Go champion Lee Sedol.

• (Tech) May 2016: Google announces it has been using custom-built Tensor Processing Units (TPUs), specialized hardware for deep learning, in its data centers.

• (Tech) September 2016: Facebook AI Research (FAIR) releases PyTorch, which becomes a major deep learning framework.

• (Science) June 2017: Researchers at Google publish “Attention Is All You Need,” introducing the Transformer architecture.

• (Social) March 2018: The Cambridge Analytica scandal breaks, revealing that the personal data of millions of Facebook users was used for political advertising, sparking a global conversation on data privacy and the ethics of machine learning.

• (Social) March 2018: Geoffrey Hinton, Yann LeCun, and Yoshua Bengio are awarded the ACM Turing Award for their foundational work on deep learning.

• (Social) December 2018: DeepMind’s AlphaFold makes its stunning debut at the CASP13 competition.

The Generative Era (2019 - 2025)

• (Tech) February 2019: OpenAI announces GPT-2 but initially withholds the full model due to safety concerns.

• (Product) November 2019: OpenAI releases the full version of the GPT-2 model.

• (Product) June 2020: OpenAI releases GPT-3 via a private API.

• (Social) November 2020: AlphaFold 2 achieves revolutionary accuracy at the CASP14 competition, effectively solving the protein folding problem.

• (Product) January 2021: OpenAI introduces DALL-E, a model that generates images from text.

• (Product) June 2021: GitHub Copilot is launched as a technical preview.

• (Tech) April 2022: Google announces its Pathways Language Model (PaLM).

• (Social) June 2022: Google engineer Blake Lemoine publicly claims the LaMDA model is sentient.

• (Product) August 2022: The open-source release of Stable Diffusion democratizes high-quality image generation.

• (Product) November 2022: OpenAI releases ChatGPT to the public.

• (Tech) February 2023: Meta releases the first Llama model to the research community.

• (Product) March 2023: Anthropic releases its first Claude model.

• (Tech) July 2023: Meta releases Llama 2 with a commercial-use license, sparking the open-source “Llamaverse.”

• (Product) December 2023: Google releases Gemini, its first natively multimodal model.

• (Social) October 2024: Nobel Prizes are awarded to Geoffrey Hinton, John J. Hopfield, Demis Hassabis, and John Jumper for their work in AI.

• (Product) January 20, 2025: DeepSeek AI releases its DeepSeek R1 model and chatbot, marking a turning point in the global AI race.

• (Product) August 2025 (GPT-5): OpenAI releases GPT-5, with a focus on more autonomous, “agentic” capabilities, and a rather underwhelming reception.

What? Still here? Ok, here’s another nice button for you to click. Thanks!

Subscribe now

Note: This article is based on the Prologue and Introduction chapters of my in-progress book Mostly Harmless AI, which deals with how to harness the power of Artificial Intelligence for good. You can get the early draft at a 50% discount in the link below.

Get Mostly Harmless AI (50% off)

For centuries, we humans have been captivated by the wild idea of a thinking machine. This isn't some modern tech obsession, even if it seems nowadays no one talks about anything else. No, the ancient dream of thinking machines goes way back, whispered in myths about automatons and golems in religions galore, and later, famously brought to life (or at least, cleverly faked) by feats like the Mechanical Turk.

One of the most fascinating aspects of the history of Artificial Intelligence has been this often dramatic back-and-forth between two core, seemingly antagonistic approaches to building intelligent machines. On one hand we have logic and rules (what’s often called symbolic AI), and the other hand, data and patterns (or statistical AI). In a deep way, this mirrors that age-old philosophical tug-of-war between rationalism (figuring things out through pure reason) and empiricism (learning from experience).

In this article, I want to explore the history of AI from this lens of rationalism (or symbolic, rule-based AI) versus empiricism (data-driven, statistical AI). Come with me into this deep dive to learn how these seemingly opposite philosophies have shaped AI’s past, defined its present, and are now finally starting to team up for its future.

Subscribe now

Prologue

The AI dream didn't kick off with microchips or code. Nope, it began way before, with grand philosophical ambitions and some seriously imaginative leaps, all thanks to incredibly brilliant and diverse minds.

Our journey begins in the 17th century, at the hand of Gottfried Wilhelm Leibniz. By this point in his life, Leibniz was already a superstar: a philosopher, mathematician, logician, and diplomat, who invented (or discovered) calculus independently of Newton. He actually invented the notation that we use today, with the integral symbol and the upper and lower limits.

Leibniz was a true polymath, totally immersed in the Enlightenment's big project of organizing all knowledge and reason. His drive wasn't just academic; he genuinely believed that logic could solve every human argument. He imagined a world where disagreements weren't settled by yelling or endless debates, but by calm, undeniable calculations. Inspired by how algebra and calculus, by means of clever notation, could make even the trickiest problems appear simple, Leibniz fueled his grand dream of universal computation in a time where even the simplest calculation machines where considered a marvel.

What if, he mused, we could formalize all human reasoning in a similar way? He dreamed up a characteristica universalis—a universal language for thought—and a calculus ratiocinator—a mechanical way to reason with it. In this, Leibniz was a rationalist: he believed the all human thought was a grand, logical machine. Unknowingly, he was laying the intellectual groundwork for logic, which paved the way for symbolic AI centuries later.

Fast forward to 19th-century England, we find Lady Ada Lovelace. The daughter of the famously rebellious poet Lord Byron, Ada was a formidable brain, tutored in math and science by some of the most prominent thinkers of her time. By the time she met Charles Babbage, the great inventor, he was working on his Analytical Engine, an abstract machine that could, in principle, do anything a modern computer can do. Ada was already known for her sharp mind and amazing mathematical insights, but she also had this poetic and imaginative side that shaped her view of technology.

While Babbage saw his Analytical Engine mostly as the ultimate number-cruncher, Ada's mind took flight beyond mere arithmetic. She famously wrote that the Engine "might compose elaborate and scientific pieces of music, or in any other extent, generate new content." She was more than a century ahead of Generative AI, dreaming of the days machines would usher a new era of synthetic creativity.

As a side note, Charles Babbage would never finish constructing an actual, physical embodiment of his Analytical Engine. He kept imagining improvements over improvements, never quite settling on something that he could actually construct and use. It would have been the first true computer, but it forever remained as an unfinished project. This serves as a cautionary tale against the all too common syndrome—aptly called the Babbage Syndrome—of intellectualizing ad infinitum without actually testing out your ideas in the real world.

A few decades later, mid-20th century, as the dust settled from war and the digital age dawned, came the man who's probably the most important figure in the history of Computer Science at large, the great Alan Turing. By the time his groundbreaking work on machine intelligence came out, Turing was already widely considered among the greatest logicians and mathematicians of his time.

He's basically the Father of Computer Science, having come up with the abstract model of computation we know as the Turing Machine—the theoretical blueprint for every modern computer—and proving not only its potential but its intrinsic limitations. His wartime experience, where he played a key role in breaking the Enigma code, gave him also a very practical grasp of the power of computing. He actually built the first electromechanic general purpose computer, but this massive milestone was kept secret for years after his death.

Turing was a man of quiet brilliance. He wasn't just curious about what a real machine could do; he was fundamentally wrestling with the very definition of thinking itself. In his famous 1950 paper, "Computing Machinery and Intelligence," he dare ask if machines could think, like, for real. He proposed a brilliant, practical yet deeply philosophical way to assert it: what he called The Imitation Game, but the world came to know as the Turing Test.

If a machine could chat with a human, he suggested, in such a way that the human couldn't tell if they were talking to a machine or another human, then, for all intents and purposes, the machine could be considered to be thinking. This wasn't just a practical experiment, though; it was a functional definition of thinking that sparked the computational theory of mind. The implications of his hypothesis are at the core of the most profound discussions in the field of Philosophy of Mind, even today.

But crucially, in that same paper 80 years ago, Turing looked beyond the test and tossed out several ideas for how such an artificial intelligence might actually be achieved. These included the concept of a learning machine, raised like a human child, soaking up knowledge from experience instead of being preprogrammed to know everything beforehand; and even hinted at using bio-inspired algorithms to mimic how evolution works.

These ideas foreshadowed major pillars of modern AI systems, like neural networks and metaheuristic search algorithms, showing his amazing foresight and his deep understanding of both rationalist and empiricist paths to intelligence. Tragically, he wouldn't live to see his dream materialize into the massive body of knowledge and practice that is the field of Artificial Intelligence.

The Foundational Era (1950s - Late 1960s)

The history of Artificial Intelligence as a scientific field formally begins in the summer of 1956. A small group of brilliant minds, including John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon, got together at Dartmouth College. It was right there, at the Dartmouth Summer Research Project on Artificial Intelligence, that the term "Artificial Intelligence" was officially coined. This workshop wasn't just a meeting; it was a declaration, setting up AI as a legitimate academic discipline with a huge goal: to build machines that could think like humans do.

In these early days, the predominant paradigm was symbolic AI. Researchers believed that machines could become intelligent by putting human knowledge and reasoning into explicit rules.

One of the first, and most impressive, demonstrations of this ethos was The Logic Theorist (Newell & Simon, 1956). This program could prove math theorems, not by brute force, but by using symbolic logic, kind of mimicking how humans solve problems. It was a clear sign that machines could actually do some form of abstract reasoning if instructed correctly.

Game-playing also became a hot area for symbolic AI research. The popularization of the Minimax algorithm in board games like chess and checkers enabled early computers to play optimally by exploring all possible moves.

These were the days of automatic reasoners and early forms of knowledge representation, with projects like the General Problem Solver aiming to tackle all formally decidable math problems. The idea was simple, yet powerful: if we could just write down all the rules, the machine would be smart enough to solve them.

It was also during this time that a seemingly simple program really captured people's imaginations: ELIZA, the first chatbot. Developed by Joseph Weizenbaum in the mid-1960s, ELIZA was a barebones linguistic interface designed to pretend to be a Rogerian psychotherapist. It worked by using simple pattern matching and rule-based responses, often just turning what you said into a question ("You say you are sad. What makes you say you are sad?").

Even though it was incredibly simple, many users found themselves opening up to ELIZA, believing it actually understood and empathized with them. This phenomenon became known as the ELIZA effect, a powerful reminder of how easily we humans tend to see human qualities in technology. ELIZA, despite being a purely symbolic, rule-driven system, sparked a persistent dream in the AI world: the quest for truly conversational AI, for machines that could talk with us naturally. This dream, born from simple rules, would keep pushing AI's boundaries for decades.

But even with symbolic AI leading the way, a different kind of idea was quietly taking root: connectionism, an early form of statistical AI. This approach drew inspiration from biology, specifically how neurons, despite their simplicity, could become exceedingly intelligent when connected in just the right way. The Perceptron, introduced by Frank Rosenblatt, was an early artificial neural network built to learn patterns directly from data.

The initial excitement was huge; these "learning machines" seemed to offer a path to intelligence without needing every single rule programmed explicitly. Imagine, a machine that could learn just by seeing examples, like a human brain! This perspective leans heavily into the empiricist tradition, where knowledge is gained through sensory experience and and data.

However, the honeymoon period didn't stick around. Both approaches ran into big problems and ultimately failed to scale beyond toy problems.

Early symbolic systems, while impressive in their specific areas, turned out to be quite brittle. They struggled with common-sense knowledge and couldn't easily adapt to new situations outside their carefully programmed rules. Trying to teach a machine absolutely everything it needed to know, one fact at a time, was an insurmountable challenge.

Meanwhile, perceptrons hit their own walls. Marvin Minsky and Seymour Papert's 1969 book, Perceptrons, famously pointed out their inability to adequately represent even the simplest nonlinear relationships in training data. They couldn't become complex enough, no matter how many neurons you connected.

This period became known as the First AI Winter: a big drop in funding and public interest as those initial grand promises didn't pan out. And this early struggle between explicit rule-based systems and pattern-based approaches set the stage for the dynamic tension that would define AI's whole history.

The Knowledge Era (1970s - Mid 1990s)

Coming out of the first winter, AI didn't vanish; it just regrouped, with symbolic methods making a strong comeback. The 1970s and 80s saw the development and initial commercial success of expert systems. These were AI programs designed to mimic how a human expert makes decisions in a very specific, narrow field.

Examples of these are systems like MYCIN, which helped diagnose blood infections, or XCON, which configured computer systems.

The main focus here was on capturing human expertise for a specific area and representing that knowledge using handcrafted rules and facts. Imagine writing down a comprehensive set of rules for medical diagnosis, for example. All the possible questions and follow ups, and all consequences of the possible answers. These systems used sophisticated inference engines (basically, automatic reasoners) to apply those rules and draw conclusions.

This era was the pinnacle of the rationalist approach to AI, aiming to formalize and apply human expertise through clear logical structures.

While expert systems grabbed the headlines, research into neural networks quietly kept going. A major algorithmic breakthrough during this time was the popularization of backpropagation (thanks to Rumelhart, Hinton, and Williams in the mid-1980s). This algorithm finally gave us an efficient way to train multi-layered neural networks, letting them learn much more complex patterns, thus breaking free of their primary limitation.

Just as expert systems hit their peak, their own limitations became painfully obvious. They were super expensive to build and maintain, needing human experts to painstakingly put in all their knowledge. They were also incredibly brittle, like all purely symbolic approaches; even a tiny change outside their programmed domain could break them entirely.

And so, the Second AI Winter arrived. This time, it was clear that while symbolic AI had done impressive things, it just couldn't scale to the complexity of the real world. At the same time, statistical AI wasn't really working yet, as the available data and computational infrastructure was insufficient. But this was about to change.

The Internet Era (Late 1990s - Early 2010s)

The internet changed absolutely everything, and AI wasn't the exception. Suddenly, data was everywhere, and statistical approaches were perfectly positioned to take advantage.

The huge growth of the internet in the late 1990s and early 2000s, combined with more and more computing power, led to an unprecedented explosion of digital data. Every click, every search, every photo uploaded contributed to a massive ocean of information. This Big Data was the fuel that statistical AI had been waiting for.

With tons of data available, statistical machine learning algorithms devised decades earlier really suddenly started to work. Techniques like support vector machines (SVMs), decision trees, and ensemble methods became extremely popular. They weren't just theoretical curiosities anymore; they were powering real-world applications. Search engines used them to rank billions of web pages, email providers deployed them to filter millions of spam messages, and e-commerce sites implemented them to recommend thousands of products to millions of users.

These were all problems perfectly suited for statistical machine learning, which could find subtle patterns in huge datasets without needing explicit rules for every single situation. It just need massive data and computational resources to work, and now we had both, in excess.

But symbolic AI didn't disappear. While statistical methods took center stage, symbolic approaches found new roles. The popularization of domain-specific ontologies (formal ways to define concepts and relationships) gave rise to the ideal of semantic web (an interconnected network of different data sources) which provided ways to structure and link information in ever growing knowledge bases, in a completely distributed and emergent process.

While the promise of a fully inter-connected semantic web hasn’t exactly panned out yet (and might never will), the underlying notion of organizing the world’s knowledge into networks of concepts stuck. These symbolic tools often worked alongside statistical methods, giving structured data that machine learning algorithms could then use, or making the results of statistical methods easier to understand. The Google Knowledge Graph is a prime example of this interplay between statistical and symbolic methods. It allowed Google to absolutely dominate the search industry for more than a decade and counting.

It was also during this period that the widespread use of AI in recommendation systems (like YouTube and Twitter) using purely algorithmic feeds started to show some of the earliest downsides of AI. While these systems were designed to personalize experiences and filter information, they also began creating filter bubbles and echo chambers.

The algorithms, often reflecting biases already in their training data, could also subtly strengthen existing prejudices or even be used to spread misinformation really fast. This early peek into AI's societal impact highlighted that even seemingly harmless applications could have big, sometimes negative, effects on how humans think and interact, setting the stage for the more complicated ethical discussions we have today.

The Deep Learning Era (Mid 2010s - Early 2020s)

If the Internet Era was just the warm-up, the mid-2010s brought the main event: Deep Learning. This wasn't just a step forward; it was a giant leap. The first big breakthroughs came from training much deeper neural networks than anyone thought possible before, often using clever tricks like layer-wise unsupervised pre-training.

The turning point arrived in 2012 with the ImageNet Large Scale Visual Recognition Challenge. A team led by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton entered a deep convolutional neural network called AlexNet. Its performance was jaw-dropping, totally blowing away all previous attempts.

AlexNet showed off the immense power of Convolutional Neural Networks (CNNs) for recognizing images. It proved that deep neural networks, given enough data and computing power, could learn incredibly complex features. This was a huge win for the empiricist approach, demonstrating that some form of intelligence could eventually pop up from massive data and complex, learned patterns, rather than needing explicit programming.

From that point on, deep learning just exploded. Quick advancements led to super sophisticated deep architectures like ResNets and Inception, plus crucial innovations like attention mechanisms, which let models focus on the important parts of the input. Deep learning quickly spread beyond just computer vision. It totally changed natural language processing, speech recognition, and even reinforcement learning.

Remember when AlphaGo, a Google DeepMind AI, beat the world's best Go players? That wasn't just a symbolic search algorithm; it combined Monte Carlo Tree Search (MCTS) with deep learning. It was a powerful statistical approach that could efficiently explore the huge and complex game spaces of Go, where traditional symbolic search (like Minimax) was simply intractable. It was a clear demonstration of statistical AI's ability to tackle problems previously considered beyond reach.

This period of fast progress also brought a critical realization, often called The Bitter Lesson. Coined by Rich Sutton, the Bitter Lesson basically says that over the long haul, general methods that really lean into computation (like just making neural networks bigger and feeding them more data and processing power) tend to be more effective and robust than trying to build in human knowledge or super detailed, hand-crafted features.

While this insight powerfully highlighted the benefits of huge, data-driven learning (the empiricist path), it's important to get that it's not a total dismissal of human understanding. Instead, it suggests that how human knowledge gets integrated matters—less about rigid, fixed rules, and more about creating architectures and environments where learning algorithms can discover patterns and rules for themselves, often at scales beyond what any human could intuitively grasp. This solidified the move towards data-driven, statistical approaches, showing that raw computing power and general learning algorithms were often the real keys to unlocking more advanced AI capabilities.

Then came 2017, and with it, the Transformer architecture. This was a game-changer for Natural Language Processing (NLP), mainly because of its innovative attention mechanism, that allowed ML models to process whole chunks of text way more efficiently to understand long-range connections. This paved the way for the rise of Large Language Models (LLMs) like BERT and early GPT versions, which started showing an uncanny ability to understand and generate human-like text.

None of this would have been possible without the absolutely massive development of hardware. The exponential growth in computing power and the development of specialized hardware like Graphics Processing Units (GPUs) and later Tensor Processing Units (TPUs) have completely changed the landscape that made statistical AI fail in the 80s. These advancements allowed for the huge parallel processing needed to train deep neural networks on truly enormous datasets, unlocking their true potential.

Finally, the explosion of open-source frameworks like TensorFlow and PyTorch, along with platforms like Hugging Face that shared pre-trained models, dramatically sped up deep learning research and adoption. This has fostered a truly collaborative global community, letting innovations spread fast and build on each other. It was a collective effort that truly launched AI into its current era.

The Generative Era (Early 2020s - Present)

And that brings us to today. The early 2020s have ushered in an era that has captivated the public like nothing else. In 2022, ChatGPT burst onto the scene, quickly followed by other groundbreaking generative models like DALL-E and Midjourney. These models can create novel and mostly coherent content across all sorts of formats: text, images, code, audio, and even video. Suddenly, AI isn't just analyzing or predicting; it is creating, as Lady Ada envisioned—and the implications of it are still unfolding. This shift has put large-scale generative models squarely in the spotlight, forcing us to rethink what machines are truly capable of.

This era also brings us full circle to that early dream sparked by ELIZA. Remember that simple, rule-driven chatbot from the 1960s? While ELIZA relied on hand-coded patterns and clever tricks to fake conversation, ChatGPT works on a totally different scale and principle. It's a purely statistical marvel, having learned the ins and outs of human language from massive amounts of data, rather than explicit rules. ChatGPT, and its generative cousins, represent a stunning realization of that long-held dream of conversational AI, really pushing the boundaries of what we thought was possible. It's a testament to how far we've come, from basic, rule-based chatbots to incredibly fluent, statistically driven ones.

But here's where the historical pendulum swings again, with a cool twist. While these statistical deep learning models achieve incredible scale and performance, they also show some inherent limitations. They can hallucinate facts, struggle with real common-sense reasoning, and often lack explainability. You might ask them why they made a certain decision, and they can't always tell you in a way that makes sense.

These limitations have sparked a renewed interest in neuro-symbolic AI. This isn't about picking sides anymore; it's all about integration. This emerging field aims to combine the best of both worlds: the pattern-recognition power of statistical models with the logical reasoning and structured knowledge of symbolic AI. Imagine using ontologies to "ground" a large language model's outputs, making sure its generated text sticks to factual consistency, or adding logical rules to make AI systems more robust, reliable, and easier to understand.

The historical struggle between symbolic and statistical AI is evolving into a quest for effective synthesis, aiming to combine the strengths of both paradigms to create something truly greater than the sum of its parts.

Conclusion

We've journeyed through decades of ambition, breakthroughs, and tough realizations. What we've seen is a constant back-and-forth, a dynamic dance between two powerful ideas: the precise, rule-based, inflexible logic of symbolic AI and the adaptable, pattern-based, unreliable power of statistical AI. This dance, as we've explored, often mirrors the philosophical tension between rationalism and empiricism.

Today, AI stands at a fascinating crossroads. While purely statistical systems have achieved incredible feats, especially in areas like conversational AI—where the dream that began with ELIZA now thrives in ChatGPT—, their inherent limitations are becoming clearer.

This brings us to a crucial realization: the future of AI likely isn't about one approach winning out over the other, but about intelligently combining them. Hybrid approaches, particularly neuro-symbolic AI, hold immense potential.

However, as we push the boundaries of what AI can do, we also have to face the serious challenges and ethical questions that come with it. The sheer power of these systems brings risks, from spreading misinformation and amplifying societal biases (which are often baked into their training data) to more complex issues around accountability and even the long-term, existential implications of creating truly autonomous and superintelligent entities.

By integrating symbolic reasoning and structured knowledge with the power of deep learning, we can build AI systems that are not only smart but also robust, explainable, and truly capable of common-sense reasoning, all while carefully navigating these potential problems.

The history of Artificial Intelligence is far from finished. AI is a living, breathing, civilization-wide project that encompasses all human endeavors, with the potential to transform society for the better—or, some believe, to become our ultimate doom. Everyone has a place here: technologists, yes, but also humanists, economists, historians, artists, politicians… The next few years, if anything, promise to be extremely exciting, and you can be a part of it.

Unless you've been living under a digital rock for the last three years, you've almost certainly heard the whispers, or perhaps the shouts, that Artificial General Intelligence (AGI) is almost here. This idealized form of AI, often pitched as the ultimate problem-solver, promises to revolutionize everything —from curing diseases to managing global economies, while also improving your coffee and doing your dishes—making our current AI advancements look like child's play. While predictions on its exact arrival diverge wildly, there's a palpable feeling that we might be on the brink of a major milestone for humanity: the emergence of truly intelligent AIs that can perform anywhere humans can, equally or even better than human experts.

And while those grand visions have their place in sparking imagination, I want to propose a more grounded, perhaps more pragmatic, path forward. My thesis here is simple: while the pursuit of AGI remains a fascinating and worthy long-term aspiration, a clearer, more achievable, and ultimately more impactful goal for our immediate future lies in what I call "General-Purpose AI" (GPAI). Think of GPAI not as the ultimate destination, but as a vital compass, guiding us toward meaningful AI development and real-world impact right now.

In this article, I'll delve into why the singular focus on AGI can be a distraction, define what I mean by General-Purpose AI, break down its essential components, and outline a realistic roadmap for how we can get there. My aim is to shift our collective gaze towards a more actionable and universally beneficial future for AI.

Challenges in Defining and Achieving AGI

Let's be honest, the concept of AGI, as it's often discussed, is at least somewhat problematic.

One of the biggest hurdles for achieving AGI is its sheer vagueness. What does it really mean to achieve "human-level intelligence across all relevant domains"? The definitions are often slippery, lacking the clarity needed for a concrete engineering goal. It frequently leans into those fantastical, almost mystical, connotations of sentience, consciousness, and an all-encompassing intelligence. Because of this, proving definitively that AGI has been "achieved" becomes incredibly difficult, almost unquantifiable.

Furthermore, a significant challenge lies in the lack of clear measurability. How would we even know when we've truly arrived at AGI? What are the metrics, the benchmarks that would declare its presence? This contrasts sharply with traditional software development, where goals are typically practical and eminently measurable. Without clear, agreed-upon ways to measure progress or success, AGI risks remaining an elusive phantom.

Most critically, the singular focus on AGI can inadvertently pull our resources and attention away from the tangible. It can divert brilliant minds and significant investment from impactful AI applications that are not only achievable but are already solving real problems today. This pursuit of a distant ideal risks creating unrealistic expectations and, potentially, widespread disillusionment if those grand promises don't materialize in the near term.

For me, the emphasis should be on useful AI, the kind that drives genuine innovation and addresses pressing challenges in our world right now. This is where GPAI offers a more grounded and actionable target. It's about building tools that work, that empower, and that deliver measurable value, without getting lost in the theoretical infinite.

What is General-Purpose AI (GPAI)?

So, if AGI is the abstract ideal, what then is GPAI?

The core idea behind GPAI is a fundamental shift in how we perceive AI. Instead of seeing AI primarily as a finished, consumer-facing application, I view it as a foundational set of tools, capabilities, and infrastructure. Think of it like electricity or the internet, not an end in itself, but a pervasive utility that empowers countless other creations and innovations. This infrastructural view brings immense benefits like scalability, ubiquity, and incredibly broad applicability across diverse domains.

Crucially, GPAI isn't primarily designed for the end-user. Its true audience is the developer. GPAI is about empowering developers to effortlessly integrate sophisticated AI capabilities into their own applications. This democratizes AI, enabling a new generation of problem-solvers to leverage powerful AI tools without necessarily needing to be deep AI experts themselves. It's about bringing AI out of the specialist's lab and into the everyday toolkit of every software engineer.

Let me try then to define GPAI more concretely:

GPAI is a state of the AI infrastructure, tools, and techniques, that enables any developer to incorporate narrow AI for any specific problem on any specific domain, within any reasonable set of infrastructure constraints.

To put it more succinctly, GPAI aims for "anything, anywhere, anytime," in contrast to AGI's often implied "everything, everywhere, all at once". It's about practical, pervasive utility rather than an all-encompassing, perhaps unattainable, general intelligence.

Unpacking GPAI

So, what are the core ingredients that I think will make GPAI a reality?

1- Foundation Models

At the heart of GPAI are what we call Foundation Models. These are vast collections of pre-trained models, already capable of understanding and generating content across various modalities. They are essential because they provide powerful, ready-to-use intelligence, significantly reducing the need for developers to train models from scratch for every new application.

We're already seeing this with text, audio, and image models. But for true GPAI, we need a crucial extension into more challenging modalities: think 3D scenes with realistic physical interactions, complex tabular and relational data, time series, and general graph-like structures. Expanding foundational models into these areas will enable developers to tackle an even more diverse set of problems without requiring deep, specialized AI expertise for each domain.

2- Comprehensive Interoperability

For AI to become truly infrastructural, it needs to speak the same language as everything else in our digital world. This is where standardized communication methods, like MCP and alternatives, become critical. These protocols will allow AI systems to connect seamlessly with traditional software components.

This is about breaking down the silos that currently exist between AI and our existing IT infrastructure. It's about enabling AI features to be integrated just as easily into legacy systems as they are into cutting-edge modern applications. Imagine AI that can effortlessly plug into databases, operating systems, file systems, and web browsers. It also means ensuring different types of AI models—generative, discriminative, pre-trained, custom-trained—can work together harmoniously.

3- Simplified AutoML

Building custom AI solutions often requires specialized knowledge in machine learning. GPAI will change that by making the training and fine-tuning of narrow, task-specific ML models astonishingly simple, directly integrated into standard development environments.

The goal here is to drastically reduce the complexity and specialized knowledge typically required for model development. We want to make custom AI solutions accessible to a vastly wider base of developers. Picture configuring a machine learning model with the same ease as setting up a database or a Docker runtime. This will involve heavy integration of AutoML techniques —automated model selection, hyperparameter tuning, and the like— directly into our IDEs. The aspiration is to shift ML from a niche, specialist domain to a standard, everyday development task for any programmer.

4- Accessible GOFAI

While modern machine learning is transformative, we shouldn't forget the rich history of AI. GPAI will also include tools that expose classic symbolic AI methods, often referred to as Good Old-Fashioned AI (GOFAI), such as constraint satisfaction and search algorithms, all with modern ease of use.

Many real-world problems—think scheduling, logistics, or complex planning—are remarkably well-suited for these time-tested techniques. Providing these simply and effectively adds a powerful, complementary set of tools to the developer's arsenal. It’s about integrating these methods with the same configuration simplicity as other components, ensuring developers have a comprehensive toolkit that combines the best of both modern and classical AI approaches.

5- Pervasive Hardware/OS Support

Finally, for AI to truly be "anywhere, anytime," GPAI systems must be incredibly robust operationally. This means they need to run effectively under a vast array of circumstances, even offline or on mobile devices.

This implies robust hardware support for in-service training and inference of traditional models directly on devices, and at least the ability to perform inference of sufficiently good generative models on those devices. This on-device capability is absolutely essential for AI to become a truly ubiquitous utility, woven into the fabric of our digital lives.

How We Get There

Achieving GPAI won't be a single, monolithic event. It will be a journey with several overlapping phases.

It's important to understand that these phases won't unfold one after another in a strict sequence. Instead, they will run concurrently, with progress in each area accelerating at varying speeds. We should also acknowledge upfront that some challenges, particularly in extending foundation models to very specific and complex modalities, will naturally take longer to solve than others.

Phase 1: Pervasive Protocol Integration and Tooling Development: This initial phase focuses on laying the foundational plumbing. We'll see the widespread development of protocols like MCP designed to seamlessly integrate AI systems with all the traditional boundaries of software—databases, operating systems, file systems, and web browsers. This is about creating the conduits through which AI can flow freely. This is ongoing as we speak.

Phase 2: Comprehensive Framework and Tooling Accessibility: Concurrent with the plumbing, this phase is about creating the robust AI frameworks and tools themselves. These will be accessible across most programming languages, encompassing both modern machine learning techniques and the classical GOFAI methods like constraint satisfaction and search. The emphasis here is on simplifying complex AI tasks, making them approachable for the broader developer community. This is also ongoing in some domains, but we still lack developer-friendly tooling for the most common GOFAI techniques and traditional ML.

Phase 3: Extension of Foundation Models to Challenging Modalities: As the frameworks mature, we'll see a significant push to advance foundation models beyond common text and image data. This phase will focus on effectively processing and reasoning with more complex data types such as 3D spaces, tabular data, time series, and graphs. This tackles the "less-solved" domains, opening up new frontiers for AI application. We’re seeing right now an increased interested in World Models that can reason in terms of the physical properties of objects and better predict their interactions. We also need more work in models that can understand heavily structured data.

Phase 4: Mainstreaming of AutoML Techniques: This phase will mark the maturation of AutoML. These techniques, currently more prevalent in academic or specialized domains, will become sufficiently developed and integrated to move into widespread, mainstream development practices. This means that optimizing and selecting machine learning models will become a standard, almost automated, part of the development workflow. We already have pretty strong AutoML frameworks, but rarely any developer knows them, and they are lagging behind in integration (think AutoML running in your CI/CD) and deployment.

Phase 5: Ubiquitous On-Device AI Execution: Finally, a truly pervasive GPAI requires that AI can run almost anywhere. This phase is about ensuring that virtually any device can execute a "good enough" foundation model. Crucially, this will involve establishing robust operating system-level support for on-device AI inference, making local AI development and deployment a standard, seamless experience. This is already happening, but I think we need some breakthrough in hardware capabilities to make running a somewhat capable model, similar to what a 24B MoE can deliver today, natively in Android or iOS.

Given these phases and the inherent complexities, I believe we're looking at a realistic timeframe of 5 to 10 years for GPAI to reach a point where any developer on Earth can easily integrate AI into any reasonably complex application, much like they currently add a database or a graphical user interface today.

It believe this isn't science fiction. It’s ambitious, yes, but also a pragmatic and achievable goal.

Final Thoughts

Focusing on GPAI offers immense, tangible benefits: it promises genuine, measurable progress, the democratization of AI development, and ultimately, a much wider societal impact. It grounds our ambitions in what is actionable and truly useful, contrasting sharply with the often abstract and distant nature of AGI.

This isn't to say that the pursuit of AGI isn't worthy – it absolutely is, as a grand, long-term scientific endeavor. But while AGI might be the distant star we navigate by, GPAI is an immediate, well-lit path beneath our feet, a compass guiding us to real-world impact.

I'm personally committed to these ideas and to helping develop this vision. If you're interested in collaborating, please feel free to send me a direct message or leave a comment below. Let's build this future together.

Donald Knuth, the father of algorithm design and analysis and one of the most impactful, insightful, and influential people in Computer Science, said science is what we understand well enough to explain to a computer, and art is everything else we do. In his view, computer programming is a form of art, and he is undoubtedly a masterful artist.

Whether you subscribe to this broad notion of "art" or not, you won't deny there are a handful of ideas in Computer Science that are so elegant, creative, innovative, powerful, or just downright beautiful that they at least blur the boundary between science and art.

In this post, I want to kick off a new series exploring the idea that some algorithms are inherently beautiful, regardless of their technical details. I will suggest what I consider the most beautiful algorithms ever designed and give you a short rationale for their selection. Then, in future posts, I will discuss each of these in detail and, hopefully, convince you that they are, indeed, beautiful.

Subscribe now

What makes an algorithm beautiful

Many believe, myself included, that beauty is in the eye of the beholder. Thus, deciding what the most beautiful algorithms ever are is a highly personal and subjective endeavour. Still, I want to justify my personal criteria for selecting some algorithms among others. You may agree to some extent with these points or disagree entirely, but I hope you'll at least understand where I'm coming from when I say some algorithms are beautiful.

First and foremost, I value elegance. Anyone can solve a complicated problem with a similarly complicated solution, full of special cases and tricks. But there is beauty in finding a simple, clear, elegant solution that happens to be as effective or more than anything else you can think of.

Second, I value cleverness. Simple ideas, yes, but also powerful ideas that exploit some hidden insight or reveal something unique about the problem they're solving. Bonus points if they involve clever mathematics.

Finally, I also consider the impact of these algorithms. Many of the most beautiful algorithms are broadly applicable and unlock entire branches of knowledge that were previously hidden.

However, there are also algorithms so unique and weird that they deserve special mention, if only because they show how far the human intellect can go to find solutions to interesting problems. I have a few of these.

The List of Beautiful Algorithms

Based on the above criteria, I created a long list of candidate algorithms. Then, I narrowed it down to 16 because it is a beautiful but still manageable number. Of course, this limitation is totally arbitrary, but so is everything else in this article.

When deciding what to include, I also prioritized diversity. For example, I left out many beautiful graph algorithms, not necessarily because the other algorithms in this list are more beautiful. We could fill the whole list with clever ideas from graph theory, and I preferred to go wide rather than deep here.

All of these algorithms have some unique, clever insight that makes them work. All of them are paradigmatic of a particular technique for algorithm design. In general, I think they cover a broad spectrum of the type of reasoning involved in crafting algorithms, and I hope they'll give you a good taste of what it means to be creative in computer science.

So, without further ado, here is my list of the most beautiful algorithms ever designed. Except for the first one, this list is in no particular order.

Binary Search

If there is an algorithm that should be number one in every list of beautiful algorithms, that is Binary Search. It is the simplest, most clever algorithm ever designed. It deals with the most straightforward version of the most important problem in computer science: search. And it does so in the most marvellous and efficient possible way.

Binary Search is the first serious algorithm you will probably ever learn in a CS major, and the first one that does something truly clever: by optimally exploiting the structure of the data, Binary Search manages to find any element in a very, very large collection in the blink of an eye. It is so fast that if you had to search the whole universe (all  10^80 or so particles in it), you would need no more than 300 operations—probably much less than a millisecond in any modern computer.

To achieve this, Binary Search requires only one key assumption: the data must be sorted. If that's the case, Binary Search is perfect. It prunes the search space in a way that can be mathematically proven optimally efficient—in the sense that it makes the most efficient use of the information available in the ordering of the search space. It is impossible to be any faster when all you can rely on is the order of elements.

Binary Search teaches us something profound about the nature of search and problem-solving in general: it teaches us that structure matters a lot. If you can organize the search space in a meaningful way, you can search through it in a breeze. In a sense, this is the most crucial insight in computer science: all computer science problems are ultimately search problems, and all clever solutions to search problems are, in a sense, finding ways to organize the search space for effective pruning.

QuickSort

If search is the most important problem in CS, then sorting is probably the most well-studied one. Among the many flavours of sorting, the simplest incarnation is sorting a finite list of numbers (or comparable items in general). And here, QuickSort is undoubtedly the king of the pack.

QuickSort is not only among the fastest general-purpose sorting algorithms—its name alone says so—but it is also probably the most elegant. It cleverly uses randomization to ensure that, on average, we make the optimal selection that leads to the fastest possible sorting strategy.

Quicksort is the simplest example of a clever use of randomization aimed not at producing an approximate result but rather at making an algorithm robust to all possible inputs. The algorithm's balance of speed and simplicity epitomizes beauty in design and demonstrates the harmony between theoretical elegance and practical application.

A* Search

A* is an extension to the most beautiful graph algorithm—Dijkstra’s shortest path—to incorporate domain knowledge as a heuristic function. A* achieves extraordinary efficiency by balancing the cost to reach a node (what we know for sure) with a heuristic estimate of the cost to reach the goal (what we can guess). This synergy allows it to explore pathways intelligently, often finding optimal solutions with significantly fewer computations than uninformed methods.

Its adaptability makes A* valuable in theoretical contexts and real-world applications, such as route planning in navigation systems and AI for games, where understanding and optimizing paths is critical.

This algorithm beautifully balances exploration and exploitation, always finding the shortest path while minimizing cost. It is a prime example of the combination of formal and heuristic thinking.

RSA

RSA is a paradigmatic algorithm in public-key cryptography. It is the first algorithm to achieve something that intuitively seems paradoxical: sending a message through a public channel to another party that is impossible for anyone except them to decipher, even though everyone else knows all the details of the communication protocol and can see all messages.

RSA relies on the difficulty of factoring prime numbers, which for a very long time was thought to be a pure mathematical pursuit devoid of any practical applications. But then cryptography came with a simple but very powerful insight: that some functions are easy to compute but almost impossible to invert—aptly called one-way functions.

RSA was the first algorithm to show how one could build truly unhackable systems using clever math (at least before quantum computing was a thing). Nowadays, stronger cryptographic frameworks work even under the threat of ubiquitous and cheap quantum computing. But all of them owe their birth to the unique insights unlocked by RSA.

Fast Fourier Transform

The Fourier transform is undoubtedly the most important operation in signal processing. It allows the decomposition of any signal into its individual frequency components, and it has uncountable applications, from compression to pattern recognition to synthesis.

However, for a very long time, it was considered an intrinsically expensive operation, applicable only to shorter sequences. The FFT is a revolutionary algorithm that transforms signals from their original domain to the frequency domain, allowing for efficient data analysis and processing. It's hard to oversell its importance; for many, it's the single most impactful algorithm ever designed.

FFT is instrumental in numerous applications, including digital signal processing, image analysis, and solving partial differential equations. Its profound impact on both mathematics and engineering underscores its elegance, showcasing how computational efficiency can radically enhance data interpretation and manipulation.

K-means

K-means is a straightforward but powerful clustering algorithm that partitions data into distinct groups. It answers how to classify different objects when you know almost nothing about them. The key insight is that similar objects should be bundled together and separated from other bundles of similar objects.

Despite its simplicity, K-means can provide insights into the underlying structure of complex information. When K-means is not enough (because the data is too complex), we often resort to methods that, deep down, are basically a transformation of the data followed by a K-means-like approach.

KMP

KMP, or the Knuth-Morris-Pratt algorithm, is a highly efficient string-searching method that finds the occurrences of a pattern within a text in linear time. The clever idea in KMP is that past searches, even failed ones, provide information about future searches. By preprocessing the pattern to create a partial match table, KMP optimally skips unnecessary comparisons, enhancing the searching process.

This clever design mitigates the performance issues seen in naive approaches, making it particularly suitable for applications involving large datasets such as text editing, data parsing, and bioinformatics.

Minimax

Minimax is a fundamental algorithm used in decision-making and game theory, particularly in zero-sum games. It systematically evaluates possible moves by minimizing the possible loss in a worst-case scenario. Minimax answers the question of what one should do to behave optimally in a sequence of decisions, assuming everyone else is also behaving optimally. This strategic approach not only aids in deriving the optimal move for a player but also prevents adverse outcomes against an opponent’s best strategy.

Despite its power, Minimax stands out for its simplicity. It is the backbone of many AI implementations in competitive gaming contexts. Although it is no longer the algorithm of choice for practical implementations of optimal AI players, its theoretical properties still inform the design of every decision-making algorithm in adversarial environments.

Monte Carlo methods

Monte Carlo methods are a family of extremely powerful statistical techniques used to understand and quantify uncertainty in prediction and forecasting models. The purpose is to find some optimal solution, e.g., an optimal strategy in a sequence of decisions that involve uncertainty or inherent randomness.

Monte Carlo exploits the intuitive idea of random sampling but with a clever twist: as you obtain more information about some specific outcome, the uncertainty about it decreases. By balancing the expected potential of each outcome with its uncertainty, Monte Carlo efficiently finds the best solutions while minimizing wasted effort by exploring unpromising ones.

It is hard to overstate the importance of Monte Carlo. Its many variants are widely applied anywhere from finance and engineering to computer science, AI, and physical sciences, allowing analysts to model scenarios that would be infeasible to compute deterministically. Its versatility and capacity for simulating the unpredictable make it an invaluable tool for decision-making.

Genetic Algorithms

Genetic Algorithms (GA) are a family of general-purpose optimization strategies inspired by the principles of natural selection and genetics. They employ mechanisms such as selection, crossover, and mutation to evolve solutions over time. This adaptive search technique is incredibly effective for optimization problems, exploring a vast solution space through iterative improvements.

GAs are used everywhere we find complex optimization problems: in engineering, economics, or artificial intelligence, just to mention a few cases. Their ability to emulate evolutionary processes reveals a profound connection between biological principles and computational strategies, underscoring their elegance and utility.

Raytracing

Raytracing is the quintessential computer graphics algorithm. It's a straightforward approximation of the rendering equation that can simulate how light interacts with different objects to create photorealistic images. Raytracing is a recursive algorithm that follows light rays as they bounce through a virtual scene and are reflected, refracted, or absorbed.

Every time you see a CGI effect in a Hollywood movie, whether live-action or animated, Raytracing is working behind the scenes. Modern rendering algorithms are significantly more complex but ultimately based on the same principles.

SVD

SVD, or Singular Value Decomposition, is easily the most important linear algebra algorithm for data representation and analysis. At its core, SVD is about finding an optimal representation of the values in a matrix that preserves as much information as possible. When used on real data, SVD reveals intrinsic structures that enable processes like noise reduction and feature extraction.

This method is critical in machine learning applications like image classification and natural language processing. Its ability to simplify complex datasets highlights its significance, demonstrating how advanced mathematical techniques can lead to practical solutions in various applications.

Deflate

Deflate is a widely used compression algorithm that reduces data size to optimize storage and transmission. It cleverly combines the two most paradigmatic approaches to data compression: LZ77 (pattern-based) and Huffman (information-theoretic) coding.

Deflate is the underlying algorithm for many widely used file formats, including ZIP files and PNG images, where maintaining data integrity during compression is vital. The algorithm's ability to balance efficiency and compression ratio makes it a valuable tool in applications ranging from software distribution to data archiving.

HyperLogLog

HyperLogLog is the kind of algorithm that looks like magic. It solves what seems like an unsurmountable problem with a genuinely clever idea. The purpose of HyperLogLog is to allow very accurate estimates of the total number of items in a very, very large dataset when items are added asynchronously in real time. Think about counting all likes in a hugely viral YouTube video.

Using hash functions and a clever data summarising technique, HyperLogLog maintains a minimal memory footprint while providing accurate estimates of unique elements. Anywhere you see massively distributed databases, you can probably find HyperLogLog or a variant of it underneath the most basic counting operations.

PageRank

PageRank is the algorithm that propelled Google to dominate the online search market. It assigns a numerical score to each page, reflecting its importance based on the number and quality of links pointing to it. The quirk is that the Internet is huge, so effectively computing this value for all web pages is unfeasible.

PageRank builds a probabilistic model that estimates the likelihood of a user landing on a particular page. It uses clever numerical approximation techniques to avoid processing the entire Internet graph every time a new web page is added or removed. Beyond search, PageRank is useful for estimating the importance of individual nodes in any type of network, such as social media or knowledge graphs, so it is also widely used in research.

Backpropagation

Backpropagation is probably the most important algorithm of the AI revolution. It is the algorithm that allows the training of artificial neural networks efficiently by computing gradients of the loss function with respect to the model's weights and propagating errors back through the network layers.

Its discovery in the 80s allowed for the first time to train arbitrarily deep neural networks, which are behind most, if not all, modern AI applications, including image recognition, natural language understanding and generation, and even reinforcement learning.

Closing remarks

And there you go. Again, keep in mind this is just my personal selection, and many of you can and most likely will disagree with me on some or maybe all of them--well, except Binary Search, that's a hill I'm willing to die on!

In future posts, I will explore each of these algorithms, explaining not only how they work in the most intuitive terms possible but also highlighting what makes them stand out as some of the most beautiful ideas ever conceived in computer science.

Until then, I would love to hear your thoughts on this topic. Do you think algorithms, and abstract ideas in general, can be beautiful? Why, or why not? And, if you answer yes, then what do you consider the most beautiful algorithms ever designed?

Abstraction is the fundamental ability that powers the way computer scientists think through and solve problems. It is the most important tool in the computer scientist's arsenal and often the hardest skill to master.

It is also a very powerful tool for general-purpose problem-solving. The right abstraction can often unlock unforeseen solutions to otherwise intractable problems. In some rare cases, it even allows for fundamentally novel insights only possible by thinking at the right level of abstraction.

In this article, I will explain abstraction in the context of solving problems, why it is such a powerful tool, and finally, give a concrete tip on how to improve at it.

This article is part of a series on computational thinking called How to Think like a Computer Scientist. In this series, I explore techniques from the computer scientist’s toolset and how they can be applied by anyone to solve problems in any domain.

If you want to learn more, subscribe for free to get future articles directly in your inbox.

Subscribe now

Abstraction in action

Abstraction is the process of taking a problem, situation, or concept in general, removing all irrelevant details, and leaving a simplified, more general version that captures what is truly essential to the thing being abstracted.

Abstraction is a form of purposeful simplification—but not one that stems from laziness or a lack of capacity. It's a meaningful simplification that makes a problem easier to solve but also more general and, thus, more relevant.

What is relevant or not about any given concept or situation is very contextual. Depending on the type of analysis you want, the same details may or may not be important.

That is, the why determines the what.

To see this in action, let's begin with a typical example from a typical computer science class.

An example problem

Suppose you want to solve the problem of getting as fast as possible from home to work. How could we solve this?

We first need to understand two things: what we are exactly asking for and what our decision space is, that is, what we can do.

For example, this problem is very different if you use public transportation or a personal vehicle. That is, the means by which you can move is an important detail, as it will determine your decision space. Can you travel through any street, and stop anywhere? Or do you need to pick from a predesigned set of routes and stop at one of the predetermined bus stations?

Likewise, the problem changes if we want to minimize total travelling time, fuel cost, time spent waiting at red lights, or the total number of swears thrown at cyclists.

Suppose you pick a goal: minimize total time spent on the road. Furthermore, assume you have a personal vehicle that can, in principle, travel anywhere in the city.

This is already an exercise in abstraction because we are deciding, for example, that the level of risk of a given road is irrelevant. We want the fastest route, not the safest. We could have done the opposite or somehow tried to find something in between.

The point is that deciding what matters immediately makes some otherwise important details irrelevant to this particular instance of the problem.

Making the problem tractable

Are we ready to solve the problem? Well, not yet. There are still a thousand factors that can determine how fast you can move around the city: traffic, semaphore lights, pedestrians, quality of the road... And worse, all of this changes daily, perhaps even every second! If we want to have a chance to tackle this problem at all, we'll have to make some further simplifications.

So, let's assume we can model each road segment as a straight line between intersections, with an average time cost associated. That is, we throw away all the details of a given road except for the average time it takes to traverse it. It doesn't matter how many lanes the road has or how bumpy it is. Also, it doesn't matter which time of day or year we happen to be.

This is a huge simplification, but one that makes the problem tractable with a very efficient computational algorithm—Dijkstra's algorithm, to be precise.

The details of the solution don't matter for the purpose of this article. What matters is that by simplifying this initially seemingly unsurmountable problem in a meaningful way, we obtain a tractable problem that we can solve with clever math.

Back to the real world

Now comes the real question: How well does the answer to the artificial, simplified, abstract problem we ended up solving apply to the real-world problem we started with?

The answer, of course, is that it depends. It depends mostly on how important the details we left out were. Every abstraction is a simplification, and thus, any solution to an abstract problem is, at best, an approximate solution to a real problem.

But approximations are all we can have in practice. All real-life problems are infinitely detailed if you take everything into consideration. If you want an exact answer to a real-life problem, you may need to consider the physics of the entire universe!

So, the answer to how good the approximation is, is this: as good as your abstraction is at capturing what is truly essential to the problem and throwing away what doesn't matter.

In this particular case, we'll have a pretty good solution, assuming the average times we assign to each road are really characteristic of said road and not just random values. But there is something even more valuable in what we've done, which is why abstraction is such a powerful tool.

Let's take a look at the bigger picture.

Why abstraction is so powerful

The power of abstraction can be seen in at least three different levels.

Abstraction as simplification

We've already seen the first level: abstraction makes problems easier to solve. Since abstracting is simplifying—as we are, by definition, removing details—we end up solving a problem that is, in some sense, smaller.

In our path-finding example, we simplified things a lot by deciding we didn't care for the width of the roads, the elevation of the terrain, the pavement quality, the weather, and many other details that would make the original problem impossible to solve.

In this simplified version of the problem, we can actually do some math and come up with a sensible solution. Perhaps, as in this case, we can even prove we have an optimal solution (to the simplified problem, that is). Thus, abstraction can make hard problems tractable or at least a bit easier.

Abstraction as generalization

But the second level is even more profound. By abstracting away the things that made our example problem related to cars, roads, and pedestrians—since, ultimately, those things weren't essential—we actually ended up solving a more general problem: finding a minimum-cost route in a graph.

In doing so, we not only solved our mundane go-to-work problem but also solved all the problems of going from anywhere to anywhere else with minimum cost, as long as we only cared about the cost of each connection. And since we didn't define cost in any particular way—just some number that must be minimized—we can apply this solution to time, money, fuel, and any other magnitude we care about!

Now, instead of a car in a city, think of a network of airports worldwide where you want to reach a destination by paying the minimum price. The abstract version of that problem is the same as this one if the cost of each route is the ticket price.

Likewise, we can think of minimizing transmission time in a computer network or even maximizing the probability of a message reaching a destination. All of these are instances of the minimum-cost path-finding problem we just solved!

Thus, since an abstraction is also a generalization, in another sense, you end up solving a bigger problem than you originally had and get almost for free the solution to a potentially infinite number of similar problems.

Abstraction as conceptualization

Finally, there is a third level, but you must trust me here because it's hard to give you a simple example. The thing is, by abstracting problems, we can sometimes find answers to questions that weren't even conceivable in the concrete problems.

This works because abstraction is also a form of concept creation: you create new concepts and relations that unlock new ways of thinking about things. These new concepts, in turn, let you discover new truths that were invisible before.

Some of the most important ideas in computability theory, such as the existence of specific, non-trivial, and relevant problems that cannot be solved at all, are only possible because we have very abstract notions for the concepts of algorithm, computer, and problem.

So, if abstraction is such a powerful tool, how can one get better at it?

Mastering abstraction

The short answer is that there is no shortcut, no easy formula. You get better at problem-solving in any domain by solving a lot of problems in that domain.

However, I can give you a concrete tip here. It's going to sound like a plug, but hear me out.

Abstraction is a very powerful tool for solving real-life problems in any domain. But there is one particular domain where it shines: computer programming.

The reason is that computer programming is, by definition, an exercise in abstraction. Digital computers are almost incapable of dealing with the fine-grained nature of real-life problems. Programming languages require very precise, clear instructions and definitions. So, any solution to any problem using a computer is a solution to an abstract problem.

If you want to improve at coming up with good abstractions, whatever you work on, you can start by learning to code. Getting any level of expertise in coding will force you to learn to abstract problems as a side effect.

In a sense, learning to code is like going to the gym, but for the mind.

Some people are professional athletes. Some even compete at the Olympic level in weightlifting. Some go to the gym for fun or just to show off. But almost anyone, regardless of their daily job, could benefit from a healthier body, stronger muscles, and more agility.

Likewise, some people code for work or fun. But almost anyone would benefit from thinking better, and being better at abstract thinking is one of the best ways to improve your general problem-solving skills.

This would be just about the right time to promote my programming book or course, but I won't. Truth is, you don't need me for that. Search online, pick the top 3 or 4 results, try them out, and choose the one that best fits your learning style and pace.

Now, don't get me wrong. I've said before most learning resources out there suck, but that is about learning to code at a professional level. Just like professional athletes need state-of-the-art equipment and world-class trainers, but if all you want is to get a bit healthier, buying a couple of dumbbells or even just doing half a dozen push-ups every day gets you most of the way there.

So there you have it. Learn to code. Doesn't matter which language or development stack. Just grab a coding tutorial and get used to doing a couple of programming exercises every week. That alone will make you a much better thinker in time.

Closing remarks

In a previous article, I described the main principles that guide my educational ethos. One of these principles is show, then tell. The basic premise is that explaining a complex idea is best achieved by first showing things and then explaining how they work.

Some of you asked me for examples of these principles. This article is one example of show, then tell. First, I showed you how abstraction works in a concrete example and then explained the general concept.

Actually, this article is a meta-example of show, then tell. First, I showed you how show-then-tell works, and now I'm telling you about it. Hope you enjoyed it ;)

Going back to the main topic, this article is part of a series I'm writing on how to bring the tools of computational thinking into your daily life. If you want to learn to think like a computer scientist, stick around. I'll be sharing a lot more in future articles.

Subscribe now

I've been teaching at college for close to 15 years now. In this time, I've changed a lot, as a person and educator. I've developed my own mentality about teaching, why it matters, and how to best do it.

This article attempts to organize and structure my teaching. What follows are the principles, practices, and tools I use. (Actually, this one is only about the core principles. The rest will follow soon.)

This article is very personal and does not claim anyone else should do the same. These ideas work for me, and my teaching style, and my primary reason for sharing them is to give some of you a couple of ideas you may want to try—nothing more.

Fair warning, none of what you'll read in this article is entirely original. I got most of these principles and practices from other educators I've had the privilege of knowing and learning from. Great ideas are meant to be stolen.

However, all of this is tinted and limited by the lens of my own experiences, motivations, and ideals. So, feel free to disagree with anything and everything I share in this article, and take whatever you find relevant or interesting for your practice. As usual, I'd love to know your thoughts!

Principles

These principles inform my teaching practices. I've tried to distil them down to core ideas that can be shared, but of course, there is a lot of nuance in between that got lost in the translation from lived experience to words.

In general, I think these principles may apply to education as a whole across all disciplines and levels, but my experience is almost completely limited to college-level STEM or adjacent disciplines. Needless to say, take all of this with a spoonful of salt.

To me, the most essential teaching principle, and the one that guides it all, is that...

Learning happens in an environment. Basically, there is nothing an educator can directly do to ensure learning occurs inside a student's brain. You cannot simply dump your knowledge there and call it a day. All we can do is influence the environment around the students so that learning emerges as a natural and almost inevitable result.

That environment is primarily composed of the relationships between students and teachers and among students themselves within a shared space—the school—which is why for many, the school, teachers, and students are all that matters.

However, there is much more going on that influences and may ultimately determine the quality of learning. There are parents and friends and other people around students, social media, TV, and video games, and the social and political landscape in which students live. Hell, there is even climate change and wars on the other side of the world—or across the street. You get it.

As educators, we need to be cognizant of these factors, even if we cannot directly modify many of them, so as to design learning experiences that effectively support the type of environment the students live in. Otherwise, no matter how good our lessons and books are, learning cannot happen.

For this reason, lessons need to be at least somewhat ingrained in that environment. I've found that the best way to achieve that is to...

Start with Why. Nobody learns anything if they're not paying attention; people only pay attention to things they care about. As we just said, the environment ultimately determines the outcome of any learning experience. So you have to start by convincing students they care about the stuff they're about to learn.

One way—the worst way—to do so is by invoking exams or grades. A slightly better but still awful way is to invoke some abstract future need—the very common "believe me, this will be useful when you're graduated". Even worse if that future need is just another subject this one is a prerequisite for.

A much better way is to find relevant problems students care about today and build from there—problems that are part of their environment. This is easier said than done, though. Students don't care about many of the really important problems they maybe should care about, often simply because they don't know why they should care about them.

So start with that why. Make it personal, if possible, or at least as close to their personal experience as reasonable. Not all problems can be framed in terms of the current state of society, but many times, we can find some connections, even if the thing you're actually teaching is somewhat abstract.

Don't overdo it, though. A poorly framed connection can be counterproductive because students realise this is just an artificial motivational device rather than a real problem they should care about.

But crucially, in the quest to find pragmatic motivations, don't forget to...

Ask the Big Questions. Even if we prefer to ground the motivation for learning something in personal experience as much as possible, that doesn't mean we shy away from asking big, existential questions.

Almost all practical matters can be abstracted and extended to encompass the meaning of life, metaphorically and literally. And even if we don't get to the big answer in the end, asking the big question elevates the subject and shows students that we are not only concerned with practical matters—although practical matters are often the priority.

Also, the big question can sometimes be the primary motivation. Especially in senior years, once students have learned lots of practical skills in other subjects, having at least one subject matter focused on the big questions can be a great change of pace and a chance to engage in different kinds of critical thinking and learning.

This leads me to what I think is the best order to present complex ideas, that is...

Intuition before formalization. Many classes will begin with the formalization—give you some abstract concepts and definitions someone else thought—and then try to justify why those make sense. Or worse, just take the formalization for granted and move on to solve some artificial problems.

I've found it much easier and useful to tackle a specific formalization once students already have an intuitive grasp of the concepts. Even more, we can build the formalization together, as if they were rediscovering the theory. With a little nudge from the teacher, students will often arrive at the same theory because good theories just make more sense than bad ones.

This principle can be seen as a particular case of a more general principle, that is...

Show, then tell. The original sin of modern education is the "do as I say, not as I do" model. We teach students they should behave in a way—they should think in a way—but rarely show them that behaviour ourselves.

In formal sciences, for example, we tell students that real-life problems are nuanced, that modelling is the most important part of problem-solving, and that they should think critically and creatively. Yet whenever we show them how to solve one problem, we already have a preconceived "correct" modelling and a scripted solution. They rarely see us teachers thinking critically and creatively.

I don't subscribe entirely to the anti-traditionalist ideal of “show, don't tell”, though. Educational environments are purposefully artificial because they are controlled environments where it's safe to fail. So, there is a limit to how much we can show, and we must complement it with telling. But we should embrace the mindsets we advocate as much as possible.

However, in the end, imitation is just one part of learning and not even the primary one. The best way to learn almost anything is to...

Learn by doing. Some subjects can be learned passively, but most essential skills students need are best acquired via active and deliberate practice.

This means any effective learning environment must encourage and support lots of practising. This includes both showing and telling students what to practice and, perhaps more importantly, how to practice effectively.

Plenty of studies, plus common sense, tell us that time spent practising is far less important than how one actually practices. Mindlessly solving the same type of math exercise over and over, for example, is probably a waste of time if you want to build problem-solving skills.

This last part is often forgotten, and thus, students are inundated with resources to practice—exercises, term projects, past exams—but no instructions as to how best to distribute their time or how to evaluate their practice performance. This is why frequent and personalized feedback is the most valuable thing students can get from us.

And if doing is the best way of learning, doing things in teams is even better, because...

Cooperation beats competition. Some students love competition and get a lot of energy from it, and that's good. But more often than not, students prefer cooperation.

Cooperation allows for richer interactions and is often even more challenging than competition. Managing team roles and distributing tasks in a term project is an additional skill that complements whatever technical knowledge they'll gain by just doing the project alone.

Also, cooperation allows for tackling bigger problems—not only bigger in quantitative terms but bigger in scope. Projects that involve background research aren't precisely defined and require some blue/red teaming, for example.

This doesn't mean competition is completely out of the picture. Healthy competition can be a strong motivator. However, when there's competition in my classes, I prefer that it be against an objective standard rather than against other students and that students compete in teams.

Finally, when designing team projects and evaluating work in cooperation, don't forget that...

Every student is different. Some thrive at analytical reasoning, others are better planners and organizers, and still others excel at remembering and recalling large swaths of knowledge.

While we want to form generally well-rounded professionals with a balanced set of relevant skills, this doesn't mean we must strive for, nor desire, an equal mix of abilities in all students. On the contrary, the best teams are those whose members complement each other.

Yet, formal education curricula tend to be designed for one particular type of student—the type of student who is good at gaming exams—and often leave little space to accommodate diverging needs and abilities that may also be as relevant and useful as those that are the primary goal of the educational system.

As educators, we can alleviate this narrow-minded view of schools as factories by designing learning environments that cater to varied learning styles and ensuring there is more than one valid path to success in our courses.

Moving on

This article is already quite long, so I'll leave the practices and tools for a follow-up. Those are even more personal and limited in scope, so they might not be terribly interesting to many of you. If there's interest, please let me know in the comments, and I'll prioritize that article.

In the meantime, I'd love to know your thoughts on these matters. Whether you're an educator yourself or a student (now or a while ago), I'm sure there's something we can all gain from your experience.

Until next time, stay curious.

Leave a comment

You should learn to code.

Yes, you! No, I don’t know who you are or what interests you, but I know you can and should learn to code. There are many obvious and not-so-obvious reasons why learning to code is one of the best intellectual investments you can make in yourself.

The most obvious reason is that coding is a high-paying skill, and being good at it pretty much guarantees you can find a good job at any of the million software companies hiring like crazy out there.

Yes, I’m sure you’ve heard many stories of developers struggling with coding interviews and getting rejected dozens of times. And sure, even some of the best coders out there can fail at a given time for a myriad of reasons, including bad luck.

But the thing no one tells you is how few of the people applying for a coding interview today can actually code. And it’s not their fault. Many have learned superficial coding practices from dubious YouTube instructors or cheap online blogs. But learning to code, in the sense I’m using the term, implies an entirely different level of understanding.

To be fair, there are also plenty of cases where the interviewer doesn’t understand what a good programmer looks like. Some coding interviews focus way too much on programming language trivia or algorithm design challenges, which are, at best, imperfect proxies for measuring good coding skills.

In any case, learning to code well is a very marketable skill. If you want a job in the tech industry, being a reasonably good programmer will get you halfway there.

But even if you don’t want a job in the software industry, there is a very good reason why learning to code is one of the best intellectual investments anyone can make today. The reason is that coding teaches you to think in a completely new way, making you extremely good at solving problems in all domains, even without a computer.

But to explain why, let me first tell you what coding is really about.

What is coding about

Coding is part science, part engineering, and part art. It is a science because it requires understanding and applying well-established, formally defined principles and techniques to solve complex problems. It is engineering because it requires the ability to design and build systems with complicated trade-offs. It is art because it requires—and boosts—creativity and imagination to create beautiful solutions to challenging problems.

Beautiful, right? But, in practice, what the hell does this mean? At a surface level, coding seems similar to any other technical skill, like, I don’t know, using advanced industrial machinery and tools. In a sense, coding is about using computers to do stuff, right? Why isn’t this just the same as using, say, a hammer and some nails to build a house?

For starters, yes, the computer is a kind of tool, but it is a completely different kind of tool. Most tools extend our physical capabilities or help us overcome our physical limitations. They make us stronger, faster, more precise.

The computer, on the other hand, is a tool to extend our minds. Computers help our brains do stuff they cannot do easily by themselves. They help us overcome our cognitive limitations.

But programming is far less about computers than you think. The great Edsger Dijkstra is often quoted claiming that computer science is as much about computers as astronomy is about telescopes. Computers are just a tool—an incredibly powerful tool—but the important part is not the tool you use, but what you do with it.

Programmers use computers to solve problems.

This is the deeper reason why learning to code is a different type of skill—a more general skill than it initially seems. Coding is general-purpose problem-solving. In fact, there is no problem that can be solved at all that cannot be solved with a computer—this is the Church-Turing thesis. Learning to code is, ultimately, learning to solve problems.

Why? Coding relies on one fundamental ability called abstraction, which is the process of taking a problem, situation, or concept in general and stripping it down to its core defining features, forgetting everything that is irrelevant or useless for the concrete purpose you have at hand.

And the thing is, abstraction is the most useful cognitive ability in the modern world. It is at the core of problem-solving. Any sufficiently complex problem requires you to think at several levels of abstraction and switch between them constantly. In fact, the ability to think at the right level of abstraction at any given point in the process of solving a concrete problem is probably the most critical ability to master to become a really good programmer.

Let me draw a cheap analogy here. Consider driving a car from home to work. Many things happen inside your car, including a bunch of physical and chemical processes, that are fundamental for the car to work correctly. But thinking about the combustion process going inside your engine is useless for reaching your destination—if your car is working correctly, at least. Actually, worrying about your engine's chemical reactions while driving arguably makes you a worse driver, not a better one.

Instead, when driving in the city, you often need to deal with three different levels of abstraction. The lowest level is just to make the car move: shifting gears in due time, pressing the pedals, etc. The middle level is about not crashing: avoiding incoming traffic, staying in your lane, etc. The higher level is about navigation: picking the right path to reach your destination most efficiently.

When people learn to drive, they often learn to think at the lowest level of abstraction first. They struggle to make the car move. But once they master this level, it becomes almost automatic, and they can start to think in terms of what other cars are doing and how to negotiate the road. And finally, they are able to navigate around town seemingly without effort, while casually talking about, I don’t know, programming maybe.

Similarly, in computer programming, you will often deal with three simultaneous levels of abstraction.

First, we have the code level. At this level, all you care about is expressing the solution to a problem in a language that a computer can understand. Computers are very dumb machines, you see. For all their power, you need to talk to them in very, very precise terms using a programming language. This may seem daunting at first, because it requires a very strict use of syntactic rules, but almost everyone can learn to write and read computer code, just like almost everyone can learn to accelerate, break, and turn. This is the easier task.

The second level is the algorithm level. At this level, you care about actually solving a problem by designing what is called an algorithm. In short, an algorithm is a very precise set of instructions that is guaranteed to provide a solution for a given problem. Coming up with the best algorithm for a problem is a pretty advanced skill, one that is almost always left for college-level advanced programming courses. But fear not, is one skill that can be mastered with enough practice.

The third level is the system level. At this level, you think about the design of a complex system made out of multiple interacting components, often hundreds of tiny algorithms working together to solve a big problem. Also, in this level, you have to think about the end user of your code, either a human on the other side of the display or a computer on the other side of the planet. Systems thinking is probably the hardest skill in this combo, but it can also be mastered in due time, and it is tremendously useful even outside computer programming.

In larger organizations, people tend to specialize in roles that focus most on one level of abstraction. For example, software architects might spend most of their time thinking about the system as a whole and far less about low level details. Still, the ability to move through abstraction layers is there.

So, learning to code will teach you to think at several levels of abstraction simultaneously: to decompose problems into parts, solve each in the best possible way, and then make those parts work together to become a solution to the original problem. It will teach you to think rigorously about why some potential solution doesn’t actually work and to detect subtle reasoning flaws in code—and, by extension, in other people’s behaviours as well as your own.

In short, coding makes you a better thinker. And thinking better is arguably the best recipe for success in general life, all other things being equal.

Why learning to code (well) is hard

But if learning to code is such a great cognitive investment and automatically makes everyone better at solving real-life problems, why are so many computer programmers out there not… being great at life?

Just like learning to drive, most people will master the lowest level first. They will learn how to express known solutions to known problems in computer lingua. They will learn to write working code.

But then, most people stay here. They can make the car move, but they can’t navigate to their destination, nor can they plan the best route and avoid the morning traffic. And the reason they don’t advance more is, often, because they don’t even know there is something else to be learned.

You see, most resources to learn how to program out there will actually just teach you how to write code. They will not teach you how to think about problems in the way good programmers do, so you can come up with clever solutions to these problems. There are books on this, too, of course. But most are college textbooks or advanced technical books that aren’t marketed at or even written for the first-time learner.

On the other hand, you will find a lot of books on how to do software engineering, and how to design systems that are reliable, maintainable, and a bunch of other fancy adjectives. But these are most often, I kid you not, written as if for people who already know this stuff. Much like college-level math books—that you often need to already know the math to even be able to read the book—the majority of books on software engineering are written by software engineers for software engineers, and they are talking among themselves, repeating the same mantras they already agree on.

Don’t get me wrong, there are good resources out there. But there are vastly, vastly more bad resources that won’t get you very far down the road that really matters: learning to think like a computer programmer. We already said this requires thinking at different levels of abstraction, and most learning resources never touch more than one of these levels.

But beyond the lack of good holistic learning resources, the deeper reason why learning to code is hard is precisely because it requires that you rewire your brain to think in a different, novel, more abstract and rigorous way. And your brain doesn’t want to do that.

Your brain evolved to keep you alive—actually, to transmit your genes—and for that, it needs to focus on mastering only two skills: getting food and getting laid. All else is secondary.

So, you have to trick your brain into believing that this is a critical skill to acquire. And that takes a lot of time and a lot of patience. No one learns to code well in 24 days or two months or even a full college semester. And, more importantly, no one learns to code by watching others code. That is a small part of learning, but you have to do the chores.

Coding is much more an acquired skill than learned knowledge. There is some knowledge involved, for sure. There are algorithms you can memorize. There is syntax and semantics and rules. There are design patterns and reusable ideas. But more than all of that, coding is the process of taking a complex problem, breaking it down into manageable parts, and then explain in the most precise possible language how to solve each of those parts, and how to put them together again.

That is, coding is about doing stuff much more than it is about knowing stuff. And there is only one way to learn how to do something: practice, lots of practice. And most people who think they can code out there simply haven’t put enough time into it.

But what about AI?

Yes, the elephant in the room! Will AI make coding skills irrelevant? Aren’t programmers coding themselves out of job?

Good that you ask. I’ve written many times before about why the current language modelling paradigm is, on the one hand, incapable of true reasoning and, on the other hand, unlikely to replace programmers anytime soon. So let me just repeat a few arguments here for the sake of completion.

First, code generators are indeed powerful tools that any professional programmer would do well to learn how to integrate effectively into their workflow. They can automate some of the most boring tasks and provide some much-needed unblockers from time to time.

However, code generators, at least those based on large language models, are inherently unreliable. So anyone using them to solve a given coding problem without a deep understanding of the problem and the proposed solution is shooting themself in the foot. AI-generated code can and will have errors, often subtle ones, that you need to be sufficiently experienced to catch.

On the other hand, language models struggle to deal with context. They often have a very limited context size compared to a reasonably large software project. But even if context size wasn’t a limitation, LLMs are known to arbitrarily ignore relevant parts of context and overfocus on irrelevant parts. They fail to grasp the big picture when things become complex enough.

In the end, this means that you can trust LLMs to generate small chunks of code that you can easily verify and understand, and that are relatively atomic and independent of the rest of the system.

They can definitely help at the code level—e.g., so you don’t need to remember or even search how that specific method or endpoint is invoked. They can help a little bit in the algorithm level, especially for applying relatively well-known strategies that you might have forgotten or not heard about. But they are almost useless at the system level.

The reason is that, the higher the level of abstraction, the more context matters. When you’re thinking about the whole architecture of your application, you need to understand how it all fits together, and even tiny details can be decisive for system-wide design choices.

So far, LLMs are incapable of such high-level reasoning. They can provide abstract, standard advice on software architecture, but the cannot reason that deeply about your specific application context and propose truly useful, novel, and accurate solutions for system-level design. And it seems we need a paradigm shift to overcome this limitation.

Thus, the better you are at coding—the better you are at thinking at the highest levels of abstraction—the more you can get out of an AI code generator. LLMs don’t make bad programmers better. In fact, they make them worse.

How should you learn to code

Finally, let me address the question of how. What follows is a very personal opinion—based on years of experience, but an opinion nonetheless.

There are two schools of thought about how to learn to code. The first one is the foundations-first school. In this school, you start by learning the syntax of a programming language and then learn how to write successively more complex programs in that language. This is the more traditional bottom-up approach, and it is often the design in college-level programming courses in engineering or computer science majors.

The upside with this approach is you get very strong foundations. After a full college year learning this way, you’re ready to tackle some reasonably complex coding problems. The downside is it takes a long time before you can do anything exciting that you can showcase. You spent most of your time solving abstract problems rather than building useful apps.

The second approach is championed by the applications-first school of thought. It is the complete opposite approach. You are thrown into a fully working application—often a rather simple one, but still far more complex than a single algorithm—and you get a high-level explanation of how the whole things work, as well as some deep dives into the important parts. This top-down approach is most common in online tutorials, bootcamps, and videos.

The upside is, of course, the pragmatism. You learn something immediately useful that you can replicate and maybe tweak for your own use case. And you get to see a lot of (hopefully) well-written code. However, if you only learn like this, you can end up with lots of disconnected ideas and no sense of the big picture. And more importantly, you don’t get a chance to develop the skill to think on your own and come up with your own ideas.

It probably won’t sound too bold or innovative to claim that I think the right way is a combination of these two. Of course it is! But the devil is in the details.

Good programming pedagogy is more than just doing algorithms one day and applications another. It’s not just an heterogenous mix. It requires a thoughful merge of these two approaches in a way that you are doing both things in unison: you’re thinking at the high level and coding at the low level simultaneously.

In my programming classes, I try to do precisely this: I create a fully working application—often a small demo, but something that has a concrete purpose and is designed for a real, human user.

We first spend some time thinking at a very high level about how such a system could be designed and discussing possible architectures. Then I crack open the code and show the parts that involve new content: maybe it’s a new instruction, a design pattern, or an algorithm.

At this point, I go back and forth between showing actual code and working on abstract ideas on the blackboard. This is time to think about why the code works, and to generalize these concrete ideas to more abstract patterns that can be learned.

Finally, I will challenge the students to make changes to the code. Some changes are superficial and only there for the coding level, so they get to practice with the new syntax. Other exercises tackle adding functionality or modifying methods to do extra stuff. These require the ability to think at a higher abstraction and understand what’s going on under the hood.

This is the best way I’ve found to get students to think about their code at several levels of abstraction simultaneously. If you’re interested in seeing this process in action, let me know in the comments. If there is enough interest, I may come back with some beginner-level tutorials using this approach.

When my second daughter was 45 days old, she got a common respiratory virus. Within hours, she got admitted to the UCI, and we spent the scariest fifteen days of our lives watching her fight for hers. Fortunately, she got the best treatment modern medicine can give and the most incredible team of caring, tireless doctors and nurses. She was saved by the doctors, but she is only alive because of modern science.

This is not an isolated story. Most of us alive today would be dead without science. Before the scientific revolution, the odds of living beyond 40 years were abysmal. The leading cause of death in the middle ages was curable diseases like flu, diarrhea, and mild infections, which ravaged populations with little understanding of cause or prevention. Raising one of your children beyond 10 was a coin toss, very biased towards the negative. Let's just say the bets weren't on your side.

The scientific method has enabled many of the most impactful discoveries in the last few centuries—in terms of improving human life—including antibiotics, vaccines, electricity, surgery, brain scans, cars, trains, airplanes, the internet, … you name it. These advancements have transformed humanity, extending longevity and improving quality of life, both physically and intellectually.

Science has allowed us to explore the mysteries of the universe, from the smallest particles to the vast expanse of space. It has expanded our understanding of the natural world to the point we can now summon and control the primal forces that make and break atoms. Science has raised us talking monkeys to heights our hairy ancestors would consider godlike.

Many, including myself, believe that the scientific method is the most important discovery in mankind's history, and the collective body of knowledge it has allowed us to build is our most valuable treasure. Still, some of the most important questions we have are forever outside the realm of what science can answer. And that's a feature, not a bug.

In this article and subsequent articles, I will explore how modern science works, why it is so effective, and its intrinsic limitations, both conceptually and pragmatically. In the end, I aim to convince you that science is our best tool for understanding the natural world, but we still need other tools to have the broadest possible view of the human condition.

For this, we will analyze the nature of knowledge to understand the types of questions we may want to ask about the universe. Then, we'll review how science works at different levels of organization, from the process of making experiments and finding new discoveries to the complexities of publishing research and the biases involved in the industrialization of science. Finally, we'll criticize the scientific method and argue there are some fundamental questions it can't answer. Then, we'll briefly look at alternative epistemic systems, which we may also need to encompass in a fuller understanding of reality.

What is knowledge?

Knowledge is a tricky thing. We all intuitively know what we mean when we say we “know” something, right? But we know all sorts of stuff for different reasons. For example, I know 2+2=4 because that’s basically the definition of those symbols. On the other hand, I know water boils at around 100 C because I’ve seen it happen over and over. I also know that electrons are negatively charged, that the speed of light is constant in a vacuum, and more or less how DNA works, but I have no direct experience with any of those things.

At the most abstract level, the study of knowledge is the job of a field of philosophy called epistemology. A complete introduction to epistemology is far beyond the scope of this article and well above my pay grade, but we can try a very brief overview.

The most commonly accepted definition of knowledge —not without its own problems— is that knowledge is justified true belief. Let’s unpack that.

Obviously, to truly claim you know something, you have to believe it. I mean, you can definitely claim you know something is true even if you believe it is false—you can give a false testament in a trial, but you would be lying. On the other hand, if you believe in something that is objectively false—like the Earth is flat—then you are simply mistaken, no matter how strongly you believe it. You can believe that you know, but you don’t truly know, because you’re wrong. So far so good? Now comes the tricky part.

Say you buy a lottery ticket, and you believe with all your heart that you will win. Then, it so happens that you win the lottery. So you had a true belief. Would you say you knew you were going to win? Most rational people would claim you didn’t actually know. You just got lucky. Whatever reason you had to support that belief—maybe the ticket had your preferred numbers in some specific pattern, or maybe that day you were dressed with your favourite underwear—, that reason is unjustified. The lottery is absolutely random, so you couldn’t know you would win.

Similarly, you might believe you know the solution to a complex math problem, but if you made subtle errors that happened to cancel each other out, you would arrive at the correct answer for the wrong reasons. In this scenario, most rational people would argue that you don't truly know the answer, even if you happen to stumble upon it.

So, justified true belief. The question then becomes, how do we attain knowledge? That is, how can we ensure we are justified in believing that something is indeed true? This is what epistemic systems are for.

Epistemic systems

In short, an epistemic system is a set of rules that determine when a given claim is true. Different epistemic systems assert the truth value of claims with different strategies. For example, you can think of mathematics as an epistemic system that asserts all claims that can be logically deduced—using a very narrow set of inference rules—from a given set of non-contradictory axioms.

This is an example of a formal epistemic system, where the truth or falsity of a given claim is decided only by symbolic manipulation. There is no need to look at the real world. Triangles are abstract things, and we can know everything there is to know about them by thinking really hard. There are no triangles out there in the woods.

Another possible epistemic system is what I call truth-by-authority. This system asserts all claims that some predefined entity—e.g., your mother—decides, regardless of… well, anything. Jokes apart, many religions fall into this broad category, choosing as authorities either ancient scriptures of dubious origins or self-appointed messiahs.

Not all epistemic systems are equally trustworthy in every domain. For instance, I don't rely on my mother's authority for most issues—though there are some cases where she holds the absolute truth—but I trust mathematics in all relevant situations. The critical question is this: How do we determine the applicability of an epistemic system to specific types of claims? What questions can we justifiably believe based on a given system?

You can probably see where I’m going with this. To understand when science is justifiable as an epistemic system, we need to understand what types of claims can be answered with the scientific method. For the purpose of this article, I want to divide claims into three categories, one of which will be the prime domain of scientific inquiry.

First, let’s consider the questions that can be answered via pure reasoning. Questions like “how many prime numbers are there?” or "what is the fastest way to sort these numbers?" fall into this category. These questions can be resolved through mathematical processes and logical deductions without empirical observation. These aren't necessarily easy questions, though. Rather, some of the hardest questions in math and computer science are in this set.

Next, we have claims that rely on empirical evidence, such as “what is the boiling point of water?” or “is the light next room turned on?” These require observational methods and experimentation. You simply cannot reason your way into knowing the precise temperature at which water molecules have enough kinetic energy to escape from each other’s electromagnetic attraction. You have to go out there and measure it, just as you have to open that door to see if the light is turned on. This is the domain of science.

Finally, the third category encompasses questions that can’t be answered either way. These include subjective or normative claims, such as “what is the best ice cream flavor?” and “is there an objective moral system?” which may not be effectively addressed by either logical reasoning or observation alone. This is the domain of philosophy, arts, and spirituality. Some of the most important questions in life, including the meaning of all this absurd existence, are very likely in this realm.

How does science work?

Now that we intuitively understand what types of questions we want to answer with science let’s try to explain how science works from the ground up.

In short, science is a systematic approach to investigating the natural world. It relies on formulating hypotheses based on observed phenomena and testing them through experiments. The results of these experiments either support or refute the initial ideas. This process is iterative, meaning each outcome may lead to new questions and hypotheses, continually refining our understanding of reality.

Crucially, science can almost never give a definite answer. All we can do is get an increasingly more accurate approximation of the true nature of reality. How close we can get is unclear. Some think the universe is fully understandable, and others claim there is some fundamental level of incomprehensibility we may never surpass. But, more importantly, this very question—the limits of science—is, for the most part, outside the realm of the types of questions that science can answer.

But let’s see how it works.

Science as an Epistemic System

At the lowest level, the scientific method is an epistemic system that attempts to assert the truth or falsity of objective claims about the natural world. By objective, I mean claims whose truth value is independent of the observer, so whether chocolate is the best flavor of ice cream, despite being a claim about a very natural thing, is outside the interest and reach of science because its truth value will depend on the observer. Most people would say yes, but some sociopaths might prefer vanilla.

How do we justify beliefs about the natural world? The most obvious answer is to verify them through direct experience. If I claim the light in the next room is on, you just need to open the door and check, right? This is the paradigm of verifiability, formally put forward in the early 20th century by a school of thought called Logical Positivism.

However, this approach has two problems. The easier one is with claims that cannot be directly observed or measured but still have measurable indirect effects. For example, suppose I claim the Moon exerts a small gravitational force over you. In that case, it is hard to conceive of an easily verifiable experiment that can readily convince you this is true. Almost everything we want science to talk about is outside of our direct, shared, macroscopic experience.

The harder problem, though, is much more fundamental: it is about universal claims. For example, if I say all electrons are negatively charged, there is no way to verify that claim. Even if you could directly measure the electric charge of any particular electron, you would still not be able to prove that all electrons in the universe, past, present, and future, have the same properties.

So, instead of verifiability, which seems to impose an absurdly high constraint on the types of claims—about the natural world—we can study, we want a more flexible paradigm that gives us, perhaps not absolute certainty, but as close as pragmatically possible.

This is what Karl Popper proposed with his concept of falsifiability. Rather than requiring claims to be provable, he suggested that scientific theories must be testable and, importantly, falsifiable. This means that in order for a statement to be considered scientific, there must be a possibility of it being shown false through observation or experiment.

According to this view, a theory is scientific if it can, in principle, be tested and potentially refuted by evidence. While no amount of positive evidence can convince us a theory is definitely true, if we test it long and hard enough and never find falsifying evidence, then we have to grant that, as far as evidence goes, this theory is the current best explanation for a given phenomenon. It becomes an accepted scientific consensus.

Shifting from verifiability to falsifiability also shifts the burden of proof from the person who makes the claim to the person trying to falsify it. At face value, this seems like a bad move, right? Now, whenever I make a claim, I don’t have to give you a way to verify it. Instead, it becomes your responsibility to demonstrate that my claim is false.

So what happens if my claim is impossible to falsify? I claim there is an invisible purple unicorn in my garage, undetectable by all instruments and producing no perceivable effect in the environment, yet it is there, watching over you and judging every decision you make. This claim is impossible to prove false, by definition. What can we do about it?

This is the problem of demarcation in the philosophy of science, which basically determines which claims are considered “scientific” to begin with. Simply put, if your claim is untestable or unfalsifiable, the scientific community won’t even pay attention to it. We simply ignore it. It is not in the interest of science to talk about magical, undetectable unicorns, right?

But it sounds easier than it really is because sometimes determining what counts as a possible falsification procedure becomes a heated debate. For example, some serious scientists consider string theory unscientific because, so far, it hasn’t produced any falsifiable predictions. Other prominent scientists instead claim we simply need more sophisticated epistemic principles.

However, barring these extreme cases, for the most part, it is often evident when a simple enough hypothesis is a scientific claim. Your hypothesis needs to make some testable predictions that can be verified. As long as you keep making predictions that are proven right, the trust in that hypothesis grows until it eventually becomes a mainstream confirmed theory. But the minute a prediction fails, your hypothesis is immediately rendered false, right?

Well, not necessarily.

Science as an Iterative Approximation of Truth

One of the most famous scientific theories of all time is classical (e.g., Newton’s) mechanics. It was the dominant explanation for most of the physical world for a long time. It is also one of the most beautiful theories in the mathematical sense, having but a few simple and coherent formulas that unify everything from how apples fall from trees to how the Moon rotates around the Earth.

We lived in the dark ages, and Newton gave us the light. Then Einstein came and made everything dark again.

Around the beginning of the 20th century, a bulk of evidence was piling up showing that something fishy was going on with Newton. The most important: the orbit of Mercury wasn’t precisely as predicted by classical mechanics. However, this had happened before.

The discovery of Uranus in 1781, the first new planet to be discovered since ancient times, was initially celebrated as a triumph of Newtonian mechanics. However, as astronomers observed Uranus's orbit, they noted irregularities that could not be reconciled with Newton's predictions. What is more likely—astronomers of the time asked themselves—that almighty Newton, who has worked perfectly so far, is actually wrong, or that we are missing something here?

This led to the hypothesis that another massive body was exerting gravitational influence on Uranus. Mathematicians Urbain Le Verrier and John Couch Adams calculated where such a planet should be based on the observed perturbations in Uranus's orbit. Their predictions were remarkably accurate, leading to the discovery of Neptune in 1846, which was found within one degree of the predicted position. So, Newton was right all along! The theory predicted that something was missing, and we found it.

Mercury's orbit was a different story, though. The planet's perihelion—the point in its orbit closest to the Sun—was observed to shift over time, a phenomenon known as perihelion precession. According to Newton's laws, this precession should be minimal and consistent; however, observations indicated a tiny discrepancy that no other explanation—like a missing planet—could account for. Something was seemingly wrong with Newton.

Then Einstein came along and presented a much more complex, precise theory that accounted perfectly for all the discrepancies. So, Newton was wrong all along! Right?

Well, the story doesn’t end there, either. In the early 20th century, more precise observations of Neptune revealed that its orbit also exhibited deviations from Newtonian predictions. Again, astronomers said there must be another planet out there, and indeed, they pointed their telescopes and found Pluto in 1930. However, subsequent studies revealed that Pluto's mass was insufficient to account for the observed perturbations in Neptune's orbit. So astronomers thought it must be yet another planet. They called it “Planet X”.

However, the search for planet X was stagnant. All observations of the night sky where planet X should have been failed to show anything as obvious as a missing planet. Then, astronomers went back to the original data that showed Neptune’s irregularities. They discovered one of the key observations was performed after some telescope maintenance, which might have introduced calibration errors. After removing those few noisy data points, the resulting data perfectly matched Newton’s predictions again—no new planet or fancy new theory was necessary.

So, is classical mechanics right or wrong? If we are being absolute, they are indeed wrong. They fail to predict how matter moves close to the speed of light and near supermassive objects like stars and black holes, yes. But, for most practical matters—hell, even for sending astronauts to the Moon—we can still rely on old Papa Newton. We still teach classical mechanics as scientific facts in high school, and that’s because, for 99% of the problems we’ll ever have, it really works!

How can this be? Aren’t falsified theories wrong? Shouldn’t we be done with Newton once and for all, especially since we have a more powerful theory that does all the previous one did, and more?

Well, the answer is that it depends. Some hypotheses are falsified with evidence showing they are way off, totally false, and useless. However, other hypotheses might just be approximations of the true nature of reality, and pretty good approximations if you are congnizant of their limitations. This is the case with classical physics. It isn’t right or wrong. It’s an approximate model of actual physics, and if you use it within reasonable—and well-known—constraints, it really works.

For all we know, general relativity is just another, more precise approximation of reality, but it might also break at some point. Right now, for example, no one knows how to unify general relativity—the physics of the large—with quantum mechanics—the physics of the very small—and it might be the case that these two theories are simply two approximations that break when you stretch them bad enough.

So, truth in science is always approximate. We seek not necessarily the ultimate truth but a better approximation of it, which accounts for all the phenomena our current instruments, math, and intelligence can comprehend.

Moving on

Science is still much more complex than what the previous story shows. Beyond the pure, rational search for truth and knowledge, other forces at play make science a complicated social phenomenon, where the winning theory isn’t always the one that most closely matches reality. And then there are economic incentives, careers at stake, institutions, governments… It’s a huge mess. But this article is already long enough, so I will leave the rest of this story for future entries.

Until then, stay curious.

Is the brain a computer? At first glance, the answer might seem obvious: no. Brains and computers are very different. Brains are biological, made from organic material—gooey stuff—while computers are electronic devices made from non-organic materials—clicky stuff—practically the opposite of brains. Settled, right?

Is this a meaningful difference, though? When we ask if two things are equivalent, the answer may be widely different depending on how we compare them. We can focus on what things are made of or how things look, but these are often the most superficial—and less interesting—comparison criteria.

Let’s take a different approach and ask where two things are alike regarding what they can do, that is, comparing their functionality. Is this a meaningful comparison? Here’s an example of why there may be some meat to this. Think of the concept of a chair. How many shapes, materials, colors, sizes, and styles can you imagine a chair could have?

A chair made of wood and a chair made of steel are very different in composition, fabrication process, durability, etc., but they are still equivalent in terms of what chairs are used for—sitting down. Is a stool a chair? Is a rocking chair a chair? What about a gaming chair, the place you sit in your car, or the thing astronauts sit on inside a rocket ship? Heck, if we are talking about stuff you can sit on, even a flat enough boulder is a chair!

But wait, why are we talking about chairs? What does this have to do with brains and computers? That is a fair question, and here’s the point I want to make. When comparing things, one often significant point of comparison is regarding what things can do. That is, comparing things in terms of their function. If two things perform the same function, they are, in a sense, equivalent. Let’s call this a functionalist paradigm and get back to whether brains and computers are equivalent.

In this article, I want to tackle this question from the point of view of computational functionalism---a specific form of functionalism that I will lay out in more detail later. What I want to do in this article is, first, convince you that this question is much more profound than it seems at first sight, and second, give you some arguments on both sides of the discussion. As usual with these philosophical articles, I won’t (can’t) give you a definite answer, but I hope you come back on the other side more informed and perhaps have a bit of fun in the process.

This article is heavily inspired by incredible series on the philosophy of mind, and especially the article Will AI Ever Be Conscious, published in . While I tried to make this article self-contained, if you really want to get the full context of this discussion, and learn a hell lot more from an actual expert in neuroscience, please check ’s work and subscribe to her substack. You're welcome.

Computational functionalism in a nutshell

We will begin by dissecting the question “are brains and computers equivalent” and define precisely what we mean by “brain,” “computer,” and, well, “equivalent”, for the matter. Then, we will go over some of the most common counter-arguments to the computationalist hypothesis and, finally, present some arguments for why one could believe brains are indeed “just” computers.

What is a brain?

At first glance, this seems like an obvious question. A brain is that gooey stuff inside your skull where all cognition happens. It’s where, in some way we still don’t quite understand, you think your thoughts and feel your feelings. It’s also where—so we are told—something as elusive as consciousness resides.

However, let’s go back to our chair example. Many things out there don’t look like human brains but still have a similar function. Other animals have brains—granted, very similar to ours in many cases. But then you have octopuses (or octopi, whatever floats your boat). They have something like 2/3rds of their neurons in their arms! But almost no one would claim they don’t have at least some limited form of cognition. And what about aliens? Do we think any living, sentient, self-aware being out there will have something strikingly similar to this gooey, wrinkled piece of meat we call the brain?

As we’ve seen, for a functionalist, these differences are unimportant. What we care about when talking about brains is the function they perform—that’s the whole deal with functionalism! The functionalist will claim that whatever a brain is, is what a brain does. Cognition, sentience, consciousness, and all subjective experiences are all entirely defined in terms of their function. So, when functionalists summon the concept of a brain, they think about what a brain does: in short, hosting a mind.

So, let’s define the brain as the kind of physical substrate that can host a mind, with all the complex cognitive processes, subjective experiences, emotions, qualia, and everything else you want to claim a mind does.

What is a computer?

Like before, this seems like an obvious question, but at this point, we know better than to rush to a conclusion. Intuitively, a computer is something that does some advanced forms of calculation. You can have mainframe computers, desktops, laptops, smartphones, smartwatches, mini-computers like a Raspberry Pi, and very weird computers like what goes inside a self-driven car or a spaceship.

Furthermore, even though most actual computers we have today are made of silicon-based transistors laid out in tightly-knit circuit boards, this is hardly the only way to implement a computer. Before fully electronic computers, we had electromechanical computers with valves and moving parts that sounded like trains. Crazy dudes have proposed more than one design of a fully working hydraulic computer. We even have bioelectrical prototypes that actually mix gooey stuff with traditional circuits.

So, just as before, let’s consider the function of a computer. This is actually way easier than with brains because unlike brains—which are things out there in nature—computers are something we made up. And we have a whole field in computer science called computability theory dedicated precisely to studying what a computer can do.

In short, a computer is an abstract mathematical construct capable of computing any effectively computable function. An effectively computable function is any mathematical function that can be calculated with a finite series of mechanical steps without resorting to guessing or magical inspiration—in other words, an algorithm. Modern electronic computers are one possible physical embodiment of such abstraction, but they are hardly the only possibility.

So, let’s define a computer as any device capable of computing all computable functions.

What is computationalism

Now, we are ready to reframe our original question in more precise terms.

Let’s start with functionalism and build our way up. When talking about theories of mind, functionalism is the theory that all cognitive processes are completely characterized by their function. This means that, for example, whatever consciousness is, is just about what consciousness does. In other words, if you have some way to reproduce what consciousness, or any other cognitive process does, it doesn’t matter which substrate you use, what you will have is genuinely the same thing.

Put more boldly, any system that performs the same functions as a mind is a mind, period.

Computational functionalism goes one step further and claims that all cognitive processes are actually computable functions. That means a sufficiently complex and suitable programmed computer could perform these functions to the same extent a brain does, and so it would presumably host a mind. More formally, it claims that all cognitive processes are computational in nature and thus can, in principle, be implemented in a substrate other than biological brains, as long as it supports general-purpose computation.

In other words, computationalism is precisely the claim that brains are computers, understanding both these terms with all the nuances we have already discussed.

To be clear, computationalism doesn’t claim modern computers are conscious or even that our current most advanced AI systems are on the right path to ever becoming truly intelligent or conscious. It just claims there is some way to build a computer, at least in principle, that is indeed self-aware, fully intelligent, and conscious, even if we have no clue what it takes to build one.

Now, there are at least two forms of computationalism; let's call them weak and strong (these are my definitions, not standard). Weak computationalism is just the claim that cognition is computation. This means that all forms of intelligence—understood as problem-solving, reasoning, etc., irrespective of whether there is a sentient being in there—are just advanced forms of computation. In other words, there is nothing super-computational in humans, animals, aliens, or any other form of intelligence. A sufficiently powerful computer can be as intelligent as anything else.

Strong computationalism goes further and claims that consciousness, sentience, self-awareness, and qualia—that is, all subjective experiences—are also computational in nature. Thus, a sufficiently powerful computer with the correct software would indeed be conscious and experience an inner world, just as we presume all humans and many animals do.

The roots of computationalism can be traced back to McCulloch and Pitts, the fathers of connectionism, a competing theory of mind. They were the first to seriously suggest that neural activity is computational and to propose a mathematical model for it, a precursor of modern artificial neuronal networks. However, it wasn’t until well into the 1960s that computationalism started to be developed as a philosophical theory of mind.

However, perhaps the most famous instance of a computationalist perspective in popular culture is the Turing Test. In a seminal paper in 1950, Alan Turing, recognized by everyone as the forefather of computer science (Turing machines are named after him!), proposed what he called “the imitation game”, a thought experiment to determine whether a computer was thinking.

In this thought experiment, a computer and a human are placed behind closed doors, able to communicate with a second human—the judge—only through a chat interface. The judge can ask both participants anything, and it must ultimately decide which is the computer and which is the human. If the computer manages to confuse the human judge more often than not, Turing claims we must acknowledge the computer is performing something indistinguishable from what humans call thinking. According to functionalism, then the computer is thinking.

Now that we've sorted our concepts, let's tackle the question.

Is computational functionalism true?

We've already dismissed the most obvious counterargument to computationalism—that brains are gooey while computers are not. But even if we disregard this superficial difference in composition, there are still important structural differences between existing brains and computers that we cannot just gloss over. And then, once we clear those more direct counterarguments, we will turn into more nuanced and profound differences between brains and computers.

Mind independence of substrate

Computers have a distinctively hierarchical structure, from hardware to software, starting at transistors and building up to registers, microprocessors, kernels, operating systems, and applications—in a very simplistic description. If any part that performs any specific function breaks, it all falls apart.

On the contrary, the brain seems to have some structure, but it is much more fluid and flexible. You can remove entire brain regions, and it will often rewire itself and learn to perform the affected functions almost to perfection.

On the other hand, there seems to be a clear distinction between software and hardware in computers, to the point where you can move software around independently of the hardware. There is no such thing in the brain—as far as we know, we can't simply paste your thoughts into some portable device, load them into a freshly minted meat suit, and get a second copy of you.

Maybe this is it. Perhaps the interconnected, seemingly intrinsically inseparable nature of thoughts and substrate in the brain is fundamental to consciousness and self-awareness. Maybe as long as you can move the mind out of the brain, it cannot be a true mind.

Well, if this were the case, and the mind is inseparable from the brain, this realization would obliterate most known religions. Forget about any form of transcendental survival of the soul. As soon as your brain dies, your mind is dead. No uploading to the cloud, metaphorically or literally. However, while this might certainly upset a bunch of people, I'm not a religious person, so I'm not bothered by this particular argument.

And even if this were the case—a true mind must be inseparable from the substrate—, this seemingly explicit separation between hardware and software is, first, just an implementation detail, and second, kind of a useful lie as well.

If you look inside a modern microprocessor, it is not trivial to distinguish where hardware ends and software begins. Not only there is programmable code running at the microprocessor level but, even more importantly, the simplest logical circuits inside a computer are both hardware and software at the same time. Just the clever disposition of transistors in some specific layout is enough to make a circuit that adds, or multiplies, or does basically any other computable thing. The hardware is the software in these cases.

So, while some things—like cables—are clearly hardware and other things—like the browser you’re using right now—are clearly software, it is simply not true that there is a clear-cut point of separation between these two concepts. It’s just a useful abstraction.

And then we have artificial neural networks, like the ones powering ChatGPT. These are universal approximators, meaning they can compute any function to an arbitrary degree of precision. And they are much closer to this idea of a flexible architecture where software and hardware intertwine in ways that are hard to clearly differentiate. While modern artificial neural networks are far from a precise simulation of brains—and that’s by design, they are not even trying to simulate brains—there is no a priori reason why we can’t build a silicon computer that perfectly mimics the physical processes inside a real, biological brain.

This means any attack on computationalism based on structural differences between actual instances of brains and computers is futile. We cannot generalize a negative claim from a finite set of negative examples. No matter how many computers you find that are not brains, you can never be sure a computer cannot, by definition, ever be equivalent to a brain. This is just the typical problem of induction, so we must find another angle of attack, something that targets the fundamental definition of computer rather than any concrete implementation.

Syntax versus semantics

The most famous attack on computationalism is John Searle's Chinese room experiment. It is meant as a counterargument to Turing's imitation game, showing that even if a system can flawlessly simulate understanding, it may not be understanding at all. Here is a quick recap.

Suppose a man is placed inside a room without communication with the outer world except via an "input" and "output" window. Through the input window, the man receives, from time to time, sheets of paper with strange symbols on them. Using a presumably very big book, the man's only job is to follow a set of dead simple instructions that determine, for any combination of input symbols, what other strange symbols he must write on a new sheet and send it through the output window.

Now, here is the plot twist. The input symbols are well-written questions in Chinese, and the output symbols are the corresponding, correct answers. The huge book is designed such that an appropriate answer will be computed for every possible input question. So, when seen from the outside, it seems this system understands Chinese and can answer any plausible question in this language. However, Searle argues that neither the man, the book, nor any part of the system actually understands the questions. (If this sounds eerily similar to ChatGPT, give the man a round of applause; he came up with this thought experiment half a century before.)

Searle's point is that syntax and semantics are two qualitatively different layers of understanding, such that no degree of syntax manipulation—which is, according to Searle, all a computer can do—can amount to actually understanding the semantics of a given message.

For all experts today, it is evident that ChatGPT is only manipulating vectors in a way that happens to produce mostly coherent answers, but there is no real "mind" inside ChatGPT doing any sort of "understanding". In a similar sense, Searle believes any computer is fundamentally incapable of true understanding in any meaningful sense of the word simply because computation acts at the syntactic level and is incapable of bridging the gap to semantics.

Searle's attack is pretty strong. There is not much a computationalist can show as a defense other than trying to point to some place in the Chinese room where understanding happens. The best such defense is to claim that even if no single part of the system—neither the man, nor the book, nor the room itself—can be said to understand anything, the system as a whole does understand. In the same way, no single part of your brain is conscious, but you are as a whole, maybe? Still, not a very strong defense if you ask me.

Qualia and knowledge

Another interesting challenge to computationalism is the thought experiment known as "Mary's Room", conceived by philosopher Frank Jackson. It deals with the nature of subjective experiences—or qualia—and how important they are for grounding knowledge about reality. The story goes like this.

Mary is a neuroscientist who has lived her entire life in a black-and-white room, learning everything there is to know about color vision through books and scientific literature. She knows all the physical facts about color and how the human brain processes visual stimuli. However, she has never experienced true color.

One day, she finally exits the room and sees color for the first time. The question is whether Mary learns something qualitatively new about color, something that can only be learned through direct experience rather than by reading or indirectly studying the phenomenon of color perception.

For many, Mary's experience is indeed qualitatively new. If you think so, this raises critical questions for computationalism. If computationalism asserts that all cognitive processes can be reduced to computational functions, then one might wonder whether knowing all the physical facts about a phenomenon is equivalent to experiencing it.

Mary's Room suggests a gap between knowledge and experience—one that cannot be bridged by computation alone. This challenges the idea that a computer, no matter how advanced, could genuinely "know" or "experience" qualia in the same way a human does. At least, it tells us ChatGPT cannot truly understand what seeing red, feeling warmth, or being in love feels like.

The implications of this thought experiment are profound for the computationalist perspective. It suggests that even if a computer could simulate all the functions of a brain, it might still lack the intrinsic, subjective experiences that characterize human consciousness. In other words, while a computer might process information about colors and respond appropriately, it would not "know" what it is like to see red or feel the warmth of sunlight—experiences that are inherently qualitative and personal.

This distinction between knowledge and experience underscores a fundamental limitation of computationalism: it may account for the mechanics of cognition but fails to address the richness of conscious experience. As such, the Mary's Room thought experiment serves as a poignant reminder that understanding the brain as a computer may overlook the essential nature of qualia.

On the other hand—you may argue—if Mary indeed knew everything that could be known about color perception other than "how it feels," how is that any different from actual perception? In a sense, the brain doesn't truly "experiences" color. It just perceives some electrical impulses correlated with the frequency of the light waves coming through our eyes. Can't color perception be explained as just another layer of simulation on top of an actually blind, purely computational brain?

In any case, Mary's room doesn't preclude us from posing extremely advanced mechanical entities that emulate all the physical aspects of color perception. Nothing is inherently unmechanical in our current understanding of how light frequency stimulates some sensors that produce an electric signal in the brain. It is the subjective experience of what it feels like to see red that we cannot reduce to that mechanistic explanation. And this is precisely what functionalism attempts to capture: if two mechanisms perform the exact same functions, they are the same.

Other Attacks

In addition to the prominent arguments against computationalism, several other critiques have emerged that challenge the notion of equating brains with computers. Two notable lines of reasoning come from Roger Penrose's insights on mathematical understanding and the connectionist perspective on reasoning.

Non-computability of human creativity

Roger Penrose, a renowned physicist and mathematician, argues that certain mathematical insights are inherently non-computable. He posits that human mathematicians can grasp concepts and solve problems that transcend algorithmic computation, suggesting that there are aspects of human thought that cannot be replicated by any computer, regardless of its complexity.

As an example, Penrose believes that while we have a formal proof that some mathematical problems are indeed unsolvable using any method of effective computation, human mathematicas can summon some hyper-computational abilitites to gain insights into these problems. In the same sense, many propose that creativity and artistic expressions in humans are clear examples of non-computable thought processes.

Connectionist arguments against symbol manipulation

Connectionism is an alternative theory of mind that posits mental processes can be understood through networks of simple units (like neurons) rather than through traditional symbolic manipulation. Proponents of connectionism argue that reasoning does not necessarily require the explicit representation of symbols or the manipulation of those symbols according to formal rules. Instead, they suggest that cognitive processes can emerge from the interactions within a network, where knowledge is distributed rather than localized.

This perspective challenges the computationalist assumption that reasoning must rely on structured representations and symbol manipulation, proposing instead that cognition may arise from more fluid and dynamic processes akin to those found in neural networks, and that symbolic manipulation of the kind that happens in a traditional algorithmic procedure is but an emergent phenomenon in the brain, at best, but not a necessary feature of an actual mind.

Moving forward

Is the brain a computer? We honestly don't know. It is a tough question, perhaps the hardest question we can ever conceive, because it challenges our most valuable and effective tool for discovering truths about the world: science. Stay with me for a second.

Science—understood as the process of generating hypotheses, producing predictions from those hypotheses, and testing the results of those predictions via experiments to falisify or validate the hypotheses—is fundamentally a computational process. Any procedure for experimental verification must be laid out in such a simple, unambiguous set of steps as to be replicable by other scientists all around the world even if they don't speak our language.

Furthermore, the language of science is mathematics, the most strict and computable of all human-made-up languages. And computers are everywhere in science too. It is inconceivable today to perform any mildly complex experiment without the help of computers to run simulations, find patterns, and, well, compute stuff. Science is deeply computational, and it has always been. The forefathers of science, Galileo, Newton, Leibniz, and the rest, explicitly focused on quantifiable, measurable phenomena; the only things we could all agree were certain.

But the inner workings of the human mind and the nature of consciousness may be neither measurable nor quantifiable. Even when we finish mapping all neuronal pathways in the brain and discovering everything that happens at all physical levels in the gooey stuff, and load that into a computer, we might end up creating just another Mary, knowing everything about how a mind works, but incapable of truly experiencing what having a mind feels like.

And the sad part is that, if that's the case—if the mind is indeed non-computable—then we might actually never know for sure. After all, the best tool we have to understand how stuff works, Science, may be nothing more than a fancy algorithm.

Or maybe, that's the fun part.

Next semester, I’ll teach the introductory course in Compilers again, after a couple years, in the CS major at the University of Havana. It was an excellent opportunity to dust my notes on formal languages, parsers, analyzers, code generators, etc. Going over the course material, I decided I needed to rewrite most lecture notes because my understanding of the fundamental issues in formal languages has dramatically changed since I last taught this course.

So, as I prepare for the upcoming course, I’ll be sharing with you some of the most intriguing and mindblowing ideas from one of the areas of Computer Science with some of the most profound results. We will begin with a very intuitive introduction to formal language theory and build our way up to understand how compilers, text editors, virtual machines, and all the associated components work.

Buckle up!

What is a (formal) language?

Intuitively, a language is just a collection of correct sentences. In natural languages (Spanish, English, etc,), each sentence is made up of words, which have some intrinsic meaning, and there are rules that describe which sequences of words are valid.

Some of these rules, which we often call “syntactic” are just about the structure of words and sentences, and not their meaning–like how nouns and adjectives must match in gender and number or how verbs connect to adverbs and other modifiers. Other rules, which we call “semantic”, deal with the valid meanings of collections of words–the reason why the sentence “the salad was happy” is perfectly valid syntactically but makes no sense. In linguistics, the set of rules that determine which sentences are valid is called a “grammar”.

In formal language theory, we want to make all these notions as precise as possible in mathematical terms. To achieve this, we will have to make some simplifications which will ultimately imply that natural languages fall outside the scope of what formal language theory can fully study. But these simplifications will enable us to define a very robust notion of language for which we can make pretty strong theoretical claims.

So let’s build this definition from the ground up, starting with our notion of words, or, formally, symbols:

Definition 1.1 (Symbol) A symbol is an atomic element with an intrinsic meaning.

Examples of symbols in abstract languages might be single letters like a, b or c. In programming languages, a symbol might be a variable name, a number, or a keyword like for or class. The next step is to define sentences:

Definition 1.2 (Sentence) A sentence (alternatively called a string) is a finite sequence of symbols.

An example of a sentence formed with the symbols a and b is abba. In a programming language like C# or Python, a sentence can be anything from a single expression to a full program.

One special string is the empty string, which has zero symbols and will often bite us in proofs. It is often denoted as 𝜖.

We are almost ready to define a language. But before, we need to define a “vocabulary”, which is just a collection of valid symbols.

Definition 1.3 (Vocabulary) A vocabulary 𝑉 is a finite set of symbols.

An example of a vocabulary is { 𝑎,𝑏,𝑐 }, which contains three symbols. In a programming language like Python, a sensible vocabulary would be something like {for,while,def,class,…} containing all keywords, but also symbols like +, ., etc.

What about identifiers?

If you think about our definition of vocabulary for a little bit, you’ll notice we defined it as finite set of symbols. At the same time, I’m claiming that things like variable and function names, and all identifiers in general, will end up being part of the vocabulary in programming languages. However, there are infinitely many valid identifiers, so… how does that work?

The solution to this problem is that we will actually deal with two different languages, on two different levels. We will define a first language for the tokens, which just determines what types of identifiers, numbers, etc., are valid. Then the actual programming language will be defined based on the types of tokens available. So, all numbers are the same token, all identifiers are another token, and so on.

Given a concrete vocabulary, we can then define a language as a (possibly infinite) subset of all the sentences that can be formed with the symbols from that vocabulary.

Definition 1.4 (Language) Given a vocabulary 𝑉, a language 𝐿 is a set of sentences with symbols taken from 𝑉.

Let’s see some examples.

Examples of languages

To illustrate how rich languages can be, let’s define a simple vocabulary with just two symbols, 𝑉={ 𝑎,𝑏 }, and see how many interesting languages we can come up with.

The simplest possible language in any vocabulary is the singleton language whose only sentence is formed by a single symbol from the vocabulary. For example, 𝐿𝑎 = { 𝑎 } or 𝐿𝑏 = { 𝑏 }. This is, of course, rather useless, so let’s keep up.

We can also define what’s called a finite language, which is just a collection of a few (or perhaps many) specific strings. For example,

𝐿1 = { 𝑏𝑎𝑏, 𝑎𝑏𝑏𝑎, 𝑎𝑏𝑎𝑏𝑎, 𝑏𝑎𝑏𝑏𝑎 }

Since languages are sets, there is no intrinsic order to the sentences in a language. For visualization purposes, we will often sort sentences in a language in shortest-to-largest and then lexicographic order, assuming there is a natural order for the symbols. But this is just one arbitrary way of doing it.

Now, we can enter the realm of infinite languages. Even when the vocabulary is finite, and each sentence is also a finite sequence of symbols, we can have infinitely many different sentences in a language. If you need to convince yourself of this claim, think about the language of natural numbers: every natural number is a finite sequence of, at most, 10 different digits, and yet, we have infinitely many natural numbers because we always take a number and add a digit at the end to make a new one.

Similarly, we can have infinite languages simply by concatenating symbols from the vocabulary ad infinitum. The most straightforward infinite language we can make from an arbitrary vocabulary 𝑉 is called the universe language, and it’s just the collection of all possible strings one can form with symbols from 𝑉.

Definition 1.5 (Universe language) Given a vocabulary 𝑉, the universe language, denoted 𝑉∗ is the set of all possible strings that can be formed with symbols from 𝑉.

An extensional representation of a finite portion of 𝑉∗ would be:

𝑉∗ = { 𝜖, 𝑎, 𝑏, 𝑎𝑎, 𝑎𝑏, 𝑏𝑎, 𝑏𝑏, 𝑎𝑎𝑎, 𝑎𝑎𝑏, 𝑎𝑏𝑎, 𝑎𝑏𝑏, 𝑏𝑎𝑎, 𝑏𝑎𝑏, 𝑏𝑏𝑎, 𝑏𝑏𝑏, ... }

We can now easily see that an alternative definition of language could be any subset of the universe language of a given vocabulary 𝑉.

Now, let’s take it up a notch. We can come up with a gazillion languages just involving 𝑎 and 𝑏, by concocting different relationships between the symbols. For this, we will need some way to describe the languages that don’t require listing all the elements–as they are infinitely many. We can do it with natural language, of course, but in the long run, it will pay to be slightly more formal when describing infinite languages.

For example, let 𝐿2 be the language of strings over the alphabet 𝑉 = { 𝑎, 𝑏 } with the exact same number of 𝑎 and 𝑏.

𝐿2 = { 𝜖, 𝑎𝑏, 𝑎𝑎𝑏𝑏, 𝑎𝑏𝑎𝑏, 𝑏𝑎𝑏𝑎, 𝑏𝑎𝑎𝑏, 𝑎𝑏𝑏𝑎, ... }

We can define it with a bit of math syntax sugar as follows:

𝐿2 = { 𝜔 ∈ { 𝑎,𝑏 }∗ | #(𝑎,𝜔) = #(𝑏,𝜔) }

Let’s unpack this definition. We start by saying, 𝜔 ∈ { 𝑎,𝑏 }∗, which literally parses as “strings 𝜔 in the universe language of the vocabulary { 𝑎,𝑏 },” but is just standard jargon to say “string made out of 𝑎 and 𝑏. Then we add the conditional part #(𝑎,𝜔) = #(𝑏,𝜔), which should be pretty straightforward: we are using the #(<symbol>,<string>) notation to denote the function that counts a given symbol in a string.

𝐿2 is slightly more interesting than 𝑉∗ because it introduces the notion that a formal language is equivalent to some computation. This insight is the fundamental idea that links formal languages and computability theory, and we will formalize this idea in the next section. But first, let’s see other, even more interesting languages, to solidify this intuition that languages equal computation.

Let’s define 𝐿3 as the language of all strings in 𝑉∗ where the number of 𝑎 is a prime factor of the number of 𝑏. Intuitively, working with this language—e.g., finding valid strings–will require us to solve prime factoring, as any question about 𝐿 that has different answers for string in 𝐿 than for strings not in 𝐿 will necessarily go through what it means for a number to be a prime factor of another.

But it gets better. We can define the language of all strings made out of 𝑎 and 𝑏 such that, when interpreting 𝑎 as 0 and 𝑏 as 1, the resulting binary number has any property we want. We can thus codify all problems in number theory as problems in formal language theory.

And, as you can probably understand already, we can easily codify any mathematical problem, not just number theory. Ultimately, we can define a language as the set of strings that are valid input/ouput pairs for any specific problem we can come up with. Let’s make this intuition formal.

Recognizing a language

The central problem in formal language theory is called the word problem. Intuitively, it is about determining whether a given string is part of a language. Formally:

Definition 1.6 (The Word Problem) Given a language 𝐿 on some vocabulary 𝑉, the word problem is defined as devising a procedure that, for any string 𝜔 ∈ 𝑉∗, determines where 𝜔 ∈ 𝐿.

Notice that we didn’t define the word problem simply as “given a language 𝐿 and a string 𝜔, is 𝜔 ∈ 𝐿”. Why? Because we might be able to answer that question correctly only for some 𝜔, but not all. Instead, the word problem is coming up with an algorithm that answers for all possible strings 𝜔—technically, a procedure, which is not exactly the same.

The word problem is the most important question in formal language theory, and one of the central problems in computer science in general. So much so, that we actually classify languages (and by extension, all computer science problems) according to how easy or hard it is to solve their related word problem.

In the next few chapters, we will review different classes of languages that have certain common characteristics which make them, in a sense, equally complex. But first, let’s see what it would take to solve the word problem in our example languages.

Solving the word problem in any finite language is trivial. You only need to iterate through all of the strings in the language. The word problem becomes way more interesting when we have infinite languages. In these cases, we need to define a recognizer mechanism, that is, some sort of computational algorithm or procedure, to determine whether any particular string is part of the language.

For example, language 𝐿2 has a very simple solution to the word problem. The following Python program gets the job done:

def l2(s):
    a,b = 0,0

    for c in s:
        if c == "a":
            a += 1
        else:
            b += 1
    return a == b

A fundamental question in formal language theory is not only coming up with a solution to the word problem for a given language but, actually, coming up with the simplest solution–for a very specific definition of simple: how much do you need to remember. In other words: what kind of algorithms can solve the word problem for what kind of languages?

For example, we can solve 𝐿2 with 𝑂(𝑛) memory. That is, we need to remember something proportional to how many 𝑎’s and 𝑏’s are in the string. And we cannot solve it with anything less than that, as we will prove a couple chapters down the road.

Now, let’s turn to the opposite problem of generating strings from a given language and wonder what, if any, is the connection between these two.

Generating a language

Suppose you want to generate all strings from a language like 𝐿2. To make things simpler, let’s redefine it as 𝐿2′, the language of strings over {𝑎,𝑏} with the same number of 𝑎’s and 𝑏’ but where all 𝑎’s come before all 𝑏’s. This means 𝑎𝑎𝑏𝑏 is a valid string in 𝐿, but not 𝑎𝑏𝑏𝑎. This language is also called 𝑎𝑛𝑏𝑛, that is, 𝑛 symbols 𝑎 followed by 𝑛 symbols 𝑏.

Here is a simple Python method that generates infinitely many strings from 𝐿2′:

def generate_l2():
    s = ""

    while True:
        yield s
        s = "a" + s + "b"

Let’s unpack this. We start with the empty string 𝜖, defined in code as s = "". Then, we enter an infinite cycle where we yield the current string, and then attach an 𝑎 to the front and a 𝑏 to the back. Take a moment to convince yourself that any string in the form 𝑎𝑛𝑏𝑛 is eventually generated by this method and, furthermore, only those strings are generated by the method.

This method is actually pretty neat because it not only generates (eventually) all of 𝑎^𝑛 𝑏^𝑛; it does so in increasing length order. It isn’t immediately obvious why this is such a good thing but here’s a bold claim: if you have a generating method for any language 𝐿, then you have a recognizing method too.

Wait, what!? Yep, you heard it right. And actually, it goes both ways. If you have a recognizing algorithm, you also have a generating one. Let’s make this our first theorem in formal language theory.

Theorem 1.1 Let 𝐿 be a formal language. There exists an algorithm 𝐴 for generating all strings in 𝐿 (in increasing length order) if and only if there also exists another algorithm 𝐴′ for solving its word problem.

Proof. To prove this, let’s first understand what the theorem is saying. If we have an algorithm 𝐴 that generates all strings in a language, we can also come up with another algorithm 𝐴′ (presumably using 𝐴) that solves the word problem, and vice-versa.

To prove this type of theorems, the most usual approach is to assume you have 𝐴 (or 𝐴′) as some kind of abstract, black-box algorithm, and try to construct the other. Let’s do it from generation to recognition first, as the other way around will be fairly easy once this is done.

⇒ Suppose we have an algorithm 𝐴 that generates all strings in 𝐿, and we are given an arbitrary string 𝜔. Let 𝑛=|𝜔| be the length of 𝜔. We just need to run 𝐴 until we either see 𝜔, in which case the answer is true (𝜔 ∈ 𝐿) or until we see one string with length greater than 𝑛, in which case the answer is false (𝜔 ∉ 𝐿). Since 𝐴 generates strings in increasing length order, one of these must happen in a finite time for any 𝜔.

Now, let’s do it the other way around.

⇐ Suppose we have an algorithm 𝐴′ that solves the word problem from 𝐿. Then we do the following. Define 𝐿∗ as the universe language associated with 𝐿. We can very easily code a generating algorithm 𝐴∗ for 𝐿∗ in increasing length order, simply by permuting all symbols. Now, run 𝐴∗ and, for each string 𝜔 generated, run 𝐴′(𝜔). If the output is true, then yield 𝜔. Otherwise, skip it.

So there you have it. Generating (in increasing order) and recognizing are two faces of the same problem. Cool, right? But why does this matter? For starters, it gives us a tremendously powerful connection between two sub-branches of formal language theory that we will explore in the following chapters.

Moving on

We are just scratching the surface of what formal language theory can do, and we have already touched upon several areas of computer science.

We have defined a super general notion (language) that is ultimately as profound and powerful as the very notion of algorithm. We have identified a central problem in formal language theory (the word problem) that is as deep as the very question of what problems can be solved, at all, with a computer. We connected two fundamental problems in languages (recognizing and generating) and discovered they are but two sides of the same coin. And we left hanging the question of which languages can be solved with which types of algorithms, which is ultimately a question about complexity theory. Phew!

In the following few articles, we will continue exploring the world of formal languages. We will dive into the different classes of languages according to the complexity of their generating and recognizing algorithms. We will find many intriguing unsolvable problems that have deep connections with other areas in computer science, from the most practical to the most esoteric. When we finish this dive, we will have a much more solid understanding of what computers can ultimately do. And then, will turn to programming languages and apply all these ideas to solving the more practical problem of building a compiler.

Yesterday, I had a delightful conversation with from on the general topic of language models and creativity. We went from a somewhat technical explanation of the most essential sampling parameters in LLMs (temperature and top-p) to practical advice on leveraging these tools to control the randomness or variability of the output, as well as more philosophical discussions on the nature of creativity and whether machines can ever be truly creative.

You can watch the whole 1-hour conversation here. But if you prefer a TL;DR, here are the main takeaways.

TL;DR:

The two most important parameters to control how an LLM produces text (other than the actual prompt) are temperature and top-p. These two parameters are found in almost any LLM API provider, including OpenAI and competitors.

In short, temperature is a factor that controls the shape of the token distribution. Higher temperature values tend to flatten the probabilities, making the model more likely to pick from a wider range of tokens, thus making it more random. Lower values make the model pick the most likely continuations, reducing randomness.

On the other hand, the top-p parameter cuts the distribution after some number of tokens with an accumulated probability mass. The lower this value, the fewer tokens are available for the model to pick as a continuation. Here is a simple illustration of this explanation.

They seem similar, but they have different effects. The temperature is a soft control, which tends to make the model pick more or fewer tokens, but any token still has a chance to be selected. In contrast, top-p restricts the selection to just a handful of tokens, effectively constraining the model so it won’t pick the less probable tokens.

Thus, you can increase the temperature to make the model more random and decrease top-p at the same time to avoid it going too far off the rails.

Playing around with these two parameters is an excellent exercise to help you grok how LLMs actually “think” (spoiler alert, they don’t, at least not in any human-like way). I actually made a demo, and you can try it out to see this effect in action.

Finally, one of the best ways to leverage this randomness in practice to produce novel but quality ideas or solutions to problems is to use a diverge-converge workflow. You take any LLM, crank the temperature up to make it go wild, and generate 10 solutions to whatever problem you want. Then, take those 10 and make a new prompt asking the model—now with a sensible temperature—to evaluate them and pick the best ones.

But this is just scratching the surface of what Lance and I discussed. Please check the full video (it’s free) and leave any comments or questions below.

This article is part of my in-progress ebook How to Train your Chatbot, jam-packed with practical advice and fully working applications on how to use LLMs for building all sorts of cool stuff. You can get early access for half the usual price.

How to Train your Chatbot - 50% off

So far, we’ve connected LLMs with other systems, extending their knowledge and enhancing their capacity to solve complicated problems by leveraging different tools. But all the tasks we can solve so far are essentially single-shot, requiring little to no planning once the task is well understood and the right tools are selected; regardless of how complex the tasks we’ve seen, the number of steps they require is fixed from the beginning.

To handle the most complex tasks, we must give LLMs the ability to pick and use arbitrary tools and actively reason about the task while performing it, incorporate new insights, and change their minds during the solution process. LLMs must become active agents rather than passive tools. This is what the agent paradigm is supposed to unlock. Agents are not a new thing in AI or computer science in general, though. In fact, it is one of the most important software design paradigms since the primordial ages—back when Turing was thinking about German codes.

In computational terms, an agent is just a software component that acts autonomously, interacting with a (computational) environment and constantly making choices to further some goals. The difference between traditional software and agent-based software is that, in traditional software, all changes to a system are in response to some user input: that is, the user is the active element. In agent-based software, the software itself is active in the sense that it is always making adjustments and producing effects on the computational system, regardless of whether there is a user.

For example, consider the difference between a passive AI assistant, like Alexa or Siri, and an active one. In the former case, the assistant is essentially dormant until you make a request. Then, it turns on, parses your query, and takes any actions needed to fulfill your request before going back to sleep. All interactions are started by you, the user.

In contrast, an active AI assistant would always be processing stuff, reading the news, scanning your emails, searching in Google, etc. It could forward you any interesting links it finds online and update your calendar as new tasks arrive in your inbox. But, more importantly, it could be working on a specific task for as long as necessary without your continued input.

Now, you can stretch the definition so that any piece of software looks agent-based. Your operating system is always on, doing stuff, ready to ping you when there’s a problem. It has a goal—keeping the system running—and it can interact with an environment—hardware drivers, peripherals, hard drives, etc.—and exert actions to make changes to the system. But this is seldom a helpful way to think about your operating system, mainly because its goals are pretty straightforward, and the actions it can perform are very restricted.

When you give a software agent long-term goals without immediately apparent steps, things turn interesting. The agent must then observe the environment, gather information, maybe ask you for clarifications, and make a plan to further those goals. Moreover, as it discovers new information, it may refine those goals, split them into lower-level subgoals, and generally navigate around a complex decision-making process that wasn’t explicitly programmed or even obvious to an external observer from the beginning.

Adding language models to the mix unlocks exciting new possibilities. Now, your agent can not only communicate with you in natural language but also talk to other agents easily—there is no need to define formal communication protocols or APIs—and even reason and argue with itself in natural language.

Anatomy of an LLM-powered agent

An LLM agent is, ultimately, just a plain old AI agent with some linguistic capabilities. So, let’s review the general definition of an agent in the AI field and explore some of the basic agent architectures invented.

In its most straightforward conception, an agent is a software component embedded within a larger computational system we call the environment. This environment can be anything from a computer simulation to the whole Internet, including other agents (computational or otherwise). The agent always has some goals—which can be as abstract and ambiguous as you want—that it must further by performing some actions. For this purpose, the agent has some sensors by which it can obtain information from the environment and some actuators that allow it to exert changes in the environment. Finally, the agent has some internal reasoning process to decide which actions to perform given the current environment state.

This picture is complete from the outside, but we can further refine it if we look inside the agent. Here is where literature is—quite literally—overflown with variants of “standard” agent architectures. So, I won’t attempt to cover the majority of variants or even be too formal. Instead, I’ll explain the main ingredients in most used agent architectures, using a casual nomenclature.

The most basic agent that does some form of reasoning needs to do two separate things: keep track of the state of the world (which may include its own state) and then reason about that world to decide which actions to take. This reasoning can be cyclical, updating its own state with new information inferred from the previous state. We can thus come up with a sensible basic agent architecture.

This architecture is sufficiently detailed for us to discuss different variants while being sufficiently abstract to allow many different use cases and implementations. For example, we haven’t explicitly defined what goes on in the internal representation of the world: it could be anything from a structured set of values to a list of natural language facts to a full-blown database. We also haven’t specified what sensors or actuators actually are, which depend heavily on the concrete use case.

Adding natural language

So far, we haven’t used the fact our agent is LLM-powered, and for good reason. The basic agent architecture we just defined is agnostic to implementation details. Adding an LLM to the agent architecture just means some or all of the modules are implemented using language models: that is, with prompts, RAG, function calling, and everything else we’ve seen so far.

First, perception may be LLM-based: your agent can receive information from the environment in the form of natural language documents. Likewise, the actuators can be LLM-based, e.g., via function calling. More importantly, the reasoning process and internal representation may also involve language models. The internal representation can be a straightforward collection of natural language claims about the environment, which the agent updates and refines via reasoning. The reasoning can be guided by some clever prompting and thus occurs entirely within a linguistic framework. Finally, the environment can contain other language models or be implemented as a language-based simulation with clever prompts.

Use cases for LLM agents

The most obvious use case for an LLM-based agent is some sort of always-on assistant, your personal Jarvis if you will. Unfortunately, most AI assistants today are more tools than agents. They turn on after a user command, do their thing, and return to idle. You start all interactions between you and Siri, Cortana, or Google Assistant.

Moving towards more agentic behavior, we should expect an AI assistant who has long-term goals—e.g., keeping you informed or keeping your calendar well organized—and is always consuming information and taking actions to further those goals. So here are a few possible, very broad use cases for such an agent.

The planning agent

This is one of the most useful and yet unexplored use cases. An agent connected to all your input sources—email, work and family chat groups, etc.—collecting all information about upcoming events and tasks. Its goal would be to help you maintain a healthy work-life balance by keeping your calendar current. The agent could reply to a meeting email and ask for a reschedule if it knows you’ll be busy that day. Now imagine everyone has such an agent, and they all negotiate with themselves and find the optimal schedule for the group meeting.

The research agent

This is an agent whose goal is to answer a complicated, nuanced research question. It can be related to your work, like mapping a business opportunity, or more akin to a typical scientific research question. The agent would search online for relevant sources, extract insights, update its internal representation, and generate new follow-up questions, building a web of interconnected information for a given domain.

The coding agent

This is the software engineering holy grail: an agent that continuously scans and improves an existing codebase, perhaps adding unit tests, comments in code, or refactoring existing functions. Additionally, this agent could pick low-complexity issues (like obvious bug fixes) and attempt to solve them automatically, even crafting a pull request when all test cases pass. Furthermore, it could revise existing pull requests and suggest improvements or general comments before a human revisor takes over the final merge.

The gossip agent

This is more tongue-in-cheek, but imagine an agent monitoring all your social accounts: Facebook (who has that?), Twitter/X, LinkedIn, your friends’ WhatsApp group chats, etc. Every time some event happens—e.g., someone announces they got a job or are traveling somewhere, etc.—the agent updates its internal representation of your social network. And periodically, or whenever something significant changes, it will send you updates on who’s doing what or going where.

Beyond the single-agent paradigm

The above are reasonably straightforward applications of the agent paradigm using LLMs. While the devil is in the details, and implementation is often much harder than first thought, I can see all of the above examples working in the near future, at least within reasonable constraints. However, single-agent architectures are just scratching the surface of what this paradigm can bring. The real deal is in multi-agent environments.

In a multi-agent setup, different agents coexist, collaborating and/or competing to achieve common or distinct goals. The power of multi-agent architectures is that a complex problem may be very hard to model with a single agent but feasible when subdivided into distinct goals and skills. Take, for example, the coding agent scenario mentioned before. Instead of a single agent taking all these tasks one at a time, imagine multiple agents working in parallel, some optimized to fix bugs, others to comment code, and others yet to revise, add tests, make suggestions, etc.

And this is more than just N different agents working in parallel on a codebase. These agents may have to coordinate to solve a big task. For example, maybe one “software architect” agent will take a complex issue and make a plan, dividing it into subtasks for different features. Then other “implementation” agents will work out the simpler tasks and do the actual coding.

Furthermore, you can have gatekeeper agents that revise code and emit some sort of qualitative evaluation. They can even ask questions, and the implementors must convince them of their solution. A lot of old and recent research suggests that, when combined, multiple agents can solve problems that none of them could tackle on their own.

Why would this work better than just a super coder LLM? You can consider it an advanced form of ensembling mixed with self-reflection, which are, in turn, advanced forms of chain-of-thought. Combining multiple LLMs with different tasks (i.e., different prompts) in iterated interactions gives them more chances to revise and regenerate, which translates into more computational power. At the same time, we’re leveraging the ensemble effect of pooling different solutions to the same problem, which tends to produce better solutions on average as mistakes are smoothed out.

Symbolic learning in LLM agents

Putting a bunch of LLM-powered agents to work together is neat, but we can go further. The next step is to make our agents learn from their interactions with the environment.

In the classic agent architecture we’ve discussed so far, no explicit learning is happening inside the agent. To some extent, you can think of updating the internal representation as learning about the environment. Still, there is no learning about the strategies the agent can use. The reasoning module, however, is implemented, has no way (at least so far) to update itself so that, given the same environment, it could perform better in future occasions instead of repeating the same mistakes.

The standard solution to this problem is reinforcement learning. Assume some optimizable inference architecture inside the agent’s reasoning module (e.g., a neural network) and use feedback from the environment to update its parameters. But what if we could achieve something similar using a purely linguistic approach?

One way to achieve this is to give the agent the ability to reason about its own performance and generate hypotheses—natural language claims—describing what it has learned. These hypotheses can be stored in a private knowledge base (separate from the environment description) and retrieved during reasoning.

In the simplest case, this is a long-term memory of what the agent has tried in different situations and whether it has worked. In future situations, a RAG approach could provide examples of things that have worked before. However, this can be refined further by progressively asking the LLM to produce high-level hypotheses that abstract and summarize similar situations, eventually building operative knowledge more general than any specific past situation.

What’s incredible is that this approach has improved reasoning and problem-solving skills in domains as complicated as answering medical questions. In a recent paper, researchers put agents in a simulated hospital environment and generated thousands of cases of patients with different conditions. The whole simulation is run by linguistic agents, from patients to nurses to doctors, who interact with each other and decide on a treatment. External rules (from a medical knowledge base) simulate whether the treatment is effective or not. After each case is completed, the doctor agents write a detailed report, which is stored in their internal knowledge base.

After ten thousand or so iterations, the doctor agents had built a case database that contained distilled knowledge about every possible rundown of diseases, symptoms, treatments, etc., they had seen. This is equivalent to what medical residents see in two years of training. And incredibly, when provided to a GPT-4 single-shot question answering bot, this database improved its results in standard medical question benchmarks by almost 10%. In a sense, the LLMs taught themselves medicine by interacting with each other in a simulated medical scenario.

This is akin to reinforcement learning but is parameter-free, 100% transparent, and explainable. Instead of updating some opaque weights in some neural networks, your LLM-powered agent is learning like humans: by analyzing its actions, taking notes, and constructing hypotheses. While highly experimental, this is one of the most exciting use cases for LLM agents I’ve seen recently.

Caveats and limitations

As in everything related to LLMs, you must beware of hallucinations and biases. Ensembles, in general, and agents, in particular, are more robust to random errors because of their distributed nature. This means that unlikely hallucinations that could be deal-breakers in single-LLM scenarios might get smoothed away when you pool together multiple LLMs, taking turns to make choices.

However, biases are not random but systematic errors. Using the same model to power all your agents means they are all susceptible to making the same judgment mistakes over and over. There is no algorithmic solution to reducing biases that doesn’t attack the original problem: biased data.

Another major caveat of LLM agents is that they are significantly more costly compared to a single zero-shot or a simple chain-of-thought prompt. Your agent is always running, which means you’re constantly calling the LLM API, even if there is nothing new to be done. This underscores the need for in-house, open-source language models that can be cost-effectively scaled to these heavy-duty scenarios.

Finally, agents in AI have a long history of being abused as an overengineered solution to problems that could otherwise be solved with more direct methods. Do your own experiments and test whether simple prompts or more straightforward augmentation techniques are sufficient for your use case before building an entire simulated environment.

Conclusions

The agent model is one of the most exciting paradigms in artificial intelligence, with a long and rich history of architectures and approaches. It is also one of the hardest to implement, at least until now, because it always required far more intelligence than was possible with previous approaches. However, with LLMs, agents are seeing a rebirth as one of the most promising approaches to building reliable and truly intelligent AI solutions.

I am, however, fairly biased. This is one of my dearest research topics, and I may be over-excited. So take everything I said here with a grain of salt. Modern agent architectures are still brittle and hard to deploy on a large enough scale for their distributed strength to be noticeable. Still, they hold great promise and are one of the few paths forward that I believe may take us all the way to general-purpose AI.

We will build some agent-based simulations in Part 3 of the book, especially for the final chapters. Due to their complexity and cost, we won’t spend as much time building agents and simulations as I would like, but you’ll get a taste of how powerful this paradigm can be in chapters 24 and 25.

If you want to learn more, grab your early access to How to Train your Chatbot in the link below.

How to Train your Chatbot - 50% off

Last week, we saw how function calling is one step towards enabling LLMs to interact with external systems by giving them more flexibility in choosing which operations to perform—i.e., which methods to call—based on the conversation context. But, in the end, function calling still limits the model's capabilities. It is restricted to the preprogrammed functionalities we give it access to.

This may be precisely what you want because it gives you total control over the action space of the LLM. However, for some of the most complex tasks, it may be too hard to develop a flexible enough API.

Suppose you’re making a data analysis bot, that will read a CSV file and answer questions about it. You can ask it to group, filter, or otherwise process the data in myriad ways. You could conceivably come up with a set of functions that cover your entire question space, but you would end up coding something that resembles the pandas API. What you actually want, is for your chatbot to write and run pandas code!

Enter code generation.

Instead of developing a very flexible and broad set of API functions, simply let your bot write Python code (or any other programming language) and run it. If the model is sufficiently trained in code generation, it can often solve most of the low-level coding tasks you would end up encapsulating anyway.

There are many flavors to code generation, ranging in complexity from single instructions to fully working applications. There is also the question of how that code gets used. One option is to execute it immediately, maybe even transparently to the user, to obtain a response. In this case, the result from running the code is what matters rather than the code itself. On the other hand, you might want the code as the end result, maybe to copy and paste it somewhere else.

In this article, we will explore the most interesting use cases for code generation, and some tips and strategies to get the most out of language models that can code.

This article is part of my in-progress book How to Train your Chatbot, currently over 150 pages of practical advice to build all sorts of cool stuff with LLMs, including prompt engineering principles, deep but intuitive theoretical explanations of how LLMs work, and hands-on coding tutorials building actual working applications.

You can get the early access today and help Mostly Harmless Ideas grow.

How to Train your Chatbot - 50% off

How code generation works

In the simplest case, you can think of code generation as a subset of text generation. If your model is trained on mixed natural language and code input and/or fine-tuned with coding instructions, it will naturally learn to answer some prompts with actual code. For example, you can train an LLM in programming contests, where the input is a problem statement, and the output is the code of the solution in some programming language.

It is at least somewhat surprising that vanilla LLMs, trained on code, can learn to write code at all. The reason to be skeptical is that programming languages have very strict syntax rules, which make it hard, at least in principle, for a purely statistical language model to produce parseable code, not to mention semantically correct. Failing to produce a single semicolon in the exact location can make an otherwise perfect piece of code incorrect. Yet, LLMs learn to code, almost without additional effort.

In fact, most general-purpose models now available have at least some general capabilities for code generation, if only because they are trained of vast datasets that contain, among the many types of text modalities, lots and lots of code. And even if you don’t want an LLM explicitly for code generation, training on code and text (rather than just text) has been shown to improve the general reasoning capabilities of a model, even for text-only tasks!

But although you can get reasonably good code generation almost for free, the best coding LLMs are fine-tuned on precise text-and-code datasets.

There are many reasons to prefer a model fine-tuned for coding to a general one. The most straightforward argument is that contrary to natural language, highly plausible code can still be incorrect. Fine-tuning a model specifically on code reinforces the syntax rules and makes it much less likely to generate almost correct but still syntactically wrong code.

In the same vein, since programming languages are much more rigid in syntax than natural language, fine-tuning can make a smaller model as good or even better than larger, general models if focused on a specific language. Likewise, even if your general LLM can code Python, it may not know the specific framework you’re interested in or code with the exact style you want.

Code generation use cases

In this section, we’ll look at code generation from a high-level perspective to understand what are the most interesting use cases it unlocks. We won’t go in-depth into technical details of how to make these use cases work in practice, as we will have plenty of time in Part 3 of the book to see concrete examples in action.

Code completion

The simplest use case for code generation is straightforward code completion. This can be as simple as asking ChatGPT the code to solve a concrete problem without any additional context. However, this use case becomes much more interesting when you can provide the LLM with relevant context (existing code, class hierarchies, function declarations, etc.) and have it produce code that fits right into an existing codebase.

At its core, code completion is just a form of text completion with all the quirks and caveats. The same prompt can produce different results, and slightly different prompts can vary widely in output quality. So far, all prompt techniques we’ve discussed apply: format your prompt carefully, be precise, don’t overshare, use examples whenever possible (no negative examples, please), and be intentional and cognizant of all usual gotchas.

Code maintenance

Code maintenance is a slightly more advanced form of code completion, in which we ask the model not code to support new functionality but rather to modify existing code (or add to the existing code) to improve its quality, maintainability, clarity, etc. A typical example is automatic refactoring: asking the model to, e.g., extract some functionality into its own function or otherwise abstract or encapsulate relevant fragments of code.

This process can be enhanced with prompt templates for everyday tasks, including generating boilerplate code or running typical refactors like splitting methods, encapsulating functionality, or changing style from, e.g., a recursive implementation to an iterative one.

Another form of code maintenance is adding unit tests. A careful explanation of the relevant functional requisites in natural language might be enough to have a model generate reasonably good unit tests for the most common use cases.

Translation and explanation

The previous use cases are mostly examples of language-to-code generation. On the other hand, we can have code-to-code and code-to-language generation.

The first case is helpful for code translation. A simple example is translating code from one programming language to another, perhaps because you found the solution to your problem online, but it’s not in the programming language you need. But you can also translate between two versions of the same language, say, from Python 2 to Python 3, to update an implementation. Or translate between different frameworks or different implementation styles.

The second case is helpful for automatically adding comments to existing code or otherwise generating explanations in any level of detail. As usual, the accuracy of the explanation is subject to how powerful the model is, and how complex the code. In general, it is impossible to understand what a given code will do without executing it, but you can get pretty far, at least in the most common scenarios.

Using external tools

Now, we get into the domain of code as a means to an end rather than the end in itself. You can use code generation to interface with external tools that either don’t have a high-level functional API or that, by their nature, are code-based.

An example of the former is using any of the myriad utilities in your terminal via Bash scripts. If you want your LLM to be capable of, e.g., creating files, making commits, downloading stuff, compressing and moving things around, etc., it is very likely that a reasonably good LLM can generate a Bash one-liner or small script to work these tasks out.

In the latter case, you may want to interface with code-based tools, such as SQL databases, or any special-purpose computing engines, from Wolfram Alpha to a SAT solver or an in-house tool. If the language used by that tool is not mainstream—meaning, the LLM won’t be trained on it—you’ll need to fine-tune on it.

And finally, you can interface with programming frameworks that have, e.g., a Python-based API. Again, unless the framework is very well-known—e.g., sklearn—you may need to fine-tune your model to teach it how to use that concrete API. But in many cases, the model might generalize from its basic Python knowledge to specific APIs with a small set of carefully curated k-shot examples.

Enhanced reasoning

LLMs are lousy at mathematical and logical reasoning. This is somewhat surprising at first because computers are supposed to be precise mathematical machines. However, when you understand how LLMs “think”, you realize they don’t have any explicit mechanism for even the simplest mathematical operations. But you know what does? Python! (and any other programming language).

Code generation is the most effective way to enhance the mathematical skills of LLMs. Instead of having the model directly answer questions involving mathematical operations, make it generate a short code expression that computes the right formula, run it, and feed the result back to the LLM. This way, you can “teach” an LLM to solve complex mathematical problems by doing the same thing we humans do: using the right tool.

But the possibilities go far beyond simple (or fairly involved) mathematical operations. You can pair an LLM with any of the many specialized inference engines the AI community has invented over the decades. Make your LLM generate Prolog code and voilá, you have a general-purpose logic reasoning engine from natural language. Or make it generate SymPy expressions, and you have a symbolic reasoner.

But this is no silver bullet, of course. Your LLM can simply fail to generate the right code. So, even if you have the perfect computing engine that solves the right problem, getting a language model to generate semantically correct code for that engine is an open problem and one which is ultimately unsolvable according to basic computability theory.

However, for many practical cases, given enough examples for k-shot or a small fine-tuning process, you can get an LLM to learn how to solve interesting problems reasonably well. This is an active area of research, so we can only expect these capabilities to improve in the near future.

Prompting tips for code generation

In many cases, you can make an LLM write code simply by asking. A prompt like “Generate a Python function to find the third maximum element from a list” will work almost flawlessly in any sufficiently capable language model you can find today. This works fine for many use cases where the code is all you need. At least, it is no worse than searching for a similar snippet of code online.

However, there are several drawbacks to this KISS approach. First, most commercial LLMs are fine-tuned for chat, so they are… chatty. Instead of the raw code, they might answer with something like “Sure, here is a function in Python to do ….” and then the code. This makes it hard to integrate them with external tools that need just the code because then you have to parse the response.

In many cases, you can get away by adding an explicit instruction like “Please answer just with the source code”, but still, some models may refuse to comply. And even if they comply, different models output code in different formats. Some will enclose the code in markdown-style code block annotations, while other models might indent the code. It depends heavily on their training data.

Another problem you may face is when asking for one-liners, i.e., single instructions or expressions that you want to evaluate with, e.g., the eval function in Python. If you ask for a single pandas expression to, say, group and filter a dataset, the model may sometimes produce a proper expression—e.g., df.groupby(...).agg(...)—and other times an instruction—e.g., df = df.groupby(...). You may work around these issues by doing some checking and post-processing of the response, like removing anything before the last = sign, but this is a very brittle approach.

In these cases, some of our well-known prompt techniques also apply. Be very intentional with the prompt and provide positive examples of the exact response format you expect. While none of this will 100% guarantee you’ll get a response in the format you need, when paired with a try-and-repeat strategy, you can often get away with the performance you need. For example, if the model makes a mistake 10% of the time, you’ll need to redo one in ten queries on average, which is not terrible.

In many cases, when retrying the same code generation task, it helps to include the previous answer and the error in the prompt. This can often be automated simply by trying to run the code, capturing any exceptions, and feeding the model back with the exception message, asking it to try to fix it.

Finally, with some tricks, we can force the LLM to produce syntactically correct code—even if it is not guaranteed semantically valid. The trick is to restrict the sampling step to only select among the tokens that would be syntactically valid.

Some open-source LLM inference engines, like llama.cpp, support passing a formal grammar that defines the programming language's syntax. During sampling, the engine will select only those among the top-k tokens that are valid according to the grammar's production rules. This can be done efficiently with a linear-complexity automaton constructed automatically from the formal grammar definition. While this is a relatively novel and, arguably, rather advanced feature, some commercial APIs are starting to support it.

Limitations and caveats

Needless to say, code generation is full of subtle and not-so-subtle problems. For starters, some hallucinations are going to happen, and this might result in several different types of problems. The simplest case is getting code that is not syntactically correct, that is, code that doesn’t parse. If this is your main problem, then you’re lucky because this is simple to check. Just run a linter for your target language and retry if you find any syntax errors.

A more complicated issue is when your model generates syntactically correct code that throws an exception. This is still not terrible because you can run the code and check for exceptions. However, running code generated by an LLM is a bad idea if you don’t have some guardrails. For all you know, the code may have an instruction to wipe out your hard drive. So, you must always run LLM-generated code in a sandboxed environment. This is especially true when running code generated by a user-facing LLM. Someone will hack that LLM to generate some system-breaking instruction.

The third level of problem is when your LLM generates code that runs without exceptions but doesn’t do the right thing. This is, in general, impossible to detect beforehand. However, depending on your specific use case, you may be able to check the result is what you expect and even roll back any potential side effects if that isn’t the case. For example, you can have your code work on a copy of the relevant data and check for any unexpected changes before merging that data.

However, this is the most important open problem in program synthesis from natural language. To fully solve it, I believe a new paradigm that goes beyond statistical language modeling is required.

Grammar-restricted output is one of the most effective ways to make code generation more robust. Still, this process is ad-hoc and has not been baked into the training process. Thus, the LLM can get stuck simply because it doesn’t give a high probability to any valid tokens. If the LLM wouldn’t naturally produce the correct response, at least with some non-trivial probability, no ad-hoc filtering mechanism can force it to generate the correct code.

This means adequate prompting and possibly fine-tuning for specific domains will remain relevant strategies in the near term.

Conclusions

Code generation is one of the most surprising (weakly) emergent capabilities in the current language modeling paradigm. The fact that pure statistical correlations between tokens give rise to something that can mostly code—granted, at the level of maybe a 2nd year CS college student, in the best of cases—is something I wouldn’t have expected to be possible even three years ago. On the other hand, strong code generation is one of the most powerful and versatile capabilities in language models. It opens the door for a myriad of integrations with existing and newly created tools.

A general-purpose language model paired with a general-purpose code interpreter is one step closer to AGI, there is no doubt. If Turing is right—and would you bet he isn’t?—, any problem that can be solved with an effective, repeatable, systematic method can be solved with any modern general-purpose programming languages. An LLM that can code at the level of the best human programmers would, almost by definition, have a general intelligence.

The only gap that needs bridging is getting the model to produce the right code. But this might well be the hardest problem in computer science. We know that it is generally impossible to know what a program will do without running it. But this doesn’t mean machines are necessarily less capable than humans. Who says our brains aren’t just very powerful machines?

But all of that is hypothetical. In the meantime, even with the many limitations of modern LLMs, code generation is one of the most useful tools available to build practical, useful applications using language models.

We will spend a lot of time in Part 3 of the book playing around with different strategies to turn LLMs into effective coders. Remember, you can get your early access copy today at a laughably low price. It is the best way to support these free educational articles.

How to Train your Chatbot - 50% off

I'm pleased to bring you this guest post by from , a deep dive into the very sensitive yet crucial topic of human-AI relationships. I hope you enjoy this post as much as I have, and that it sparks some interesting thoughts in your brain. It surely has for me.

Please consider subscribing to and support Riccardo. If you do subscribe, please send me a screenshot, and I'll give you a 100% discount coupon to one of my books of your choice. Sounds like a deal?

— Alejandro.

By .

AI surprises us every day. We appreciate its capabilities or we are frightened by it, we try to find the best use for us, and, above all, we experiment. What is the point? It is that in all this time of experimentation and use, what we do is, trivially, interact with a technology. I use the term interact because it is no coincidence that this has a different weight for an entity that we as human beings naturally tend to anthropomorphize, that is, to give human characteristics while knowing that there is nothing but a machine behind it. A machine that, although more and more 'intelligent' than our coffee machine or our washing machine, is still not a conscious entity - unlike what one might sometimes think.

Yet, we build relationships based on interactions with it, just as we do with human beings. This topic has become so important in research - as well as for technologist owners - for several reasons:

1. First, because it is important to understand what is peculiar or different from human interactions;

2. To understand what aspects and how they impact on our eventual adoption and use of this technology;

3. And, finally, because this 'AI' will be (and is being) hugely popular, and it is crucial to understand the implications for human nature.

In this regard, I am really happy today to be able to talk to you about this topic in 's newsletter, one of the publications that I read with the most involvement and with which I immediately interacted since I started writing mine, The Intelligent Friend, focused on humans-AI relationships. My issues are based only on scientific papers and naturally, even today, we will be "helped" by some (three specifically) works.

We’ll try to explore a specific aspect of relationships: how our "social" interactions change depending on whether we are dealing with a human or with AI? First we will try to answer specifically thanks to a paper that was a pioneer of the topic, published five years before (2017) ChatGPT arrived at the beginning of its popularity (2022). Then we will move on to the intelligent virtual agent that has probably accompanied people the most in recent years, Siri, and we will try to understand how we humans perceive collaboration with AI in a very particular context: a game.

Are you ready? So let's dive in!

Who am I chatting with?

As you will immediately understand and/or by reading my newsletter, I like to explain scientific concepts with concrete and impactful examples. So, let's start with one of these.

Let's say you find yourself chatting with a friend of yours on social media. Share jokes, news, opinions. Then you find yourself interacting with a chatbot, to talk about different things. Do you think this changes your level of extroversion, openness? ‘There shouldn't be any difference,' you might be thinking. Well, that's not really the case. The study by Mou & Xu (2017) tried to understand how we show different personality traits depending on whether we interact with a human friend and with a chatbot. Where did the scholars start from?

First, when we talk about human-machine interactions, there are some fundamental paradigms. One of these, also at the basis of this study, is the 'Computers Are Social Actors' paradigm (abbreviated as 'CASA'). It was developed by Nass et al. (1994, 1996) who tried to replicate some results from social science research with computers or televisions. The crucial concept is as intuitive as it is powerful: we treat computers as if they were real people.

Now, this paradigm has seen a huge following in research due to its various applications and its importance (I advise you, if you are interested, to read the dedicated paragraph of the paper, it is really interesting!) which naturally I will not deepen further here. The important thing is to keep in mind the concept underlying the paradigm, because it is also the conceptual basis on which Mou & Xu (2017) based their study. The researchers investigated interactions with chatbots and with humans on the same platform in the context of a social interaction, hypothesizing that people exhibited different levels of certain personality traits and characteristics.

Different subjects, different reactions

For this, one of the first widely used interactive chatbots, Little Ice, developed by Microsoft, and WeChat, where Little Ice was embedded, were chosen. Ten volunteers, just regularly using WeChat, got roped into sharing their chat transcripts, one with a chatbot named Little Ice and the other with a human buddy. They had no clue they were part of a study; these were just their everyday chats.

I would like to point out that these people were asked to select a conversation with a regular friend, not a close friend or significant other. Furthermore, only the first conversations with Little Ice and friends were analyzed to avoid relationship development bias. These volunteers were evaluated by 277 viewers on personality and communication attributes and also filled out self-assessment measures.

The results were very interesting: distinct behavioral patterns were observed when WeChat users interacted with Little Ice compared to human friends. Users exhibited higher levels of openness, agreeableness, extroversion, and conscientiousness when conversing with humans, while showing increased neuroticism with AI.

This suggests a preference for socially desirable traits in human interactions. Furthermore - and this is one of the things that I found most interesting - the exploratory study conducted suggests that users may apply social scripts from human interactions to human-computer interactions, but, it should be underlined, less openness and extroversion.

As also specified by the authors, this was one of the first exploratory studies to try to understand the differences between human interactions and interactions with a chatbot in a social media context. Among other things, at a time (2017) from which ChatGPT and such advanced technologies now appear very far away. Thus, this study, as well as many others, has begun to ask important questions about integrating relationships with a virtual agent or chatbot into our daily lives. How did this relationship evolve?

It's not just a matter of who, but also of what

To analyze a possible answer to this question, I will present the results of a different study, in which the protagonist is a "person" we know well. You talk to her to remember things, to help you while you cook, maybe to call someone. Yes, you understand, I am talking about Siri. "She", together with Alexa, has been the subject of numerous studies on the factors that could impact interactions with consumers. The study by Lee, Kavya and Lasser (2021) is very interesting in my opinion because it is one of those that put a very recurring theme at the center: trust.

If you think about it, trust is a fundamental factor in human relationships. We also think about our friends on the basis of how much we trust them, on the basis of how much we can confide in them. Here, this centrality of trust is also present in human-machine relationships.

In this study, it is defined as "the attitude that an agent will help achieve an individual's goals in a situation characterized by uncertainty and vulnerability” (Hancock et al., 2011). However, the authors also investigate a fundamental component of the relationship: the type of task.

To talk about it, let's stop for a moment.

Think about three things you interact with Siri for on a daily basis. Probably among your choices there will have been information search, scheduling, mathematical calculation. However, you could also have fun conversations with Siri or even look for support.

The former are defined as "functional" tasks, the latter as "social" tasks. The authors' objective was to understand how trust is associated in relationships with Siri with different types of tasks. In reality, the study is very elaborate and explores the interaction of various factors, such as gender, social presence and the actual type of relationships with AI. Nevertheless, since we have seen quite a few things, and we will also see another interesting paper shortly, let's focus on these aspects which are still very interesting (if you want, find the paper here to learn more about the rest).

So we were saying: how is trust in Siri interrelated to a functional vs. social task? To understand this, the scholars measured five dimensions of trust regarding participants' interactions with Siri. On these three, namely perceived reliability, technical competence, perceived understandability, there were significant differences: people have more trust in Siri in these dimensions when dealing with a functional task.

However, there is another interesting theme, constituted the part of trust connected most to the emotional sphere and which includes the remaining two dimensions that I have not mentioned so far: "affective trust".

The other two dimensions of the five, namely "faith" and "personal attachment", more linked to the emotional sphere, indeed did not seem to have seen a significant difference in the social task. From this it could be deduced that, at least in this context, the participants once again perceived the social task as something separate

AI, games and collaboration

Well. We saw a study from a few years ago that tried to explore the differences in how we interact in some conversations with chatbots. We saw the (partial) results of a more recent study which significantly revealed to us that it is not only who we interact with (Siri) which is relevant, but also what we do (functional vs social task) in this interaction. The third paper I chose for this issue that I had the pleasure of writing here on Mostly Harmless Ideas will be very intriguing for several of you. To introduce it, let's think about this question: What is something we have often interacted with a 'computer' for fun, without it actually being a 'physical computer'?

It won't take long for the gamers among the readers to respond.

Naturally, I’m talking about video games! Video games are a very interesting area for AI. Both for the related commercial developments and for the opportunities it offers to study human-machine relationships and interactions. In this paper by IBM researchers, the game is used as a context to measure and evaluate the effects of collaboration between AI and players.

In particular, the scholars tested different dimensions of the perceptions of players (rapport, intelligence, creativity, likeability) who were told that they would collaborate with a human, with AI or without specifying it.

To understand user perceptions of collaborators, the authors created a two-player cooperative game called Wordgame. In Wordgame, one player (the "giver") provides clues to another player (the "guesser") who tries to identify a hidden target word. 

The game starts with an AI-provided hint (e.g., "car" for target word "engine"). The guesser has 10 attempts, with each guess following a hint. As a cooperative game, players work together for the guesser's success, fostering openness and honesty towards a shared goal.

What were the results? People who believed their partner was human find them more likeable, intelligent, and creative, even if the interactions are identical to those with the AI. This effect is further shown in how people described their partners – using more positive words when they think they were playing with another human and more negative ones when they believed their partner was AI.

A final note

I wrote this issue by briefly including some concepts and results from different papers regarding interactions and relationships between humans and AI for several reasons.

First of all, I wanted to highlight how the research into human-machine interactions and relationships, although it sees fundamental paradigms and theories in constant development, is evolving very quickly and will see increasingly surprising developments.

Speaking of papers relating for example to the period 2022-2024, there is evidence on how people feel "love" for Alexa, on how your partner could be jealous of ChatGPT if used to find his birthday present and on how people may be more or less willing to share sensitive information with a chatbot than a doctor.

Secondly, I wanted to highlight the variety and facets of this topic and, above all, its importance when we often focus a lot on the technical aspects, considering the relational ones, which are of incredible interest, a step backwards.

Third, I chose three studies that, honestly, I thought could intrigue you and give you a first idea of how the topic of relationships can hide a thousand nuances yet to be discovered.

Finally, and this is one of my recurring mantras, I wanted to underline the beauty of research and how many things can be discovered by reading and sharing the results of scientific advances. Sharing knowledge is fundamental, and this is why I like to enhance the work of many researchers by writing about it in my newsletter.

I thank Alejandro again for giving me the opportunity to write this piece for you. If the topic of relationships has interested or intrigued you in the slightest, stop by my newsletter, The Intelligent Friend, and take a look. See you soon!

This is a guest post by one of my favorite writers, from . I’m priviledged to bring you a deep dive into the many facets of the man behind some of the most important scientific discoveries in the last century, and the undisputable founding father of the youngest of the major sciences.

Nick writes on crucial topics at the intersection of artificial intelligence and education. Subscribe to to read more of his work.

— Alejandro Piad Morffis

By .

I have been deeply reflecting on Alan Turing lately, especially in the context of this third era of Artificial Intelligence. Recently, I delved into Aaron Hodges’s classic biography of Turing, a monumental work that took me the better part of a month to absorb thoroughly.

Hodges, a quantum physicist who studied under Roger Penrose and was active in the Gay Rights movement within the English academy during the late 1970s, stumbled upon Turing’s story through his activism. This discovery launched him into a rigorous six-year investigation, piecing together the extraordinary life of a man who left behind a surprisingly sparse historical footprint. Hodges's method—rooted in a blend of interviews and exhaustive archival work—culminated in what has been heralded as “a new kind of biography,” weaving together mathematics, science, computing, war history, philosophy, and gay liberation into a singular narrative of a complex individual.

As I navigate the landscape of the third AI era, I am struck by the stark contrast between the unassertiveness of Turing’s public persona and the definitive proclamations flooding our current discourse. Educators wholeheartedly reject AI in classrooms or endorse wholesale handover of curricula to algorithms. Product designers and builders tout their creations as revolutionary, as epochal as the internet itself, while consultants offer ironclad guarantees of success. Consultants jockey to position themselves as experts in a field that is continuously evolving and changing. 

In the magisterial final chapter of his biography, Hodges portrays Turing as a figure of “iconoclastic originality.” Yet, Turing himself never sought the limelight or hawked his achievements. He did not deal in guarantees about the future, nor did he ever tether himself too long to any single project. This humility and restraint seem all the more poignant when juxtaposed with the current era’s rush towards an AI-integrated future, often championed with a certainty that Turing would likely have questioned.

Alan Turing: An Immersive, Experiential Worker

First and foremost, Alan Turing was an immersive and experiential worker. He delved deeply into every project, thoroughly understanding its nuances. Hodges emphasizes throughout the biography that Turing never quite fit the traditional academic mold, preferring instead to test theories through practical application. As a teenager, Turing's fascination with chemistry led him to study the gaseous emissions of rotting seaweed. Later, as a mathematician, his wartime code-breaking work provided a foundation of practical skills that he applied in the development of early computers in the late 1940s.

Turing’s hands-on approach offers a cautionary lens through which to view the current enthusiasm for "language" coding with LLM chatbots. Perhaps if we as AI literacy experts fail to engage directly with coding, or at least with platforms like GitHub where the process becomes more visible and discrete, we risk detaching ourselves from a crucial level of experience with these new tools. The irony is palpable: social media is awash with self-proclaimed AI experts, yet only a few truly understand the inner workings of these technologies. Here, dear readers, I acknowledge my own gaps in knowledge as much as anyone’s—and it's why I'm gearing up to take coding classes this summer.

Extended Periods of Deep Immersion Fueling Extraordinary Bursts of Imagination and Creative Achievement

Turing's extended periods of deep immersion fueled extraordinary bursts of imagination and creative achievement. I would like to highlight a few of Turing's remarkable and imaginative breakthroughs:

The Halting Problem: Validity vs. Veracity

In the mid-1930s, Turing's deep engagement with set theory and Gödel's incompleteness theorem, combined with his profound affection for mechanical devices, set the stage for his seminal breakthroughs. Focusing on the "Entscheidungsproblem," which challenges the existence of an algorithm to determine the universal validity of statements in first-order logic, Turing addressed this through his work on the halting problem. 

In 1936, he ingeniously demonstrated that no such algorithm could exist, using a hypothetical solution and designing a special Turing machine to expose inherent contradictions. Check out this video for an amusing, but accurate explanation. This fusion of theoretical mathematics and practical insight not only propelled Turing’s own groundbreaking contributions but also laid the foundational framework for modern computing and the discipline of computer science.

Practical Computing: Finite Solutions

In the mid-1940s, Turing applied the lessons he learned from code-breaking and developing voice-encryption machines like Delilah to the practical task of building the first computer. The resulting machine, the Ace, proved to be exceptionally powerful. Yet, much like during his wartime projects, Turing found himself increasingly marginalized from the project's core team, due to his reluctance to wield power and promote himself. Instead, Turing thrived on direct interaction with the Ace, becoming its most adept operator.

Observers often likened him to a concert organist as he masterfully manipulated its dials and gears. During this period, Turing also penned his most visionary thoughts on the future of computing, transitioning his focus from universal to finite machines and highlighting the potential of limited memory and storage. It was while programming the Ace that he first formulated the idea of machine learning.

Morphogenesis: The Mathematics of Life 

In the late 1940s and early 1950s, Alan Turing revisited his lifelong fascination with nature by exploring the formation of patterns in biological matter. From a young age, Turing had observed the Fibonacci sequence manifesting in various natural forms, such as leaves, shells, pinecones, and flowers. Like many scientists of his era, he sought to explain these patterns at the molecular level. 

Turing hypothesized that morphogenetic fields might influence the development of chemical solutions, guiding their pattern formation. His experimental approach combined mathematics with tightly defined chemical states, focusing on scenarios involving one to four molecules. 

The numerical complexity of these interactions quickly exceeded the computational capabilities of the era's early computers, challenging Turing to balance intricate theoretical constructs with the practical limitations of technology. Turing's exploration into the patterning of biological matter marked his final project and evidenced a mind that remained vitally active, perpetually in search of its next major endeavor.

In each of these ventures, Turing derived little personal gain from his extensive experiments and labors. Though he occasionally ascended to positions of influence, he readily relinquished them whenever they hindered his pursuit of a compelling idea or project. Turing's approach starkly contrasts with the highly commercialized landscape of today's AI industry. This begs the question: how many groundbreaking ideas are we losing in our haste to transform them into marketable, viral, and transactional products? Turing’s example challenges us to reflect on the true cost of prioritizing profit over the unbridled pursuit of knowledge.

The Profound Creative Insights Stemming from Turing's Immersive Work Habits Were Underpinned by What Might Be His Most Crucial Trait: A Resolute Detachment from Criticism

This Stoic indifference—the essence of what Hodges refers to as Turing’s "iconoclastic originality"—is central not only to his intellectual freedom but also to the great tragedy that overshadowed his life.

Turing's indifference shielded him from the harsh criticisms of family, public school teachers, professors, and employers. Choosing not to follow a traditional academic path, Turing didn't receive the accolades that many of his contemporaries did. His pivotal contributions to the war effort could not be recognized during his lifetime, and for years after his death, he was remembered primarily for resolving Hilbert's 10th problem. It wasn't until the rediscovery and renaissance of his work in the 1960s and 1970s, as his visions of a computer age were realized, that his true impact began to be appreciated.

In the 1940s, Turing was relatively open about his homosexuality with friends, colleagues, and employers. His forthright nature led to severe consequences after he reported a theft by the friend of a lover, inadvertently exposing himself to criminal charges. 

Choosing hormone treatment over imprisonment, Turing continued to pursue his personal and professional life unabated, traveling to Sweden to be with a lover. Hodges speculates that Turing’s liaisons attracted intense scrutiny from the British government, possibly playing a role in Turing’s untimely death via suicide, though historical records on this are scant.

We ought to approach the task of distilling the intricate circumstances of Turing’s life into digestible insights with considerable caution. Yet, it's undeniable that there is something profoundly remarkable and admirable about Turing's steadfast resistance to external pressures.

Unlike so many of us in this digital age, Turing was not swayed by rumors, hype, personal gain, or even comfort. He was a stalwart figure, embodying honor and ethical integrity—an emblematic figure for us as we navigate the complexities of the 3rd AI Era. 

1. How do we stand firm against the latest news cycle? 

2. Are we cultivating a posture of resoluteness that allows us to commit to truly impactful work? 

These are the questions that Turing's legacy prompts us to consider today.

Thank you for reading this piece. If you found it meaningful, please leave a note. And consider exploring Hodges’s biography to delve deeper into the life of this remarkable individual.

By contrast, we find ourselves carried this way and that depending upon the changing winds of the news cycles and each other’s opinions.

For more captivating insights into how artificial intelligence is reshaping the way we teach, work, write, communicate, and uphold our humanity, subscribe to my Substack . Please consider becoming a paid subscriber so that I can continue to publish incredible content for my readers every Monday. I am always excited to hear from my readers!!!

Drop me a line at potkalitsky@gmail.com.