Artificial Neural Networks Are Nothing Like Brains

Busting the biggest public misconception in AI

Unless you've lived under a rock for the last couple of years, you already know that neural networks are the workhorse of AI. They’re everywhere—driving advancements from voice assistants to self-driving cars. But what makes them so unique?

Neural networks are not just a passing fad. They are incredibly powerful and flexible tools that can learn complex patterns from vast amounts of data. While they were initially inspired by how our brains work, today’s neural networks have evolved far beyond that simple analogy.

But before we further explore neural networks' power, we need to clarify one thing: artificial neural networks are nothing like the brain. This crucial distinction is often overlooked.

In this article, I will attempt to demystify the notion that artificial neural networks are anything like biological brains. And then, in a follow-up article, we’ll dive into the mathematical characteristics that give them flexibility and discuss how they scale with data and compute power. By the end, you’ll have a clearer picture of why these models are at the forefront of machine learning today.

Let’s get started!

Differences between Artificial Neural Networks and Biological Brains

Believing that NNs function similarly to biological neurons can lead to misconceptions about AI. For instance, it might make you think that artificial general intelligence (AGI) is just around the corner when, in reality, we are still far from achieving that level of complexity. It can also lead to anthropomorphizing AI, attributing human-like qualities and emotions to models that are fundamentally mathematical constructs. Understanding this difference is vital for grasping both the capabilities and limitations of neural networks.

Let’s explore how these models diverge from their biological counterparts.

Historical Inspiration of Neural Networks

Neural networks trace their roots back to a time when researchers sought to understand the brain's workings. In 1943, Warren McCulloch and Walter Pitts published a groundbreaking paper titled "A Logical Calculus of the Ideas Immanent in Nervous Activity." Their goal was not to create artificial brains but to construct a mathematical model that represented the behaviour of individual neurons. They aimed to explore how simple computational units could lead to complex behaviours, laying the groundwork for what would eventually become neural networks.

This idea sparked interest in connectionism in the early days, which focused on how interconnected simple units could replicate sophisticated cognitive functions. Researchers began to realise that these artificial neurons could work together to produce intricate activity patterns, similar to how biological neurons operate in the brain. However, this is where the biological inspiration largely ends.

As the field progressed, it became clear that while NNs were inspired by biology, they diverged significantly from it. The models developed were not intended to accurately replicate the brain's structure or function. Instead, they evolved into powerful computational tools that utilize mathematical constructs and algorithms, often bearing little resemblance to their biological counterparts. Understanding this distinction is crucial as it helps dispel myths about AI and clarifies what neural networks can and cannot do.

Differences in Structure

Artificial neural networks (ANNs) are constructed from mathematically simple units that perform differentiable computations. Each artificial neuron takes inputs, applies weights, and produces an output through a mathematical function. This process is significantly simpler than what occurs in a biological neuron, which involves complex electrochemical signalling and intricate interactions with other neurons.

When we examine the scale of the brain, the differences become even more pronounced. The human brain contains approximately 86 billion neurons, each forming thousands of connections with other neurons—estimates suggest there are around 100 trillion synapses in total.

In contrast, even the largest neural networks today, like LLaMA 3, operate as directed acyclic graphs with far fewer connections. For example, LLaMA 3 has about 400 billion parameters, which is more directly comparable to the number of inter-neuron connections in the brain than the number of neurons. This figure is nothing compared to the trillions of synapses in human brains.

Moreover, the complexity of brain connections is far more diverse than that seen in ANNs. Biological neurons can form various types of synapses and exhibit different firing patterns and neurotransmitter types, leading to a rich tapestry of connectivity that supports complex cognitive functions. In contrast, ANNs typically rely on a small set of fixed mathematical functions.

Differences in Learning Mechanisms

Most importantly, the learning mechanisms in NNs differ fundamentally from those in the brain.

The human brain learns in a complex and dynamic process. It adapts by forming new connections between neurons, a phenomenon known as synaptic plasticity. This allows the brain to strengthen or weaken connections based on experience, enabling it to learn from and adapt to new information. Additionally, the brain can create new neurons through a process called neurogenesis, particularly in regions like the hippocampus, which is associated with memory and learning. This lifelong capacity for learning and adaptation is a hallmark of biological intelligence.

In contrast, artificial neural networks (ANNs) operate under a different paradigm. The primary learning mechanism for NNs is backpropagation, which uses a straightforward mathematical operation to adjust weights based on the error of predictions. During training, the network calculates the gradient of the loss function with respect to each weight, allowing it to update those weights in a direction that minimizes error. This process relies on gradient descent, a method that iteratively adjusts weights to find the optimal solution.

However, gradient descent is biologically implausible for several reasons. First, there is no evidence that the brain stores gradients in any form. Instead, learning in biological systems occurs through more nuanced mechanisms involving complex biochemical processes and feedback loops. Second, while the human brain can learn continuously throughout life, modern ANNs typically have a distinct training phase followed by an inference phase. Once trained, these models do not adapt or learn from new data unless they undergo retraining.

Additionally, while some promising research into lifelong learning algorithms and dynamic neural networks aim to mimic the brain's ability to adapt and reorganize, this area remains largely experimental and is nowhere near mainstream application. The neural networks used in practical AI systems today are predominantly static; they do not create new neurons or modify their structure based on experience. This static nature further underscores the limitations of current neural networks compared to the brain's remarkable capacity for continuous learning and adaptation.

Some Necessary Nuances

Convolutional Neural Networks (CNNs) provide an interesting case of biological inspiration in artificial intelligence. While they are indeed inspired by the architecture and essential functions of biological vision, it's important to note that CNNs are not fully biologically plausible, nor do they serve as accurate simulations of the brain's visual processing systems. Instead, they represent a pragmatic approach to leveraging insights from the natural world while optimizing for computational efficiency.

CNNs mimic certain aspects of the visual cortex, particularly in processing spatial information. They employ local connectivity and shared weights, allowing them to detect patterns and features in images effectively. This design draws from how biological neurons respond to localized areas of visual stimuli, making CNNs well-suited for tasks like image recognition. However, while they capture some essential characteristics of biological vision, they do not replicate the full complexity or diversity of neural architectures in the brain.

The key takeaway is that CNNs exemplify how we can take valuable inspirations from biology and apply them to create effective computational systems. By focusing on what is useful for solving specific tasks—like recognizing images or classifying objects—researchers can forgo unnecessary biological complexities that do not translate into computational advantages. This pragmatic approach allows us to harness the strengths of both biological insights and modern computing capabilities, paving the way for advancements in AI while acknowledging the limitations of current models compared to their biological counterparts.

Conclusions

Believing neural networks are direct analogs of biological systems can lead to dangerous myths and misconceptions. This misunderstanding can create unrealistic expectations about what AI can achieve and foster a false sense of security regarding its capabilities.

For example, one common myth is that neural networks are on the verge of achieving artificial general intelligence (AGI), which is far from the truth. In reality, the complexities of human cognition are not just a matter of scaling up current models; they involve intricate processes that we have yet to fully understand.

Another misconception is that neural networks learn and adapt like humans. While they can be trained on vast amounts of data, they cannot learn continuously throughout their lifetime or create new connections like the brain does. This static nature means that once trained, most neural networks cannot adapt to new information without retraining.

Additionally, there is a tendency to anthropomorphize AI systems, attributing them human-like qualities. This misrepresents their capabilities and obscures the ethical implications of deploying such technology in society.

As we continue to explore the powerful potential of neural networks in modern AI, it’s essential to approach these technologies with a clear understanding of their limitations and differences from biological systems. If you're interested in learning more about why neural networks are so powerful and prevalent in today's AI landscape, stay tuned for the follow-up article!