Predictive Mind Theory and AGI 

Predictive Mind Theory is a very charming theory introduced by Jakob Hohwy, which aims to identify a simple mechanism with the ability to explain the fundamental workings of our brain, both overall and in detail. We will look into how this theory explains some interesting perks of human intelligence and how this can be transferred both to understand existing AI algorithms, as well as to future algorithm designs to come closer to AGI.

It is the theory that the brain is a sophisticated hypthesis-testing mechanism, which is constantly involved in minimizing the error of its predictions of the sensory input it receives from the world.

Jakob Hohwy, The Predictive Mind

Background

Jakob Hohwy's theory of the predictive mind emerges from two foundational concepts in neuroscience and cognitive science: Bayesian inference and the free energy principle. Together, these ideas provide a mathematical and theoretical basis for understanding how the brain functions as a dynamic prediction engine.

Bayesian Inference: The Brain as an Inference Machine

At its core, Bayesian inference describes a formal method for updating beliefs based on evidence. It involves calculating the probability of a hypothesis given prior knowledge and new sensory data. This process can be expressed in terms of a simple equation:

P(H∣D) = ( P(D∣H) ⋅ P(H) ) / P(D)

Where:

• P(H∣D) is the updated probability of a hypothesis H after observing data D.
• P(D∣H) is the likelihood of observing the data if the hypothesis is true.
• P(H) is the prior probability of the hypothesis.
• P(D) is the probability of the data under all possible hypotheses.

Hohwy's theory extends this idea to cognition, suggesting that the brain constantly updates its internal models (hypotheses) to match sensory data. For example, when you hear a rustling sound in the bushes, your brain might initially hypothesize that it's the wind. However, if you then see a squirrel, the brain updates its model to accommodate this new evidence.

This Bayesian approach highlights the brain's probabilistic nature, always weighing prior knowledge against new evidence to make sense of the world. However, it leaves an open question: how does the brain implement this process biologically and at scale?

The Free Energy Principle: The Brain as an Energetic System

The free energy principle, introduced by neuroscientist Karl Friston, offers a unifying framework for understanding how Bayesian inference operates in the brain. Free energy here is not the same as energy in the physical sense; instead, it refers to a measure of surprise or prediction error in the brain's internal model.

Hohwy adopts this principle, stating that the brain minimizes free energy to maintain a state of equilibrium. In other words, the brain seeks to reduce the gap between its predictions and actual sensory inputs. This aligns with the idea that organisms must resist disorder (entropy) to survive, and one way they do so is by reducing uncertainty about their environment.

How Bayesian Inference and Free Energy Combine

The integration of Bayesian inference and the free energy principle creates a cohesive picture of cognition. Bayesian inference provides the mathematical rules for updating beliefs, while the free energy principle explains the biological drive to reduce prediction error. Together, they form the backbone of Hohwy's predictive mind:

1. Bayesian Updating: The brain continuously refines its internal models based on prior beliefs and new sensory input.
2. Minimizing Free Energy: The process of updating models serves to minimize prediction error, keeping the system stable and adaptive.
3. Active Inference: The brain not only passively updates but actively seeks to test its models through action, further minimizing free energy by influencing sensory inputs to align with predictions.

What does this mean for how we learn to navigate in this world?

Imagine a baby just beginning to explore the world. At first, its brain receives a chaotic stream of sensory input—light, colors, and patterns shifting unpredictably. Over time, through trial and error, the baby begins to notice a connection: when it moves its eyes, the "pixels" in its visual field shift in unison. This simple cause-and-effect relationship becomes the first building block of its internal world model. It can now with a certain accuracy predict how the sensory input changes depending on certain action output. Further exploration reveals another pattern—moving its arm causes certain pink blobs (its hand) to shift within the visual input. Yet, intriguingly, there are other blobs that move without any action from the baby itself, hinting at the existence of independent agents in the environment. These realizations form distinct modules in the baby's mind: one for controlling eye movement, another for limb coordination, and a rudimentary distinction between "self" and "others." Each module grows more refined with experience, forming a robust and adaptable foundation for learning more abstract concepts later in life.

Prediction, Accuracy, and Model Updates in the Predictive Mind

A key feature of the predictive mind is that the brain doesn't just generate predictions—it also estimates the accuracy or precision of these predictions. This estimation plays a crucial role in determining how the brain updates its internal models when faced with surprising sensory input (or surprisal). Precision is a measure of confidence in the prediction, derived from the variability or reliability of the incoming sensory data. For example, a clear and consistent signal, such as the sight of a brightly colored object in good lighting, carries high precision, whereas a blurry or ambiguous signal, such as a shadowy shape in the dark, has low precision.

When surprisal occurs—sensory data deviates from the brain's prediction—the amount of model updating depends not only on the magnitude of the error but also on the precision assigned to the prediction. If the model assigns high precision to its prediction, even a small amount of surprisal triggers significant model updates. Conversely, if the prediction is deemed unreliable (low precision), the brain weights the error less heavily and is less inclined to adjust its model. This dynamic weighting ensures that the brain adapts effectively in environments where signals vary in reliability, focusing its computational resources on the most trustworthy inputs.

Where do we already see this reflected in modern machine learning algorithms?

Hohwy's predictive mind theory, particularly the role of precision-weighted predictions and probabilistic updates, resonates with how many modern machine learning algorithms operate. A crucial shift in machine learning has been the move from strictly rewarding correct answers to using probabilistic approaches that account for the likelihood of all possible outcomes. By not treating predictions as binary (right or wrong), but rather as distributions of probabilities, algorithms can learn more effectively from their errors and improve generalization.

Softmax and Cross-Entropy: Precision in Probabilistic Outputs

One of the simplest examples is the use of the softmax function in neural networks, which transforms raw output scores (logits) into a probability distribution over all possible classes. Rather than merely selecting the most likely class, the network uses the entire distribution to assess its prediction accuracy.

The cross-entropy loss function further refines this process by comparing the predicted probability distribution to the true labels (represented as a one-hot vector). The loss penalizes incorrect predictions more heavily when the network is overly confident about them (assigning high probability to an incorrect class) and less when the network is uncertain. This mirrors how the brain weighs the precision of its predictions: confident predictions that result in errors trigger larger updates than uncertain ones.

Bayesian Neural Networks: Uncertainty in Prediction

Bayesian neural networks (BNNs) explicitly incorporate uncertainty into their predictions, aligning closely with the precision-weighted updating in the predictive mind. In BNNs, the weights themselves are represented as probability distributions rather than fixed values. This allows the network to express uncertainty about its predictions, making it particularly effective in tasks requiring probabilistic reasoning or when encountering out-of-distribution data.

For instance, a BNN predicting whether an image contains a cat would not only output the most likely label but also provide a measure of confidence based on the variance of the posterior distribution. Low-confidence predictions lead to less substantial updates during training, akin to the brain assigning less weight to uncertain sensory data.

Reinforcement Learning and Policy Gradients: Rewarding Probability Distributions

In reinforcement learning (RL), algorithms like policy gradient methods (e.g., Proximal Policy Optimization or PPO) optimize not just for the correct action but for the probabilities of all potential actions. The agent outputs a policy—a probability distribution over actions for a given state—and the algorithm uses this distribution to calculate gradients that adjust the model's parameters.

For example, an RL agent navigating a maze might predict several plausible actions, each with a certain likelihood. When the agent receives feedback (reward or penalty), the update is proportional to the likelihood of the chosen action, adjusted by the overall policy gradient. This probabilistic backpropagation aligns well with the predictive mind's principle of refining models based on weighted evidence, ensuring that the agent improves not just for specific actions but for its overall understanding of the action space.

Attention Mechanisms: Precision-Weighted Inputs

In transformer architectures like GPT or BERT, attention mechanisms assign weights to different input tokens based on their relevance to the task at hand. This process resembles how the brain assigns precision to sensory inputs. Tokens with higher attention weights contribute more to the final prediction, ensuring the model focuses on the most reliable or salient information. This dynamic weighting of input relevance mirrors the predictive mind's precision-weighted error minimization.

The Success of Self-Supervised Learning (SSL)

Hohwy's predictive mind aligns closely with self-supervised learning (SSL), which trains models to predict parts of data from other parts, mimicking how the brain builds models through prediction and error minimization. Like the brain, SSL thrives on unlabeled data, constructing hierarchical representations and refining internal models by reducing surprisal.

For example, language models like BERT predict missing words, while vision models like MAE reconstruct image patches. Both processes mirror the brain's precision-weighted updates, where errors in high-confidence predictions lead to greater model refinement. Similarly, contrastive learning methods like SimCLR refine representations by aligning similar inputs and separating dissimilar ones, reflecting how predictive coding emphasizes relationships in data.

The success of SSL highlights the predictive mind's principles: learning through prediction, minimizing error, and building transferable representations—core mechanisms for creating adaptable and human-like AI systems.

Toward AGI: The Missing Modular Structure of Internal Models

While the predictive mind framework offers a powerful basis for understanding intelligence, it also highlights significant gaps between current AI and human cognition, particularly in the modular structure and adaptability of internal models. Achieving AGI requires systems capable of developing modular, hierarchical models that function both independently and in concert, much like how human cognition operates.

Modularity in Human Internal Models

From infancy, humans build modular models to navigate the world:

• Early Layers: Basic models of motor control and sensory integration, such as eye movement causing visual shifts or the distinction between "self" and "other." These form a stable foundation for understanding the environment.
• Later Layers: More abstract and domain-specific models, like language comprehension, social reasoning, or advanced mathematics. These layers build on earlier ones but remain distinct, functional, and adaptable within their own realms.

Critically, these modules can:

• Override Each Other: High-level reasoning can suppress lower-level perceptions (e.g., recognizing an optical illusion as false despite its appearance).
• Collaborate Across Domains: Modules lend their capabilities to others (e.g., using spatial reasoning in solving a math problem).
• Support Counterfactual Thinking: Humans explicitly probe models to reason through hypothetical scenarios, refine predictions, or adapt models based on new evidence.

What Current AI Systems Lack

1. Unified Yet Modular Architectures: Most AI systems lack the modular design seen in the human brain. Models are often monolithic, with no clear separation between layers for distinct cognitive tasks. This leads to inefficiencies and brittleness when applying knowledge across domains.
2. Cross-Layer Adaptation: While humans adapt one layer (e.g., motor control) without destabilizing others (e.g., language comprehension), AI struggles with such targeted updates, often requiring retraining or risking catastrophic forgetting.
3. Counterfactual Reasoning: AI systems rarely probe their own models or generate explicit hypotheses for testing. Counterfactual reasoning—imagining "what if" scenarios to refine understanding—is still rudimentary in AI.
4. Precision Flexibility: Unlike humans, AI struggles to adjust the confidence (precision) of its predictions based on new evidence. Humans can "loosen" confidence in assumptions after receiving contradictory information, but AI models often require extensive retraining to achieve similar adaptability.

Path Forward to AGI

To bridge the gap, AGI systems need:

• Modular and Hierarchical Learning: Architectures that develop distinct modules for specific tasks, allowing independent adaptability while maintaining global coherence. Techniques like neural symbolic systems or dynamically reconfigurable neural networks could help.
• Cross-Domain Generalization: Mechanisms to enable modules to share knowledge efficiently across domains, while preserving their primary functions.
• Explicit Hypothesis Testing: Embedding reasoning systems that can actively test and refine their internal models, inspired by human counterfactual thinking.
• Precision-Weighted Updating: Dynamic control over model confidence, enabling systems to adjust predictions and adapt selectively based on the reliability of new evidence.

By focusing on modularity, adaptability, and the ability to reason and refine models explicitly, AI can move closer to replicating the layered, flexible intelligence that underpins human cognition. This would not only advance AGI but also make AI systems more robust, interpretable, and relatable.

Invitation to further discussion

Much of this is my personal take on the idea as I fell in love with the theory back in 2015 and over my time in AI research saw much of it come true. I'd love to discuss these ideas further and invite you to do so on my Omega space.

Hohwy, Jakob. The predictive mind. OUP Oxford, 2013.