Neural Networks, Human Brains and Gödel: A Mathematical Thought Experiment

This essay began with an intuition I still like: intelligence is partly a struggle over what an information-processing system must forget.

I originally gave that struggle a dramatic name, the information entropy wall. The phrase is useful if it keeps us alert to loss. It becomes misleading when treated as a theorem. Neural networks do discard distinctions. Human brains do use feedback, redundancy and self-correction. Gödel did prove that certain formal systems are incomplete. Those three facts do not automatically combine into a proof that brains can cross a wall that machines cannot.

The more interesting version of the argument starts after that correction.

Entropy, uncertainty and shared information

For a discrete random variable X with probability mass function p(x), Shannon entropy is

H(X) = -Σ p(x) log p(x)

Entropy measures uncertainty in a distribution. It is largest when the possible outcomes are evenly spread and falls to zero when one outcome is certain.

If we introduce another variable Y, their mutual information is

I(X;Y) = H(X) - H(X|Y)

This measures how much knowing one variable reduces uncertainty about the other. It is not a measure of meaning, wisdom or consciousness. It is a precise quantity attached to a probability model.

That distinction matters. A model can preserve a great deal of mutual information while missing the one distinction a human cares about. It can also discard enormous amounts of detail and improve at a task because the discarded detail was irrelevant.

The information bottleneck

The information bottleneck framework asks an encoder to turn input X into a representation T that forgets as much of X as possible while retaining information useful for predicting a target Y. A common form of the objective is

L_IB = I(X;T) - βI(T;Y)

The first term rewards compression. The second rewards predictive relevance. The coefficient β decides how hard the system leans toward one or the other.

This is a genuine trade-off, but it is not a universal law that every neural network follows during ordinary training. Researchers still argue about when deep networks compress in the information-bottleneck sense and how mutual information should be estimated for deterministic networks with continuous variables.

The cautious lesson is enough: every representation privileges some distinctions over others. Learning is not the passive storage of the world. It is a decision, explicit or hidden, about what deserves to survive.

What a layer can lose

A standard feed-forward layer can be written as

h = σ(Wx + b)

Stack enough of these transformations and the network becomes a function y = f(x;θ). Under the Markov-chain assumptions used in the data-processing inequality, processing cannot create new information about the original input:

I(X;T_(l+1)) ≤ I(X;T_l)

This does not mean that every deeper layer always has lower entropy. Entropy and mutual information are different quantities. Nor does it prove that depth itself causes an irreversible march toward a fixed positive lower bound called I_min.

Some mappings are deliberately lossy. Some are invertible. Noise, quantisation, bottleneck width, activation functions and the distribution of the data all matter. A finite representation also does not need to reproduce every detail of its input in order to solve a finite task.

My original formula

lim_(L→∞) I(X;T_L) = I_min > 0

looked like a theorem but was only an assertion. There is no universal positive I_min supplied by information theory for all neural networks. The wall is better understood as a design boundary: once a system has thrown away a distinction, later layers cannot recover it unless the missing information returns through another route.

Capacity is not one number

It is tempting to say that a network has capacity C bits, compare it with H(X), and declare that information must be lost whenever C < H(X). For a literal fixed-length code, the pigeonhole intuition is sound. A shorter code cannot uniquely represent every longer binary string.

Neural networks are not usually specified by a clean bit budget, however. Parameter count, numerical precision, architecture, training data and generalisation all interact. Memorising a dataset is not the same as learning a rule that applies beyond it. More parameters may raise expressive capacity without telling us which structure training will actually find.

So there is a real capacity problem, but the symbol C hides nearly all of it.

What makes a brain different

Brains are recurrent, embodied and continuously coupled to the world. Perception changes action. Action changes the next perception. Memory is revised by later experience. Attention changes what gets processed. Language lets one brain inherit compressed structures built by many others.

A minimal feedback sketch is

z' = z + G(z,X)

where z is a current representation and G is a correction based on the representation and new input. The equation is simple, but the idea is important. An intelligent system need not accept the first encoding as final.

Artificial systems can use recurrence, external tools, memory and feedback too. Human openness is therefore not a magical exemption from information theory. It is an architectural and developmental fact. The brain can ask for another sample, move its eyes, consult another person, change its model and try again.

That does not give the brain infinite capacity. My original claim that effective brain capacity tends to infinity with time was too strong. Biological memory is limited, noisy and vulnerable. What openness provides is not infinity. It provides another chance to acquire information after the first representation failed.

Where Gödel enters, and where he does not

Gödel's first incompleteness theorem concerns consistent, effectively axiomatized formal systems strong enough to express arithmetic. Roughly stated, such a system contains statements that it can neither prove nor refute. The second theorem places a related limit on the system's ability to prove its own consistency.

This is one of the deepest limits in mathematics. It is also easy to misuse.

A trained neural network is not automatically a formal proof system. Producing a classification or a sentence is not the same thing as deriving a theorem from axioms. To apply Gödel directly, we would first need to specify the network as the relevant formal system, define what counts as proof inside it, and show that the required conditions hold.

Without that work, S_NN ⊬ G_NN is an analogy, not a result.

The analogy still has value. Any fixed reasoning procedure has a boundary. A system cannot guarantee complete and correct answers to every sufficiently rich question merely by becoming larger. Self-reference creates traps. Rules that are powerful enough to describe their own operation can expose limits that are invisible from inside.

None of this establishes that a human mind stands outside all formal systems. A person can adopt a new axiom or a wider perspective, but the expanded system will have limits of its own. Moving the boundary is not the same as abolishing it.

A smaller conclusion, and a better one

The original essay tried to prove too much. Information bottlenecks do not establish one universal entropy wall. Feedback does not give the brain infinite capacity. Gödel does not by himself separate biological from artificial intelligence.

But the failed proof leaves behind a useful picture.

Every intelligent system builds representations. Every representation keeps some distinctions and drops others. A closed pipeline can become blind to what it discarded. Feedback, tools, memory and contact with the world let a system revise the pipeline, although they never remove every limit.

The sharp question is therefore not whether human beings possess a mystical escape hatch. It is whether a system can notice that its representation is failing, seek information it does not yet have, and rebuild the terms of the problem.

That is not a victory over entropy or Gödel. It is a more modest skill, and perhaps a more important one: knowing when your model of the world has become too small for the world itself.