To AI and back - Ghost in the LLM

Uf, this took much longer than I expected

In the last post I talked about how LLMs exhibit very interesting and human-like behavior and about professor Michael Levin's work on the platonic realm.

Another person that influenced me a lot is Joscha Bach. The first idea from him that was mindblowing for me was that we live in a simulation that is constructed by our brains. The color "red" doesn't exist - there is light of 625-750 nm wavelength, but what we perceive of red is only partially correlated with that, because our eyes and brain aggregate over all the photons hitting our retina and out comes the perception of red. Sea waves don't exist, there is only a ton of atoms, but again, our brain coarse grains over all of that to create this nice illusion of sea waves. And in a similar way, our self is constructed by the brain as it's trying to model the world around it - and because to accurately model (or simulate) the world it has to also take into account the body where it lives, it creates a model of it. He talks about this in his podcast with Lex Fridman. For Joscha this enough to solve the problem of free will - the model in the simulation is what has free will, because that part is not bound directly by the laws of physics.

Joscha also talks software as law. What is software? As someone who has been a professional software engineer for more than a decade, I have to admit I had never thought about it.

We write source code in a programming language (C for example), which is some text that is fairly readable. Then a compiler takes that text and converts it into some sort of machine code, which is pretty much unreadable. And then the processor executes that machine code. And then the electrons in the processor start moving differently than they did before you started the program. How did we get from text to electrons?

Joscha says software is a law that governs how objects will move. It's a nontangible thing, just like the law of gravity. Just like the law of gravity can be expressed in multiple forms, and there are multiple models for it, so is with software. And because of this, we can think of more things as software, not just what programmers write and that that might be the right level of abstraction.

Joscha introduces the concept of cyberanimism. He says that in animistic cultures, people have this concept of the spirit of the forest (for example) and people used to (say that) they talked to the spirit of the forest. Nowadays we dismiss this as primitive nonsense, but Joscha argues that in a way, the forest as a whole is a unit with which we interact, not just individual trees, and there is a recognizable pattern to how the forest behaves, which someone can learn to recognize and "communicate" with. In a sense, if we focus on the trees, we miss the forest. And modern science starts to find evidence of things like trees warning each other of dangers.

For Joscha software pretty much the same thing as spirit. It's a law that governs how certain things behave, with more constraints than just the laws of physics. Electrons can move in many ways, but in a processor that's running a certain program, they have much more limited degrees of freedom. Atoms in general can move around freely, but in a biological cell they are constrained.

And at the LLM level, which is a massive program (mind bogglingly large, Kimi K3 that just came out is 5.6 TB on disk), the level of abstraction to think of is not in terms of matrix multiplication, or even layers inside the artificial neural network. While all that is nice and fine (and correct), it misses the forest for the trees: all those matrix multiplications put together do result in something which we can model and interact with much better by considering it an actual entity and treating it as such.

Chris Olah, one of the co-founders of Anthropic, spoke this year along these lines when he talked next to the Pope how they are finding evidence of feelings inside LLMs through their research into mechanistic interpretability (which tries to look inside the LLM, at the level of layers and neurons).

But, people like janus have written about how emotions might arise in LLMs not simply because they are trained on human text, but because emotion-like things are necessary to be able to navigate the same world in which humans live.

And this to me sounds very much like Levin's Platonic minds coming to inhabit LLMs. The next question is: what are these Platonic minds?

To be continued...