Some children are born blind and deaf. They can still be taught language, and some of them have gone on to study at Moscow State University. Nobody put the sun in front of their eyes or its warmth in a sentence they could hear. They learned the world through words.
Mikhail Burtsev keeps these children in mind when he thinks about chatbots. He is an AI researcher, the founder of the DeepPavlov project at MIPT, and he builds language models for a living. A language model is in the children's position. It has no eyes, no ears, no skin. It gets text, and one job: guess the next word. Burtsev calls that job "absolutely dumb."
A job that dumb should produce a parrot. It produces something people ask for advice, argue with, and sometimes fall in love with. When a model finishes your sentence, what is on the other end? A shelf of memorised phrases, or something that has rebuilt a small piece of the world from text? The answer decides how much to trust what it tells you about that world.
Burtsev's bet is on the second. Text is not a random pile of words. Humans evolved, then built a language, and the language exists to pass around the state of the outside world. Someone says "the sun is shining". A string of sounds goes into another head and sets off the activity that matches a sunny day. A sentence is a compressed report of a situation.
Now take the report and ask what comes next. Guessing the next word well means guessing the situation behind the words. A model trained on this task, in his telling, has to build a picture of the world those words describe. It reconstructs the world from the words, which is the position the poor model is in.
To predict a sequence of words well, you have to predict this state of the environment that is tied to those words.
Для того, чтобы эффективно предсказывать последовательность слов, нужно предсказывать это состояние среды, которая связана с этими словами.
— Mikhail Burtsev, Think Like a Scientist, 55:30
Researchers have started looking inside to see whether anything like this is there. A 2023 paper on arXiv by Wes Gurnee found that language models hold linear representations of space and time, and even individual "space neurons" and "time neurons" whose activity tracks where and when. A 2024 study in Nature Human Behaviour, led by James Strachan, put GPT-4 models through theory-of-mind tests. They matched or sometimes beat people at spotting false beliefs, indirect requests and misdirection. They stumbled on faux pas.
Burtsev takes the idea back to the amoeba. His favourite philosopher, Karl Popper, liked to say that an amoeba's ideas die with the amoeba, while Einstein could kill his own ideas without dying. Burtsev adds a thought of his own. Every living thing has to predict its surroundings to stay alive, so the more conditions an organism can foresee, the more places it survives. Knowledge, on this view, is a survival imperative. Training a language model puts the same principle at its centre: watch, predict, adjust.
He would like biology to have a theory that ties evolutionary fitness to the power to predict the future. He says it does not exist yet. It might even draw a line between the living and the non-living, though he offers that only as a possibility.
Here Burtsev stops short. If predicting words gets you human-level cleverness, then intelligence would be word generation, and he declines to put forward "such a bold hypothesis." He sees the parallel, and he thinks it is now a fair question for philosophy and for research.
The test case is already at hand. People have goals, some of them unconscious. A language model has none, and whatever purpose it seems to have is set by how humans trained it and by the context they give it. So what is left in a person once you subtract prediction? That is the question his idea leaves open.
