Somewhere, a clerk types a birth year as 1880 instead of 1980. The loan model reads it and concludes that the applicant won't live long enough to pay anything back. Application refused.
Sergey Markov uses this small slip to show what he thinks AI danger looks like. Markov is a managing director at SberDevices, part of Sberbank, and leads its experimental machine learning systems. He tells it as an illustration, not a case file. The point is that the applicant has no way to find out what went wrong.
Think about how many decisions about you are already made this way. You apply for a loan, you send a résumé, you pass through a thousand processes you never hear about. In each one, Markov says, algorithms hand down verdicts, and their weight in your life is often comparable to a court's. A refused loan can mean the health of your family hangs on it.
He is not talking about the Terminator. Hollywood shows a different danger from the one that is already here. His own example is deliberately crude, and he admits he is exaggerating: a new system goes live tomorrow, and people with red hair start getting refused more often than people with black hair. A company would have created a mass of oppression along some arbitrary trait without noticing. The exaggeration is the hair colour. The mechanism, he says, is real, and algorithmic bias is a serious problem precisely because these models now sit everywhere.
Researchers have measured versions of it. In a study presented at the AAAI/ACM Conference on AI, Ethics, and Society in October 2024, Kyra Wilson tested text-embedding models that simulate résumé screening across nine occupations. The models favoured names associated with White applicants in 85.1% of cases, and for Black men the disadvantage ran as high as 100%. An audit posted to arXiv in December 2025 by Madison Coots looked at about 80,000 personal loans from a major U.S. fintech platform. The underwriting model overestimated credit risk for women and underestimated it for Black borrowers.
Now compare the two rooms you might end up in. In a courtroom you get an adversarial process and a qualified lawyer. You can see every piece of evidence gathered against you. At the loan desk, the model is a commercial secret. You can't see how it is built or what data went into it, and you can't write to an ombudsman, because there isn't one. Markov's picture of the mistyped 1880 makes the point: you could be refused for a clerk's keystroke and never know.
Algorithms hand down their verdicts, yes, and in how much they affect your life, what these algorithms do is often comparable to a court's decision.
свои вердикты выносят алгоритмы, да, и по силе влияния на вашу жизнь действия этих алгоритмов сопоставимо с решением суда зачастую.
— Sergey Markov, Gleb Solomin Podcast, 42:31
Here Markov makes a concession that keeps the argument honest. A machine should not be measured against zero risk. It should be measured against the human process it replaced, and in many cases automation lowers risk. A model can be run through millions of simulated tests to get an objective figure, which you can't do with a taxi driver whose head you can't see into. What goes wrong is carelessness, as with a badly tested radiation-therapy machine in the 1980s whose faulty software gave several patients lethal doses.
So the remedy he asks for is dull. Factories already live under thick binders of safety rules, and the same is owed to systems that decide who gets credit. Frameworks for managing those risks have to be built, or companies will shoot themselves in the foot.
Markov knows the problem with this. People fall asleep when the talk turns to risk regulation. They would rather watch the film about the Terminator.
