IN THE PRISON OF DIGITAL ILLUSION: Is AI Deliberately Slowing Down Scientific Progress?

Photo by Mathew Schwartz on Unsplash

The scientific journal Nature is sounding the alarm on this issue. Historians of science have long noted that the pace and scale of significant scientific discoveries have steadily declined over the past decades. At the same time, funding, the number of publications, and research staff have all grown. Against this backdrop, breakthroughs in the field of AI injected a considerable dose of optimism into the scientific community. But now scientists are once again concerned: the growing role of artificial intelligence in science may bring more harm than good.

IS AI WORSENING STATISTICAL METHODS?

rom 2012 to 2022, the average share of scientific papers dedicated to the use of artificial intelligence in the 20 leading scientific fields increased fourfold. AI has been widely applied in forecasting all sorts of things: the outcomes of implementing economic models, the spread of disease outbreaks, and even civil wars. Yet, this hype has had certain consequences. Statistical methods themselves are not free from serious errors.

AI adds even greater risks to them due to its «black box» nature. These errors are further amplified when ready-made tools are used by researchers with limited knowledge of computer science. People tend to overestimate the capabilities of AI models, and this overestimation fatally affects forecasting, creating an illusion of progress while holding back real breakthroughs.

CHAINSAWS INSTEAD OF AXES

There are many ways to use AI in science — for instance, for efficient analysis of work produced by natural human intelligence. One common application is machine learning, which can be seen as an improvement over traditional statistical modeling. If conventional manual statistics are like an axe, machine learning modeling is like a chainsaw. This automated tool is undoubtedly powerful, but also very dangerous and potentially harmful if misused. That is why modeling, in which AI is employed to forecast or test hypotheses, raises the greatest concerns.

«LEAKAGE» MAKES MODELS USELESS

One of the most common sources of error is the so-called «leakage». This problem arises when a machine learning model memorizes the patterns of evaluation data rather than the patterns of the actual phenomenon scientists are interested in. It was recently discovered that papers in at least 30 scientific fields that used machine learning were affected by such leakage.

Errors generated by AI can be found across a wide range of scientific work — from psychiatry and molecular biology to computer security. For example, during the COVID-19 pandemic, hundreds of studies claimed that AI could diagnose the disease using only chest X-rays or CT scans. Yet only 62 out of 415 such studies met basic quality standards.

Even in those 62, incorrect evaluation methods, data duplication, and diagnostic confusion were widespread. In about 12 studies, researchers used a training dataset where all positive COVID cases were from adults and all negative cases were from children aged 1 to 5. As a result, the AI model simply learned to distinguish adults from children based on this feature. Yet the researchers claimed they had developed a COVID-19 detector!

CRISIS OF REPRODUCIBILITY

Unfortunately, there are no standards yet for evaluating the accuracy of predictions. Computer codebases consist of thousands of lines, making errors difficult to detect. And the cost of even a single error can be extremely high. Thus, we are only at the very beginning of a reproducibility crisis in machine learning–based science. But it could grow to considerable proportions. For example, the use of large language models as surrogates for participants in psychological experiments has recently become popular. Most of these experiments turn out to be irreproducible, since the models are sensitive even to the smallest changes in input data.

IN THE PRISON OF SELF-DECEPTION

The triumphant invasion of machine learning into science is nothing more than a form of self-deception. The fact is that the stream of discoveries made with the help of AI, even if free of errors, may not lead to genuine scientific progress. This was first pointed out back in 2001 by Leo Breiman, who described the cultural and methodological differences between the fields of statistics and machine learning. Yet public opinion preferred to deal with a beautiful utopian dream rather than reality.

AI propagandists still prefer to talk only about its potential rather than the long-known and quite significant limitations of machine learning. Breiman argued that models based on it may work well in engineering but are hardly applicable in the natural sciences, whose very essence is to explain Nature. Alas, AI is unlikely to «explain» anything here without producing an error. But too many researchers, lured by the commercial success of AI, ignore this limitation.

By joining the Huxley friends club, you support philosophy, science and art

Join the friends club

AI — A BRAKE ON SCIENTIFIC PROGRESS

The reason is simple: to use the results of models for gaining knowledge about the surrounding world requires a great deal of work — and it must come from natural, not artificial, intelligence. Machine learning tools merely simplify the process of building models, but extracting real knowledge about the world from them, on the contrary, becomes more difficult.

As a result, we produce more scientific content with ever less understanding of the world. And here fertile ground arises for conspiracy theorists: could it be that the «black box» is doing this on purpose? Yet, if we step away from apocalyptic fantasy, we must admit that in this situation, the fault lies with humans themselves.

At some point, we began to see science incorrectly — as a mechanical set of facts or discoveries. In reality, scientific progress unfolds differently. Without the explanatory function of human thought, it does not work. Science advances from discoveries to theories and paradigms, which serve as conceptual tools for understanding and investigation. Along this path, scientific ideas become more abstract and resistant to automation. That is why the rapid spread of AI-based discoveries does not accelerate but instead slows down scientific progress.

«THE RUT» LEADING TO A DEAD END

Do not think this is such an unusual phenomenon! The history of science is full of similar examples: from alchemy to chemistry, from astronomy to the Copernican revolution, from geology to plate tectonics. Entire scientific fields have repeatedly and for long periods become stuck in a rut. And this rut often led scientists to a dead end, even if they succeeded in achieving individual results.

In the history of astronomy, for instance, an important place is occupied by the concept of «epicycles», which held that planets move in circles around the Earth. This model was fairly accurate in its predictions of planetary motion. And even after it no longer corresponded to scientific knowledge, modern planetarium projectors continued to use this very method for calculating trajectories.

AI is the modern equivalent of such epicycles. Perhaps its ability to squeeze more predictive power from imperfect theories and inadequate paradigms will allow it to stay afloat for quite some time. But the longer this continues, the more AI will hinder genuine scientific progress.

HOW TO AVOID THE ILLUSION OF PROGRESS?

The answer lies in honest scientific dialogue. As starting points for this dialogue, the following theses can be proposed:

Machine learning is not a ready-to-use technology for scientists, but merely a set of tools. Applying these tools requires deep knowledge, the study of quantitative methods, and at least an understanding of common pitfalls and limitations. Closer collaboration between subject-matter experts and machine learning specialists is essential.

It may also be necessary to find ways in which researchers do not evaluate their own work. Since the number of AI-based results is enormous, there is a need for synthetic methods that encompass different forms of analysis.

Finally, organizations that fund science should focus on quality rather than quantity, encouraging reproducibility — the verification and replication of results by other researchers — as well as evidence synthesis, combining data from different sources for more reliable conclusions.

Whether the seductive illusion of the «black box» or science and common sense will prevail, only time will tell…

Original research: