«THE RANDOM WORLD»: Was Our Civilization Shaped by Scientific Discoveries No One Expected?!

Photo by Michael Schiffer on Unsplash

There is a common belief that scientists are highly rational individuals — and therefore, the process of scientific discovery must also be rational and predictable. Unfortunately, that’s far from the truth. A recent study analyzed 1.2 million biomedical publications and assessed the «unexpectedness» of their findings. It turned out that a vast number of discoveries were made entirely by chance. This suggests that we live in a «random world» — and it could have been completely different if not for the role of serendipity in science.

SCIENCE — A MATTER OF CHANCE?

cience is full of accidental discoveries. One day, Alexander Fleming noticed that mold from the genus Penicillium was killing the bacteria he had been cultivating in a Petri dish. This chance observation led to the discovery of antibiotics and saved millions of lives. It’s frightening to imagine what might have happened to humanity if not for the intervention of His Majesty, Chance!

Yet Fleming’s stroke of luck is far from unique in the history of science. In fact, it’s quite possible that science is largely made up of serendipitous discoveries. At least, research in the field of biomedicine suggests as much. Similar studies haven’t yet been conducted in other scientific disciplines, but it’s unlikely the situation would be radically different elsewhere.

The authors of a paper recently published in Research Policy found that the overwhelming majority of biomedical studies contain results that are significantly different from those outlined in scientists’ original funding proposals.

CHANCE IS STATISTICS, NOT JUST ANECDOTE!

Ironically, the project that set out to measure how «unexpected» scientific results are ended up delivering unexpected results itself. Its lead author, Ohid Yaqub — a British biochemist and sociologist from the University of Sussex — believes the role of chance in the scientific discovery process remains severely underestimated. Even though, empirically, it has long been clear that surprises in science are not exactly rare.

Italian philosopher of biological sciences Telmo Pievani from the University of Padua, commenting on his British colleagues’ work, said that it moves researchers «beyond the anecdotal view of randomness in science» and «for the first time confirms it at a quantitative and statistical level».

THE GRANT PARADOX: RESULTS DON’T MATCH EXPECTATIONS

Yaqub and his colleagues examined over 1.2 million scientific publications. Their analysis drew from more than 90,000 biomedical science grants awarded by the U.S. National Institutes of Health (NIH) between 2008 and 2016. As expected, artificial intelligence played a key role in the process.

To begin with, the British team developed a machine learning algorithm capable of analyzing article texts and classifying studies according to scientific categories defined by the NIH — for example, «chronic pain» or «climate change».

The researchers then compared these results with the categories originally indicated in the grant proposals. The data they uncovered stunned even the study’s authors. The final results of research projects rarely matched what funders initially «ordered».

THE «UNEXPECTEDNESS» MEASURE

About 70% of biomedical publications included content in at least one category that had not been mentioned in the original grant proposal. The researchers then refined their analysis by excluding closely related terms — for example, «liver disease» and «liver cancer».

This adjustment improved the picture somewhat, but even then, 58% of the articles reviewed by Yaqub’s team contained outcomes that no one had anticipated. On average, about one-third of the findings across all scientific categories turned out to be unexpected.

MORE SPECIFICS — MORE SERENDIPITY!

Interestingly, unexpected results were most often found in large-scale studies. The bigger the research grant and the longer the project timeline, the higher the likelihood of random outcomes. And it was fundamental science grants that generated the highest number of «surprises».

In contrast, applied projects — those related to clinical trials or disease research — tended to yield slightly fewer unexpected results. Logical enough, right? Yet even here, there was a twist: applied grants where applicants had clearly defined goals were more likely to produce surprising outcomes.

MORE TIME AND MONEY — MORE SURPRISES!

When grant-funded projects were initiated by researchers themselves, the degree of unexpectedness was somewhat lower. So does this mean that chance favors funders over performers?

According to Pievani, Yaqub’s project carries important implications for research policy: «Both fundamental and applied research can be funded, as long as they remain open to unexpected results and don’t eliminate anomalies too quickly». He urges not to fear chance — even if it comes at a cost.

SCIENTISTS SEEKING FREEDOM

Yaqub, for his part, believes that if sponsors are open to encountering the unexpected, they should provide larger grants over longer periods. However, some, like Samantha Copeland — a philosopher at Delft University of Technology in the Netherlands — argue that chance has little to do with it.

She maintains that greater research freedom doesn’t necessarily increase randomness. Rather, when scientists are given more time and flexible funding, they simply make better use of their opportunities. Copeland also reminds us that researchers are often under immense pressure to conform to the «scientific method» and thus want to defend their right to make «unexpected discoveries».

IT’S TIME TO LISTEN TO THE SCIENTISTS…

In any case, Yaqub’s research is one of the first to examine randomness in scientific results. For now, the project has a number of limitations. Still, the team doesn’t plan to stop there — they aim to explore how often researchers in different fields explicitly reference serendipity in their publications and to learn about scientists’ personal attitudes toward chance itself.

Original research: