There are reasons to be skeptical about the paper The nature, and scholars who wrote in response were quick to point them out. First, the authors are not scientists in the common sense; they have doctorates in business management, not biology. Second, the data they used were not surveys of those in the field, but a survey of patent filings.
This has fostered criticism that the paper is more about the lack of new ideas that open up new areas for business than about truly evaluating changes to basic science. However, the real data seems compelling, and the impact could hardly be greater.
We find that papers and patents are increasingly less likely to break with the past in ways that push science and technology in new directions. This model holds for all fields and is robust to many different values based on quotes and text. We subsequently relate this decrease in disturbance to a narrowing of the use of prior knowledge, allowing us to reconcile the patterns we observe with the “shoulders of giants” view. We find that the observed declines are unlikely to be driven by changes in the quality of published science, citation practices, or field-specific factors. Overall, our results suggest that slowing rates of disruption may reflect a fundamental change in the nature of science and technology.
Most of the paper is about how they determined the “disruption” of papers and patent filings (which is where many of those offended by the idea find action in challenging the general theme), but the bottom line is this: the number of publications has increased, many of those works are of very good quality, some remain disruptive, but many only confirm the status quo. Or, at best, they provide new insights that lead to little potential for scientific or economic impact.
This immediately caused many publications covering the story to go in one of two directions. First, there’s Doom mode (if not DOOM mode), which expresses the concern that new ideas might dry up and that we’re really running out of new things to discover. This is sometimes followed by thinking about which STEM fields will die first.
The second answer usually starts with the phrase “in the late 19th century” before explaining how relativity and quantum mechanics upset the Newtonian apple cart. The last time we were convinced we knew everything, by this line of reasoning, it turned out we knew next to nothing.
To this second idea, the only possible answer is: boy, I wish! Nothing excites a scientist, any scientist, more than results that do not match the prediction. For decades, thousands of researchers have worked diligently to poke holes in the Standard Model of particle physics, the limits of relativity, and the fundamental frustrations inherent in the quantum. However, whenever a potential deviation from the models’ predictions occurs, further research seems to serve only to uncover flaws, not in the underlying theories, but in the previous work. We certainly live in an age where some of the predictions made a century ago have been tested on an apparent equation…and stubbornly righted again and again.
There are, of course, some well-known holes left. The marriage quantum with relativity remains elusive, even if schemes to make it work on (uncontrolled) paper are never in short supply. On a large, cosmological scale, making the visible universe obey our equations requires the belief that the vast majority of everything exists as invisible matter and inexplicable energy. These are giant Fudge factors of the first water, and it seems as likely that Dark Energy and Dark Matter will be driven out of existence by some future insight in mathematics as it is that either will be “discovered” in any meaningful way. However, some of the most appealing theories that might offer new insights, often woven together by thousands of scientists working over decades, continue to fail to stand up to real-world tests (ie, supersymmetry and string theory).
There is an important precursor to this work that many media outlets seem to have omitted from this discussion, and that is the 1996 book The End of Science by science journalist John Horgan. Horgan is a prolific author and columnist for american scientistwhose interview subjects included a list of scientists who could be seen as the most disruptive crew of the past three generations, from EO Wilson and Roger Penrose to Richard Dawkins, Stephen Jay Gould and Stephen Hawking. Horgan spoke to all of them and hundreds more.
Horgan’s book was a surprise bestseller (any time a science book hits the bestseller list, it’s a surprise), but there was a tough group of both scientists and science journalists who were shadow the core idea of the book: we should expect less and less. important scientific discoveries as time passes.
The reasoning behind this was simple. At the beginning, All was available to be discovered. Scientists could make a discovery about the scale of the Earth with a vertical stick. They could learn about the speed of sound by watching someone chop wood. However, with each passing year, as the great book of facts has become more full of learning, the difficulty of making new fundamental discoveries increases. In the 19th century, the electron was discovered by a guy using equipment that could have been found in a high school science lab (or a rich naturalist’s basement). It took an international effort with a $4 billion+ collider to shut down the Higgs boson particle zoo.
Seriously, how hard is it to believe that the easy stuff has been picked clean, and the discoveries that remain will come at an ever-increasing cost? It seems perfectly reasonable. But it was the next step that left a sore lump in many throats: What happens when the cost of a new discovery becomes so high that it’s simply unattainable? Horgan saw that day, if not already at hand, certainly just around the corner. Horgan laced this with a troubling reminder of something that echoes the discussion between God and Job: if there are rules by which the universe works, there’s no guarantee that we evolved apes can understand those rules in any meaningful way.
… given the limits that constrain further research, science will have difficulty making truly profound additions to the knowledge it has already generated. Further research may not bring more revelations or big revolutions, but only incremental returns.
There is an extremely uncomfortable agreement between this conclusion of Horgan’s and the overview of the work published in The nature. Or at least, the way that work is presented in most media. However, this is not a very accurate description of what is actually hidden at the end of the patent and works review.
Because while the number of groundbreaking papers published may have declined as a percentage of the total, and the impact of new research may even be fading somewhat overall, there is still a healthy vein of disruption in what is being published today.
… the stability we see in the large number of disruptive papers and patents suggests that science and technology do not seem to have reached the end of the “endless frontier.”
The implications of this go beyond just breathing a sigh of relief for finding new features in the iPhone 15. It may not be obvious, but the fundamental scientific theories that underpin our view of the universe remain open to attack. And that’s a good thing.