15  The Future of Science in the Age of Machine Learning

We have reached the end of our little journey. Remember how we started with examples of scientists using machine learning to gain insights? From there, we evaluated machine learning from the perspective of different scientific goals, such as prediction, control, explanation, and understanding. We argued that machine learning has great potential to become a mainstay of scientific problem-solving. However, bare-bones supervised machine learning does not do the trick – we need add-ons such as interpretability, uncertainty quantification, and the integration of causal knowledge, all of which we systematically introduced in dedicated chapters. One by one, we addressed how to “fix” the shortcomings of machine learning for science: Generalization, domain knowledge, interpretability, causality, robustness, uncertainty, reproducibility, and reporting.

In this final chapter, we speculate about the future of science in the age of machine learning. How will scientists integrate machine learning into their research? Will some sciences remain untouched by machine learning? What will be the role of future scientists in the face of machine learning automation?

Time for some speculation.

Many years later, an aging but happy Rattle invited the stony-faced Krarah into her giant nest to talk about old times. Since then, several scientific problems have been solved, many with the help of machine learning. So, was it all good? Their smart feathers begin to vibrate. Rattle received a targeted ad from a radical political movement, and Krarah received a warning of an approaching storm. Krarah winked at Rattle: “No! Not all of it, but some of it for sure.”

15.1 No science without machine learning

In early 2023, we started brainstorming and conceptualizing this book. By this time, the increasing use of machine learning models in science was becoming apparent, even in fields we wouldn’t have thought to find machine learning in, such as anthropology and theoretical physics. Around the same time, in November 2022 to be exact, ChatGPT was released. One of many large language models that could be used as chatbots and for many natural language tasks. Today, these models can even integrate other inputs such as images. These so-called foundation models have a lot of potential and we can already see an impact on science:

• Similar to how doctors make judgments, multi-modal foundation models allow for the integration of medical data from multiple sources [1].
• Chatbots are opening the door to personalized education with high-quality feedback at a large scale [2].
• Large language models are even changing the way scientists write [3].

Foundation models have opened up new types of problems for the application of machine learning in science. We believe that in the future all sciences – not just the natural and social sciences but also the humanities – will integrate machine learning into their research. Machine learning will even be used in disciplines that have been beyond quantitative treatment like literature sciences, history, and philosophy. We do not claim that this development will always be for the better.

But especially with large language models like ChatGPT in mind, what role is left for humans in science anyway?

15.2 No science without humans

For decades, we have heard stories of theory-free science or dreams of the automation of science [4]. And, indeed, some of the tasks that scientists spend most of their time on have changed:

• Scientists no longer have to solve numerical equations by hand.
• Sensors collect and transmit data automatically, and there are fewer instances where scientists need to collect the data themselves.
• Plotting data can be done digitally and doesn’t require pen and paper.

Many other tasks are now changing with machine learning.

• Labeling wildlife images may no longer be an issue for future animal ecologists [5].
• Machine learning models can help physicists design quantum experiments [6].
• Designing proteins with desired functions will no longer require the intuition of biologists, but can rely on the power of big data models like AlphaFold [7].

But may foundation models like ChatGPT at some point suggest all the experiments we should run and steer the direction of science?

It is true that more and more tasks that used to be done by scientists can be automated. But that doesn’t mean we are moving toward a science without scientists. If you look at history, the job profile of scientists has always changed. In Aristotelian times, scientists were often collectors of knowledge, writing and reading were essential skills. Scientists like Copernicus or Galileo had to perform precise calculations using their brains instead of CPUs and a piece of paper instead of computer memory. Physicists in Einstein’s time couldn’t enter the field without strong training in pure mathematics. And in many fields of modern science, scientists without basic skills in programming and data analysis skills have a hard time. Similarly, the profile of future scientists will require basic machine learning modeling skills.

Automating a task can free up time for other tasks. Potentially, it can free up time for scientists to study new phenomena, which requires a different set of skills. That is, if writing grants and reviewing and submitting papers doesn’t eat up all the extra time – but that’s another story. Scientists are constantly reinventing their roles – discovering new things seems to be the only job description that never expires. In the end, science is what scientists find interesting, and for something to be interesting, there has to be a being with goals and desires who finds it interesting. In fact, we could start talking about artificial general intelligence and machine consciousness right now, but we prefer not to…

15.3 From predictive tool to scientific method

To become part of the scientific toolkit – as programming and data analysis already are today – machine learning skills must go beyond building a powerful predictive model. The perception of machine learning as a purely associative, predictive tool reflects a growing phase. As we have shown in this book, many other skills must be part of the scientist’s machine learning toolkit to work for science:

• Nobody just gives you a prediction task, you need domain knowledge to identify a meaningful problem.
• Data is not given, it is constructed. Understanding your data is necessary to build robust and generalizing models.
• Uncertainties of predictions and causal dependencies are not a side quest but essential, especially when machine learning predictions guide real-world actions.
• Models that are complete black boxes, not reproducible, and poorly reported will not be used in practice.

Machine learning methodology needs to mature and take into account all aspects of science: We need best practices for formalizing scientific questions; frameworks for data acquisition, cleaning, and engineering; a theory of learning algorithms that explains why some learning algorithms are successful; and, finally, standards for evaluating deployed models.

We are optimistic that future machine learning will take these steps, otherwise we would not have written this book. Our optimism is supported by the fact that other scientific methods have followed a similar path. For example, early statistics was primarily a tool for summarizing demographic data or making questionable claims, as in phrenology, before it turned into the multi-faceted and widely accepted scientific methodology it is today.

15.4 Machine learning knowledge with a long shelf-life

Machine learning is a fast-moving field. New trends and hype every year can make it difficult to keep track of the topics that are here to stay. Specific algorithms such as random forests, support vector machines, convolutional neural networks, generative adversarial networks, graph neural networks, or transformer models always have a half-life. They may be state of the art for two to five years before the next competitor comes along.

How do you succeed in such a fast-paced environment? Of course, sometimes you have to learn a new tool. But ideally, you also learn the principles that remain. Learning to program involves learning both general concepts like functions and the specific syntax of a programming language. The concepts often live for a long time: Someone who learned about functions 30 years ago still finds that knowledge relevant today, but they may no longer write their applications in Delphi or Pascal.

What remains relevant in science is not the concrete methods for solving problems, but the problems themselves and the general strategies for solving them. This is what we tried to convey in this book. We introduced ideas and formalisms for describing questions as machine learning problems and provided taxonomies for solutions.

Ultimately, the future of science in the age of machine learning is not about replacing scientists but, about enhancing their ability to explore the unknown. And while the tools may change, the curiosity and drive that fuel scientific discovery will remain timeless. That, after all, is the essence of science: to keep asking, exploring, and understanding.

[1]

M. Moor et al., “Foundation models for generalist medical artificial intelligence,” Nature, vol. 616, no. 7956, pp. 259–265, 2023.

[2]

E. Kasneci et al., “ChatGPT for good? On opportunities and challenges of large language models for education,” Learning and individual differences, vol. 103, p. 102274, 2023.

[3]

D. Kobak, R. G. Márquez, E.-Á. Horvát, and J. Lause, “Delving into ChatGPT usage in academic writing through excess vocabulary,” arXiv preprint arXiv:2406.07016, 2024.

[4]

C. Anderson, “The end of theory: The data deluge makes the scientific method obsolete,” Wired magazine, vol. 16, no. 7, pp. 16–07, 2008.

[5]

M. S. Norouzzadeh et al., “Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning,” Proceedings of the National Academy of Sciences, vol. 115, no. 25, pp. E5716–E5725, 2018, doi: 10.1073/pnas.1719367115.

[6]

M. Krenn, M. Malik, R. Fickler, R. Lapkiewicz, and A. Zeilinger, “Automated search for new quantum experiments,” Physical review letters, vol. 116, no. 9, p. 090405, 2016.

[7]

J. Jumper et al., “Highly accurate protein structure prediction with AlphaFold,” Nature, vol. 596, no. 7873, pp. 583–589, 2021, doi: 10.1038/s41586-021-03819-2.