Eamon Duede (University of Chicago), "Deep Learning Opacity in Scientific Discovery"
Philosophy of Science, 2023
Artificial intelligence is now (nearly) ubiquitous in the sciences. Though there are many reasons for this, the most straightforward among them is that AI is incredibly useful for achieving scientific aims. Of course, there are many such aims. Some are epistemic and include achievements like accurate prediction. Others are non-epistemic and include pursuits such as health and well-being. While scientists’ enthusiasm for AI is undoubtedly influenced by the prestige conferred upon those working in ‘sexy’ new areas, it would require a rather extreme brand of cynicism to boil this all down to hype. Genuine scientific breakthroughs that leverage AI are plentiful and easy to find.
However, there is an equally genuine problem. Philosophers (and scientists) have recently focused on critical, epistemological challenges arising from the opacity of deep neural networks. The black-box nature of today’s most powerful AI systems frustrates equally laudable epistemic aims such as understanding and explanation, both of which are traditionally taken to serve justificatory roles not only for our scientific claims (epistemic) but also our practical aims (non-epistemic). In general, the philosophical literature on AI (whether in the philosophy of science or the ethics of technology) has been concerned with running down and clarifying the nature of the challenges that arise from opacity and has worked diligently to set in place novel frameworks for both the epistemically and socially responsible use of AI.
However, reading that literature, one gets the sneaking suspicion that using epistemically opaque AI systems must be zero-sum. So, you could be forgiven for concluding that using AI for science must just be exceptionally risky, and that doing justifiably good science might not be possible. However, this sentiment is difficult to make sense of when one considers the flood of recent AI-enabled breakthroughs and near-ubiquitous excitement for AI-infused science.
In a new paper recently published in Philosophy of Science, I argue that the disconnect between philosophical pessimism and scientific optimism is driven by a failure to examine how AI is actually used in science. I show that, to understand the epistemic justification underlying AI-powered breakthroughs, philosophers must examine the role played by deep learning as part of a much broader process of discovery. Much of scientific pursuit involves stumbling about, proposing and testing hypotheses, making sense of new observations, integrating or rejecting anomalous results, and generally making a mess of things until something resembling codified, rigorously justifiable knowledge hits the academic press. This aspect (which is, in fact, most of scientific activity) was, as you might recall, painfully transparent and on fully unvarnished display in the early days of the pandemic.
The distinction between that heated period of exploration and the cold, rigorous process of acceptance is captured by an old philosophical distinction (proposed initially by Hans Reichenbach) between the ‘context of discovery’ and the ‘context of justification’. In the paper, I argue that this distinction is most helpful in making sense of the apparent disconnect between philosophical and scientific work in AI.
In the context of discovery, the outputs of AIs can be used to guide attention and scientific intuition toward more promising hypotheses. Yet, these outputs do not themselves require justification. Instead, the outputs of opaque models provide reasons or evidence for pursuing some particular paths of inquiry over others. While the process ultimately leads to scientific claims that stand in need of justification, the part played by an opaque model in that process can, itself, be epistemically insulated from the strong sort of evaluation applied to findings in the context of justification.
I demonstrate the importance of attending to this distinction with two cases drawn from the recent scientific literature. The first involves the use of AI to guide mathematical intuition toward the formulation of a conjecture for and proof of the relationship between algebraic and geometric properties of low-dimensional knots. The second marshals the predictive power of deep learning to drive investigative attention in geophysics to empirical quantities most salient for the accurate prediction of earthquake aftershock timing and location. Both cases result in new and deeper scientific understanding. Neither is epistemologically frustrated by opacity.
This paper does not deny the legitimacy and seriousness of the epistemological concerns raised by philosophers. When scientists, doctors, or mortgage brokers treat the outputs of opaque systems as claims to be believed or acted upon, these claims require justification. If the only justification for them is hidden in the depths of an otherwise opaque model, we’ve got trouble. But, this paper aims to show that there are other, perfectly responsible ways to leverage today’s most powerful systems. And the result is that epistemic opacity need not diminish AI’s capacity to lead scientists to significant and justifiable breakthroughs.