Find the Gap: AI, Responsible Agency and Vulnerability

Abstract The responsibility gap, commonly described as a core challenge for the effective governance of, and trust in, AI and autonomous systems (AI/AS), is traditionally associated with a failure of the epistemic and/or the control condition of moral re­ sponsibility: the ability to know what we are doing and exercise competent control over this doing. Yet these two conditions are a red herring when it comes to under­ standing the responsibility challenges presented by AI/AS, since evidence from the cognitive sciences shows that individual humans face very similar responsibility challenges with regard to these two conditions. While the problems of epistemic opacity and attenuated behaviour control are not unique to AI/AS technologies (though they can be exacerbated by them), we show that we can learn important lessons for AI/AS development and governance from how philosophers have re­ cently revised the traditional concept of moral responsibility in response to these challenges to responsible human agency from the cognitive sciences. The resulting instrumentalist views of responsibility, which emphasize the forward-looking and flexible role of agency cultivation, hold considerable promise for integrating AI/ AS into a healthy moral ecology.

Introduction. The responsibility gap has long been viewed as a core challenge for the effective governance of autonomous systems (hereafter, AS). The concept of an autonomous system can be defined in many ways, but for the purposes of this paper we generally refer to “a system involving software applications, machines, and people, that is able to take actions with little or no human supervision.”1 While the literature is split on the nature and true extent of responsibility gaps, they are typically attributed to a failure of the epistemic and/or the control condition of moral responsibility.2 That is, they are thought to arise when no moral agent in the right relation to an AS action has the ability to know what the system is doing, and/or exercise competent control over this doing. It is widely agreed that AS today cannot be moral agents, that is, they cannot be morally responsible for their own actions. Yet we argue that this is not because AS are mere machines.

Discussion / Conclusion. Where does this leave us? We have established that attributing responsibility to AS in some ways faces surprisingly similar challenges as in the case of human individu­ als and collectives. This is because individual human agency is both far more socially embedded and increasingly more distributed than traditional accounts of responsibil­ ity assumed. While evidence from the cognitive sciences suggests that the knowl­ edge and control gaps typically seen as the roots of the problem for AS are in fact reproduced in mundane human contexts, AS do widen existing responsibility gaps created by increasing corporate and institutional distribution of agency. They also increasingly drive the disintegration of agency, which places even greater pressure on traditional responsibility norms and practices. The good news is that agency cultivation approaches allow us to reconceive responsibility practices as compatible with widespread individual deficits in knowl­ edge and control.