Biomedical AI CDT seminar series 13 January 2023, Rob Procter, University of Warwick
Abstract:
The need for AI systems to provide explanations for their
behaviour is now widely recognised as key to their adoption. In this paper, we
examine the problem of trustworthy AI and explore what delivering this means in
practice, with a focus on healthcare applications. Work in this area typically
treats trustworthy AI as a problem of Human-Computer Interaction involving the
individual user and an AI system. However, we argue here that this overlooks
the important part played by organisational accountability in how people reason
about and trust AI in socio-technical settings. To illustrate the importance of
organisational accountability, we present findings from ethnographic studies of
breast cancer screening and cancer treatment planning in multidisciplinary team
meetings to show how participants made themselves accountable both to each
other and to the organisations of which they are members. We use these findings
to enrich existing understandings of the requirements for trustworthy AI and to
outline some candidate solutions to the problems of making AI accountable both
to individual users and organisationally. We conclude by outlining the
implications of this for future work on the development of trustworthy AI,
including ways in which our proposed solutions may be re-used in different
application settings.
Speaker bio:
Rob Procter is professor of Social Informatics and co-chair
of the AI and Human Centred Computing theme in the Department of Computer
Science, University of Warwick. He is also a Fellow of the Alan Turing
Institute for Data Science and AI. His research takes an interdisciplinary,
socio-technical approach to the study of factors that shape the design,
development and adoption of digital technologies. Health informatics has been a
longstanding theme in his research. Current work includes challenges for the adoption
of AI systems in primary, secondary and social care, with a particular interest
in methodologies and practices for the development and deployment of
trustworthy and ethical AI systems. He was editor of the Health Informatics
Journal from 2004 to 2018.
This is joint work with Mark Rouncefield and Peter Tolmie.