ADM in the Public Sector: The Workshop
AlgorithmWatch Switzerland was invited to give a workshop as part of an event series on AI & Ethics organised and presented by Swiss Digital Initiative (SDI). Together with participants from various backgrounds, we explored how Artificial Intelligence (AI) could be applied to the public sector, where and in what forms such applications are already used and what questions should be asked when considering the use of AI in the public sector.
based on a blog post by Nicolas Zahn (SDI)
What are the potential issues with AI in the public sector?
Before the workshop started, host Nicolas Zahn, Senior Project Manager of SDI, welcomed the participants and afterwards cleared the stage for AlgorithmWatch Switzerland. Dr. Anna Mätzener, Dr. des. Angela Müller and Tobias Urech then introduced the organisation and pointed to several challenges regarding the use of algorithmic decision-making (ADM) systems in the public sector:
- Such systems are introduced in ever more fields with great speed and little debate.
- The systems themselves as well as the processes surrounding them are often non-transparent.
- The potential effects on humans and their fundamental rights are not clearly understood.
- The potential effects on democratic societies are not clearly understood.
Against this background, we developed an impact assessment tool for ADM systems used in the public sector that aims at identifying risk signals and enabling transparency over the entire life cycle of such systems.
Systems already in use
The participants were split up into three groups to discuss one use-case each under the guidance our three team members consulting a simplified version of the checklist. The first use-case concerned the education system and revolved around a system that assigns children in a district to kindergarten and primary schools. The second use-case involved a system used in the judicial system to evaluate the risk of relapse for criminals. The final use-case showed a system in use in the field of social security. It tries to identify potential welfare fraud.
Similar systems are or were in use in various European countries and some of them have already led to public outrage due to issues such as non-transparent communication by authorities about the use or bias in data leading to discriminatory decisions.
No easy answers
The participants discussed the impact assessment tool and each use-case vividly in their groups and it became quickly apparent that while serious issues have been identified with each system, there are no easy, fit-anything answers. Nevertheless, it is clear that particularly for AI applications in the public sector, questions need to be raised and answered. To do so, we need to introduce more transparency on which systems are currently used where, by whom and for what purposes. A registry of ADM systems and data access for researchers can yield important insights that help to enable an evidence-based debate on these open questions. Given the potential benefits of ADM systems, participants agreed that it is not an option to assume that they will not be a relevant factor. This is why we need to have an inclusive debate on how we want to design the governance for ADM systems at the Swiss, European and global levels.