Algorithmic systems, Artificial Intelligence included, are increasingly used within the public sector as diagnostic and predictive tools to support decisions that affect the quality of people’s lives. For example, they are used in social security to flag potential benefit fraud; in unemployment policy to determine the access to publicly funded programmes, and in the justice system to support decisions about sentencing or bail.

Yet, high-profile scandals have raised serious concerns about use of these technologies as decision aids in the public sector. Investigations by civil society organisations, journalists and researchers have uncovered systems that produced significant errors, discriminated against vulnerable groups, and violated human rights. In Europe, legal frameworks such as the General Data Protection Regulation (GDPR) and the AI Act aim to protect people from these risks. The most commonly used safeguard in the public sector—required by the GDPR—is the supervision of algorithmic systems by frontline staff, often referred as “human in the loop.”

This seminar will examine whether this safeguard is truly effective. Drawing on a mixed-methods study of a profiling algorithm used by the Public Employment Services in Poland, Karolina Sztandar-Sztanderska will present evidence of frontline staff over-relying on algorithmic advice and explain how policy-, organisation-, professionalism-, and technology-related factors limited their ability to oversee such systems.

These research findings have practical implications. They challenge the overly optimistic assumptions about the human in the loop and show how difficult it is to ensure conditions that facilitate meaningful supervision of algorithmic systems in public administration. The talk will argue that additional safeguards are essential to better protect people from algorithmic harms.

Bio Karolina Sztandar-Sztanderska