Algorithmic discrimination in Europe – Challenges and opportunities for gender equality and non-discrimination law by Janneke Gerards and Raphaële Xenidis published by European Commission (2020).
In recent years, media stories and scholarly articles about algorithmic discrimination have flourished. In 2019, for example, the AppleCard algorithm was found to grant higher credit limits to men than to women despite the latter having higher credit scores.1 In 2014, Amazon developed an algorithmic hiring prototype, which was later found to discriminate against women and had to be abandoned.2 A recent empirical study showed how the targeting of online ads can reinforce stereotyping and segregation on the labour market: during the experiment, researchers used the Facebook advertising platform to neutrally disseminate various employment ads. In the end, cashier positions in supermarkets reached an audience composed of 85 % women, while ads for taxi driver positions reached a 75 % black audience and ads for lumberjack positions reached an audience that was 90 % male and 72 % white.3 A global consensus has emerged among both researchers and policy-makers that risks of algorithmic discrimination are pervasive and multifaceted. In this context, understanding these risks and the types of legal challenges they create is key to ensuring equality and combating discrimination.
Indeed, in 2019, the European Commission published a white paper on artificial intelligence, which recognised that the increasing use of algorithms in Europe poses specific risks in terms of fundamental rights protection and in particular in terms of equality and non-discrimination.4 Such risks are also recognised by the Commission’s recent Gender Equality Strategy 2020-2025, which acknowledges that ‘AI […] risks intensifying gender inequalities’.5 In response, the EU has called for the creation of an ‘ecosystem of trust’, which demands that ‘European AI is grounded in [EU] values and fundamental rights’ among which the right to equality and non-discrimination is central.
This report investigates how algorithmic discrimination challenges the set of legal guarantees put in place in Europe to combat discrimination and ensure equal treatment. More specifically, it examines whether and how the current gender equality and non-discrimination legislative framework in place in the EU can adequately capture and redress algorithmic discrimination. It explores the gaps and weaknesses that emerge at both the EU and national levels from the interaction between, on the one hand, the specific types of discrimination that arise when algorithms are used in decision-making systems and, on the other, the particular material and personal scope of the existing legislative framework. This report also maps out the existing legal solutions, accompanying policy measures and good practice to address and redress algorithmic discrimination both at EU and national levels. Moreover, this report proposes its own integrated set of legal, knowledge-based and technological solutions to the problem of algorithmic discrimination.
