Why we need to audit algorithms and AI from start to finish

We would therefore like to share our thoughts on what we mean by this ‘end-to-end auditing’ of algorithmic or AI systems and why we are taking this approach, using some specific examples that we are considering exploring further. An idea to keep in mind is that a complicated picture can be thought of as a jigsaw puzzle: it can be broken down into many parts, which can be worked together by many people to fit into the right places.

The end-end chain

Behind every AI output lies a long chain of actions, technologies and decisions. From determining the purpose of a tool; planning and exploring the tool; up to and including data collection; training and refining the models; creating interfaces for users; create and use outputs; to build them into a business model; and iterating on the tools after their implementation. There will be constant trade-offs to be made, policies and decisions to be made, and safeguards and restrictions to be implemented. Different actors will be involved, and different risks and harms may arise. So before we explain further what we mean by end-to-end, let us first clarify that we do not view “AI auditing” as merely a technical process or approach. Instead, the diverse nature of potential harm must always be assessed in relation to the social context in which it manifests, which requires diverse perspectives and expertise.

To limit the scope, let’s focus on one specific form of AI: Generative AI (“GenAI”). These systems can create new content (text, images, audio, video) based on user-defined input. Let’s look at the risks and harms that occur at both ends of the chain. Let’s start by considering an early “upstream” part of the value chain: tuning the models so they can recognize and avoid malicious content. This is known as ‘detoxing’ the models. If not done properly, this led to some early and high-profile PR disasters, such as Microsoft’s Tay Chatbot in 2016, which became famous for tweeting racism and conspiracy theories. While it is certainly possible to get newer chatbots like ChatGPT to produce such content, it is significantly more difficult due to the detoxification involved.

One of the most effective ways to detox at scale is to outsource it to humans, who look at the content and label it as ‘harmful’ – which can also include content deemed offensive, violent, etc. – so that a model is created. trained to ‘recognize’ characteristics of this content. News media report this as: Time magazineCompanies like OpenAI and Meta had outsourced this to a company called Sama. Sama employed people in Kenya to take care of the labelling. Some had been hired under false pretenses (vacancies claimed to offer call center work) and moved to Kenya for work. The content they labeled, for $1.30 to $2 an hour, included harm up to and including extreme (sometimes sexual) violence. Here we see upstream decisions that prioritize the speed and cost savings of model training over ensuring people’s well-being – potentially even in violation of those workers’ most basic rights.

We could point to further upstream issues, especially the use of energy and water required to train models. Energy costs and CO2 emissions are increasingly discussed, including in AI ‘model maps’ designed for transparency. But as made clear by the UN Sustainable Development Goals, and explored in our own SustAIn project, this does not include all dimensions of sustainability – environmental (e.g. water, resource extraction, disposal), social (e.g. workers’ rights) and economic (e.g. concentration of power and oligopolies in AI markets).

Let’s consider one now downstream risk of GenAI – the use of tools to create sexualized and degrading images of real women (you could think of other examples). This can be seen in examples ranging from high-profile figures such as Taylor Swift or Giorgia Meloni, members of parliament in Germany and Switzerland, to young women at school. Not only is this inherently terrible and a potential violation of a person’s right to dignity, but in some circumstances it can also contribute to the continued harassment of women from important roles in public life, mental health issues for young people, and a range of other systemic risks that governments around the world want to address.

Intervene along the chain

It is clear that there is no single solution to the wide-ranging harms caused by AI, but there is to many of the problems can are addressed with specific measures. As an advocacy organization, we believe that well-created and well-enforced legislation, which effectively uses civil society input, can create protection and accountability. This is of course not the only answer. For example, the African Content Moderators Union and the Data Workers Inquiry are supporting workers detoxing AI models, including by increasing the visibility of their experiences and filing lawsuits against the companies involved. Organizations such as HateAid and the Apolitical Foundation support women at risk of abuse from technology. However, legislation can help, and the EU is certainly trying to provide a range of tools to protect against damage caused by technology. Using the metaphor of a puzzle, it’s about finding the right piece that fits in the right place, while not losing sight of the big picture and how the pieces should all fit together.

Relevant instruments may not even be in the most obvious legislation. For the above case studies, one could immediately consider the AI ​​Act. However, as we at AlgorithmWatch have previously explained, the AI ​​law has several gaps when it comes to addressing risks to fundamental rights. Instead, the upcoming Corporate Sustainability Due Diligence Directive (also known as the Supply Chain Law) may be better placed to tackle the exploitative use of labor in the creation of GenAI tools deployed in the EU. The EU’s proposed new legislation to combat violence against women and girls can and should take into account the harms facilitated by GenAI; in the meantime, the Digital Services Act may also be an appropriate tool to study and address the distribution of such content. By keeping the end-to-end chain of potential harms in mind, we can therefore expand the range of relevant tools available for different contexts. This can help engage relevant actors with diverse expertise – whether in human rights, labor organization or the protection of women and girls – to intervene at different points along the value chain. But this involves careful coordination and communication with groups that may be very far removed from technology law, and already very overburdened.

Power inequality and lack of responsibility

An end-to-end approach can also consider how the problems are connected, and how they all relate to the big picture behind them: the enormous concentration of power and lack of accountability behind AI. For example, in the above cases, detoxifying the models can play a positive role in reducing the ability to produce derogatory content against women (and other groups). However, this cannot be used to justify the arbitrary exploitation of the workers who train the models. By balancing the risks at different ends of the supply chain, we can consider fairer and more equitable ways to support those doing the detox work while ensuring that detox is carried out effectively. To give another example, in the words of legal scholar Phillip Hacker, perhaps certain “downstream” applications of AI – for example, tools that have been shown to be medically useful – can justify certain levels of resource consumption in the upstream production of models , while other uses need to be justified. held to much stricter standards.

To rigorously consider and enforce the balancing actions outlined above, reliable and transparent information is needed. That brings us to our final point. By looking at GenAI’s end-to-end chains, we can identify common issues – the most obvious being the balance of power between the companies and everyone else, and the lack of transparency and accountability, which hampers any approach to addressing them will confuse you to solve. fronts will be most needed, as the European Center for Non-Profit Law recently pointed out. In our puzzle, many of the most important pieces are hoarded by Big Tech companies (and, to some extent, regulators). Other issues are being fought over by too many groups, such as the disproportionate interest in ‘deep fakes’ and elections. Some pieces may come from groups with little involvement in the world of technology policy, yet who still have the experience and expertise to contribute to these discussions. These are all practical challenges on which civil society organizations and others must coordinate carefully.

Building the puzzle

If this sounds like a complex picture, it is. But this is the world that big tech companies gave us by releasing GenAI the way they did. We must face this problem as it currently exists and not try to reduce it to overly simple standardized frameworks. We cannot examine it at one level, for example by making models “explainable”, or by forming a rescue team in laboratories. The trick is to break it down into smaller ideas, which, when added together, reveal a bigger picture: the political economy behind AI dominated by a handful of billion-dollar tech companies. So we need to break this down into smaller ideas, while at the same time not losing sight of this bigger, overall picture of what’s in and behind the technology that’s currently being presented to us.

At AlgorithmWatch, we combine broader research and advocacy work – whether that’s around the Digital Services Act, the AI ​​Act, surveillance or other topics – with specific empirical projects, such as our work on chatbots and elections or AI at the Frontiers of the EU, and investigative journalism. Turning to the two examples outlined above – exploitation in detoxification and degrading content against women – we explore how we can explore these in ways that reveal the possibilities for intervening “along the value chain” with a range of tools. We are approaching partners working in these areas and considering possible participatory research approaches. And, as is often the case with NGOs, important ideas don’t pay for themselves and we need to find ways to fund such work sustainably. We believe it is important that complex, seemingly abstract issues such as end-to-end auditing can – and should – be addressed by looking at the real-life issues that face people with the toughest demands of the technology.