How Should AI Decisions Be Explained? Requirements for Explanations from the Perspective of European Law

Abstract

This paper investigates the relationship between law and eXplainable Artificial Intelligence (XAI). While there is much discussion about the AI Act, which was adopted by the European Parliament in March 2024, other areas of law seem underexplored. This paper focuses on European (and in part German) law, although with international concepts and regulations such as fiduciary duties, the General Data Protection Regulation (GDPR), and product safety and liability. Based on XAI-taxonomies, requirements for XAI methods are derived from each of the legal fields, resulting in the conclusion that each legal field requires different XAI properties and that the current state of the art does not fulfill these to full satisfaction, especially regarding the correctness (sometimes called fidelity) and confidence estimates of XAI methods.