Scientific article
OA Policy
English

Evaluating GenAI for automated EU AI Act compliance against human experts

Published inDiscover Artificial Intelligence, vol. 6, no. 1, 458
First online date2026-04-20
Abstract

The growing adoption of Artificial Intelligence (AI) in critical sectors raises increasing concerns about regulatory compliance. The European Union Artificial Intelligence Act (EU AI Act) represents the first comprehensive legal framework specifically designed to govern AI systems, classifying them according to their associated risk levels. This paper presents an AI-based compliance checker developed to support the assessment of AI systems under the EU AI Act. The tool is designed to help developers understand legal obligations and identify potential compliance risks through structured, contextualized analysis. We describe the system’s architecture, which combines a generative language model with legal document retrieval techniques to interpret regulatory requirements. The tool was tested on a range of real-world use cases, including algorithmic hiring, medical diagnostics, smart city traffic management, and facial recognition in retail environments. Evaluation was conducted through both automated analysis and expert review. The results confirm the system’s effectiveness in understanding and reconstructing core regulatory obligations, particularly in assessing risk levels and suggesting improvements. While the tool performed strongly in central legal areas, some limitations emerged in handling highly technical or cross-regulatory content. Expert feedback highlighted its utility as a pre-audit tool, emphasizing its depth of legal reasoning, transparency, and contextual adaptability.

Keywords
  • Artificial intelligence regulation
  • EU AI act compliance
  • Automated legal assessment
  • High-risk AI systems
  • Generative language models
  • Regulatory risk evaluation
Citation (ISO format)
CAPPELLI, Maria Assunta et al. Evaluating GenAI for automated EU AI Act compliance against human experts. In: Discover Artificial Intelligence, 2026, vol. 6, n° 1, p. 458. doi: 10.1007/s44163-026-01196-1
Main files (1)
Article (Published version)
Identifiers
Additional URL for this publicationhttps://link.springer.com/10.1007/s44163-026-01196-1
Journal ISSN2731-0809
19views
399downloads

Technical informations

Creation30/05/2026 12:34:48
First validation02/06/2026 14:46:19
Update02/06/2026 14:46:19
Status update02/06/2026 14:46:19
Last indexation02/06/2026 14:46:19
All rights reserved by Archive ouverte UNIGE and the University of GenevaunigeBlack