Bridging Socio-Technical Gaps in Bias Detection

January 27th, 2025

Artificial intelligence (AI) models are increasingly autonomous in decision making, making the pursuit of responsible AI more critical than ever. Responsible AI (RAI) is defined by its commitment to transparency, privacy, safety, inclusiveness, and fairness. But while the principles of RAI are transparent and shared, RAI practices and auditing mechanisms are still incipient. A key challenge is establishing metrics and benchmarks that define performance goals aligned with RAI principles. This paper presents how the ITACA AI auditing platform incorporates demographic benchmarking for AI recommender systems to identify and measure bias. We propose a Demographic Benchmarking Framework to measure populations potentially affected by specific models, set acceptable performance ranges, and guide policymakers and developers. Our approach integrates socio-demographic insights directly into AI systems, reducing bias while also improving overall performance. The main contributions of this study include: 1. Defining control datasets tailored to specific demographics so they can be used in model training to quantify sampling bias; 2. Comparing the overall population with those impacted by the deployed model to identify discrepancies and account for structural bias; and 3. Quantifying drift in different scenarios continuously and as a post-market monitoring of deployment bias.

Source URL: https://arxiv.org/html/2501.15985v1