arXiv:2611.03257v1 Announce Type: cross
Abstract: Artificial Intelligence, and Machine Learning in general, is becoming pervasive across all domains and disciplines. By means of this data-driven approach, algorithms can learns complex task just by learning from samples. However, if the data-set is biased, they might lead to the development of unfair computer systems, that is, characterised by a prejudice towards specific individuals or groups, which might get an unmotivated output from the system. This circumstance lead to the development of multiple metrics designed to quantify the level of unfairness of an algorithm. However, a generalised assumption in the literature is that the sensitive features and the output labels are binary. When the data is not binary, it is forcefully binarized using a one-versus-the-rest simplification. In this work, we discuss the limitations of existing metrics and propose novel, mathematically sound, and generalized multi-group and multi-class formulations of the metrics. Ultimately, our results highlight the empirical assessment distortion caused by binarization strategies, which reveals the limitations of binary fairness metrics in the multi class setting.
arxiv
Rethinking Fairness Metrics for Multi-Class and Multi-Group Assessment
This paper critiques the common practice of binarizing sensitive features and labels in fairness metrics, which distorts assessments in multi-class settings. The authors propose generalized multi-group and multi-class formulations of fairness metrics that are mathematically sound. Empirical results show that binarization strategies cause significant assessment distortion, highlighting the limitations of binary metrics.
Provides a concrete, recent research contribution addressing a real gap in AI fairness evaluation, with practical implications for model auditing.
Comments & Questions (0)
No comments yet
Be the first to comment!