BENN: Bias Estimation Using a Deep Neural Network

doi:10.1109/TNNLS.2022.3172365

BENN: Bias Estimation Using a Deep Neural Network

2024-01-01

2

OA

AI

DOI：10.1109/TNNLS.2022.3172365

Original

Original request for help

Save

Abstract

En 中文

Utilizing existing methods for bias detection in machine learning (ML) models is challenging since each method: 1) explores a different ethical aspect of bias, which may result in contradictory output among the different methods; 2) provides output in a different range/scale and therefore cannot be compared with other methods; and 3) requires different input, thereby requiring a human expert's involvement to adjust each method according to the model examined. In this article, we present BENN, a novel bias estimation method that uses a pretrained unsupervised deep neural network. Given an ML model and data samples, BENN provides a bias estimation for every feature based on the examined model's predictions. We evaluated BENN using three benchmark datasets, one proprietary churn prediction model used by a European telecommunications company, and a synthetic dataset that includes both a biased feature and a fair one. BENN's results were compared with an ensemble of 21 existing bias estimation methods. The evaluation results show that BENN provides bias estimations that are aligned with those of the ensemble while offering significant advantages, including the fact that it is a generic approach (i.e., can be applied to any ML model) and does not require a domain expert.

Keywords:

Data models

Maximum likelihood estimation

Predictive models

Feature extraction

Ethical aspects

Ethics

Neural networks

Bias estimation

deep neural network (DNN)

ethics

fairness estimation

machine learning (ML)

unsupervised learning

BENN: Bias Estimation Using a Deep Neural Network

Abstract

Journal

Organization