Genetic Algorithm for Interpretability in Vision Language Models

Marcelo H. L. Barreto

doi:10.1007/978-3-032-15987-8_40

Genetic Algorithm for Interpretability in Vision Language Models

2026-01-01

0

PRE

AI

M

Marcelo H. L. Barreto *

C

Cristiano L. Oliveira

Leonardo Nogueira Matos

A

André Britto

DOI：10.1007/978-3-032-15987-8_40

Original

Original request for help

Save

Abstract

En 中文

The rapid advancement of deep learning has highlighted the need for methods capable of explaining model decisions to ensure transparency and trust in AI systems. In this context, Explainable Artificial Intelligence (XAI) has emerged as a key research area. This work introduces an optimization-based approach to interpretability, identifying the most and least semantically relevant regions of an image concerning textual descriptors. We formulate the task as an optimization problem and employ a genetic algorithm to explore the solution space, maximizing or minimizing the semantic alignment between image regions and text. We used a customized dataset derived from ImageNet, containing 1.000 classes, each with 5 images and 6-8 textual descriptors to evaluate our method. Several established interpretability methods were employed for comparison, including Saliency, Guided Backpropagation, Grad-CAM, Gradient Shap, Integrated Gradients, and InputXGradient. Experimental results show that our approach consistently outperforms baseline methods, achieving higher similarity scores between selected regions and their associated descriptors. Our findings demonstrate the effectiveness of genetic algorithms in solving interpretability problems, offering a flexible and scalable method for XAI.

Keywords:

Explainable artificial intelligence

Genetic algorithm

Vision language models

Optimization

Genetic Algorithm for Interpretability in Vision Language Models

Abstract

Journal

Organization