arrow
Return

Neural Logic Vision Language Explainer

delete2024-01-01
delete1
delete
OA
AI
X
Xiaofeng Yang
F
Fayao Liu
G
Guosheng Lin *
DOI:10.1109/TMM.2023.3310277delete
deleteOriginal
deleteOriginal request for help
deleteShare
deleteSave
Abstract

Abstract

En 中文
If we compare how humans reason and how deep models reason, humans reason in a symbolic manner with a formal language called logic, while most deep models reason in black-box. A natural question to ask is Do the trained deep models reason similar as humans? or Can we explain the reasoning of deep models in the language of logic?. In this work, we present NeurLogX to explain the reasoning process of deep vision language models in the language of logic. Given a trained vision language model, our method starts by generating reasoning facts through augmenting the input data. We then develop a differentiable inductive logic programming framework to learn interpretable logic rules from the facts. We show our results on various popular vision language models. Interestingly, we observe that almost all of the tested models can reason logically.
Keywords:
Explainable artificial intelligence
vision language pretraining

Journal

IEEE Transactions on Multimedia cover
IEEE Transactions on Multimedia
IF:
9.7
Papers:
4.6K
Citations:
2.4W

Organization

N
Nanyang Technological University
Scholars:
4.9W
Papers: 4.8W
Citations: 8.1W
A
agency for science technology & research (a*star)
Scholars:
2.2W
Papers: 1.9W
Citations: 57
Cited Papers

Cited Papers

Intercellular Adhesion Molecule 1 (ICAM-1) Gene Variant is Associated with Coronary Artery Calcification Independent of Soluble ICAM-1 Levels
err2004-12-01
err0
PREAI
errMuredach P. Reilly; Megan L. Wolfe; Jennifer Dykhouse; Karthik Reddy; A. Russell Localio; Daniel J. Rader
errShare
errSave
Markov logic networks
err2006-01-27
err1.6K
errOAAI
errRichardson, M; Domingos, P
errShare
errSave
Internal control in PCR for Mycobacterium tuberculosis: usefulness and improvement of the diagnosis
err2008-08-01
err0
errOAAI
errElizabeth Cortez-Herrera; Rosa Dea Sperhacke; Daniela Becker; Afrânio Kritski; Arnaldo Zaha; Maria Lucia Rosa Rossetti
errShare
errSave
errShare
errSave
errShare
errSave
researcher View more