arrow
Return

Learning differentiable logic programs for abstract visual reasoning

delete2024-10-26
delete0
delete
OA
AI
H
Hikaru Shindo *
V
Viktor Pfanschilling
D
Devendra Singh Dhami
K
Kristian Kersting
DOI:10.1007/s10994-024-06610-2delete
deleteOriginal
deleteOriginal request for help
deleteShare
deleteSave
Abstract

Abstract

En 中文
Visual reasoning is essential for building intelligent agents that understand the world and perform problem-solving beyond perception. Differentiable forward reasoning has been developed to integrate reasoning with gradient-based machine learning paradigms. However, due to the memory intensity, most existing approaches do not bring the best of the expressivity of first-order logic, excluding a crucial ability to solve abstract visual reasoning, where agents need to perform reasoning by using analogies on abstract concepts in different scenarios. To overcome this problem, we propose NEUro-symbolic Message-pAssiNg reasoNer (NEUMANN), which is a graph-based differentiable forward reasoner, passing messages in a memory-efficient manner and handling structured programs with functors. Moreover, we propose a computationally-efficient structure learning algorithm to perform explanatory program induction on complex visual scenes. To evaluate, in addition to conventional visual reasoning tasks, we propose a new task, visual reasoning behind-the-scenes, where agents need to learn abstract programs and then answer queries by imagining scenes that are not observed. We empirically demonstrate that NEUMANN solves visual reasoning tasks efficiently, outperforming neural, symbolic, and neuro-symbolic baselines.
Keywords:
Neuro-symbolic AI
Differentiable reasoning
Inductive logic programming
Graph neural network

Journal

Machine Learning cover
Machine Learning
IF:
2.9
Papers:
2.7K
Citations:
3.4W

Organization

T
Technical University of Darmstadt
Scholars:
1.3W
Papers: 10.0K
Citations: 1.2W
Cited Papers

Cited Papers

errShare
errSave
The machine learning algorithm for the diagnosis of schizophrenia on the basis of gene expression in peripheral blood
err2021-02-01
err0
PREAI
errLulu Zhu; Xulong Wu; Bingyi Xu; Zhi Zhao; Jialei Yang; Jianxiong Long; Li Su
errShare
errSave
Identification and Characterization of Latex-Specific Proteins in Opium Poppy
err1985-10-01
err0
errOAAI
errCraig L. Nessler; Randy D. Allen; Samuel Galewsky
errShare
errSave
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
err2017-02-06
err3.1K
errOAAI
errKrishna, Ranjay; Zhu, Yuke; Groth, Oliver; Johnson, Justin; Hata, Kenji; Kravitz, Joshua; Chen, Stephanie; Kalantidis, Yannis; Li, Li-Jia; Shamma, David A.; Bernstein, Michael S.; Li Fei-Fei
errShare
errSave
researcher View more