arrow
Return

Gaze-guided contrastive unsupervised representations learning

delete2025-08-05
delete0
delete
OA
AI
J
Joseph P. Distefano
H
Hemanth Manjunatha
C
Chaithanya Thammineni
K
Kristian Dalland
E
Ehsan T. Esfahani *
DOI:10.1007/s00521-025-11526-6delete
deleteOriginal
deleteOriginal request for help
deleteShare
deleteSave
Abstract

Abstract

En 中文
This study explores the integration of information-rich prior knowledge, specifically human gaze data, to enhance representation learning through contrastive methods. We propose gaze-guided contrastive unsupervised representation learning, a novel framework harnessing human gaze data to guide the selection of positive and negative samples for contrastive learning. By leveraging human gaze information, we capture meaningful patterns in visual task dynamics, enabling the agent to acquire effective strategies from demonstrations and achieve superior performance. Our findings demonstrate significant improvements over baseline algorithms, highlighting the value of gaze-guided representation learning in reducing data requirements and accelerating learning. This approach offers broad applicability to vision-based tasks, emphasizing the critical role of human gaze in improving task efficiency and generalization.
Keywords:
Contrastive learning
Reinforcement learning
Gaze prediction
Human attention networks
Human in the loop systems

Journal

Neural Computing and Applications cover
Neural Computing and Applications
IF:
4.5
Papers:
887
Citations:
3.2W

Organization

D
department of mechanical and aerospace engineering
Scholars:
332
Papers: 160
Citations: 2
Cited Papers

Cited Papers

Emerging Properties in Self-Supervised Vision Transformers
err2021-10-01
err0
errOAAI
errMathilde Caron; Hugo Touvron; Ishan Misra; Herve Jegou; Julien Mairal; Piotr Bojanowski; Armand Joulin
errShare
errSave
Atari-HEAD: Atari Human Eye-Tracking and Demonstration Dataset
err2020-04-03
err0
errOAAI
errRuohan Zhang; Calen Walshe; Zhuode Liu; Lin Guan; Karl Muller; Jake Whritner; Luxin Zhang; Mary Hayhoe; Dana Ballard
errShare
errSave
Gaze and Eye Tracking: Techniques and Applications in ADAS
err2019-12-14
err0
errOAAI
errMuhammad Qasim Khan; Sukhan Lee
errShare
errSave
Human Gaze Assisted Artificial Intelligence: A Review
err2020-07-01
err0
errOAAI
errRuohan Zhang; Akanksha Saran; Bo Liu; Yifeng Zhu; Sihang Guo; Scott Niekum; Dana Ballard; Mary Hayhoe
errShare
errSave
Random Erasing Data Augmentation
err2020-04-03
err0
errOAAI
errZhun Zhong; Liang Zheng; Guoliang Kang; Shaozi Li; Yi Yang
errShare
errSave
Selective eye-gaze augmentation to enhance imitation learning in Atari games
err2021-08-13
err0
PREAI
errChaitanya Thammineni; Hemanth Manjunatha; Ehsan T. Esfahani
errShare
errSave
AGIL: Learning Attention from Human for Visuomotor Tasks
err2018-10-06
err0
errOAAI
errRuohan Zhang; Zhuode Liu; Luxin Zhang; Jake A. Whritner; Karl S. Muller; Mary M. Hayhoe; Dana H. Ballard
errShare
errSave
researcher View more