Closed-Loop Brain Machine Interface System for In-Vehicle Function Controls Using Head-Up Display and Deep Learning Algorithm

doi:10.1109/TITS.2023.3345855

Closed-Loop Brain Machine Interface System for In-Vehicle Function Controls Using Head-Up Display and Deep Learning Algorithm

2024-07-01

4

PRE

AI

H

Hossein Hamidi Shishavan

DOI：10.1109/TITS.2023.3345855

Original

Original request for help

Save

Abstract

En 中文

Human-vehicle interfaces have evolved with technologies such as touchscreens, voice commands, and advanced steering wheels with control panels. However, these technologies can increase the driver's cognitive workload and distraction, potentially compromising road safety. Brain-machine interfaces (BMI) offer an alternative way of controlling in-vehicle features with minimal distraction. This study presents a closed-loop steady-state visual evoked potentials (SSVEP) based BMI for controlling in-vehicle features via a windshield head-up display to control multiple vehicle features such as music, temperature, settings, and navigation system. The custom software synchronizes visual stimuli, processes electroencephalogram (EEG) signals, and selects target icons in real-time. A convolutional neural network (CNN) based on the SE-ResNet architecture efficiently detects SSVEPs and the driver's intended target icon in under 1 second. Comparative evaluations against traditional and deep learning methods using time and frequency features show superior performance. With just 225 seconds of calibration data, the proposed compact deep-learning model enables drivers to adjust their environment within 0.5 seconds, achieving an Information Transfer Rate (ITR) of 93.43 +/- 7.26 bits/min. This SSVEP-based BMI system demonstrates multi-layer in-vehicle feature control with ten human participants, highlighting its potential to enhance road safety by allowing drivers to make adjustments without diverting their attention from the road. Our code and dataset can be downloaded from https://github.com/hosseinhamidi92.

Keywords:

Brain-machine interface (BMI)

brain-computer interface (BCI)

deep learning

electroencephalogram (EEG)

heads-up display (HUD)

steady state visual evoked potentials (SSVEP)

Closed-Loop Brain Machine Interface System for In-Vehicle Function Controls Using Head-Up Display and Deep Learning Algorithm

Abstract

Journal

Organization