Multimodal multi-instance learning for long-term ECG classification

doi:10.1016/j.knosys.2023.110555

Return

Multimodal multi-instance learning for long-term ECG classification

2023-06-01

PRE

DOI：10.1016/j.knosys.2023.110555

Original

Original request for help

Save

Abstract

En 中文

Recently, deep learning-based models have been widely used for electrocardiogram (ECG) classification tasks. Most ECG signals are long-term time series that contain a large number of sample points. However, existing deep learning-based models resize or crop the original long-term ECG signal due to the limitation of input size and hardware, which results in information loss. To address this issue, a multimodal multi-instance learning neural network (MAMIL) is proposed for long-term ECG classification. The proposed MAMIL has three major components. First, the original ECG signal and Gramian Angular Field (GAF) image converted from the ECG signal are utilized as multimodal inputs, which enables the model to learn complementary information between different modalities. Second, multi-instance learning (MIL) is introduced to avoid information loss. Each long-term ECG signal and GAF image are treated as bags, and each heartbeat from a long-term ECG signal and each patch from a GAF image are treated as instances. Convolutional neural networks (CNNs) are utilized to extract instance features from different modalities. Third, a novel attention mechanism-based feature fusion method is proposed to aggregate the instance features from multiple modalities to obtain the bag feature for final classification. Our feature fusion method adopts pooling to obtain positive instances, which can effectively eliminate redundant information and achieve low computational complexity. The proposed MAMIL is evaluated on both intrapatient and interpatient patterns of two commonly used ECG datasets. Experimental results show that our model not only outperforms common deep learning-based models, but also outperforms previous MIL-based models. (c) 2023 Elsevier B.V. All rights reserved.

Keywords:

Long-term ECG

Multimodal learning

Multi-instance learning

Attention mechanism