.png){kind=icon}
.png){kind=icon}
Return
A crisis event classification method based on a multimodal multilayer graph model
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2024.129271.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Quickly obtaining and classifying relevant information about crisis events via social media platforms, such as Twitter and Weibo, plays a critical role in the subsequent rescue operations and post-disaster reconstruction. Current crisis event classification technologies are unable to comprehensively acquire data and are not applicable to a wide range of scenarios. To address this issue, this paper proposes a crisis event prediction method based on a multi-information multimodal deep graph model. Exploring multimodal graph information innovatively and deeply enhances the effectiveness and broadens the applicability of predicting crisis events. First, Bi-LSTM and a GCN structure based on word graphs are used to obtain the sentence and grammatical information of the text, constructing a text feature vector. The image feature vector is constructed by combining a CNN and a Transformer. Second, a graph of the text group and a graph of the image group are constructed on the basis of cosine similarity, and a multilayer structure combining autoencoders and a GCN is used to mine the integrated information of the data. Finally, multimodal feature vectors are converted into a multimodal semantic network, and a multilayer GCN structure with an attention mechanism is employed for crisis event classification prediction. The experimental results on the CrisisMMD dataset and its generated datasets demonstrate that the proposed model outperforms the current state-of-the-art models.
Keywords:
Multimodal crisis event prediction
Grammatical features
Attention mechanism
Autoencoder
Integrated information
Multilayer
Multimodal semantic network
Journal
IF:
6.5
Papers:
2.5W
Citations:
6.5W