.png){kind=icon}
Return
Multimodal learning with missing modality for chemical process system
.png){kind=icon}
PRE
AI
DOI：10.1016/j.compchemeng.2025.109196.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Multimodal learning, which integrates data from diverse modalities such as images, text, and acoustic signals, has gained increasing importance for its ability to extract richer information from various sources. In chemical process systems, where complex phenomena such as reactions and heat flows involve data across multiple modalities, effectively leveraging this diverse information is crucial for accurate system interpretation and decision-making. Despite its potential, existing multimodal approaches have largely been developed for vision and natural language processing domains, with limited attention to the unique challenges in chemical processes, particularly the common issue of missing modalities in real-world data. To address this, we propose a novel Multimodal learning framework with Missing modality for Chemical processes (MMC). Our framework handles image and tabular data using patch projection and LSTM layers for embedding, while a multimodal transformer identifies interactions across modalities. We incorporate prompt learning to mitigate the missing modality problem by fine-tuning the model with missing-aware prompts. Through extensive experiments on a plug flow reactor system for fault diagnosis, we demonstrate the robustness and improved performance of MMC, especially in scenarios with missing data. An ablation study further investigates the effectiveness of the model architecture and prompt design.
Journal
C
IF:
3.9
Papers:
8.1K
Citations:
1.7W
Organization
No organization information available
Cited Papers
ShareSave
ShareSaveShareSaveHybrid multimodal fusion with deep learning for rolling bearing fault diagnosis.png)
MEASUREMENT
MEASUREMENTIF5.6
PREAI
ShareSaveShareSaveShareSave
View more