.png){kind=icon}
返回
SLAPP: Subgraph-level attention-based performance prediction for deep learning models
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neunet.2023.11.043.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The intricacy of the Deep Learning (DL) landscape, brimming with a variety of models, applications, and platforms, poses considerable challenges for the optimal design, optimization, or selection of suitable DL models. One promising avenue to address this challenge is the development of accurate performance prediction methods. However, existing methods reveal critical limitations. Operator-level methods, proficient at predicting the performance of individual operators, often neglect broader graph features, which results in inaccuracies in full network performance predictions. On the contrary, graph-level methods excel in overall network prediction by leveraging these graph features but lack the ability to predict the performance of individual operators. To bridge these gaps, we propose SLAPP, a novel subgraph-level performance prediction method. Central to SLAPP is an innovative variant of Graph Neural Networks (GNNs) that we developed, named the Edge Aware Graph Attention Network (EAGAT). This specially designed GNN enables superior encoding of both node and edge features. Through this approach, SLAPP effectively captures both graph and operator features, thereby providing precise performance predictions for individual operators and entire networks. Moreover, we introduce a mixed loss design with dynamic weight adjustment to reconcile the predictive accuracy between individual operators and entire networks. In our experimental evaluation, SLAPP consistently outperforms traditional approaches in prediction accuracy, including the ability to handle unseen models effectively. Moreover, when compared to existing research, our method demonstrates a superior predictive performance across multiple DL models.
Keyword:
Deep Learning (DL)
Graph neural networks (GNNs)
Performance prediction
Computation graph optimization
Attention mechanisms
期刊
IF:
6.3
论文数:
8.2K
被引数:
3.0W
机构
引用论文
分享收藏Determination of the optical band-gap energy of cubic and hexagonal boron nitride using luminescence excitation spectroscopy使用发光激发光谱法测定立方和六方氮化硼的光学带隙能
OAAI
OAAI分享收藏
分享收藏Efficient network architecture search via multiobjective particle swarm optimization based on decomposition.png)
NEURAL NETWORKS
NEURAL NETWORKSIF6.3
PREAI
分享收藏Structural study of lanthanides(III) in aqueous nitrate and chloride solutions by EXAFS通过EXAFS对硝酸盐和氯化物水溶液中镧系元素 (III) 的结构研究
PREAI
分享收藏
查看更多内容