.png){kind=icon}
返回
Democratizing protein language models with parameter-efficient fine-tuning
.png){kind=icon}
OA
AI
DOI：10.1073/pnas.2405840121.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Proteomics has been revolutionized by large protein language models (PLMs), which learn unsupervised representations from large corpora of sequences. These models are typically fine-tuned in a supervised setting to adapt the model to specific downstream tasks. However, the computational and memory footprint of fine-tuning (FT) large PLMs presents a barrier for many research groups with limited computational resources. Natural language processing has seen a similar explosion in the size models, where these challenges have been addressed by methods for parameter -efficient fine-tuning (PEFT). In this work, we introduce this paradigm to proteomics through leveraging the parameter -efficient method LoRA and training new models for two important tasks: predicting protein-protein interactions (PPIs) and predicting the symmetry of homooligomer quaternary structures. We show that these approaches are competitive with traditional FT while requiring reduced memory and substantially fewer parameters. We additionally show that for the PPI prediction task, training only the classification head also remains competitive with full FT, using five orders of magnitude fewer parameters, and that each of these methods outperform stateof-the-art PPI prediction methods with substantially reduced compute. We further perform a comprehensive evaluation of the hyperparameter space, demonstrate that PEFT of PLMs is robust to variations in these hyperparameters, and elucidate where best practices for PEFT in proteomics differ from those in natural language processing. All our model adaptation and evaluation code is available open -source at https://github.com/microsoft/peft_proteomics. Thus, we provide a blueprint democratize the power of PLM adaptation to groups with limited computational resources.
Keyword:
protein language
parameter-efficient fine-tuning
protein-protein interactions
homooligomer symmetry
quaternary structure
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
P
IF:
9.1
论文数:
10.8W
被引数:
73.5W
机构
引用论文
Expression of the low‐affinity NGF receptor during human muscle development, regeneration, and in tissue culture低亲和力NGF受体在人肌肉发育，再生和组织培养中的表达
PREAI
分享收藏Large language models generate functional protein sequences across diverse families大型语言模型生成跨不同家族的功能蛋白质序列.png)
NATURE BIOTECHNOLOGY
OAAI
NATURE BIOTECHNOLOGYIF41.7
OAAI分享收藏分享收藏Flaws in evaluation schemes for pair-input computational predictions成对输入计算预测的评估方案中的缺陷NATURE METHODSOAAI
NATURE METHODSIF32.1
OAAI分享收藏
查看更多内容