Transformers and genome language models

Micaela Elisa Consens

doi:10.1038/s42256-025-01007-9

2025-03-13

PRE

Micaela Elisa Consens

DOI：10.1038/s42256-025-01007-9

原文链接

原文求助

摘要

En 中文

基于Transformer深度学习架构的大型语言模型已经彻底改变了自然语言处理领域。受人类语言与基因组生物编码之间类比的启发，研究人员已开始基于Transformer及相关架构开发基因组语言模型（gLMs）。本文综述探讨了Transformer和语言模型在基因组学中的应用。我们调研了基因组学中适用于gLMs的开放性问题，并阐述了gLMs及其Transformer架构在这些问题上的应用动机。我们讨论了gLMs在利用无监督预训练任务对基因组进行建模方面的潜力，特别关注零样本和少样本学习的强大能力。我们探讨了Transformer架构的优势与局限性，以及当前gLMs更广泛的优势与局限性。此外，我们基于当前研究趋势，展望了超越Transformer架构的基因组建模未来。本综述为对基因组数据中Transformer和语言模型感兴趣的生物信息学家和计算机科学家提供指导。

Keyword:

DEEP

VARIANTS

SEQ

期刊

Nature Machine Intelligence

IF:

23.9

论文数:

1.3K

被引数:

1.5W

机构

暂无机构信息

引用论文

GENOMICS AND BIOINFORMATICS 基因组学与生物信息学

Introduction to Biomedical Engineering

IF0

2005-01-01

PREAI

Spencer Muse

The Algorithmic Foundations of Differential Privacy 差分隐私的算法基础

Foundations and Trends® in Theoretical Computer Science

IF0

2013-01-01

OAAI

Cynthia Dwork; Aaron Roth

A deep learning framework for neuroscience 神经科学的深度学习框架

Nature Neuroscience

IF0

2019-10-28

OAAI

Blake A. Richards; Timothy P. Lillicrap; Philippe Beaudoin; Yoshua Bengio; Rafal Bogacz; Amelia Christensen; Claudia Clopath; Rui Ponte Costa; Archy de Berker; Surya Ganguli; Colleen J. Gillon; Danijar Hafner; Adam Kepecs; Nikolaus Kriegeskorte; Peter Latham; Grace W. Lindsay; Kenneth D. Miller; Richard Naud; Christopher C. Pack; Panayiota Poirazi; Pieter Roelfsema; João Sacramento; Andrew Saxe; Benjamin Scellier; Anna C. Schapiro; Walter Senn; Greg Wayne; Daniel Yamins; Friedemann Zenke; Joel Zylberberg; Denis Therien; Konrad P. Kording

IF0

OAAI

The Sequence Read Archive 序列读取存档

NUCLEIC ACIDS RESEARCH

IF13.1

2010-11-09

2.0K

OAAI

Leinonen, Rasko; Sugawara, Hideaki; Shumway, Martin

The Genotype-Tissue Expression (GTEx) project 基因型-组织表达 (GTEx) 项目

NATURE GENETICS

IF31.8

2013-05-29

6.2K

OAAI

Lonsdale, John; Thomas, Jeffrey; Salvatore, Mike; Phillips, Rebecca; Lo, Edmund; Shad, Saboor; Hasz, Richard; Walters, Gary; Garcia, Fernando; Young, Nancy; Foster, Barbara; Moser, Mike; Karasik, Ellen; Gillard, Bryan; Ramsey, Kimberley; Sullivan, Susan; Bridge, Jason; Magazine, Harold; Syron, John; Fleming, Johnelle; Siminoff, Laura; Traino, Heather; Mosavel, Maghboeba; Barker, Laura; Jewell, Scott; Rohrer, Dan; Maxim, Dan; Filkins, Dana; Harbach, Philip; Cortadillo, Eddie; Berghuis, Bree; Turner, Lisa; Hudson, Eric; Feenstra, Kristin; Sobin, Leslie; Robb, James; Branton, Phillip; Korzeniewski, Greg; Shive, Charles; Tabor, David; Qi, Liqun; Groch, Kevin; Nampally, Sreenath; Buia, Steve; Zimmerman, Angela; Smith, Anna; Burges, Robin; Robinson, Karna; Valentino, Kim; Bradbury, Deborah; Cosentino, Mark; Diaz-Mayoral, Norma; Kennedy, Mary; Engel, Theresa; Williams, Penelope; Erickson, Kenyon; Ardlie, Kristin; Winckler, Wendy; Getz, Gad; DeLuca, David; MacArthur, Daniel; Kellis, Manolis; Thomson, Alexander; Young, Taylor; Gelfand, Ellen; Donovan, Molly; Meng, Yan; Grant, George; Mash, Deborah; Marcus, Yvonne; Basile, Margaret; Liu, Jun; Zhu, Jun; Tu, Zhidong; Cox, Nancy J.; Nicolae, Dan L.; Gamazon, Eric R.; Im, Hae Kyung; Konkashbaev, Anuar; Pritchard, Jonathan; Stevens, Matthew; Flutre, Timothee; Wen, Xiaoquan; Dermitzakis, Emmanouil T.; Lappalainen, Tuuli; Guigo, Roderic; Monlong, Jean; Sammeth, Michael; Koller, Daphne; Battle, Alexis; Mostafavi, Sara; McCarthy, Mark; Rivas, Manual; Maller, Julian; Rusyn, Ivan; Nobel, Andrew; Wright, Fred; Shabalin, Andrey; Feolo, Mike; Sharopova, Nataliya; Sturcke, Anne; Paschal, Justin; Anderson, James M.; Wilder, Elizabeth L.; Derr, Leslie K.; Green, Eric D.; Struewing, Jeffery P.; Temple, Gary; Volpi, Simona; Boyer, Joy T.; Thomson, Elizabeth J.; Guyer, Mark S.; Cathy Ng; Abdallah, Assya; Colantuoni, Deborah; Insel, Thomas R.; Koester, Susan E.; Little, A. Roger; Bender, Patrick K.; Lehner, Thomas; Yao, Yin; Compton, Carolyn C.; Vaught, Jimmie B.; Sawyer, Sherilyn; Lockhart, Nicole C.; Demchok, Joanne; Moore, Helen F.

IF0

OAAI

IF0

OAAI

On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation 通过逐层相关性传播对非线性分类器决策的逐像素解释

PLOS ONE

IF0

2015-07-10

OAAI

Sebastian Bach; Alexander Binder; Grégoire Montavon; Frederick Klauschen; Klaus-Robert Müller; Wojciech Samek

Long Short-Term Memory 长短期记忆

Neural Computation

IF0

1997-11-01

PREAI

Sepp Hochreiter; Jürgen Schmidhuber

查看更多内容