.png){kind=icon}
Return
Tokenization for Molecular Foundation Models
.png){kind=icon}
PRE
AI
DOI：10.1021/acs.jcim.5c01856.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Text-based foundation models have become an important part of scientific discovery, with molecular foundation models accelerating advancements in material science and molecular design. However, existing models are constrained by closed-vocabulary tokenizers that capture only a fraction of molecular space. In this work, we systematically evaluate 35 tokenizers, including 20 chemistry-specific tokenizers, and reveal significant gaps in their coverage of the SMILES molecular representation. To assess the impact of tokenizer choice, we introduce n-gram language models as a low-cost proxy and validate their effectiveness by pretraining and finetuning 18 RoBERTa-style encoders for molecular property prediction. To overcome the limitations of existing tokenizers, we propose two new tokenizers─Smirk and Smirk-GPE─with full coverage of the OpenSMILES specification. The proposed tokenizers systematically integrate nuclear, electronic, and geometric degrees of freedom, facilitating applications in pharmacology, agriculture, biology, and energy storage. Our results highlight the need for open-vocabulary modeling and chemically diverse benchmarks in cheminformatics.
Journal
IF:
5.3
Papers:
9.1K
Citations:
4.0W
Organization
Cited Papers
ShareSaveE(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials.png)
NATURE COMMUNICATIONS
OAAI
NATURE COMMUNICATIONSIF15.7
OAAIShareSaveFound in Translation: predicting outcomes of complex organic chemistry reactions using neural sequence-to-sequence modelsCHEMICAL SCIENCEOAAI
CHEMICAL SCIENCEIF7.4
OAAIShareSave
ShareSaveShareSaveShareSave
View more