.png){kind=icon}
返回
Material transformers: deep learning language models for generative materials design
.png){kind=icon}
OA
AI
DOI：10.1088/2632-2153/acadcd.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Pre-trained transformer language models (LMs) on large unlabeled corpus have produced state-of-the-art results in natural language processing, organic molecule design, and protein sequence generation. However, no such models have been applied to learn the composition patterns for the generative design of material compositions. Here we train a series of seven modern transformer models (GPT, GPT-2, GPT-Neo, GPT-J, BLMM, BART, and RoBERTa) for materials design using the expanded formulas of the ICSD, OQMD, and Materials Projects databases. Six different datasets with/out non-charge-neutral or EB samples are used to benchmark the generative design performances and uncover the biases of modern transformer models for the generative design of materials compositions. Our experiments show that the materials transformers based on causal LMs can generate chemically valid material compositions with as high as 97.61% to be charge neutral and 91.22% to be electronegativity balanced, which has more than six times higher enrichment compared to the baseline pseudo-random sampling algorithm. Our LMs also demonstrate high generation novelty and their potential in new materials discovery is proved by their capability to recover the leave-out materials. We also find that the properties of the generated compositions can be tailored by training the models with selected training sets such as high-bandgap samples. Our experiments also show that different models each have their own preference in terms of the properties of the generated samples and their running time complexity varies a lot. We have applied our materials transformers to discover a set of new materials as validated using density functional theory calculations.
Keyword:
deep learning
language models
generative design
materials discovery
transformer
期刊
M
IF:
4.6
论文数:
1.1K
被引数:
3.4K
机构
引用论文
Predicting materials properties without crystal structure: deep representation learning from stoichiometry预测没有晶体结构的材料特性: 从化学计量学中学习深度表示.png)
NATURE COMMUNICATIONS
OAAI
NATURE COMMUNICATIONSIF15.7
OAAI分享收藏分享收藏Graft copolymerization of methylmethacrylate with N‐substituted maleimide‐styrene copolymer by ATRPOAAI
OAAI分享收藏Ab initiomolecular-dynamics simulation of the liquid-metal–amorphous-semiconductor transition in germanium 从头算 锗中液态-金属-非晶-半导体跃迁的分子动力学模拟
PREAI
分享收藏Commentary: The Materials Project: A materials genome approach to accelerating materials innovation评论: 材料项目: 加速材料创新的材料基因组方法APL MATERIALSOAAI
APL MATERIALSIF4.5
OAAI分享收藏分享收藏A Generative Neural Network for Maximizing Fitness and Diversity of Synthetic DNA and Protein SequencesCELL SYSTEMSOAAI
CELL SYSTEMSIF7.7
OAAI分享收藏
查看更多内容