Language Models: Past, Present, and Future

doi:10.1145/3490443

Language Models: Past, Present, and Future

2022-06-21

33

PRE

AI

H

Hang Li *

DOI：10.1145/3490443

Original

Original request for help

Save

Abstract

En 中文

NATURAL LANGUAGE PROCESSING (NLP) has undergone revolutionary changes in recent years. Thanks to the development and use of pre-trained language models, remarkable achievements have been made in many applications. Pre-trained language models offer two major advantages. One advantage is that they can significantly boost the accuracy of many NLP tasks. For example, one can exploit the BERT model to achieve performances higher than humans in language understanding.8 One can also leverage the GPT-3 model to generate texts that resemble human writings in language generation.3 A second advantage of pre-trained language models is that they are universal language processing tools. To conduct a machine learning-based task in traditional NLP, one had to label a large amount of data to train a model.

Language Models: Past, Present, and Future

Abstract

Journal

Organization