A Knowledge-Enriched Ensemble Method for Word Embedding and Multi-Sense Embedding

Lanting Fang

doi:10.1109/TKDE.2022.3159539

A Knowledge-Enriched Ensemble Method for Word Embedding and Multi-Sense Embedding

2022-01-01

5

OA

AI

DOI：10.1109/TKDE.2022.3159539

Original

Original request for help

Save

Abstract

En 中文

Representing words as embeddings has been proven to be successful in improving the performance in many natural language processing tasks. Different from the traditional methods that learn the embeddings from large text corpora, ensemble methods have been proposed to leverage the merits of pre-trained word embeddings as well as external semantic sources. In this paper, we propose a knowledge-enriched ensemble method to combine information from both knowledge graphs and pre-trained word embeddings. Specifically, we propose an attention network to retrofit the semantic information in the lexical knowledge graph into the pre-trained word embeddings. In addition, we further extend our method to contextual word embeddings and multi-sense embeddings. Extensive experiments demonstrate that the proposed word embeddings outperform the state-of-the-art models in word analogy, word similarity and several downstream tasks. The proposed word sense embeddings outperform the state-of-the-art models in word similarity and word sense induction tasks.

Keywords:

Task analysis

Context modeling

Semantics

Bit error rate

Knowledge engineering

Wheels

Vocabulary

Word embedding

multi-sense embedding

knowledge graph

ensemble model

A Knowledge-Enriched Ensemble Method for Word Embedding and Multi-Sense Embedding

Abstract

Journal

Organization