A distributed memory algorithm for lexicon building

doi:10.1006/jpdc.1997.1344

Return

A distributed memory algorithm for lexicon building

1997-07-01

PRE

David Hawking *

DOI：10.1006/jpdc.1997.1344

Original

Original request for help

Save

Abstract

En 中文

A parallel algorithm for preparing word frequency concordances over two specified sets of documents from a collection is presented. Good parallel efficiency is demonstrated on a 128-node distributed memory machine using sets whose combined size exceeds one gigabyte. It is demonstrated that efficiency is heavily influenced by hashing and communication strategies. A two-stage hashing algorithm is proposed to reduce communication overhead. Ways of increasing capacity are considered, and the applicability of the algorithm to other text-processing functions such as index and symbol-table building is outlined. (C) 1997 Academic Press.