.png){kind=icon}
返回
A fast algorithm for constructing inverted files on heterogeneous platforms
.png){kind=icon}
PRE
AI
DOI：10.1016/j.jpdc.2012.02.005.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Given a collection of documents residing on a disk, we develop a new strategy for processing these documents and building the inverted files extremely quickly. Our approach is tailored for a heterogeneous platform consisting of multicore CPUs and highly multithreaded CPUs. Our algorithm is based on a number of novel techniques, including a high-throughput pipelined strategy, a hybrid trie and B-tree dictionary data structure, dynamic work allocation to CPU and GPU threads, and optimized CUDA indexer implementation. We have performed extensive tests of our algorithm on a single node (two Intel Xeon X5560 Quad-core CPUs) with two NVIDIA Tesla C1060 CPUs attached to it, and were able to achieve a throughput of more than 262 MB/s on the ClueWeb09 dataset. Similar results were obtained for widely different datasets. The throughput of our algorithm is superior to the best known algorithms reported in the literature even when compared to those run on large clusters. (C) 2012 Elsevier Inc. All rights reserved.
Keyword:
Indexer
Inverted files
Multicore
GPU
Pipelined and parallel parsing and indexing
期刊
IF:
4
论文数:
3.8K
被引数:
4.8K
.png){kind=icon}
没有更多内容