.png){kind=icon}
返回
Scalable thread based index construction using wavelet tree
.png){kind=icon}
PRE
AI
DOI：10.1007/s11042-022-13906-9.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Indexing is one of the key components of any search tool to be optimized for searching documents. Among the existing indexing techniques, inverted indexing is one of the best methods used at a larger scale for various applications. Under this method, the index is designed using a signature file, hash tree and B-tree to retrieve the required document in efficient time. B-tree is popular due to its searching efficiency, but its performance degrades with increasing data set size. The wavelet tree has become a popular and versatile data structure in the last decade, used in various domains such as sequences, indexing, compression, and grid-point with surprising results. This study proposes a parallel wavelet tree algorithm with hybridization of the Map-Reduce concept to construct an index for textual search. The proposed algorithm reduces the index construction time considerably. Experiments show that the proposed algorithm takes a reasonable trade-off with existing indexing approaches. For large data sets, index construction time has been reduced with respect to other existing state-of-art schemes. Also, results show that the algorithm performs well when the data-set scales up to up-to-the full utilization of available cores. It is possible due to the use of multiple threads working in parallel. Our experiment demonstrated consistent performance with 2-core, 4-core, 8-core, 12-core and results of 16-core show increase in index construction time due to parallel overhead when the data-set in not sufficiently large.
Keyword:
Wavelet tree
Inverted index
Map-reduce
Search time
Parallel algorithms
Dictionary searching
期刊
IF:
3
论文数:
2.0W
被引数:
3.2W
机构
引用论文
分享收藏Edge effects and intraguild predation in native and introduced centipedes: evidence from the field and from laboratory microcosms.png)
Oecologia
OecologiaIF0
PREAI
分享收藏
Observation of DNA and protein distributions in mammalian cell nuclei using STXM使用STXM观察哺乳动物细胞核中DNA和蛋白质的分布
OAAI
OAAI分享收藏Tuning sound for infrastructures: artificial intelligence, automation, and the cultural politics of audio mastering基础设施的声音调优：人工智能、自动化以及音频母带处理的跨文化政治
PREAI
分享收藏
查看更多内容