.png){kind=icon}
返回
Assessing multivariate Bernoulli models for information retrieval
.png){kind=icon}
PRE
AI
DOI：10.1145/1361684.1361690.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Although the seminal proposal to introduce language modeling in information retrieval was based on a multivariate Bernoulli model, the predominant modeling approach is now centered on multinomial models. Language modeling for retrieval based on multivariate Bernoulli distributions is seen inefficient and believed less effective than the multinomial model. In this article, we examine the multivariate Bernoulli model with respect to its successor and examine its role in future retrieval systems. In the context of Bayesian learning, these two modeling approaches are described, contrasted, and compared both theoretically and computationally. We show that the query likelihood following a multivariate Bernoulli distribution introduces interesting retrieval features which may be useful for specific retrieval tasks such as sentence retrieval. Then, we address the efficiency aspect and show that algorithms can be designed to perform retrieval efficiently for multivariate Bernoulli models, before performing an empirical comparison to study the behaviorial aspects of the models. A series of comparisons is then conducted on a number of test collections and retrieval tasks to determine the empirical and practical differences between the different models. Our results indicate that for sentence retrieval the multivariate Bernoulli model can significantly outperform the multinomial model. However, for the other tasks the multinomial model provides consistently better performance (and in most cases significantly so). An analysis of the various retrieval characteristics reveals that the multivariate Bernoulli model tends to promote long documents whose nonquery terms are informative. While this is detrimental to the task of document retrieval (documents tend to contain considerable nonquery content), it is valuable for other tasks such as sentence retrieval, where the retrieved elements are very short and focused.
Keyword:
theory
experimentation
information retrieval
language models
multinomial
multivariate Bernoulli
期刊
IF:
9.1
论文数:
1.2K
被引数:
4.7K
机构
引用论文
A TUTORIAL ON HIDDEN MARKOV-MODELS AND SELECTED APPLICATIONS IN SPEECH RECOGNITION关于语音识别中的隐马尔可夫模型和选定应用的教程.png)
PROCEEDINGS OF THE IEEE
PROCEEDINGS OF THE IEEEIF25.9
PREAI
分享收藏Two decades of statistical language modeling: Where do we go from here?统计语言建模二十年: 我们何去何从？PROCEEDINGS OF THE IEEE
OAAI
PROCEEDINGS OF THE IEEEIF25.9
OAAI分享收藏
GOT CALCIUM? A RARE CASE OF PERITONEAL SARCOIDOSIS CAUSING HYPERCALCEMIA IN A DIALYSIS PATIENT有钙吗？一例引起透析患者高钙血症的罕见腹膜肉芽肿病病例
PREAI
分享收藏
查看更多内容