.png){kind=icon}
返回
Modelling Human Word Learning and Recognition Using Visually Grounded Speech
.png){kind=icon}
OA
AI
DOI：10.1007/s12559-022-10059-7.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Many computational models of speech recognition assume that the set of target words is already given. This implies that these models learn to recognise speech in a biologically unrealistic manner, i.e. with prior lexical knowledge and explicit supervision. In contrast, visually grounded speech models learn to recognise speech without prior lexical knowledge by exploiting statistical dependencies between spoken and visual input. While it has previously been shown that visually grounded speech models learn to recognise the presence of words in the input, we explicitly investigate such a model as a model of human speech recognition. We investigate the time course of noun and verb recognition as simulated by the model using a gating paradigm to test whether its recognition is affected by well-known word competition effects in human speech processing. We furthermore investigate whether vector quantisation, a technique for discrete representation learning, aids the model in the discovery and recognition of words. Our experiments show that the model is able to recognise nouns in isolation and even learns to properly differentiate between plural and singular nouns. We also find that recognition is influenced by word competition from the word-initial cohort and neighbourhood density, mirroring word competition effects in human speech comprehension. Lastly, we find no evidence that vector quantisation is helpful in discovering and recognising words, though our gating experiment does show that the LSTM-VQ model is able to recognise the target words earlier.
Keyword:
Computational modelling
Human speech recognition
Multi-modal learning
Deep learning
Vector quantisation
期刊
IF:
4.3
论文数:
1.6K
被引数:
3.6K
机构
引用论文
.png){kind=icon}
A practical in-line FM-EDFA based on cladding pumping for MIMO-free MDM amplification and transmission一种基于包层泵浦的实用化在线FM-EDFA，用于MIMO-free MDM放大和传输
PREAI
分享收藏分享收藏Description of physical aging kinetics of glassy polymers by interpretation of parameters of the Kohlrausch-Williams-Watts relaxation function via simulation通过模拟解释kohlrausch-williams-watts弛豫函数的参数来描述玻璃状聚合物的物理老化动力学
PREAI
分享收藏
分享收藏
查看更多内容