National Centre of Competence in Research Evolving Language

Understanding speech requires mapping fleeting and often ambiguous soundwaves to meaning. While humans are known to exploit their capacity to contextualize to facilitate this process, how internal knowledge is deployed online remains an open question. Here, we present a model that extracts multiple levels of information from continuous speech online. The model applies linguistic and nonlinguistic knowledge to speech processing, by periodically generating top-down predictions and incorporating bottom-up incoming evidence in a nested temporal hierarchy. We show that a nonlinguistic context level provides semantic predictions informed by sensory inputs, which are crucial for disambiguating among multiple meanings of the same word. The explicit knowledge hierarchy of the model enables a more holistic account of the neurophysiological responses to speech compared to using lexical predictions generated by a neural network language model (GPT-2). We also show that hierarchical predictions reduce peripheral processing via minimizing uncertainty and prediction error. With this proof-of-concept model, we demonstrate that the deployment of hierarchical predictions is a possible strategy for the brain to dynamically utilize structured knowledge and make sense of the speech input.

Language aptitude has recently regained interest in cognitive neuroscience. Traditional language aptitude testing included phonemic coding ability, associative memory, grammatical sensitivity and inductive language learning. Moreover, domain-general cognitive abilities are associated with individual differences in language aptitude, together with factors that have yet to be elucidated. Beyond domain-general cognition, it is also likely that aptitude and experience in domain-specific but non-linguistic fields (e.g. music or numerical processing) influence and are influenced by language aptitude. We investigated some of these relationships in a sample of 152 participants, using exploratory graph analysis, across different levels of regularisation, i.e. sensitivity. We carried out a meta cluster analysis in a second step to identify variables that are robustly grouped together. We discuss the data, as well as their meta-network groupings, at a baseline network sensitivity level, and in two analyses, one including and the other excluding dyslexic readers. Our results show a stable association between language and cognition, and the isolation of multilingual language experience, musicality and literacy. We highlight the necessity of a more comprehensive view of language and of cognition as multivariate systems.

Abstract Understanding speech requires mapping fleeting and often ambiguous soundwaves to meaning. While humans are known to exploit their capacity to contextualize to facilitate this process, how internal knowledge is deployed on-line remains an open question. Here, we present a model that extracts multiple levels of information from continuous speech online. The model applies linguistic and nonlinguistic knowledge to speech processing, by periodically generating top-down predictions and incorporating bottom-up incoming evidence in a nested temporal hierarchy. We show that a nonlinguistic context level provides semantic predictions informed by sensory inputs, which are crucial for disambiguating among multiple meanings of the same word. The explicit knowledge hierarchy of the model enables a more holistic account of the neurophysiological responses to speech compared to using lexical predictions generated by a neural-network language model (GPT-2). We also show that hierarchical predictions reduce peripheral processing via minimizing uncertainty and prediction error. With this proof-of-concept model we demonstrate that the deployment of hierarchical predictions is a possible strategy for the brain to dynamically utilize structured knowledge and make sense of the speech input.