Using an Encoding Model to Determine How the Brain Predicts Semantic and Syntactical Units in Natural Speech

Abstract

Prior studies have found that nouns drive comprehension and predictability, and parts of speech (POS) necessary for grammatical relationships have little to no importance. These results support the dominant perspective that semantic features, rather than syntactical features, dominate language cognition. However, many of these studies do not represent POS in natural speech contexts, relying on single-word or sentence level presentations to conclude POS influence on cognition. As grammatically relevant POS rely on surrounding context to establish semantics, these conclusions are premature. This thesis examines how POS modulates predictability and neural response to an interview-style narrative podcast. The results of a continuous predictive measure reveal that despite their dismissal, semantically irrelevant POS like prepositions and determiners were more predictable than nouns and verbs. Using neural responses detected by intracranial electrocorticography electrodes (ECoG) in a passive listening task, an encoding model was trained to predict neural activity at podcast word onset. The accuracy of the encoding model in predicting correct neural responses differed by POS. A comparison of predictability and encoding accuracy revealed that while the predictability of a POS does not correlate with encoding model performance, predictable subsets within a POS have higher, and sometimes earlier peak of encoding accuracy at word onset compared to unpredictable subsets. This suggests that encoding models cannot discriminate distinguishable properties of POS but can distinguish predictability. Encoding accuracy differences by POS, predictability, and word embedding type may have implications for the conflation of predictability and comprehension and use of encoding models in language contexts.