Journal of Modern Foreign Psychology
2024. Vol. 13, no. 1, 92–100
doi:10.17759/jmfp.2024130108
ISSN: 2304-4977 (online)
The Temporal Response Function — a New Method for Investigating Neurophysiological Mechanisms of Speech Perception under Ecologically Valid Conditions
Abstract
The temporal response function is a new method that allows to investigate the brain mechanisms of perception of natural, naturalistic speech stimuli. In contrast to other methods for studying brain activity (e.g., evoked potentials), the temporal response function does not require the presentation of a large number of uniform stimuli to produce a robust brain response - recordings of narrative speech lasting 10 minutes or more can be used in experimental paradigms, increasing their ecological validity. The temporal response function can be used to study brain mechanisms of online processing of different components of natural speech: acoustic (physical properties of the audio signal such as envelope and spectrogram), phonological (individual phonemes and their combinations), lexical (contextual characteristics of individual words) and semantic (semantic meaning of words), as well as the interaction between these components processing mechanisms. The article presents the history of the method, its advantages in comparison with other methods and limitations, mathematical basis, features of natural speech components extraction, and a brief review of the main studies using this method.
General Information
Keywords: temporal response function (TRF), EEG, speech, brain mechanisms, naturalistic stimuli, ecological validity
Journal rubric: Neurosciences and Cognitive Studies
Article type: scientific article
DOI: https://doi.org/10.17759/jmfp.2024130108
Funding. This work is supported by the Ministry of Science and Higher Education of the Russian Federation (Agreement No. 075-10-2021-093; Project COG-RND-2262).
Received: 31.01.2024
Accepted:
For citation: Rogachev A.O., Sysoeva O.V. The Temporal Response Function — a New Method for Investigating Neurophysiological Mechanisms of Speech Perception under Ecologically Valid Conditions [Elektronnyi resurs]. Sovremennaia zarubezhnaia psikhologiia = Journal of Modern Foreign Psychology, 2024. Vol. 13, no. 1, pp. 92–100. DOI: 10.17759/jmfp.2024130108. (In Russ., аbstr. in Engl.)
References
- Alday P.M. M/EEG analysis of naturalistic stories: a review from speech to language processing. Language, cognition and neuroscience, 2019. Vol. 34, no. 4, pp. 457—473. DOI:10.1080/23273798.2018.1546882
- Di Liberto G.M., Peter V., Kalashnikova M., Goswami U., Burnham D., Lalor E.C. Atypical cortical entrainment to speech in the right hemisphere underpins phonemic deficits in dyslexia. NeuroImage, 2018. Vol. 175, pp. 70—79. DOI:10.1016/j.neuroimage.2018.03.072
- Broderick M.P., Anderson A.J., Lalor E.C. Semantic Context Enhances the Early Auditory Encoding of Natural Speech. Journal of Neuroscience, 2019. Vol. 39, no. 38, pp. 7564—7575. DOI:10.1523/JNEUROSCI.0584-19.2019
- Castles A., Rastle K., Nation K. Ending the Reading Wars: Reading Acquisition From Novice to Expert. Psychological Science in the Public Interest, 2018. Vol. 19, no. 1, pp. 5—51. DOI:10.1177/1529100618772271
- Crosse M.J., Liberto G.M.D., Lalor E.C. Eye Can Hear Clearly Now: Inverse Effectiveness in Natural Audiovisual Speech Processing Relies on Long-Term Crossmodal Temporal Integration. Journal of Neuroscience, 2016. Vol. 36, no. 38, pp. 9888—9895. DOI:10.1523/JNEUROSCI.1396-16.2016
- Mirkovic B., Debener S., Jaeger M., De Vos M. Decoding the attended speech stream with multi-channel EEG: implications for online, daily-life applications. Journal of Neural Engineering, 2015. Vol. 12, no. 4, article ID 046007. 9 p. DOI:10.1088/1741-2560/12/4/046007
- Di Liberto G.M., Hjortkjær J., Mesgarani N. Editorial: Neural Tracking: Closing the Gap Between Neurophysiology and Translational Medicine. Frontiers in Neuroscience, 2022. Vol. 16, article ID 872600. 4 p. DOI:10.3389/fnins.2022.872600
- Ding N., Simon J. Cortical entrainment to continuous speech: functional roles and interpretations. Frontiers in Human Neuroscience, 2014. Vol. 8, article ID 311. 7 p. DOI:10.3389/fnhum.2014.00311
- Ding N., Simon J.Z. Adaptive Temporal Encoding Leads to a Background-Insensitive Cortical Representation of Speech. Journal of Neuroscience, 2013. Vol. 33, no. 13, pp. 5728—5735. DOI:10.1523/JNEUROSCI.5297-12.2013
- Broderick M.P., Di Liberto G.M., Anderson A.J., Rofes A., Lalor E.C. Dissociable electrophysiological measures of natural language processing reveal differences in speech comprehension strategy in healthy ageing. Scientific Reports, 2021. Vol. 11, no. 1, article ID 4963. 12 p. DOI:10.1038/s41598-021-84597-9
- Mikolov T., Sutskever I., Chen K., Corrado G.S., Dean J. Distributed Representations of Words and Phrases and their Compositionality. In Burges C.J., Bottou L., Welling M., Ghahramani Z., Weinberger K.Q. (eds.), Advances in Neural Information Processing Systems: 27th Annual Conference on Neural Information Processing Systems 2013: Held 5-10 December 2013, Lake Tahoe, Nevada, USA. New York: Curran Associates Inc. Proceedings.com, 2013. Vol. 26. 9 p. DOI:10.48550/arXiv.1310.4546
- Broderick M.P., Anderson A.J., Di Liberto G.M., Crosse M.J., Lalor E.C. Electrophysiological Correlates of Semantic Dissimilarity Reflect the Comprehension of Natural, Narrative Speech. Current Biology, 2018. Vol. 28, no. 5, pp. 803—809. DOI:10.1016/j.cub.2018.01.080
- Hamilton L.S., Huth A.G. The revolution will not be controlled: natural stimuli in speech neuroscience. Language, Cognition and Neuroscience, 2020. Vol. 35, no. 5, pp. 573—582. DOI:10.1080/23273798.2018.1499946
- Klimovich-Gray A., Di Liberto G., Amoruso L., Barrena A., Agirre E., Molinaro N. Increased top-down semantic processing in natural speech linked to better reading in dyslexia. NeuroImage, 2023. Vol. 273, article ID 120072. 11 p. DOI:10.1016/j.neuroimage.2023.120072
- Kalashnikova M., Peter V., Di Liberto G.M., Lalor E.C., Burnham D. Infant-directed speech facilitates seven-month-old infants’ cortical tracking of speech. Scientific Reports, 2018. Vol. 8, article ID 13745. 8 p. DOI:10.1038/s41598-018-32150-6
- Khalighinejad B., da Silva G.C., Mesgarani N. Dynamic Encoding of Acoustic Features in Neural Responses to Continuous Speech. Journal of Neuroscience, 2017. Vol. 37, no. 8, pp. 2176—2185. DOI:10.1523/JNEUROSCI.2383-16.2017
- Crosse M.J., Zuk N.J., Di Liberto G.M., Nidiffer A.R., Molholm S., Lalor E.C. Linear Modeling of Neurophysiological Responses to Speech and Other Continuous Stimuli: Methodological Considerations for Applied Research. Frontiers in Neuroscience, 2021. Vol. 15, article ID 705621. 25 p. DOI:10.3389/fnins.2021.705621
- Maddox R.K., Lee A.K.C. Auditory Brainstem Responses to Continuous Natural Speech in Human Listeners. eNeuro, 2018. Vol. 5, no. 1, article ID e0441-17.2018, 13 p. DOI:10.1523/ENEURO.0441-17.2018
- Broderick M.P., Zuk N.J., Anderson A.J., Lalor E.C. More than words: Neurophysiological correlates of semantic dissimilarity depend on comprehension of the speech narrative. European Journal of Neuroscience, 2022. Vol. 56, no. 8, pp. 5201—5214. DOI:10.1111/ejn.15805
- Gillis M., Vanthornhout J., Simon J.Z., Francart T., Brodbeck C. Neural Markers of Speech Comprehension: Measuring EEG Tracking of Linguistic Speech Representations, Controlling the Speech Acoustics. Journal of Neuroscience, 2021. Vol. 41, no. 50, pp. 10316—10329. DOI:10.1523/JNEUROSCI.0812-21.2021
- Di Liberto G.M., Nie J., Yeaton J., Khalighinejad B., Shamma S.A., Mesgarani N. Neural representation of linguistic feature hierarchy reflects second-language proficiency. NeuroImage, 2021. Vol. 227, article ID 117586. 13 p. DOI:10.1016/j.neuroimage.2020.117586
- Brodbeck C., Bhattasali S., Heredia A.A.C., Resnik P., Simon J.Z., Lau E. Parallel processing in speech perception with local and global representations of linguistic context. eLife, 2022. Vol. 11, article ID e72056. 28 p. DOI:10.7554/eLife.72056
- Pasley B.N., David S.V., Mesgarani N., Flinker A., Shamma S.A., Crone N.E., Knight R.T., Chang E.F. Reconstructing Speech from Human Auditory Cortex. PLOS Biology, 2012. Vol. 10, no. 1, article ID e1001251. 13 p. DOI:10.1371/journal.pbio.1001251
- Lalor E.C., Power A.J., Reilly R.B., Foxe J.J. Resolving Precise Temporal Processing Properties of the Auditory System Using Continuous Stimuli. Journal of Neurophysiology, 2009. Vol. 102, no. 1, pp. 349—359. DOI:10.1152/jn.90896.2008
- Sassenhagen J. How to analyse electrophysiological responses to naturalistic language with time-resolved multiple regression. Language, Cognition and Neuroscience, 2019. Vol. 34, no. 4, pp. 474—490. DOI:10.1080/23273798.2018.1502458
- Seyednozadi Z., Pishghadam R., Pishghadam M. Functional Role of the N400 and P600 in Language-Related ERP Studies with Respect to Semantic Anomalies: An Overview. Archives of Neuropsychiatry, 2021. Vol. 58, no. 3, pp. 249—252. DOI:10.29399/npa.27422
- Crosse M.J., Di Liberto G.M., Bednar A., Lalor E.C. The Multivariate Temporal Response Function (mTRF) Toolbox: A MATLAB Toolbox for Relating Neural Signals to Continuous Stimuli. Frontiers in Human Neuroscience, 2016. Vol. 10, article ID 604. 14 p. DOI:10.3389/fnhum.2016.00604
- Luck S.J., Kappenman E.S. (eds.), The Oxford Handbook of Event-Related Potential Components. Oxford: Oxford University Press, 2011. 664 p. DOI:10.1093/oxfordhb/9780195374148.001.0001
- Gwilliams L., Marantz A., Poeppel D., King J.R. Top-down information shapes lexical processing when listening to continuous speech. Language, Cognition and Neuroscience, 2023, pp. 1—14. DOI:10.1080/23273798.2023.2171072
- Fahmie T.A., Rodriguez N.M., Luczynski K.C., Rahaman J.A., Charles B.M., Zangrillo A.N. Toward an explicit technology of ecological validity. Journal of Applied Behavior Analysis, 2023. Vol. 56, no. 2, pp. 302—322. DOI:10.1002/jaba.972
- Van Petten C., Luka B.J. Prediction during language comprehension: Benefits, costs, and ERP components: Predictive information processing in the brain: Principles, neural mechanisms and models. International Journal of Psychophysiology, 2012. Vol. 83, no. 2, pp. 176—190. DOI:10.1016/j.ijpsycho.2011.09.015
- Verschueren E., Vanthornhout J., Francart T. The Effect of Stimulus Choice on an EEG-Based Objective Measure of Speech Intelligibility. Ear and Hearing, 2020. Vol. 41, no. 6, pp. 1586—1597. DOI:10.1097/AUD.0000000000000875
- Weissbart H., Reichenbach J., Kandylaki K. Cortical tracking of surprisal during continuous speech comprehension. Journal of Cognitive Neuroscience, 2020. Vol. 32, no. 1, pp. 155—166. DOI:10.1162/jocn_a_01467
Information About the Authors
Metrics
Views
Total: 275
Previous month: 30
Current month: 18
Downloads
Total: 73
Previous month: 9
Current month: 7