A comparative study of visual and auditory perception of emotions in children of primary school age

We investigated the characteristics of perception of nonverbal emotional information in the two modalities of presentation, visual and auditory, in 32 schoolchildren of 8-9 years old. We studied the recognition of four basic emotions: «happiness», «sadness», «anger», «fear» in facial expressions and in tone of voice. We found that in visual perception, the identification of emotion is more accurate. The lack of correlation between the results of emotions recognition in different modalities indicates the independence of processes of emotion recognition in the visual and auditory modalities. The detected hierarchically coincident unevenness in forming different emotion perception mechanism in both modalities, apparently, is largely determined by external factors.

1. Anderson M.N. Vozrastnaya izmenchivost' raspoznavaniya emotsii det'mi ot 6 do 11 let [Age variability of emotions’ recognition in children of 6-11 years old]. Avtoref.diss.kand. SPb. 2013.24 p. (In Russian).
2. Anderson M. N. Eksperimental'noe issledovanie uspeshnosti raspoznavaniya emotsii det'mi ot 6 do 11 let [Experimental study of emotion recognition effectiveness in children of 6-11 years old]. Izvestiya RGPU im. A.I. Gertsena [Proceedings of Herzen SPU of Russia], 2010, no. 121, pp.l 7-24 (In Russian).
3. Banse R., Scherer K.R. Acoustic profiles in vocal emotion expression. Journal of personality and social psychology, 1996, vol. 70, no. 3, p. 614.
4. Banziger Т., Grandjean D., Scherer K. R. Emotion Recognition From Expressions in Face, Voice, and Body:The Multimodal Emotion Recognition Test (MERT) 2009. Emotion 2009, vol. 9, no. 5, pp. 691-704.
5. Barabanshchikov V. A., Maikova T.N. Zavisimost’ tochnosti identifikatsii ekspressii litsa ot lokalizatsii mimicheskikh proyavlenii [Dependence of identification accuracy of face expressions on mimic displays’ localizations]. Voprosypsikhologii [Questions of psychology], 1986, no 5, pp. 131-140 (In Russian).
6. Castro S. L., Lima C. F. Recognizing emotions in spoken language: A validated set of Portuguese sentences and pseudosentences for research on emotional prosody. Behavior Research Methods, 2010, vol. 42, no. 1, pp. 74-81.
7. Demenescu L. R., Mathiak K. A., Mathiak K. Age-and gender-related variations of emotion recognition in pseudowords and faces. Experimental aging research, 2014, vol. 40, no. 2, pp. 187-207.
8. Denisova Z. V. Mekhanizmy emotsional’nogo povedeniya rebenka [Mechanisms of emotional behavior of a child]. L. Nauka, 1978,142 p. (In Russian).
9. Dmitrieva E. S., Gel’man V. Ya. Svyaz’ vospriyatiya emotsional’noi intonatsii rechi v usloviyakh pomekh s akusticheskimi parametrami rechevogo signala u vzroslykh raznogo pola i vozrasta [The Relationship between the Perception of Emotional Intonation of Speech in Conditions of Interference and the Acoustic Parameters of Speech Signals in Adults of Different Gender and Age]. Zhumal vysshei nervnoi deyatel’nosti [Journal of higher nervous system], 2011, vol. 61, no. 3, pp. 306-316 (In Russian).
10. Dmitrieva E.S., Gel’man V.Ya., Zaitseva K.A., Orlov A.M. Vliyanie individual’nykh osobennostei cheloveka na akusticheskie korrelyaty emotsional’noi intonatsii rechi [Influence of human personal features on acoustic correlates of speech emotional intonation characteristics]. Zhumal vysshei nervnoi deyatel’nosti Journal of higher nervous system], 2009, vol. 59, no. 5, pp. 538-546 (In Russian).
11. Dmitrieva E.S., Gel’man V.Ya., Zaitseva K.A., Orlov A.M. Zavisimost’ vospriyatiya emotsional’noi informatsii rechi ot akusticheskikh parametrov stimula u detei raznogo vozrasta [Dependence of the perception of emotional information of speech on the acoustic parameters of the stimulus in children of various ages]. Fiziologiya Cheloveka [Human Physiology], 2008, vol. 34, no. 4, pp. 149-153 (In Russian).
12. Dmitrieva E.S., Zaitseva K.A., Gel’man V.Ya. Vozrastno-polovye osobennosti vospriyatiya emotsio- nal’nykh kharakteristik rechi pod vozdeistviem shuma [The age-sex traits of the perception of the emotional characteristics of speech under noise exposure]. Fiziologiya Cheloveka [Human Physiology], 1999, vol. 25, no. 3, pp. 57-64 (In Russian).
13. Ekman P. Facial expressions. Handbook of cognition and emotion, 1999, vol. 16, pp. 301-320.
14. Farber D. A., Beteleva T. G. Formirovanie mozgovoi organizatsii rabochei pamyati v mladshem shkol’nom vozraste [Development of the brain’s organization of working memory in young schoolchildren]. Fiziologiya Cheloveka [Human Physiology], 2011, vol. 37, no. 1, pp. 5-15 (In Russian).
15. HarriganJ. A. The effect of task order on children’s identification of facial expressions. Motivation and Emotion, 1984,8, pp. 157-169.
16. Hawk S. Т., Van Kleef G. A., Fischer A.H., Van Der Schalk J. “Worth a thousand words”: Absolute and relative decoding of nonlinguistic affect vocalizations. Emotion, 2009, vol. 9, no. 3, p. 293.
17. Hess U., Kappas A., Scherer K. R. Multichannel communication of emotion: Synthetic signal production. 1988.
18. Il'in E. P. Emotsii i chuvstva [Emotions and feelings], SPb.: Piter, 2001, pp. 171-180 (In Russian).
19. Izard С. Е. Psikhologiya emotsii [The Psychology of Emotions] [per. s angl. A. Tatlybaeva, V. Misnik]/ Carroll E. Izard. SPb.: Piter, 2007 (In Russian).
20. Juslin P. N., Scherer K. R. Vocal expression of affect. The new handbook of methods in nonverbal behavior research. 2005, pp. 65-135.
21. Khrizman T. P., Eremeeva V. P., Loskutova T.D. Emotsii, rech' i aktivnost' mozga rebenka [Emotions, speech and brain activities of a child]. M. 1991 (In Russian).
22. Lambrecht L., Kreifelts B., Wildgruber D. Gender differences in emotion recognition: Impact of sensory modality and emotional category. Cognition & emotion, 2014, vol. 28, no. 3, pp. 452-469.
23. Leppanen J. М., Hietanen J. K. Emotion recognition and social adjustment in school-aged girls and boys. Scandinavian Journal of Psychology, 2001, vol. 42, pp. 429-435
24. Machinskaya R. I. Funktsional’noe sozrevanie mozga i formirovanie neirofiziologicheskikh mekhanizmov izbiratel’nogo proizvol’nogo vnimaniyau detei mladshego shkol’nogo vozrasta [Machinskaya R. I. Functional maturation of the brain and formation of the neurophysiological mechanisms of selective voluntary attention in young schoolchildren], Fiziologiya Cheloveka [Human Physiology], 2006, vol. 32, no. 1, p. 26 (In Russian).
25. Matsumoto D., Ekman P. Japanese and Caucasian facial expressions of emotion and neutral faces (JACFEE and JACNeuF). Human Interaction Laboratory, University of California, San Francisco. 1988. vol. 401.
26. Montagne B. et al. Sex differences in the perception of affective facial expressions: Do men really lack emotional sensitivity? Cognitive Processing, 2005, vol. 6, no. 2, pp. 136-141.
27. Morozov V. P. i dr. Vospriyatie rechi: voprosy funktsional’noi asimmetrii mozga [Speech perception: problems of functional brain asymmetry]. Red. Morozov V.P. L., “Nauka”, 1988,135 p. (In Russian).
28. Most Т., Aviner C. Auditory, visual, and auditory-visual perception of emotions by individuals with cochlear implants, hearing aids, and normal hearing. Journal of Deaf Studies and Deaf Education, 2009, vol. 14, no. 4, pp. 449-464.
29. Most Т., Bachar D., Dromi E. Auditory, visual, and auditory-visual identification of emotions by nursery school children./оиггаа/ of Speech-Language Pathology and Applied Behavior Analysis, 2012, vol. 5, pp. 25-34.
30. Most Т., Weisel A., Zaychik A. Auditory, visual and auditory-visual identification of emotions by hearing and hearing-impaired adolescents //British journal of audiology, 1993, vol. 27, no. 4, pp. 247-253.
31. Rosenberg E. L., Ekman P. Conceptual and methodological issues in the judgment of facial expressions of emotion. Motivation and Emotion, 1995, vol. 19, no. 2, pp. 111-138.
32. Russell J. A., Bachorowski J.A., Fernandez-Dols J. M. Facial and vocal expressions of emotion. Annual review of psychology, 2003, vol. 54, no. 1, pp. 329-349.
33. Saxby L, Bryden M.P. Left-ear superiority in children for processing auditory emotional material. Dev. Psychol, 1984, vol. 20, pp. 72-80.
34. Scherer K. R., Banse R., Wallbott H. G. Emotion inferences from vocal expression correlate across languages and cultures .Journal of Cross-cultural psychology, 2001, vol. 32, no. 1, pp. 76-92.
35. Scherer K. R., Ellgring H. Multimodal expression of emotion: Affect programs or componential appraisal patterns? Emotion, 2007, no. 7, pp. 158-171.
36. Scherer K. R. Universality of emotional expression. Encyclopedia of human emotions, 1999, vol. 2, pp. 669-674.
37. Shackman J. E., Poliak S. D. Experiential influences on multimodal perception of emotion. Child development, 2005, vol. 76, no. 5, pp. 1116-1126.
38. Wellman H. M. et al. Early understanding of emotion: Evidence from natural language. Cognition & Emotion, 1995, vol. 9, no. 2-3, pp. 117-149.