Perception of Emotional State of a Communicant Based on Information about His Facial and Body Expressions

The hypothesis about the holistic nature of the perception of expressive behavior is tested using the material of posed dynamic expressions of seven emotional states. The availability of information about facial and body expressions, as well as about the context of the situation (a holistic image of human behavior; an image with blurred face; an image with blurred body and context) serves as the independent variable. The influence of perception conditions on the accuracy of expression recognition is assessed. It is shown that the accuracy of assessing expressions of fear and boredom is significantly reduced when information is available only about facial expressions. On the contrary, the neutral state of the poser is perceived most accurately when the face is blurred. The expression of disappointment is recognized better in conditions of full information than when the face is blurred. No significant changes in the accuracy of assessing the expressions of happiness, disgust, and excitement were found when the perception conditions changed. Analysis of the structure of erroneous answers allowed us to identify the main patterns of mixing perceived emotions. In particular, it was shown that in the absence of information about body movements and the context of the situation, the expression of fear is perceived as “shame” or other states of negative valence. Probably, the key to understanding situational fear is knowledge of the presence of an object that evokes this emotion. Situational manifestations of excitement and interest are similar, and therefore it is quite difficult to reliably distinguish between these emotions.

1. Barabanschikov V.A., Zhegallo A.V. Vospriyatie ekspressii tetcherizovannogo litsa [The perception of facial expression in the Thatcher illusion]. Eksperimental'naâ psihologiâ = Experimental Psychology (Russia), Vol. 4, no. 3, pp. 28—41. (In Russ., аbstr. in Engl.).
2. Barabanschikov V.A., Zhegallo A.V., Ivanova L.A. Raspoznavanie ekspressii perevernutogo izobrazheniya litsa [Recognition of expression of inverted face image]. Eksperimental'naâ psihologiâ = Experimental Psychology (Russia), Vol. 3, no. 3, pp. 66—83. (In Russ., аbstr. in Engl.).
3. Starostina E.G., Taylor G., Quilty L., Bobrov A., Moshnyaga E., Puzyreva N., Bobrova M., Ivashkina M., Krivchikova M., Shavrikova E. [et al.]. Torontskaya Shkala Aleksitimii (20 Punktov): Validizatsiya russkoyazychnoi versii na vyborke terapevticheskikh bol'nykh [Toronto Alexithymia Scale (20 Items): Validation of the Russian version on a sample of medical patients]. Sotsial'naya i klinicheskaya psikhiatriya [Social and Clinical Psychiatry], 2010. Vol. 20, no. 4, pp. 31—38. (In Russ., а in Engl.).
4. Abo Foul Y., Eitan R., Aviezer H. Perceiving emotionally incongruent cues from faces and bodies: Older adults get the whole picture. Psychology and Aging, 2018. Vol. 33, no. 4, pp. 660—666. DOI:10.1037/pag0000255
5. Abramson L., Marom I., Petranker R., Aviezer H. Is fear in your head? A comparison of instructed and real-life expressions of emotion in the face and body. Emotion, 2017. Vol. 17, no. 3, pp. 557—565. DOI:10.1037/emo0000252
6. Albohn D.N., Brandenburg J.C., Kveraga K., Adams R.B. The shared signal hypothesis: Facial and bodily expressions of emotion mutually inform one another. Attention, Perception, & Psychophysics, 2022. Vol. 84, no. 7, pp. 2271—2280. DOI:10.3758/s13414-022-02548-6
7. Aviezer H., Bentin S., Dudarev V., Hassin R.R. The automaticity of emotional face-context integration. Emotion, 2011. Vol. 11, no. 6, pp. 1406—1414. DOI:10.1037/a0023578
8. Aviezer H., Hassin R.R., Ryan J., Grady C., Susskind J., Anderson A., Moscovitch M., Bentin S. Angry, disgusted, or afraid? Studies on the malleability of emotion perception. Psychological Science, 2008. Vol. 19, no. 7, pp. 724—732. DOI:10.1111/j.1467-9280.2008.02148.x
9. Aviezer H., Trope Y., Todorov A. Body cues, not facial expressions, discriminate between intense positive and negative emotions. Science, 2012. Vol. 338, no. 6111, pp. 1225—1229. DOI:10.1126/science.1224313
10. Aviezer H., Trope Y., Todorov A. Holistic person processing: Faces with bodies tell the whole story. Journal of Personality and Social Psychology, 2012. Vol. 103, no. 1, pp. 20—37. DOI:10.1037/a0027411
11. Crawford M.T., Maymon C., Miles N.L., Blackburne K., Tooley M., Grimshaw G.M. Emotion in motion: perceiving fear in the behaviour of individuals from minimal motion capture displays. Cognition and Emotion, 2024. Vol. 38, no. 4, pp. 451—462. DOI:10.1080/02699931.2023.2300748
12. Du S., Martinez A. Image size reveals perception biases of similarity among facial expressions of emotion. Journal of Vision, 2010. Vol. 10, no. 7, pp. 605. DOI:10.1167/10.7.605
13. Ghuman A.S., McDaniel J.R., Martin A. Face Adaptation without a Face. Current Biology, 2010. Vol. 20, no. 1, pp. 32—36. DOI:10.1016/j.cub.2009.10.077
14. Hu Y., Baragchizadeh A., O’Toole A.J. Integrating faces and bodies: Psychological and neural perspectives on whole person perception. Neuroscience and Biobehavioral Reviews, 2020. Vol. 112, no. May 2020, pp. 472—486. DOI:10.1016/j.neubiorev.2020.02.021
15. Izard C.E. The Psychology of Emotions. New York: Springer Science & Business Media, 1991. 452 p.
16. Lecker M., Dotsch R., Bijlstra G., Aviezer H. Bidirectional contextual influence between faces and bodies in emotion perception. Emotion, 2020. Vol. 20, no. 7, pp. 1154—1164. DOI:10.1037/emo0000619
17. Martinez L., Falvello V.B., Aviezer H., Todorov A. Contributions of facial expressions and body language to the rapid perception of dynamic emotions. Cognition and Emotion, 2016. Vol. 30, no. 5, pp. 939—952. DOI:10.1080/02699931.2015.1035229
18. Meeren H.K.M., Heijnsbergen C.C.R.J. van, Gelder B. de. Rapid perceptual integration of facial expression and emotional body language. Proceedings of the National Academy of Sciences of the United States of America, 2005. Vol. 102, no. 45, pp. 16518—165 DOI:10.1073/pnas.0507650102
19. Mondloch C.J. Sad or fearful? The influence of body posture on adults’ and children’s perception of facial displays of emotion. Journal of Experimental Child Psychology, 2012. Vol. 111, no. 2, pp. 180—196. DOI:10.1016/j.jecp.2011.08.003
20. O’Reilly H., Pigat D., Fridenson S., Berggren S., Tal S., Golan O., Bölte S., Baron-Cohen S., Lundqvist D. The EU-Emotion Stimulus Set: A validation study. Behavior Research Methods, 2016. Vol. 48, no. 2, pp. 567—576. DOI:10.3758/s13428-015-0601-4
21. Oh Kruzic C., Kruzic D., Herrera F., Bailenson J. Facial expressions contribute more than body movements to conversational outcomes in avatar-mediated virtual environments. Scientific Reports, 2020. Vol. 10, no. 1. DOI:10.1038/S41598-020-76672-4
22. Palumbo R., D’Ascenzo S., Tommasi L. Cross-category adaptation: exposure to faces produces gender aftereffects in body perception. Psychological Research, 2015. Vol. 79, no. 3, pp. 380—388. DOI:10.1007/s00426-014-0576-2
23. Tanaka J.W., Farah M.J. Parts and wholes in face recognition. The Quarterly journal of experimental psychology. A, Human experimental psychology, 1993. 46, no. 2, pp. 225—245. DOI:10.1080/14640749308401045
24. Teufel C., Westlake M.F., Fletcher P.C., Hagen E. von dem. A hierarchical model of social perception: Psychophysical evidence suggests late rather than early integration of visual information from facial expression and body posture. Cognition, 2019. Vol. 185, pp. 131—143. DOI:10.1016/j.cognition.2018.12.012
25. Willis M.L., Palermo R., Burke D. Judging approachability on the face of it: The influence of face and body expressions on the perception of approachability. Emotion, 2011. Vol. 11, no. 3, pp. 514—523. DOI:10.1037/a0022571
26. Zant T. Van Der, Nelson N.L. Motion Increases Recognition of Naturalistic Postures but not Facial Expressions. Journal of Nonverbal Behavior, 2021. Vol. 45, no. 4, pp. 587—600. DOI:10.1007/S10919-021-00372-4/FIGURES/3