The effect of perceptual adaptation to dynamic facial expressions

We present three experiments investigating the perceptual adaptation to dynamic facial emotional expressions. Dynamic expressions of six basic emotions were obtained by video recording of a poser’s face. In Experiment 1 participants (n=20) evaluated the intensity of 6 emotions, neutral state, genuineness and naturalness of dynamic expressions. The validated stimuli were further used as adaptors in Experiments 2 and 3 aimed at exploring the structure of facial expressions perceptual space by adaptation effects. In Experiment 2 participants (n=16) categorized neutral/emotion morphs after adaptation to dynamic expressions. In Experiment 3 (n=26) the task of the first stage was to categorize static frames derived from video records of the poser. Next individual psychometric functions were fitted for each participant and each emotion, to find the frame with emotion recognized correctly in 50% trials. These latter images were presented on the second stage in adaptation experiment, with dynamic video records as adaptors. Based on the three experiments, we found that facial expressions of happiness and sadness are perceived as opponent emotions and mutually facilitate the recognition of each other, whereas disgust and anger, and fear and surprise are perceptually similar and reduce the recognition accuracy of each other. We describe the categorical fields of dynamic facial expressions and of static images of initial phases of expression development. The obtained results suggest that dimensional and categorical approaches to perception of emotions are not mutually exclusive and probably describe different stages of face information processing. The study was supported by the Russian Foundation for Basic Research, project № 15-36-01281 “Structure of dynamic facial expressions perception”.

1. Barabanschikov V.A., Korolkova O.A., Lobodinskaya E.A. Vospriiatie emotsional'nykh ekspressii litsa pri ego maskirovke i kazhushchemsia dvizhenii [Perception of facial expressions during masking and ap- parent motion]. Eksperimentalnaya psikhologiya [Experimental psychology (Russia)], 2015, vol. 8, pp. 7–27 (In Russ., аbstr. in Engl.).
2. Barabanschikov V.A., Zhegallo A.V., Korolkova O.A. Pertseptivnaia kategorizatsiia vyrazhenii litsa [Per- ceptual categorization of facial expressions]. Moscow: Kogito-Tsentr Publ., 2016. 376 p. (In Russ.).
3. Bates D., Mächler M., Bolker B., Walker S. Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software, 2015, vol. 67, no. 1. doi: 10.18637/jss.v067.i01
4. Burton N., Jeffery L., Calder A.J., Rhodes G. How is facial expression coded? Journal of Vision, 2015, vol. 15, no. 1, pp. 1–1. doi: 10.1167/15.1.1
5. Butler A., Oruc I., Fox C.J., Barton J.J.S. Factors contributing to the adaptation aftereffects of facial ex- pression. Brain Research, 2008, vol. 1191, pp. 116–126. doi: 10.1016/j.brainres.2007.10.101
6. Cook R., Matei M., Johnston A. Exploring expression space: adaptation to orthogonal and anti-expres- sions. Journal of vision, 2011, vol. 11, no. 2011, pp. 1–9. doi: 10.1167/11.4.2
7. Dobs K., Bülthoff I., Breidt M., Vuong Q.C., Curio C., Schultz J. Quantifying human sensitivity to spatio-temporal information in dynamic faces. Vision Research, 2014, vol. 100, pp. 78–87. doi: 10.1016/j. visres.2014.04.009
8. Ekman P. Expression and the nature of emotion. Approaches to emotion, 1984, no. 1963, pp. 319–344.
9. Hothorn T., Bretz F., Westfall P. Simultaneous inference in general parametric models. Biometrical jour- nal. Biometrische Zeitschrift, 2008, vol. 50, no. 3, pp. 346–63. doi: 10.1002/bimj.200810425
10. Hsu S., Young A. Adaptation effects in facial expression recognition. Visual Cognition, 2004, vol. 11, no. 7, pp. 871–899. doi: 10.1080/13506280444000030
11. Jaeger T.F. Categorical data analysis: Away from ANOVAs (transformation or not) and towards logit mixed models. Journal of Memory and Language, 2008, vol. 59, no. 4, pp. 434–446. doi: 10.1016/j. jml.2007.11.007
12. Korolkova O.A. Pertseptivnoe prostranstvo i prediktory razlicheniia emotsional’nykh ekspressii litsa [Perceptual space and predictors of emotional facial expression discrimination]. Rossiiskii zhurnal kognitivnoi nauki [The Russian Journal of Cognitive Science], 2014, vol. 1, no. 4, pp. 82–97 (In Russ.; abstr. in Engl.).
13. Korolkova O.A. The role of dynamics in visual adaptation to emotional facial expressions. The Russian Journal of Cognitive Science, 2015, vol. 2, no. 4, pp. 38–57.
14. Kurakova O.A. Sozdanie novoi bazy fotoizobrazhenii estestvennykh perekhodov mezhdu bazovymi emotsional’nymi ekspressiiami litsa [A new database of natural transitions between basic emotional fa- cial expressions]. In V.A. Barabanschikov, A.A. Demidov, D.A. Diveev (eds.), Litso cheloveka kak sredstvo obshcheniia: Mezhdistsiplinarnyi podkhod [Human face as a means of communication: Interdisciplinary ap- proach]. Moscow: Kogito-Tsentr; IPRAS Publ., 2012, pp. 287–309 (In Russ.).
15. Linares D., Lopez-Moliner J. quickpsy: An R package to fit psychometric functions for multiple groups. The R Journal, 2016, v. 8, no. 1, pp. 122–131. URL: https://journal.r-project.org/archive/2016-1/linares-na. pdf
16. Prkachin G.C., Prkachin K.M. Adaptation to facial expressions of emotion. Cognition & Emotion, 1994, vol. 8, no. 1, pp. 55–64. doi: 10.1080/02699939408408928
17. R Core Team R: A Language and Environment for Statistical Computing. Vienna, 2016. URL: https:// cran.r-project.org/
18. de la Rosa S., Giese M., Bulthoff H.H., Curio C. The contribution of different cues of facial movement to the emotional facial expression adaptation aftereffect. Journal of Vision, 2013, vol. 13, no. 1, pp. 23–23. doi: 10.1167/13.1.23
19. Russell J.A. A circumplex model of affect. Journal of Personality and Social Psychology, 1980, vol. 39, no. 6, pp. 1161–1178.  doi: 10.1037/h0077714
20. Russell J.A., Fehr B. Relativity in the perception of emotion in facial expressions. Journal of Experimental Psychology: General, 1987, vol. 116, no. 3, pp. 223–237. doi: 10.1037/0096-3445.116.3.223
21. Rutherford M.D., Chattha H.M., Krysko K.M. The use of aftereffects in the study of relationships among emotion categories. Journal of Experimental Psychology: Human Perception and Performance, 2008, vol. 34, no. 1, pp. 27–40. doi: 10.1037/0096-1523.34.1.27
22. Skinner A.L., Benton C.P. Anti-expression aftereffects reveal prototype-referenced coding of facial expressions. Psychological science, 2010, vol. 21, no. 9, pp. 1248–1253. doi: 10.1177/0956797610380702